Archived

This topic is now archived and is closed to further replies.

Please be aware that the content of this thread may be outdated and no longer applicable.

debugman18

Storing data across game saves?

Recommended Posts

debugman18    1,104

So, without going into too much detail (I'd rather not spoil the mod yet except for those who may grasp what I'm doing), I'm trying to store a table and its keys inside of the persistentdata table within the PlayerProfile class. I'm using the AddGlobalClassPostConstruct function to do this.

 

I've successfully inserted my table, but it seems that it isn't keeping the data persistent. My goal is to keep the table persistent through game loads, including keys and their values (which may be arbitrarily changed.)

 

Does anyone have any tips for cleanly (as in keeping stability to a maximum) storing arbitrary data for use across saves? I am aware was a mod called "Amanager" that popped up a while back, but I can't find its files anywhere, nor am I even sure it was able to achieve the same thing.

Share this post


Link to post
Share on other sites
Blueberrys    172

@debugman18

Amanager mod on Github

 

Would load/save time efficiency be an issue? You could probably store the keys and values as one object per pair inside a table and parse through them manually, instead of depending on the game to handle it.

Something like this, perhaps.

-- Datalocal correct_data = {    1={"some value"},    2={"new thing", "something else"},    4={3, 55, "string"},}-- Savinglocal data_to_save = {}for k, v in pairs(correct_data) do    table.insert(data_to_save, {key=k, value=v})end-- Save data_to_save to profile..-- The above should be equivalent to:---- data_to_save = {--     {key=1, value={"some value"},--     {key=2, value={"new thing", "something else"},--     {key=4, value={3, 55, "string"},-- }-- Loadinglocal loaded_data -- load from profile..local correct_data = {}for k, v in pairs(loaded_data) do    correct_data[v.key] = v.valueend

Share this post


Link to post
Share on other sites
debugman18    1,104

 

@debugman18

Amanager mod on Github

 

Would load/save time efficiency be an issue? You could probably store the keys and values as one object per pair inside a table and parse through them manually, instead of depending on the game to handle it.

Something like this, perhaps.

-- Datalocal correct_data = {    1={"some value"},    2={"new thing", "something else"},    4={3, 55, "string"},}-- Savinglocal data_to_save = {}for k, v in pairs(correct_data) do    table.insert(data_to_save, {key=k, value=v})end-- Save data_to_save to profile..-- The above should be equivalent to:---- data_to_save = {--     {key=1, value={"some value"},--     {key=2, value={"new thing", "something else"},--     {key=4, value={3, 55, "string"},-- }-- Loadinglocal loaded_data -- load from profile..local correct_data = {}for k, v in pairs(loaded_data) do    correct_data[v.key] = v.valueend

 

Thanks for the link! As I suspected, it wouldn't do what I want.

 

I'm not sure how your snippet solves the issue, though. I'm able to successfully add my table to persistdata, but closing and reopening the game loses the data.

 

I do a simple check to make sure my table doesn't already exist in persistdata before I insert it, but it always comes up nonexistent.

 

    -- Avoid redundancy and errors.    if self.persistdata then        if not self.persistdata.feats then            print("Feats will now persist.")            self.persistdata.feats = {"debug"}            self.dirty = true            self:Save()        else            print("Feats already persist.")        end    end

 

To make sure we're on the same page, I'm not doing this with a component (as that would be lost with the save) and I intend to load the table for use in a menu. I know unlocked characters are stored in persistdata as a table, but I'm not entirely certain how the game saves/loads that table.

 

I know that Save() and Load() are used, along with Set().

function PlayerProfile:Save(callback)    Print( VERBOSITY.DEBUG, "SAVING" )    if self.dirty then        local str = json.encode(self.persistdata)        local insz, outsz = SavePersistentString(self:GetSaveName(), str, ENCODE_SAVES, callback)    else        if callback then            callback(true)        end    endendfunction PlayerProfile:Load(callback)    TheSim:GetPersistentString(self:GetSaveName(),        function(load_success, str)            self:Set( str, callback )        end, false)        SaveGameIndex:GuaranteeMinNumSlots(NUM_SAVE_SLOTS)end

Set() makes this call:

self.persistdata = TrackedAssert("TheSim:GetPersistentString profile",  json.decode, str)

I'm not sure why it's not doing it correctly.

 

I can't seem to find TheSim:GetPersistentString(), either for reference, for whatever reason. I can find usecases, but those don't seem to help.

 

Edit:

 

I looked at amanager further, and I'm trying to understand how it works. It looks like it attaches a component to the Sim, but I don't think that would be accessible when the world isn't loaded? I still believe it doesn't have the capability I'm looking for.

Share this post


Link to post
Share on other sites
Blueberrys    172

@debugman18

 

Apologies, I misunderstood the problem. I thought the keys and values were not matching up correctly.

 

Judging from these posts, I believe the data needs to be serialized to save correctly.

The AddWorldCustomizationPreset function seems to be using DataDumper:

local data = DataDumper(presets, nil, false) 

I'm not sure which serialization methods are available to mods, though. Test if you can access that one, or another one of these?

If none are available, you might have to implement one manually. That also raises another question, have you tried if you can save and retrieve a string variable successfully?

 

Edit: Fixed first link

 

Edit 2: Just tested DataDumper, it is accessible but doesn't seem to solve the problem. Strings aren't working either.

Share this post


Link to post
Share on other sites
debugman18    1,104

@debugman18

 

Apologies, I misunderstood the problem. I thought the keys and values not matching up correctly.

 

Judging from these posts, I believe the data needs to be serialized to save correctly.

The AddWorldCustomizationPreset function seems to be using DataDumper:

local data = DataDumper(presets, nil, false) 

I'm not sure which serialization methods are available to mods, though. Test if you can access that one, or another one of these?

If none are available, you might have to implement one manually. That also raises another question, have you tried if you can save and retrieve a string variable successfully?

 

Edit: Fixed first link

 

No worries! I appreciate you taking the time to even reply. This seems like it's an unusual problem.

 

Simple strings don't save/load, unfortunately. Neither do booleans.

 

local function DoCharacterUnlock(inst, whendone)    GetPlayer().profile:UnlockCharacter("waxwell")  --unlock waxwell        GetPlayer().profile:SetValue("characterinthrone", SaveGameIndex:GetSlotCharacter() or "wilson") --The character that will be locked next time.        GetPlayer().profile.dirty = true    GetPlayer().profile:Save(whendone)end

 

The function used to change the character in the throne is very straightforward.

 

I'm not sure why saving/loading variables from my end isn't working. I know I'm inserting to the right table (ignoring the fact that I'm doing a globalclasspostconstruct), since I'm printing the values of the persistdata table to ensure I'm actually putting everything in its place.

Share this post


Link to post
Share on other sites
Blueberrys    172

@debugman18

 

I found a work around. It's pretty messy, bit it'll work.

 

If you replace the "Set" function and print the value of "str", you'll see that the it does indeed get saved, but does not load into the persistdata variable correctly. I'm assuming there is some sort of lock on which values are read and the rest are discarded.

self.old_Set = self.Setself.Set = function(self, str, callback)	print(str)	self:old_Set(str, callback)end

Edit: The str variable is in json format, use this to extract your variable.

data = GLOBAL.json.decode(str)local your_var = data["your_var_name"]

Edit 2: Fixed things in both code snips

Share this post


Link to post
Share on other sites
debugman18    1,104

@debugman18

 

I found a work around. It's pretty messy, bit it'll work.

 

If you replace the "Set" function and print the value of "str", you'll see that the it does indeed get saved, but does not load into the persistdata variable correctly. I'm assuming there is some sort of lock on which values are read and the rest are discarded.

self.old_Set = self.Setself.Set = function(self, str, callback)	print(str)	self:old_Set(str, callback)end

Edit: The str variable is in json format, use this to extract your variable.

data = GLOBAL.json.decode(str)local your_var = data["your_var_name"]

Edit 2: Fixed things in both code snips

 

Ah, it's not as clean as I would have liked, but it's probably as clean as it's going to get.

 

Thanks for your help! I'll use this to get things working. Hopefully it'll go smoothly.

Share this post


Link to post
Share on other sites
debugman18    1,104

@Blueberrys

I got it working thanks to your snippets. It's pretty hacky, admittedly, but it works!

 

I take the extracted table from the saved str variable, and I change a temporary nil variable into the extracted table (loaded_feats). If loaded_feats doesn't exist (or rather, if it's nil) on load, I create the empty feats table in persistdata. If loaded_feats does exist, I insert loaded_feats keys into the feats table within persistdata.

Share this post


Link to post
Share on other sites
Blueberrys    172

@debugman18 Heya. I've come up with a better solution for storing data.

 

Edit 5: Moved to downloads section: Persistent Data

 

Post preserved for reference.

Save this module in your mod's scripts directory.

--[[By Blueberrys]]local PersistentData = Class(function(self, id)	self.persistdata = {}	self.dirty = true	self.id = idend)local function trim(s)	return s:match'^%s*(.*%S)%s*$' or ''endfunction PersistentData:GetSaveName()	return BRANCH == "release" and self.id or self.id .. BRANCHendfunction PersistentData:SetValue(key, value)	self.persistdata[key] = value	self.dirty = trueendfunction PersistentData:GetValue(key)	return self.persistdata[key]endfunction PersistentData:Save(callback)	if self.dirty then		local str = json.encode(self.persistdata)		local insz, outsz = SavePersistentString(self:GetSaveName(), str, ENCODE_SAVES, callback)	elseif callback then		callback(true)	endendfunction PersistentData:Load(callback)	TheSim:GetPersistentString(self:GetSaveName(),		function(load_success, str)			-- Can ignore the successfulness cause we check the string			self:Set( str, callback )		end, false)endfunction PersistentData:Set(str, callback)	if str and trim(str) ~= "" then		self.persistdata = json.decode(str)		self.dirty = false	end	if callback then		callback(true)	endendreturn PersistentData

Example usage:

local PersistentData = require "persistentdata"local Data = PersistentData("SomeDataID")local key = "key"local data = {"Data", "Data"}local function Save()	Data:SetValue(key, data)	Data:Save()endlocal function Load()	Data:Load()	print(Data:GetValue(key))end-- Uncomment these to test-- Save()-- Load()

This will create a separate file for the data in the game's save folder. I think that's much cleaner than using the profile-data file.

 

Edit: Added data id. Save files will be named accordingly. The module can be used multiple times as long as the id is unique per instance.

 

Edit 2: Added a check for "str" upon loading. Now it won't crash if the file was not saved beforehand.

 

Share this post


Link to post
Share on other sites
seronis    130

Why use a separate module that contains the playerprofile routines,  instead of just using the playerprofile routines ?

Share this post


Link to post
Share on other sites
Blueberrys    172

@seronis

Using the playerprofile module directly posed a problem because it did not retrieve custom the data from the serialized string.

 

Previously, the solution was to overwrite a function of the playerprofile to enable retrieving custom data. While it did what was needed, it was pretty ugly and couldn't be reused as-is. If multiple scripts/mods modified the profile function that way, they would link up and do the exact same thing every time (save the old fn, replace it with new one, retrieve portion of custom data, return old fn).

 

This module also provides flexibility in using separate files. Storing all mod data into the profile's data file seems a little strange, especially for mods that have nothing to do with profile data.

 

We could also use a new instance of the profile module, but that would conflict with the original one as they read and write to the same file. Personally, I also think that would be overkill for a simple task like saving and loading.

 

The game itself also uses multiple modules for saving and loading. Namely, playerdeaths, which has a very similar format to this and playerprofile.

Share this post


Link to post
Share on other sites
seronis    130

Makes sense  -except- for the part about mods saving to profile needing to hook the save function.  The player profile has setValue/getValue functions so you can store your entire table of stuff under one key unique to a given mod.  But everything else still makes your module more useful than I was thinking at first glance.  Its elegant code either way =-)

Share this post


Link to post
Share on other sites
Blueberrys    172

@seronis The profile module does have set/get functions, but they don't actually work for custom values. The custom values are discarded somewhere in the module's "Set" function when the saved data is loaded. That's why we had to overwrite the Set function and retrieve the data manually.

Share this post


Link to post
Share on other sites
debugman18    1,104

For those interested, the mod I needed this for is in a pretty functional state now. It's not finished by any stretch, but it does what it's supposed to. 

 

GitHub page.

 

The feats screen doesn't support scrolling yet, but I plan on implementing it.

Share this post


Link to post
Share on other sites