Solution to the Bad Saving in Klei Games

[Onidotmoe]

When saving the game is basically frozen to the player, this is true for Oxygen Not Included and Don't Starve Together, tho there is a solution and that's to set a bounty for a solution, this is common practice when dealing with problems that are beyond one's own abilities.

For example :
Millennium Prize Problems, offering $1 Million for a solution to each of the remaining 6 out 7 Math problems.
Whole bunch of Prize money if one can prove the existence of the supernatural

Microsoft offers bounties for any security vulnerabilities

Google has a similar program

Netflix had its own prize for people that could create a better algorithm than their own

If there was a public prize available from Klei, people would come to solve the problem.

And no, a background worker wouldn't work.

[Yobbo]

I don't think this is a case of ability, more being literally impossible.

If you advance the game state before finishing serialization, then the saved game state will be inconsistent.

If you can remember the game state from some time in the past, allowing you to save a consistent state even while advancing the game... then you have already saved it.

You could of course spread out saving so that incremental state changes are continuously saved every game tick. However as the incremental changes of temperature and gas pressure involve pretty much the entire map, i very much doubt this would be an improvement.

Backwards simulation during serialization to determine past state... is probably not worth mentioning here.

[Onidotmoe]

1 hour ago, Yobbo said:

I don't think this is a case of ability, more being literally impossible.

If you advance the game state before finishing serialization, then the saved game state will be inconsistent.

If you can remember the game state from some time in the past, allowing you to save a consistent state even while advancing the game... then you have already saved it.

You could of course spread out saving so that incremental state changes are continuously saved every game tick. However as the incremental changes of temperature and gas pressure involve pretty much the entire map, i very much doubt this would be an improvement.

Backwards simulation during serialization to determine past state... is probably not worth mentioning here.

Other games manage well enough without freezing for a safe.

An easier solution would be of partitioning the map into different sectors and save stuff that's static, then proceeded to save more important things afterwards.

There's always a solution to these sorts of problems, you don't need to try to make the current saving method work better, you could just use a different one altogether.

[rgduck]

On 8/30/2021 at 6:19 PM, Yobbo said:

If you advance the game state before finishing serialization, then the saved game state will be inconsistent.

If you can remember the game state from some time in the past, allowing you to save a consistent state even while advancing the game... then you have already saved it.

Pause just to cache the game state in memory (fast), then spread the serialization out over the next minute?

[Yobbo]

11 hours ago, rgduck said:

Pause just to cache the game state in memory (fast), then spread the serialization out over the next minute?

I dunno, ONI already happily eats more than half of my RAM and i have a lot of RAM...

[rgduck]

16 hours ago, Yobbo said:

I dunno, ONI already happily eats more than half of my RAM and i have a lot of RAM...

Wow... you're not kidding. Mine's using 7GB right now. I'd never actually checked.

But game state can't be all of that, or even a large fraction... er... right? The save files are only 3 MB. That's compressed as hell, I realize, but still.

I'm sure I'm missing something. Otherwise this would be a really easy solution, and Klei obviously has some really talented devs.

[Yobbo]

On 9/7/2021 at 6:13 AM, rgduck said:

But game state can't be all of that, or even a large fraction... er... right? The save files are only 3 MB. That's compressed as hell, I realize, but still.

Dunno, there can be a huge amount that's kept in memory but technically doesn't need to be put in a save file. Take a duplicant for example - to save their state you probably just need to save who they are, where they are, how they are feeling, and what they are currently doing. But then when you load them into the game you might need to have megabytes of stuff to deal with how they make decisions, how they affect the world, how the world affects them, and so on. The same goes for pretty much everything.

Without access to the source code it's difficult to speculate. But most likely the time it takes right now to save things, is almost exactly the time it takes to go through every individual thing and copy only the important savable bits into memory somewhere else.

[rgduck]

10 hours ago, Yobbo said:

Without access to the source code it's difficult to speculate. But most likely the time it takes right now to save things, is almost exactly the time it takes to go through every individual thing and copy only the important savable bits into memory somewhere else.

They must be doing something more CPU intensive than that.

The game already has to go through every individual object each time it updates. Copying an object's state must be faster than updating it, since an update requires reading that object's state and then running the game's logic on it, and that logic can be pretty sophisticated. That means that copying the entire game state has to be faster than a single update. And this doesn't even account for tasks like pathfinding, which people here keep telling me is the bulk of the game's CPU time (i.e. it dramatically outweighs updates).

Since updates happen several times a second and copying the game state must be faster than that, just copying the game state instead of fully saving it should be way faster than the 3-5 seconds it takes to autosave right now.

[Yobbo]

15 hours ago, rgduck said:

The game already has to go through every individual object each time it updates.

It doesn't. You can see when you load a game that it takes several seconds for all of the plants to notice they can breathe for example. They get updated in slices, with only a small selection updated every tick. I assume many other game objects also work this way.

There are likely also many things that sit in memory but do not need to be accessed or updated until relevant. For example a building might have a bunch of things it needs to remember in order to work, but only be checked or updated when interacted with, which can happen very infrequently.

If everything were updated every game tick, then as you say it would be simple to copy the state.

[rgduck]

7 hours ago, Yobbo said:

It doesn't. You can see when you load a game that it takes several seconds for all of the plants to notice they can breathe for example. They get updated in slices, with only a small selection updated every tick. I assume many other game objects also work this way.

That's a really good point. I retract my claim.

My intuition is still that it shouldn't require several seconds to iterate over everything and write down the important bits, or that it ought to be cheap to maintain a separate game state object that gets updated every time an individual object changes state. But I think you must be right that, for whatever reason, it does and it isn't.