Why does saving creates the file bigger / game slower?

[meepmoop]

It was observed that constant saving / logging out and in a game makes the game file bigger and decreases the performance in a game.

This makes oni hard to play-in-sessions, and pushes towards a one-sit-game.

My question is what makes it so? Why cant the game run smoothly throught multiple sessions? Does anyone has an inside-insight into how the game preforms?

[Gurgel]

I have not noticed that. Any numbers on this?

[Xenotrix]

I did not quite understand what the author meant, but I noticed that the game does not clear the RAM before loading the save file. If you load saves many times, then each time the loading becomes longer, and in the game lags begin. This is fixed by exiting the game and restarting it.

Screenshot example:
1. Load the save on a fresh start
2. The game consumes ~ 5 GB of RAM
3. Then load the same save 15+ times from the game
4. After that, the game consumes already ~ 10GB RAM
5. It becomes impossible to play, lags, etc.
6. To free memory, you need to exit the game and restart

Spoiler

 

[psusi]

11 hours ago, meepmoop said:

It was observed that constant saving / logging out and in a game makes the game file bigger and decreases the performance in a game

By whom? The save does get larger the longer you play because it is storing all of the historical reports and a map screen shot every cycle.  Are you saying that if you pause the game and save it 10 times, each time the file gets larger even though no time has passed?

 

[Risu]

The daily reports that nobody cares about contributes a lot to the save bloat over time.
 

[PIRATE D KING]

17 hours ago, Xenotrix said:

I did not quite understand what the author meant, but I noticed that the game does not clear the RAM before loading the save file. If you load saves many times, then each time the loading becomes longer, and in the game lags begin. This is fixed by exiting the game and restarting it.

Screenshot example:
1. Load the save on a fresh start
2. The game consumes ~ 5 GB of RAM
3. Then load the same save 15+ times from the game
4. After that, the game consumes already ~ 10GB RAM
5. It becomes impossible to play, lags, etc.
6. To free memory, you need to exit the game and restart

  Reveal hidden contents

 

indeed, i get forced to close the game and open it u again after having to load 2 or 3 times

[RonEmpire]

I have mentioned this already during early access.    

[SharraShimada]

It is faster to save > quit > start load than load a game from another running game. So there is no benefit of loading a game while the game is already running another one. 

[MarcinW]

On 3/23/2020 at 3:35 PM, Risu said:

The daily reports that nobody cares about contributes a lot to the save bloat over time.
 

That is not true. Stats are just numbers, so if we got 16 stats (each 4 bytes of memory), then 1000 cycles of stats takes only 64kB of memory  My ~2000cycle save is only ~13MB of data big.

As you play you discover more map, more items are going to save file and more calculations need to be performed. So it may decrease performance on slower machines.
In oni to reduce save file size not everything is saved, this is why we see everything a little bit off after save load. No errands are saved or internal statuses of plenty items. They are calculated after load. On my machine it takes 4,5GB of RAM - not so much I guess.
Loading seems to be on single thread - this is why it loads so long, but it could be hard to change it at current stage.
Going to menu and resuming game may increase memory use, but this is not something user should worry about. Game is written with unity .net. In "garbage collector" programming languages if memory is not needed for anything else it is not released. It will be release only if system runs of memory or is idle.

But even in garbage collector word you can make memory leaks which are so hard to trace and it may happen that best solution is simply to reset game after longer gameplay. Combine that with multithreading and we get worst kind of bugs to fix. They can take days of weeks to trace and if they dont happen frequently then it is better to leave them alone. Simply... "it works on my machine"  
To summarize: get a machine as dev's have. On my i7-8700K, 16GB RAM & ssd drive game works perfect. Only midnight save is a little bit annoying, but changing it in options to 5 cycles helps  On my nephew i5-4500 it works so bad that playing more than 100 cycles would be real pain. You just can't do anything about it other than upgrade your machine.

Just a senior programmer, speaking. Unfortunately not at Klei  

[Chipplyman]

1 hour ago, MarcinW said:

That is not true. Stats are just numbers

In the context of the save file, stats are not just numbers. They're numbers plus the schema metadata required to serialize and deserialize them. That metadata can be quite large, especially when serializing as xml which iirc oni does. Even the less verbose json format usually requires more bytes for metadata than for data. Serializing to a binary format needs very little metadata, but serialization code must be much more complex and brittle to handle schema changes across versions. 

[MarcinW]

Yeah, but you are getting into tech stuff. But looking into .sav file I dont see any xml, just some json and .net serialization.
But that is not a point here, which is that save files are relatively small (in MB) compared to memory usage (in GB) after game load.

I only wonder what is causing save lag. Maps size is 256x384, so 98 304 tiles of data with multiple debris and building which is already in memory. Still it doesn't seems to be so much to cause that kind of lag, there must be something else happening. What a fun challenges Klei programers face every day! Not just a boring 500 Internal Server Error like most of .net programmers  

[psusi]

6 hours ago, SharraShimada said:

It is faster to save > quit > start load than load a game from another running game. So there is no benefit of loading a game while the game is already running another one. 

One really wonders what they did so wrong to cause this.

[sheaker]

11 hours ago, psusi said:

One really wonders what they did so wrong to cause this.

There are many games working like that. For example EU IV from Paradox. There must be something the can not overcome.

[psusi]

On 3/29/2020 at 3:25 AM, sheaker said:

There are many games working like that. For example EU IV from Paradox. There must be something the can not overcome.

Hiring competent programmers? 

I mean, seriously, whatever memory is allocated when you load a game, should be freed when you unload it, leaving you right back where you started.  If they would quick leaking memory like a sieve, then loading another save wouldn't suck so hard.

[Gurgel]

On 3/28/2020 at 9:12 PM, psusi said:

One really wonders what they did so wrong to cause this.

Maybe not a lot. Memory leaks like this are quite often the fault of the libraries and framework used, at least when the application coders are competent. 

1 hour ago, psusi said:

Hiring competent programmers? 

I mean, seriously, whatever memory is allocated when you load a game, should be freed when you unload it, leaving you right back where you started.  If they would quick leaking memory like a sieve, then loading another save wouldn't suck so hard.

When you load a save, you can should be able to get rid of anything you allocated yourself. That is why I think this may be a problem not in the code Klei wrote. That said, memory management is hard as soon as you have complex data with a lot of references.

[psusi]

On 4/6/2020 at 5:38 PM, Gurgel said:

When you load a save, you can should be able to get rid of anything you allocated yourself. That is why I think this may be a problem not in the code Klei wrote

I think the guys writing Unity know what they are doing.

On 4/6/2020 at 5:38 PM, Gurgel said:

That said, memory management is hard as soon as you have complex data with a lot of references.

It isn't hard; it's just easy to make mistakes.  There are plenty of good tools to help find and fix those mistakes, but I guess you have to care to do so.  When nobody gives a damn, then they don't find and fix the mistakes.  I guess that's the real problem. They don't figure that fixing memory leaks is going to sell one more copy of the game, so they don't put any time into it.