game freezes computer / client crashes

well, i been talked this months now that game is freezing, this affects also the the benchmark test at here

so what is a freeze of meaning? freeze is when some part off code stays at same place too long by not allowing continue the cpu loop inside the code

 

Edited April 7, 2022 by gabberworld

Could this be related to some network issues?

From your uploaded log:

src/steamnetworkingsockets/clientlib/steamnetworkingsockets_lowlevel.cpp (1852) : Assertion Failed: SteamnetworkingSockets service thread waited 92ms for lock!  This directly adds to network latency!  It could be a bug, but it's usually caused by general performance problem such as thread starvation or a debug output handler taking too long.

ERROR: ld.so: object '/home/skuz/.local/share/Steam/ubuntu12_64/gameoverlayrenderer.so' from LD_PRELOAD cannot be preloaded (wrong ELF class: ELFCLASS64): ignored.

assert_20220406153409_3.dmp[20370]: Uploading dump (out-of-process)
/tmp/dumps/assert_20220406153409_3.dmp

src/steamnetworkingsockets/clientlib/steamnetworkingsockets_lowlevel.cpp (1852) : Assertion Failed: SteamnetworkingSockets service thread waited 92ms for lock!  This directly adds to network latency!  It could be a bug, but it's usually caused by general performance problem such as thread starvation or a debug output handler taking too long.

src/steamnetworkingsockets/clientlib/steamnetworkingsockets_lowlevel.cpp (1852) : Assertion Failed: SteamnetworkingSockets service thread waited 1102ms for lock!  This directly adds to network latency!  It could be a bug, but it's usually caused by general performance problem such as thread starvation or a debug output handler taking too long.

src/steamnetworkingsockets/clientlib/steamnetworkingsockets_lowlevel.cpp (1852) : Assertion Failed: SteamnetworkingSockets service thread waited 1102ms for lock!  This directly adds to network latency!  It could be a bug, but it's usually caused by general performance problem such as thread starvation or a debug output handler taking too long.

src/steamnetworkingsockets/clientlib/steamnetworkingsockets_lowlevel.cpp (1852) : Assertion Failed: SteamnetworkingSockets service thread waited 1626ms for lock!  This directly adds to network latency!  It could be a bug, but it's usually caused by general performance problem such as thread starvation or a debug output handler taking too long.

Could you upload your DMP file (/tmp/dumps/assert_20220406153409_3.dmp).

Edited April 7, 2022 by Upload

28 minutes ago, Upload said:

Could this be related to some network issues?

From your uploaded log:

src/steamnetworkingsockets/clientlib/steamnetworkingsockets_lowlevel.cpp (1852) : Assertion Failed: SteamnetworkingSockets service thread waited 92ms for lock!  This directly adds to network latency!  It could be a bug, but it's usually caused by general performance problem such as thread starvation or a debug output handler taking too long.

ERROR: ld.so: object '/home/skuz/.local/share/Steam/ubuntu12_64/gameoverlayrenderer.so' from LD_PRELOAD cannot be preloaded (wrong ELF class: ELFCLASS64): ignored.

assert_20220406153409_3.dmp[20370]: Uploading dump (out-of-process)
/tmp/dumps/assert_20220406153409_3.dmp

src/steamnetworkingsockets/clientlib/steamnetworkingsockets_lowlevel.cpp (1852) : Assertion Failed: SteamnetworkingSockets service thread waited 92ms for lock!  This directly adds to network latency!  It could be a bug, but it's usually caused by general performance problem such as thread starvation or a debug output handler taking too long.

src/steamnetworkingsockets/clientlib/steamnetworkingsockets_lowlevel.cpp (1852) : Assertion Failed: SteamnetworkingSockets service thread waited 1102ms for lock!  This directly adds to network latency!  It could be a bug, but it's usually caused by general performance problem such as thread starvation or a debug output handler taking too long.

src/steamnetworkingsockets/clientlib/steamnetworkingsockets_lowlevel.cpp (1852) : Assertion Failed: SteamnetworkingSockets service thread waited 1102ms for lock!  This directly adds to network latency!  It could be a bug, but it's usually caused by general performance problem such as thread starvation or a debug output handler taking too long.

src/steamnetworkingsockets/clientlib/steamnetworkingsockets_lowlevel.cpp (1852) : Assertion Failed: SteamnetworkingSockets service thread waited 1626ms for lock!  This directly adds to network latency!  It could be a bug, but it's usually caused by general performance problem such as thread starvation or a debug output handler taking too long.

Could you upload your DMP file (/tmp/dumps/assert_20220406153409_3.dmp).

if some off that code part will hang main-thread then yes

There are some related discussions here and here. The SteamnetworkingSockets error is likely a side effect of system performance and not the root cause. I'd start by monitoring memory usage while playing and see if that tells us anything.

Could you attach your save file and .dmp file if you still have it?

first of all, i've been having trouble logging in. i'm using the steam account and since this morning i've been trying to connect to reply to you with the required files but have not been able to given the error shown below (2S119/1) - anyone knows what this is?

so here goes another player.log and dump file from after a freeze yesterday evening

50 minutes ago, EricKlei said:

Could you attach your save file and .dmp file if you still have it?

this game has some mods enabled. i'm sending the save as is, but if that's a problem for dismissal, i can load the save without any mods and resend log+dump+save after a freeze without any enabled

i just didn't do it because i'm using the "Airlock Door" mod (https://steamcommunity.com/sharedfiles/filedetails/?id=2094698134https://steamcommunity.com/sharedfiles/filedetails/?id=2094698134) and if i disable it the doors will turn to blocks of copper, and doing so will spoil the current save, but i can do it for troubleshooting (the freeze will eventually happen anyway)

thanks everyone for your time!

 

assert_20220407134437_3.dmp Player.log Junkyard Cycle 147.sav

Edited April 7, 2022 by skuz

8 hours ago, Upload said:

Could this be related to some network issues?

i forgot to add i tried running the game with steam offline and the problem persisted

in my years off experience about sockets is, if you play with them and use them wrong way then it will hang the threat where its running. of-course i not know if that is a case 

This really sounds like a swap storm, a "normal" thing on linux systems where too much memory pressure leads to the system becoming INCREDIBLY unresponsive as the kernel desperately tries to service memory requests for Peter by stealing from Paul resulting in no progress being made by anyone.

In other words: it's almost definitely related to running out of RAM. top sorted by RAM usage as near to the moment of the crash as possible would be useful to see the breakdown so we can identify next steps.

6 hours ago, nome said:

This really sounds like a swap storm, a "normal" thing on linux systems where too much memory pressure leads to the system becoming INCREDIBLY unresponsive as the kernel desperately tries to service memory requests for Peter by stealing from Paul resulting in no progress being made by anyone.

In other words: it's almost definitely related to running out of RAM. top sorted by RAM usage as near to the moment of the crash as possible would be useful to see the breakdown so we can identify next steps.

well, he does have at there low memory indeed by looking log so you may have point at there. but log say also at there that he played only 134 cycles, by default at game beginning that should not happen if you to have 8gib mem

also when you scroll bottom before the crash, then there a Error info what is releated with Sockets and also directly with steam.

also there is info about the (wrong ELF class: ELFCLASS64) what indicates by fast google search that there is 32bit what conflicts with 64bit

8 hours ago, nome said:

top sorted by RAM usage as near to the moment of the crash as possible would be useful to see the breakdown so we can identify next steps.

how could i get that done? i guess the task manager isn't good for that. for some days i'm trying to get netdata to run but i can't

1 hour ago, gabberworld said:

well, he does have at there low memory indeed by looking log so you may have point at there. but log say also at there that he played only 134 cycles, by default at game beginning that should not happen if you to have 8gib mem

also when you scroll bottom before the crash, then there a Error info what is releated with Sockets and also directly with steam.

also there is info about the (wrong ELF class: ELFCLASS64) what indicates by fast google search that there is 32bit what conflicts with 64bit

yeah an i've run ONI in with this computer, also running manjaro linux for hundreds of cycles more with a previous version of the game (pre-space out) and, as i said before, the only problem was end game lag

8 hours ago, nome said:

This really sounds like a swap storm, a "normal" thing on linux systems where too much memory pressure leads to the system becoming INCREDIBLY unresponsive as the kernel desperately tries to service memory requests for Peter by stealing from Paul resulting in no progress being made by anyone.

btw, i have no swap partition. could that be a problem here?

1 hour ago, skuz said:

yeah an i've run ONI in with this computer, also running manjaro linux for hundreds of cycles more with a previous version of the game (pre-space out) and, as i said before, the only problem was end game lag

 only thing what i could recommend is try reinstall your Steam.

for me personalty is red alert this (wrong ELF class: ELFCLASS64)  not the memory

Edited April 8, 2022 by gabberworld

7 hours ago, skuz said:

how could i get that done? i guess the task manager isn't good for that. for some days i'm trying to get netdata to run but i can't

 

I was using top as an example of a program every install likely has. Any resource monitor works though. For top you open a command line console, type "top", hit enter to run it, then use > to shift the sorting column over to memory. Now all your processes should be sorted by RAM usage (descending). Leave that running and you'll likely see Oxygen Not Included grow a bit as you play. It may be that later in the game it grows just enough to set you over the limit, especially given what you said about swap.

7 hours ago, skuz said:

yeah an i've run ONI in with this computer, also running manjaro linux for hundreds of cycles more with a previous version of the game (pre-space out) and, as i said before, the only problem was end game lag

 

More content == more animations == more memory usage. The eternal battle of live updated games, you have to be always optimizing even just to stand still!

7 hours ago, skuz said:

btw, i have no swap partition. could that be a problem here?

Yes, that means that when you run out of RAM rather than being able to evict wasteful anonymous memory to swap the kernel has no choice but to evict mmaped pages that it's probably actually using. On carefully managed servers having no swap can lead to increased performance, but carefully managed is the key part there. Swap is wiggle room, it lets performance degrade more gradually when you run out of RAM. Without it when you run out of RAM things go bad _fast_.

On 4/8/2022 at 4:54 PM, nome said:

I was using top as an example of a program every install likely has. Any resource monitor works though. For top you open a command line console, type "top", hit enter to run it, then use > to shift the sorting column over to memory. Now all your processes should be sorted by RAM usage (descending). Leave that running and you'll likely see Oxygen Not Included grow a bit as you play. It may be that later in the game it grows just enough to set you over the limit, especially given what you said about swap.

first of all thank you for the tip, i had run top a long time ago but couldn't sort it, heh. never really searched how to do it cuz i never needed to monitor memory usage!

now i have been testing it, having only a terminal window with top running beside ONI, and periodically Alt+Tabbing to check memory usage and specially when the stutter starts. what i've been noticing is when i run the game and stay on the menu, memory usage goes to the 30s% and stays there. while loading a save it steadily climbs to 62 - 64, when the game loads. as long as i don't unpause, it stays there. when i resume the simulation, is starts to grow some more and usually stays around 68 - 70, sometimes going up to 74%. now, every time the game froze and i managed to close it (Alt+Tab to terminal, right-click ONI on panel and close it), it was never higher than 74%. twice the system froze completely, but didn't seem top was updating anymore. a couple more times it just crashed, and in top i just see it go from 70 something to disappearing from the list

so, i don't know if there's some spike that doesn't even get registered in top or if the memory usage never really goes up from there

On 4/8/2022 at 4:54 PM, nome said:

Yes, that means that when you run out of RAM rather than being able to evict wasteful anonymous memory to swap the kernel has no choice but to evict mmaped pages that it's probably actually using. On carefully managed servers having no swap can lead to increased performance, but carefully managed is the key part there. Swap is wiggle room, it lets performance degrade more gradually when you run out of RAM. Without it when you run out of RAM things go bad _fast_.

ok, but i decided not to use swap based on concerns it would unnecessarily waste my SSD. in the guides i've read, a modern PC with 8GB wouldn't really need a swap partition (yes i am a noob, and i'm open to other opinions about this!)

but somehow it doesn't make sense to me that i would NEED swap (not just benefit from having it) not to get ONI freeze my system, because that would also mean another solution would be to just have more memory. i have 8GB RAM memory and ONI's recommended settings for linux are 2GB. is that really the problem?

EDIT: forgot to had that i've also noticed that i basically always get a crash whenever i try to load a save while in-game or even from the menu but after i resumed a save and quit to menu

Edited April 9, 2022 by skuz
formatting and PS

nome is right about memory usage tho, more stuff you have at game more memory it needs, in my resent test showed that for 1000 dues for example you need  somewhere 80GB memory, windows does have the virtual memory so it uses that if you run out off memory, im assuming that swap or whatever it is act similar

i don't know if this relevant or even obvious, but i kept testing and wanted to add something: trying to get a screenshot of top at the moment of the freeze (ONI showing 74,3% memory usage) with the game on background still, i decided to left the frozen system to melt by itself (lol) and actually after about 10mins the client crashed and i had my OS back. tried it two more times and the same thing happened..so i guess they're not total freezes, just very slow crashes...and the screenshots always show a perfectly empty desktop just with the terminal running

47 minutes ago, gabberworld said:

nome is right about memory usage tho, more stuff you have at game more memory it needs, in my resent test showed that for 1000 dues for example you need  somewhere 80GB memory, windows does have the virtual memory so it uses that if you run out off memory, im assuming that swap or whatever it is act similar

that makes sense, but in the current save i have 6 dupes, i'm on cycle 180 and going about quite slowly: little exploring (far from reaching oil or space biomes) and am still going for my first geyser (i'm finally gonna do my perfect NG geyser taming!!), so not many contraptions going on yet. i only have one stable with 7 hatches also..three pips that came from the printing pod disturbing my perfect storage and a few wild critters roaming about on sight

given it was in a previous version, my longest save i had close to 20 dupes, going for several hundred cycles (600 or more) most of the map explored, several ranches, making petroleum and plastic and building my surface base...never made it to space though. i want to believe it's gonna be this time!

10 hours ago, skuz said:

that makes sense, but in the current save i have 6 dupes, i'm on cycle 180 and going about quite slowly: little exploring (far from reaching oil or space biomes) and am still going for my first geyser (i'm finally gonna do my perfect NG geyser taming!!), so not many contraptions going on yet. i only have one stable with 7 hatches also..three pips that came from the printing pod disturbing my perfect storage and a few wild critters roaming about on sight

given it was in a previous version, my longest save i had close to 20 dupes, going for several hundred cycles (600 or more) most of the map explored, several ranches, making petroleum and plastic and building my surface base...never made it to space though. i want to believe it's gonna be this time!

when you start a new map does it crash as well?

i to want point that you could buy yourself more memory tho if motherboard supports that. if that is issue

but i to looked your map and its so small that im not sure its memory issue

Edited April 10, 2022 by gabberworld

3 hours ago, gabberworld said:

when you start a new map does it crash as well?

nope, not at all. i've started several new colonies recently and it hasn't crashed once

On 4/9/2022 at 2:15 PM, skuz said:

but somehow it doesn't make sense to me that i would NEED swap (not just benefit from having it) not to get ONI freeze my system, because that would also mean another solution would be to just have more memory. 

You could absolutely buy more memory, but the sad truth of modern software is that huge amounts of allocated memory just sit around doing nothing so generally swap is a good deal. Linux also has a knob you can tune for swappiness that would let you be VERY conservative about swapping stuff out in order to save your SSD.

Also noteworthy: wear levelling controllers are amazing - you hear about the low number of total writes a particular block of flash can handle and you can't help but feel scared you're going to burn out your drive in a few minutes, but then you see how many GB of writes the drive is rated to handle over its lifespan due to wear levelling algorithms and you realize you'd have to be writing absolutely ludicrous amounts to approach that.

That said they're all valid options. Or wait for more memory optimizations from the game team.

On 4/9/2022 at 4:05 PM, skuz said:

so i guess they're not total freezes, just very slow crashes...and the screenshots always show a perfectly empty desktop just with the terminal running

Yup, that's a swapstorm. The computer's not dead, it's just shuffling the tiny amounts of pageable memory it has available back and forth among the running processes, with each one making a TINY bit of progress each time, until either one makes enough progress to free up a little RAM or one asks for even more and the OS's OOM handler kills it and frees up RAM that way. Hence one trick for avoiding swap storms being using cgroups to run some processes in little enclaves with access to only some of your RAM, so that they crash early without making your window manager unresponsive. chromium based browsers even helpfully set their render processes to a high "Kill me first!" value so that your browser proper stays up and you just get an angry tab you have to reload.

i've been continuing to check out top as i play and, at the moment the stutter starts, i check top and stay looking at it: several times i've seen it getting updated while the freeze is going on, and the memory never climbs or spikes. it stays in the 68 ~ 75% range, the same as while playing.

so, i've never seen it fill up the memory (even taking into account other memory users - while playing the other tasks amount to about 10 ~ 12% memory). is it really established this is a swap storm? because i won't mind to take the time when i can and reinstall manjaro, but i'm starting to be afraid that this problem that is new with the game, not my system, won't get solved and i'll have to wait for more optimizations™ to be able to play the game as I did before...eventually

is there an official way to roll back the game to a previous version? like pre-spaced out?

EDIT: ok i've been reading and learnt i don't need to reinstall the system go get a swap partition going, and actually it seems i can just use a swap file with the same result. now i'm wondering what swappiness should i use. the manjaro wiki suggest a change to 10 (from default 60) for most desktop uses. but since i'm still in doubt that this whole thing is actually lack of memory (since i've never seen it's usage go up to 90%, even) i wonder if it will even get used. guess i'll be testing that

Edited April 11, 2022 by skuz

the problem is solved! i created a 4GB swap file on the SSD and set swappiness to 10 and have not been able to reproduce the crash/freeze or even the stutter (i think i got a very slight stutter when swap starts to get used, for 2 seconds or so). i can even load games on demand now

i'm happy that that my fear was unfounded, but still confused by the fact that top had never shown memory use of more than 85% before

the fact is that it now shows, while playing, that swap file gets used up until around 2GB (only while loading a game during a current game. it spikes heavily during that)

how can i mark the thread as fixed/solved?

thanks so much everyone, specially nome for the solution

Edited April 11, 2022 by skuz
spelling

2 hours ago, skuz said:

the problem is solved! i created a 4GB swap file on the SSD and set swappiness to 10 and have not been able to reproduce the crash/freeze or even the stutter (i think i got a very slight stutter when swap starts to get used, for 2 seconds or so). i can even load games on demand now

i'm happy that that my fear was unfounded, but still confused by the fact that top had never shown memory use of more than 85% before

the fact is that it now shows, while playing, that swap file gets used up until around 2GB (only while loading a game during a current game. it spikes heavily during that)

how can i mark the thread as fixed/solved?

thanks so much everyone, specially nome for the solution

users in windows don't need worry about the swap file or as called pagefile.sys because its enabled by default. but if there is no room they to get the popup error message that there no room left. also in my experience in windows xp days was, by disable this it also crash system if you run out off memory.

but if you pc start use that for example the game. it also reduces the game performance

Edited April 11, 2022 by gabberworld

Glad to hear adding swap space fixed the issue. We've also been working on memory optimizations as part of the next update.

6 hours ago, skuz said:

i'm happy that that my fear was unfounded, but still confused by the fact that top had never shown memory use of more than 85% before

To be clear, that 85% was what? The usage of Oxygen Not Included? The rest of your running programs, like your OS, need RAM too.

On 4/11/2022 at 8:38 PM, nome said:

To be clear, that 85% was what? The usage of Oxygen Not Included? The rest of your running programs, like your OS, need RAM too.

if you read several of my previous post you'll have the answer. 85% was referring to total memory usage, ONI was topping at 75%. all this from what I could see on top, hence my doubts