Low Proformace Issues

[chemie]

11 hours ago, melquiades said:

It is not because of compression, it is because Kley decided the save file should be encripted... for some reason.

Utterly unnecesary at best. Annoying for sure.

That makes it worse than I thought.  No reason to encrypt it...at all.  ever. 

[Lancar]

6 hours ago, cpy said:

Looks like whole piping system is on 1 core. I though that gas and liquids will use separate cores since they don't interact much?

Uhh and temperature calculations on 1 core and path finding on 1 core? Now we have like 5 cores used! I guess it's really hard to code but who knows which system uses most CPU.

Piping (including conveyors) is a good fit because its interactions with everything else is fairly limited.
Temperature system might be hard to move to its own core since it interacts with basically every other material in the game, including free flowing gas, which in mid-late game there's likely to be quite a lot of.
Pathfinding definitely is a good candidate, and also I'm thinking maybe the solids handling (debris) would work too (perhaps in conjuction, since those two things regularly interact)

[ToiDiaeRaRIsuOy]

8 hours ago, cpy said:

Looks like whole piping system is on 1 core. I though that gas and liquids will use separate cores since they don't interact much?

Uhh and temperature calculations on 1 core and path finding on 1 core? Now we have like 5 cores used! I guess it's really hard to code but who knows which system uses most CPU.

Uh... May I ask your testing methodology for that? Unless you created a vast maze of pipes, I'm not even sure you'd be able to get a read on it.

For the record, the update only said this:

Quote

• Flow of gasses/liquids in pipes is now multithreaded for improved CPU usage

So it's only about the flow, the amount of fluids moving through the pipes and possible bottle necks hindering flow. The change says nothing about thermal energy exchange with the environment of said pipes, so you have to assume it doesn't.

I wonder if there's a possibility to shift workload to the GPU. A GPU is more made to make millions of random calculations.

[LeftyRighty]

13 hours ago, melquiades said:

It is not because of compression, it is because Kley decided the save file should be encripted... for some reason.

Utterly unnecesary at best. Annoying for sure.

more likely they're just saving the object rather than wasting processing time parsing it to a readable text file... 

 

cores/threading is most like a combination of two things:

 

unity is only just getting their job/threading/batch core into beta, prior to that you're looking at integrating third party threading solutions which isn't a simple task (that said I'd love to see what a game like this running on that new job system would run like!! shame it's a total redesign from base principles and not just something you can add in at this stage of development... although there is the hybrid approach... *hint hint devs* :P)

without a threading solution built into the engine third party solutions are limited, there is always going to be one main thread taking the burden and farming out applicable tasks to a threaded sub processing script; and not all tasks can be threaded (just hire more programmers to get the project done doesn't work either  ) 

[cpy]

2 hours ago, ToiDiaeRaRIsuOy said:

Uh... May I ask your testing methodology for that? Unless you created a vast maze of pipes, I'm not even sure you'd be able to get a read on it.

For the record, the update only said this:

So it's only about the flow, the amount of fluids moving through the pipes and possible bottle necks hindering flow. The change says nothing about thermal energy exchange with the environment of said pipes, so you have to assume it doesn't.

I wonder if there's a possibility to shift workload to the GPU. A GPU is more made to make millions of random calculations.

Experience in how coding optimizations are rolled out? First you do 1 thread offload since it's easiest to do. Multithreading concurrent calculations is way harder than doing simulation and flow at the same time then syncing it all together.

Now imagine you have to sync all subgrids of pipes together. I guess it's possible they did 1 thread per subgrid since they probably won't have big problem with it.

Things to be tested is 10 long pipes vs 1 ultra long pipe performance.

Easiest would be for developer to jump in and say if they did 1 thread for it all or 1 thread per type or 1 thread per subgrid or went full retard and did 1 thread per 1 segment of a subgrid.

Last one would give you boost no matter how long your pipe would be.

Pathfinding is always CPU intensive so there's your golden chicken. Why we didn't invent some sort of super efficient pathfinding? Now with tensor cores in GPU, we actually might have chance for smart AI pathfinding. Who knows?

[SackMaggie]

12 hours ago, cpy said:

Looks like whole piping system is on 1 core.

• Flow of gasses/liquids in pipes is now multithreaded for improved CPU usage

Since QOL mk1 in optimization part.

 

[cpy]

11 minutes ago, SackMaggie said:

• Flow of gasses/liquids in pipes is now multithreaded for improved CPU usage

Since QOL mk1 in optimization part.

 

Still does not say if it's 1 core or more. Point of your post?

My growing base uses 1+0.5 cores. So yeah not much proof that it uses more than 1 for piping.

[SackMaggie]

2 minutes ago, cpy said:

Still does not say if it's 1 core or more. Point of your post?

Just to correct you that it was improved to multi thread already, And word multi is mean more than 1 ?

[cpy]

Just now, SackMaggie said:

Just to correct you that it was improved to multi thread already, And word multi is mean more than 1 ?

Piping base game runs on 1 core and piping seems to be running on 1. So yeah 1+1 is multi threading. Still 1 core for piping. This is how usually games start to introduce multithreaded optimizations. They add 1 core for some demanding system or systems that are easiest to MT.

[SackMaggie]

1 minute ago, cpy said:

Piping base game runs on 1 core and piping seems to be running on 1. So yeah 1+1 is multi threading. Still 1 core for piping.

Well I'm kinda confusing here, Are you mixing a word "core" and "thread" ? I'm sure it's not same thing.

Took me a long time to think what do you mean?

What you saying was like Piping = 1 core and Piping = 1 core -> So 1+1 = 2 (multi thread ?? It should be multi core ?). 1 core = Piping.

Is this some math?

And I've not heard of multicore coding before but I've learn that there's multi thread coding.

[cpy]

You observe multithreading by observing CPU cores. Please stay out of this discussion it's too techy for you obviously. Threading is a complex thing and I don't really want to explain how threading and cores work together.

Modern windows and CPU do lot of crazy stuff so it's not very apparent for tech newbies.

[ToiDiaeRaRIsuOy]

42 minutes ago, cpy said:

You observe multithreading by observing CPU cores. Please stay out of this discussion it's too techy for you obviously. Threading is a complex thing and I don't really want to explain how threading and cores work together.

Modern windows and CPU do lot of crazy stuff so it's not very apparent for tech newbies.

Let's not get condencending please. If you feel somebody's knowledge is not up to standard, then try to explain in simple terms. But don't exclude them from the discussion like that.

There's a simple confusion where people call a thread a core and vice versa. Obviously the game only ran, until recently with the preview patch, on a single thread.

Just for the record, I am by no means an expert and am basing myself on the very limited information I have gathered. So yes, multi threading for games is difficult. But, I will take Klei's word for it they use it for pipe flow 

[cpy]

I am REALLY BAD at teaching others anything. That's why. When I start to explain **** not even PHD docs can understand me. So yeah I'm not the person you want to be your teacher. I've been told countless times how bad I'm at teaching stuff.

Then there's complex topic about multithreading. GG FTW.

[LeftyRighty]

5 hours ago, SackMaggie said:

Well I'm kinda confusing here

The simple idea is:

multithreading is the software side (high end), the code/data needs to be setup so that they can be run in non dependent chunks (pitfalls are when the processing of one chunk requires something from another chunk and they sit waiting, or ... well there are lot of things that can go wrong, why it pays the big bucks and isn't always applicable to all cases; and why the unity engine the game is based on is taking a long time to get the "it can thread anything across all available cores" framework in place).

 

multicore is the physical side (low end) each core can execute 1 instruction at a time, more cores means more instructions per second can be handled... if the code requesting those executions can split them up safely...

 

non multithreaded code will just run on one core (it's not split up)

multithread code can be run across any number of available cores at the whim of the resource management built into the multicore controller (think voodoo where 1+1+1+1 != 4) and on how the threading is implemented. "Simpler" implementations (because non of them are "simple") create specific numbers of threads, some more dynamic ones will try to create as many threads as the hardware can handle but they have more scope for an "oops".

 

For something which is partially mutlithreaded you'd expect the main core to be loaded heavily (the main thread sits here) and a spread of load on the other cores, but given that we neither know the specific hardware controller state nor how they've implemented the threading it's a lot of guess work as to the specifics.

[SackMaggie]

11 hours ago, cpy said:

You observe multithreading by observing CPU cores. Please stay out of this discussion it's too techy for you obviously. Threading is a complex thing and I don't really want to explain how threading and cores work together.

So you are getting me wrong. What I mean is there's no single code that's will run on single core, They run on single thread in the core. So I'm confusing what do you mean by run on single core at first.

You can see that game are running on one core but the code itself run in sequential (single thread in the core), So saying running on one core is technically wrong.

6 hours ago, LeftyRighty said:

each core can execute 1 instruction at a time

No it's each clock and Its called clock speed for CPU that you see in GHz.

6 hours ago, LeftyRighty said:

more cores means more instructions per second can be handled.

More cores is directly more threads and indirectly handle more instruction per clock 

6 hours ago, LeftyRighty said:

multithread code can be run across any number of available cores

Multithread its say "thread" in the word so don't use "core" it's wrong and it's mean part of code is running in concurrent not sequential, And thread will be handle by kernel and you will need an API to access that.

In the end I'm tech newbie or you're tech newbie ? Well what I know might be wrong and if it wrong correct me with reference would be nice.

[cpy]

It's speculative observation, in quad core CPU you have 4 cores, total CPU usage under 25% = 1 core used -> probably single threaded app.

35% CPU usage is 1 full core + some extra load -> expected 1+1 threads. This is what I'm seeing with ONI right now.

You observe just one app CPU usage not total system since windows and background apps are different threads.

[ToiDiaeRaRIsuOy]

12 hours ago, cpy said:

I am REALLY BAD at teaching others anything. That's why. When I start to explain **** not even PHD docs can understand me. So yeah I'm not the person you want to be your teacher. I've been told countless times how bad I'm at teaching stuff.

Then there's complex topic about multithreading. GG FTW.

Ok, fair enough. I think it's best to post some informative videos:

 

[akrabat14]

1 hour ago, cpy said:

It's speculative observation, in quad core CPU you have 4 cores, total CPU usage under 25% = 1 core used -> probably single threaded app.

35% CPU usage is 1 full core + some extra load -> expected 1+1 threads. This is what I'm seeing with ONI right now.

You observe just one app CPU usage not total system since windows and background apps are different threads.

I have 6 cores and ONI loads my processor at 40-42%, which I can assume that it takes up completely 2 cores and 1 partially or 1 core completely and all the other 5 only partially

[cpy]

2 minutes ago, akrabat14 said:

 

I have 6 cores and ONI loads my processor at 40-42%, which I can assume that it takes up completely 2 cores and 1 partially or 1 core completely and all the other 5 only partially

6 cores no HT = 16.66% CPU usage per one core maxxed out. So 42/16.66=2.519 so I guess pipes run at least on 2 cores. Heck my CPU clock is 5Ghz so yeah it's hard to guess when you have strong single core performance. OC FTW!

Maybe I should downclock my CPU to force ONI to as many cores as possible.

[ToiDiaeRaRIsuOy]

33 minutes ago, cpy said:

6 cores no HT = 16.66% CPU usage per one core maxxed out. So 42/16.66=2.519 so I guess pipes run at least on 2 cores. Heck my CPU clock is 5Ghz so yeah it's hard to guess when you have strong single core performance. OC FTW!

Maybe I should downclock my CPU to force ONI to as many cores as possible.

It would probably only drastically decrease ONI performance. Not worth the excercise.

And yes, OC FTW. My advise would be to get a decent cheap cooler and go to town with overclocking. It can quickly become up to 10 FPS improvement.

[cpy]

Yeah, less cores more OC is still better for games in 2018. But it's slowly getting into MT territory, which I love, since city simulators now can get away with agent simulations thanks to effective MT. SimCity 4 is a single core pain and struggles even on 4770K

Disable hyperthreading overclock and GO GO GADGET!

I really hope for some more details about multithreading from devs. Like Factorio devs that post every damn optimization they do on their blog so you can see how addicted to optimizing their game they are. I think one of the coders get's high everytime he optimize game by 10%. They keep going and going and never seem to run out of options to push game performance further. Space Engineers too. So yeah if you get some really eager dev that is addicted to optimizing code, you're golden.

Did you know that even really bad minecraft server was optimized by one crazy dude with multithreading and he even multithreaded mods, networking, sound and everything that servers were unslowable! I had that installed.

Proper multithreading vs single thread can mean 1 guy dumping chest content on ground = killed server with ultra lags

vs

4 people generating new map on server + 2 guys overbreeding animals + 1 guy dropping items on ground for lols = they borked their client speed down to zero and server runs like boss.

[Cairath]

On 12/11/2018 at 2:23 AM, melquiades said:

It is not because of compression, it is because Kley decided the save file should be encripted... for some reason.

Utterly unnecesary at best. Annoying for sure.

Not encrypted. Serialized and compressed. Nothing unusual.

[ToiDiaeRaRIsuOy]

Just now, Cairath said:

Not encrypted. Serialized and compressed. Nothing unusual.

We have online save game editor tools. I don't expect those being there if there was encryption going on. So yeah, nothing unusual.

[Cairath]

Just now, ToiDiaeRaRIsuOy said:

We have online save game editor tools. I don't expect those being there if there was encryption going on. So yeah, nothing unusual.

I've messed with the code and the files too, I've no idea where melq got 'encryption' from -- games don't save saves in plain text anymore, not for years

[LeftyRighty]

2 hours ago, SackMaggie said:

10 hours ago, LeftyRighty said:

each core can execute 1 instruction at a time

No it's each clock and Its called clock speed for CPU that you see in GHz.

How fast the instructions are executing per second is irrelevant here. 1 core can only do one thing per tick, how many ticks it achieves per second doesn't change how it behaves, just how long it takes.

2 hours ago, SackMaggie said:

10 hours ago, LeftyRighty said:

more cores means more instructions per second can be handled.

More cores is directly more threads and indirectly handle more instruction per clock 

Only if you want to make use of the extra cores. Again, software written to execute on a single thread will only be able to run on 1 core. Software written to execute on 2 (and only 2 threads) may run on 1 or 2 cores depending on the resource controllers in place. 

2 hours ago, SackMaggie said:

10 hours ago, LeftyRighty said:

multithread code can be run across any number of available cores

Multithread its say "thread" in the word so don't use "core" it's wrong

Again, the whole thrust of the explanation is that a thread is in software, core is physical hardware. A thread of code is executed by a physical core.

 

3 hours ago, SackMaggie said:

In the end I'm tech newbie or you're tech newbie ? Well what I know might be wrong and if it wrong correct me with reference would be nice.

I hear there is this newfangled device called "google"... I think that'll serve you much better than my uni textbooks by post