Benchmark results, what we learned

[JohnFrancis]

Somewhere to debate and interpret the benchmark information without filling up the benchmark request thread

Benchmark file of people spec
https://docs.google.com/spreadsheets/d/1CHKRwjLfPsOZq2nVFGFoz5EgUvO64qqeYOAFnueeJxo/edit?usp=sharing

We don't have all the info yet but we can see a lot of 5xxx processors are being used.
What does not affect ONI is HDD or graphic card.
*Additionally it would appear CPU cache does not appear to have any noticeable effect.
*Edit turns out the entire 5xxx series has the same amount of cache per core. So can't say for certain that cache has no effect.

What matters most if your CPU followed really closely by your RAM speed, their is a noticeable difference between 3200, 3600 and 3600+ RAM
We managed to get four 5600X with different RAM speeds and sizes, the RAM size does not appear to matter 16 gigs looks to be enough. But do note the lower the ram speed the slower the result but not by a huge amount.

Time taken in seconds, lower is better
106    Ryzen 5600X    32 GB DDR4 3733 Mhz (4*8GB) (Overclocked) @Kerdeld
113    Ryzen 5600x (no overclocking and stock cooler) 16gb @ 3600Mhz @koobluh
116    Ryzen 5600X    32 GB DDR4 3200 MHz @ulnor
125    Ryzen 5600X    16GB at 2800MHz (Two 8GB sticks) @Tuberi

Edit For AMD The entire 5000 series looks to test around the same speed assuming no overclock and similar RAM, so the Cheap 5600x will get you the same ONI performance as a 5900x. So buy a 5600X and use the saved money to get faster RAM, target DDR 3600 CL16 and you should be getting just about top performance for your buck. If you want to overclock your memory 3733Mhz is where AMD tops out (Something to do with AMD infinity fabric), also anything below CL16 looks to give no gains.

For intel, i9 11900k is the only chip that beats the AMD 5000 series. With an honourable mention to the i7-10700K. Intel does not appear to have their RAM speed limited to 3733Mhz so you can buy crazy high speed RAM speeds of 4000Mhz+ and see gains. However form what I see you need to overclock the 11900k if you want to get superior performance to a 5600k.

[Fradow]

Super good info!

When you say "what matters most is your CPU", what part of the CPU is the most important? On the top of my head, characteristic you can shop for could be:

• having a recent CPU (recent architectures could be more performant, without that being immediately obvious with other numbers)
• AMD or Intel specifically
• single-thread performance (that's what the common wisdom says is the most important)
• overall multi-threaded performance (common wisdom says that's not important at all)
• L1/L2/L3 cache (you already pointed out it's not important, is there a minimum though?)
• Bus speed
• anything else I might have missed, I don't actually know much about hardware

[JohnFrancis]

@Fradow

With the usual disclamier that all this advice is based on the info in the spreadsheet that was submitted by the community.
For best cost benefit as far as I can see

Ryzen 5600X
16GB to 32GB of DDR 4 3600 (Keep the CL below 22) Also Please please ensure you enable XMP for your memory otherwise you will get 2133 speeds.
For everything else it does not appear to really matter.

As for L1/L2/L3 cache the 5600x, 5800x, 5900x and 5950x all have increasing levels of Cache but it appears to have had zero impact on ONI performance. The lot of them all appear to test around the same speed.

To all the players with the top results, first up congratulations that is some impressive numbers.
Would really like to get some more details from you if you have the time. The big things looks to be Processor followed by RAM speed so was wondering if we could get more info on both? Other people are going to want to ape your success so any overclock info on the CPU, all the RAM info, Make, model, number of DIMMS, any overclock info. I'll add your descriptions to a separate page of the benchmarks so people can see what you did and what hardware you did it with. If it's not to much bother maybe list cooling solution, case and anything else you think might be relevant.

105    intel i9 11900k    32GB 3200mhz    16-18-18-36    @Alex Thivierge
106    Ryzen 5 5600X    32 GB DDR4 3733 Mhz (4*8GB)    16 16 16 28 42 290 (Trfc) Command Rate 1    @Kerdeld
107    r9 5900x PBO been tweak a bit    3600 @ 3766    16-16-16-36    Claude P (this was a YT comment following up there)
110    Ryzen 7 5800x@4.6ghz    32GB 3600mhz    22/22/22/42    @Rashnar
111    Ryzen 7 5800X    32GB 3600MHz    16-19-19-39    @theich
111    Ryzen 9 5900X (non PBO i think, unless it is enabled by default).  32 GB DDR4 3600    16, 19, 19, 39. @melquiades
112    i7-10700K @5.0GHz Allcore    64GB (4*16) DDR4 3800MHz    17-22-22-42 @m.stitek

[JohnFrancis]

This post is more about some CPUS and memory combo's that potentially look to be under performing a little. 
Can the below people if they have time check to ensure they are getting the memory speeds they think they are. You can check memory speeds in task manger. Even if memory is rated for 3600 it will not automatically reach that speed. For speeds exceeding 2133 you have to enable XMP/DOCP and I think that maybe the below benchmarks do not have that enabled. Could be other things like poor ventilation causing thermal throttling, lots of other programs running, not restarting system before test, loading the save from inside another save (ONI has difficulty letting go of RAM). So if you could have a quick check of your systems maybe you could find a bottleneck.

121    Ryzen 5800X      32 GB 3600 MHz    cl 16 @ponsai 
122    5900x PBO + custom curve undervolt    16GB 3200mhz    CL 16 @ArkturusRX 
122    Ryzen 9 5950x - Base 3.4GHz 3200MHz CL16   Jared Sheppard (YT comment following up there)
123    Ryzen 9 5950x 2x 32GB 3600 MHz    16/17/17/36   @SpoonWasTaken

[Kerdeld]

49 minutes ago, JohnFrancis said:

This post is more about some CPUS and memory combo's that potentially look to be under performing a little. 
Can the below people if they have time check to ensure they are getting the memory speeds they think they are. You can check memory speeds in task manger. Even if memory is rated for 3600 it will not automatically reach that speed. For speeds exceeding 2133 you have to enable XMP/DOCP and I think that maybe the below benchmarks do not have that enabled. Could be other things like poor ventilation causing thermal throttling, lots of other programs running, not restarting system before test, loading the save from inside another save (ONI has difficulty letting go of RAM). So if you could have a quick check of your systems maybe you could find a bottleneck.

121    Ryzen 5800X      32 GB 3600 MHz    cl 16 @ponsai 
122    5900x PBO + custom curve undervolt    16GB 3200mhz    CL 16 @ArkturusRX 
122    Ryzen 9 5950x - Base 3.4GHz 3200MHz CL16   Jared Sheppard (YT comment following up there)
123    Ryzen 9 5950x 2x 32GB 3600 MHz    16/17/17/36   @SpoonWasTaken

This is a good one, I also want to add that people should check their power setting, it should not be on power saving, this can lock the CPU cores to lower clocks even under load.

I had this when I switched to a new processor from 2600 to 5600x because the 5xxx series does not include a Ryzen optimized power setting but instead can use balanced or performance power setting.

This is only when u don't reinstall windows after putting in a new processor and upgrading from 1xxx/2xxx/3xxx to a 5xxx series as far as I know.

how do you want more information about my system @JohnFrancis? Here in a post or somewhere else? I think I can write down something when I have some time today or tomorrow I guess 

[melquiades]

@JohnFrancis I have not overclocked anything, mostly plug and play (the only tweak was just to enable the XMP profile in the bios). And when comparing my CPU to the ones in the CPU-Z database, mine is a few points below average. The only things that could possibly make a difference are my cooling solution, a Corsair H150i RGB PRO XT, and my board, an X570 AORUS PRO WIFI, my PSU is old and need change (like 4 years old) 750W semi modular EVGA. the M.2 PCIE 4.0 i don't think it could matter, or can it?

[Kerdeld]

48 minutes ago, melquiades said:

@JohnFrancis I have not overclocked anything, mostly plug and play (the only tweak was just to enable the XMP profile in the bios). And when comparing my CPU to the ones in the CPU-Z database, mine is a few points below average. The only things that could possibly make a difference are my cooling solution, a Corsair H150i RGB PRO XT, and my board, an X570 AORUS PRO WIFI, my PSU is old and need change (like 4 years old) 750W semi modular EVGA. the M.2 PCIE 4.0 i don't think it could matter, or can it?

PBO Mhz override or/and PBO limits are holding u back if u compare on CPU-z with others, mine came also with auto and it's limiting  your maximum boost for single core and all core heavy loads. try Cinebench r20/23 and look at all core speed dropping down if it's on auto.

It might be different per Bios what the auto setting exactly is.

Mine came 4400-4450 (this depends on cooling and the auto pbo limit) cpu-z benchmark out of the box and a few tweaks made it to 5180 all core and 621 to 644 single core on a 5600x, here is where cooling is more important on all core load.

[dezixn]

I hope you take the time get a stable OC on your new build FJ. This thread shows the huge benefit from single core speed. Modern CPUs have a lot of headroom and you definitely have the mind to do a nice reasonable OC on your system.

Also this guy's system "112    i7-10700K @5.0GHz Allcore    64GB (4*16) DDR4 3800MHz    17-22-22-42"

Is very close to mine: 119 seconds, Intel i7-10700K @ 5.2 GHz, RAM : 32 GB DDR4 3000 MHz, CAS: 19-19-19-43

You can see even though my CPU beats his, his benchmark is better likely because of his RAM speed. Maybe core speed has diminishing returns? Pretty interesting info.

[lautasdf]

 

As everything in computer building goes, it is relative indeed. Besides that, it is amazing how some people took the time and effort to squeeze out every ounce of performance possible, given the time that it takes, as Francis said, congratulations!

[Gurgel]

Interesting results. Remember that less than around 20% difference in speed is generally not noticeable to humans, unless they have a direct comparison. That means all the results up to 120 or so are basically fine and hence price and other factors are more important in that area than performance. It also means that any real speed-ups will come from code optimizations in ONI, not from faster hardware.

As to caches, I can explain the lack of impact: Cache impact drops strongly from a certain cache size onward for specific running code. This point is much lower for single core scenarios and all cache sizes are well above it for ONI. The reason CPUs have really large caches toady is multi-core scenarios and there it makes a large difference. 

4 hours ago, dezixn said:

Maybe core speed has diminishing returns? Pretty interesting info.

It has. Caches can do only so much to mitigate that main memory is massively slower than the CPU. From a certain active memory footprint onward (i.e. the memory the code uses frequently), main memory becomes the limiting factor and your CPU just idles more if it is faster. This point is lower the fewer CPU cores get used. That point is also lower if calculations are simpler and ONI uses only 32 bit short floats for most things.

[Orzelek]

After looking at the results I tried to overclock RAM from 3200 CL 14 to 3800 CL 16 but.. no effect. It actually went worse by about 2-3 seconds.

One other thing is that I see in Ryzen master that when running sim on this save at max speed it can't load even one core fully (it doesn't reach max clock speed on averages that it shows). It would seem that memory is to much of a bottleneck for game (maybe some prefetch instructions would help like Factorio tried and managed to get some nice %% or so speed up if I recall correctly).

[Tartarus Plays]

Good info, glad so many people are posting.

I hope Klei can use learn and improve game performance.

A slight departure to a side topic, but the other big time suck in the late game is the auto save.

Sure we can set it to not every cycle, but I hate on cycle 4 of the 5 cycle autosave I get a game crash.  In the late game that is a huge amount of real time.

I also play on the testing branch mostly, so I expect crashes and leave it on every cycle saves.

Anyone played around with speeding up the auto saves?

I thought a dedicated ssd for the game would help, but not so much. 

[sakura_sk]

9 hours ago, Tartarus Plays said:

Anyone played around with speeding up the auto saves?

There is the fast save mod but if you play on testing branch (meaning frequent possible crashes) I don't know if you would want to install any mods

[FGPraxis]

RAM Timings are also important, although slightly less than CPU and RAM freq :

112 sec : i7-10700F HT off (4.6Ghz) + 16Go DDR4-3733CL16-16-16-36 CR 2T tRFC 350 @ 1.39V (MSI Motherboard)

In short no CPU oc, only a safe DDR4 oc (XMP for my RAM is 3200CL14-14-14-34@1.35V, 117sec XMP on)

RAM oc stability was tested with OCCT and HCI memtests : 0 errors

It's also dependent on AMD or Intel, see Kerheld for a very good RAM tuning for AMD Ryzen

(example of bad DDR4 timings for Ryzen : 125 sec Ryzen 9 5900X + 2x32GB 3600mhz CAS : 18/22/22/42)

(previous example partially related to CPU thermal throttle down, so another example https://forums.kleientertainment.com/forums/topic/133992-benchmark-testing-of-spaced-out/page/8/?tab=comments#comment-1499853)

 

[gabberworld]

funny thing is, allot talks in this forum that AMD is best. but here we are 2 intel cpu sitting at front list 

[KnoBuddy]

I've just spent the last week or so fiddling with RAM overclocks trying to get the most out of my mislead Corsair B-DIE revisioned with C-DIE and my 3900x.

A little info about RAM and single core speeds on Ryzen... First off, Ryzen has their CCD's binned in a very specific manner. A 5900x will have two CCD's. One above average binned CCD and one average binned CCD. Each consisting of 6 cores with 2 cores disabled. This is because the silicon is lower grade. Hence you will get worse single core performance than you will from a 5950x, which has two very well binned CCD's allowing a higher boost clock and all 8 cores active on each chiplet. This in turn means that the 5800x should have an equal or better CCD than the 5900x, because all 8 cores are turned on, however advertised boost clocks would paint a different picture, suggesting that at least 1 of the CCD's on the 5900x is better than the 5800x, at least when only 6 cores are enabled.

This is all a long winded explanation to say, if you want the best single threaded performance 5950x is the best, but if you need the best single thread performance, you probable don't need 16 cores... The better silicon will also allow for a higher fclk which matters just as much as ram speed.

Now onto the RAM. Essentially, RAM speed does matter but so does low clocks. If you move the flck higher than the memory speed, you will incur a latency penalty, however with Zen 2/3 this penalty is very small and can be overcome by the higher flck.

A real world example is my poor 16gbx2 sticks of Dominator RGB which are rated at 3200 16-18-18-18-34, which have a latency of 70ns. I can get them to 3600 18-20-10-20-38, however the latency barely moves to just below 70ns, but this allows me to run my fclk at 1800 for a 1:1:1 ratio with no latency penalty. My mobo and CPU can do 1866 FLCK but because of the average binned CCD's in my 3900x I cannot for the life of me get 1900 flck stable, nor can my RAM do anything stable above 3600. If I overclock the flck to 1866 and leave the RAM 3600, I do get a VERY SLIGHT improvement in RAM speed even with the latency penalty incurred from not running flck==memclk. The benefit isn't worth it since I have to give more voltage to the SOC than I care to run 24/7 for such a slight improvement. However if I had a 3950x and could achieve 1900 FLCK, as well as probably requiring less voltage to the SOC due to better binning I certainly would.

The 5000 series Ryzen processors are capable of achieving 2000FLCK and higher on better binned chips, and even if you are unwilling to spend the money on 4000mhz RAM, RAM that achieve 3600, with the FLCK at 2000, would certainly be a benefit, and another reason to get the 5950x.

This is all going to come down to how much you wanna take out of your pocket for a CPU. I wish I had spent the extra couple hundred and gotten a 3950x now that I know more about how the chips are binned. I actually had a 3800x and returned it when the 3900x came down in price, and that may not have been the best choice because I don't really need the extra cores, single core performance matters to me much more. But ultimately if you won't be using this CPU for workstation tasks, you may as well go team BLUE and get a 11900k. It's also much easier to get higher memory clocks with intel, Ryzen can be a bit finnicky. 

TL;DR
5950x is the best single core performance you will get. RAM speeds matter more than clocks, but both are important. 5950x will allow for higher FLCK which is beneficial even if your memory can't keep up. If you can justify the money, the 5950x is the only way to go on team RED.

[gabberworld]

33 minutes ago, KnoBuddy said:

5950x

yes. i dont know how much you gain at oni but at some games 5950x can be better than intel. but then again like always the memory matters as well

but remember i not talk like huge better but like little better

currently is this situation that AMD need release something much better that intel cant beat, otherwise AMD with his huge cpu price starts only lose

[gabberworld]

i been tested with JohnFrancis idea the performance between version 479045 and 420700

now there is defiantly performance loss between those two version, and i not talk about like sec or two but like 15-25 sec for that 300 sec cycle test

 

 

 

[Kerdeld]

4 hours ago, gabberworld said:

i been tested with JohnFrancis idea the performance between version 479045 and 420700

now there is defiantly performance loss between those two version, and i not talk about like sec or two but like 15-25 sec for that 300 sec cycle test

 

 

 

Yes there is a big performance loss with the last merge update, it's almost like a whole CPU downgrade (on the spaced out version for me).

While that update was mostly about the Main ONI getting new features already existing in the spaced out version.

I saw more people noticing this but nothing from Klei about it and if some of the performance will come back with updates.

[gabberworld]

2 minutes ago, Kerdeld said:

Yes there is a big performance loss with the last merge update, it's almost like a whole CPU downgrade (on the spaced out version for me).

While that update was mostly about the Main ONI getting new features already existing in the spaced out version.

I saw more people noticing this but nothing from Klei about it and if some of the performance will come back with updates.

that test i made in base game tho. but it affects spaceout as well because they are same games with same code now. before merge they was separated. needed download every time when switch the version

[KnoBuddy]

When I get home I will see if when I turn off SMT and disable the worst cores if I can get an improvement from a 5ghz single CCX overclock in the test. I haven't even ran the test yet, but will be interesting to see if ONI is on of the few games that will actually benefit from turning my 12c 24t 3900x into a 8c 8t CPU.