Profiling your server

[bizziboi 1933]

Since dedicated servers can use some optimization love the profiler had been enabled in the previous build, but right after release I realized it could use slight tweaks - so they went in todays patch.

 

If you're interested in getting performance up, like I am, you could help out by grabbing a profile and uploading it here - please also mention some details about hardware, OS and the situation the game was in (number of users, night, winter - especially if you feel the performance is erratic it'll help to get as much context as I can).

 

To run the profiler, open the console, and execute the command 'TheSim:Profile()' as administrator as a remote command (tap ctrl when you type the command in the in-game console).

 

This should create a file 'profile.json' in your server's document directory containing 3 seconds worth of profile data (it's kinda large, probably close to the 2mb mark, and, as @Kzisor pointed out, please zip it up as the forum won't accept json attachments).

 

Your server will lag during these 3 seconds (not freeze).

 

Looking forward to your profiles!

 

Kaj

[ToNiO55 298]

Hi Bizziboi,

 

thx again for your help

 

so i have try your command TheSim:Profile() and it's works

 

but just one problem, i can't find the file on my server dedicated

 

so i have take screenshot

 

http://nsa33.casimages.com/img/2015/02/11/150211011005899604.jpg

 

as you can see I have make the command and it's works, I had to run it twice because I can not find the file profile.json

yet according to the screenshot,the file seems to be created correctly

 

but when I look on my dedicated server linux (Debian 7.8), I find nothing

 

:~$ locate profile.json

:~$

 

:~/.klei/DoNotStarveTogether$ pwd
/*****/*****/.klei/DoNotStarveTogether
:~/.klei/DoNotStarveTogether$ ls

:~/.klei/DoNotStarveTogether$

 

someone else have try to see if this file is created?

 

thx for your help

[bizziboi 1933]

Hi @ToNiO55, 

 

Argh, I knew I'd forget something - you have to execute it as a remote command (tap control when you enter it in the console)- it has to run on the server.

 

Kaj

 

[Kzisor 1060]

@bizziboi, you should probably note that people need to upload it as a .zip file as you cannot upload .json files to the forums.

[ToNiO55 298]

@bizziboi no problem

 

so yes it's works now

 

i have take the file when we have big lag and of course status of host is "bad host"

 

when i say big lag, the player becomes uncontrollable and monsters are slow for a few seconds and this problem come back often

 

captured the 213 days, daytime, wilderness,early summer, 8 players on the server

 

mod on my server:

 

Renewable World

Nature's Still Wild

Respawnable Gears

Simple Protection

Global Player Icons

 

http://kfkserver.com/download/dont_starve/profile.json.rar

 

(I have just compress the file with winrar)

 

my server dedicated is on linux:

 

Debian 7.8, 24Gb, Kernel 3.2.0-4-amd64, 100Mbps,2 To HDD

 

if you ever need further testing on different configurations or

if I forget to mention some details,tell me do not hesitate

 

i hope that's help you and your team

[DreadPirate 7]

Hello there everyone!

 

I gathered some profiles from our public test server and zipped them up for you.

In the archive, please find the following files:

profile_1.json

profile_2_bad.json

profile_3_bad.json

profile_4_bad.json

profile_5_ok.json

profile_6_ok.json

 

I don't remember if profile 1 is Good Host or OK Host.

Profiles 2 - 4 are when it was listed as Bad Host.

Profiles 5 & 6 are when it went from Bad Host to OK Host.

 

We have this running on a pretty beefy machine.

The CPU Affinity for DST is CPU 1 - CPU 5 (5 total CPUs, DST is NOT on CPU 0)

The physical amount of RAM on this machine is 72gb, DST has plenty of room to move around.

We have 500mbps guaranteed throughput, burstable to 1gbps... network utilization was not even close to limits.

The OS is Windows Server 2012 R2.

 

I'll continue to monitor the server and generate profiles when I am able to.

 

I have noticed that when I can handle about 10 players with pings that are lower than 100ms as an "OK Host" however if I have any player with a ping of over 150ms it can only handle about 7 players and becomes a "Bad Host". The players on the server were pretty awesome in helping me out and we tried a few different things... sticking together, spreading out, etc... and eveyrthing was about equal. When the high ping player dropped the rubber-banding stopped.

 

Just some obervations, the high ping performance hit is something I have personally noticed on many occasions. Please continue to let us know if there is anything more we can do to help.

 

DST_Profiler_Logs.zip

[bizziboi 1933]

Hi @ToNiO55,@DreadPirate,  

 

Thanks It was really insightful - sadly not yet in a productive way, it exposed a flaw in the profiler which I am fixing now which my testing didn't expose but which renders the profiles fairly useless - I can only apologize for that, hoping to push an update ASAP.

 

Kaj

 

 

[DreadPirate 7]

Hi @ToNiO55,@DreadPirate,  

 

Thanks It was really insightful - sadly not yet in a productive way, it exposed a flaw in the profiler which I am fixing now which my testing didn't expose but which renders the profiles fairly useless - I can only apologize for that, hoping to push an update ASAP.

 

Kaj

 

Kaj,

 

No apologies needed! This is early access... it's for squashing bugs... sometimes for squashing bugs in tools needed to squash more bugs

 

I'm happy to run the profiler again whenever you are ready.

[ToNiO55 298]

Hey bizziboi,

 

same for me,no apologies needed

 

we're just here to help you and also help the community

as DreadPirate said it's still a early access

 

if you have need anything like help for server dedicated

 

you can count on us

[bizziboi 1933]

I have noticed that when I can handle about 10 players with pings that are lower than 100ms as an "OK Host" however if I have any player with a ping of over 150ms it can only handle about 7 players and becomes a "Bad Host"

 

 

This is something we are definitely going to investigate.

 

I might also contact you for more specifics regarding the underlying hardware depending on the profiles after the update, I am seeing strange context switching side-effects but it might be related to the way the profiler acted. 

[Atena 1]

My Server:

 

 CPU: Intel E1231 V2

 RAM: 8 G

 HDD: 500G

 OS: Windows 7 64 bit Pro SP1

 Server Name: [TW]Atena's World

 Days: 14xx

 

 Seems loading 1 CPU. The other CPU core not work.

 

 

 

profile.zip

[bizziboi 1933]

Hi @Atena, 

 

Thanks for the profile, however from what I can tell your game runs a very solid 60fps with that profile - were you experiencing lag when that profile was taken?

[ToNiO55 298]

Hi,
 
so I don't know if this'll help you
 
but i have take just little video of lag:
 

on this video we were 8 players on the server, I had a ping of 25 all along the video,

I just see the lag to start from the beginning of the day,we were within 200 days with the summer

 

on this video you can see that its happening in the twilight,
always with 8 players, 200 days and always summer, even monsters have lags

 

I know both videos will not be a great utility
just it was to show the problem more closely
on both videos,all time the status of the server is "bad host"

[bizziboi 1933]

Hey @ToNiO55, 

 

Thanks, and yes, that's definitely not what we want.

 

When the profiler is updated, can you grab a profile of when that happens? It should come with the next update and I hope it works better this time (as said, I can't reproduce the issue that the profile has on some other servers so I can only hope I fixed it....).

I would also be interested in knowing if there happens to be a player with high ping on the server at that moment (am working on storing that -and more- info in the profile but alas, I don't have that yet)

 

Kaj

[ToNiO55 298]

yes of course with pleasure

 

I play often with some friends on my server with high ping 200-300

 

http://nsa33.casimages.com/img/2015/02/12/150212085845108145.png

 

I could probably do some tests with that

 

so it's no problem to do what you ask me

[bizziboi 1933]

Hi @ToNiO55,@Atena, @DreadPirate,  

 

The new update has been released, and I hope to get more info from the profiles (fingers crossed).

 

If you feel generous, please capture a profile and upload it - preferably when the server is under load. No need to send me multiple, as I don't know yet if the profiler gives me the info I need now. My test-setup behaves decidedly different from other servers it seems.

 

Please also mention how many players were on, I'll work on storing such details in the actual profile, but at this moment I still rely on your intel.

 

Thanks!

[Atena 1]

Hi, bizziboi

 

  I already update the dedicated server to 126778, i will collect the profile file to you continuously.

 

  A large number players on-line in our server at night(GMT+8 20:00-24:00), we will collect the profile file at this time to you.

 

 But our server is too lag exceed 9 players online, after 1500 game days. We  can only provide you with 9 players on-line of profile-file.

 

 

 

 

 

 

 

6 players online, goodhost to okhost.

 

 

profile.zip

profile 20150213 0440.zip

[ToNiO55 298]

Hi bizziboi,

 

i have update my server

 

so i have take a new profile with players who have just joined the server with high ping

as before, 8 players on the server, summer, 507 days,wilderness, daytime,

see bad host on status, look like the same lag in the video I show before

 

 

http://kfkserver.com/download/dont_starve/profile.json_2.rar

 

I hope it'll help you

[Gelatous 81]

I've found something interesting about it that may be of use to you. It seems that the age of the currently loaded world has an effect on the host strength for me. Today right after resetting the world we had a peak of about 30 players and a consistent "Good Host." About a few hours later the host became "OK Host." Eventually the server could barely sustain 20 players. I don't have a profile log but I do have a video which shows having a good host right after resetting the server, and I'm wondering if this is the case for everyone:

 

 

If you need more information I'll try to provide it.

[bizziboi 1933]

Hi @Gelatous,

 

This could certainly be the case. I am extending the profiler to include more information about the active world and may request profiles in the near future that show this behaviour. 

 

Thanks for the info

[DreadPirate 7]

Team,

 

Here is another set of logs from today.

Sorry for the delay in getting back to you, it's been a crazy week at work.

 

In the zip file there are multiple profiles.

The profiles are numbered in order they were taken 01 was before 02, etc...

The profiles also have a status of good, ok or bad depending on what the server was reporting as in game.

 

I hope some of this helps, let me know if you have any questions.

 

Thanks again for taking a look at these.

DST_Profiler_Logs.zip

[DreadPirate 7]

Ok, so more info to report... I hope this helps.

 

Tonight we had a player with a ping of over 700.

He was on a satellite connection and didn't have much control over it.

 

He helped me try a few things... we profiled the server a few times as he was playing. I restarted the server, dropped it to 20 ticks and profiled it again...anytime he was playing the host was a Bad Host, and there were massive issues with rubberbanding and overall desync.

 

After a bit, we had him stop moving... once he did that we the host upgraded to OK, and the rest of us experienced no lag issues at all... until he started moving again.

 

Attached is a zip archive with some profiles, the 700 ping bad is when he was actively playing, the 700 ping ok was when he was stopped.

 

I hope this help, I'll keep trying to give you guys profiler logs when I see something odd.

DST_Profiler_Logs_700_Ping.zip

[Atena 1]

Days 17xx, 3 Players online.

profile 20150215 0909.zip

[Kzisor 1060]

@bizziboi, here is a profile with 17 players online with people constantly joining the server. Server cap was at 20 and every time a new player joined pings spiked HARD into the 600-1000. Host remained GOOD HOST. 

 

[bizziboi 1933]

@Kzisor,

 

Thanks, I'll have a look. Is this on a shared server or your own hardware? As for the ping spike (guess I could see that happen), were they disruptive and if so, for how long would you say?

 

Thanks!