[Resolved] A hopefull, vibrant and strong call to Klei

[Onirian]

Hi Klei, and hi all ^^.

First of all, Klei, I love you, and I'm absolutely sorry about this topic, because I will be the boring guy talking about a boring recrurent (fu*****) problem, probably really hard to solve. 

So. I've posted this ticket, two weeks ago, about a huge lag problem. I put in this ticket, all logs and information I can provide, including a video play (to see how major is the problem). Of course, I'm totally agree to do more tests/logs if needed ^^.

For memory, the lag happens each times I go into my (growing) (mega) base. I play in lan with two big pc (Core i7, and the host have 20Go of ram...).

So why this post ? Because, Klei, I want to convince you (and to other user reading this post, don't hesitate to express yourself if you have also encoutered lag or fps issue, call Klei with me !), that this problem is a real MAJOR problem, which makes the game just unplayable for long games and perseverant players. By far, more important than new games content ( (I read somewhere that you plan to add late game content... but we can't reach late game !))

I do a very fast research about "lag" on this forum, and I just look the firsts results page. I found this (I say that I'll be a boring guy... sorry again) :

• forums.kleientertainment.com/topic/66203-rubberbanding/#comment-764403
• forums.kleientertainment.com/topic/66660-optimization-of-dst-for-servers/#comment-756464
• forums.kleientertainment.com/klei-bug-tracker/dont-starve-together/lan-client-lagging-r738/
• forums.kleientertainment.com/topic/65377-lag/#comment-756625
• forums.kleientertainment.com/klei-bug-tracker/dont-starve-together/massive-rubberbanding-and-lag-on-local-and-online-server-r271/
• forums.kleientertainment.com/klei-bug-tracker/dont-starve-together/lag-for-windows-7-hp-home-premium-laptop-r318/
• forums.kleientertainment.com/topic/65639-walls-causing-lag/#comment-736952
• forums.kleientertainment.com/klei-bug-tracker/dont-starve-together/performance-issue-unplayable-lag-r616/
• forums.kleientertainment.com/topic/67391-lag/#comment-773029
• forums.kleientertainment.com/topic/66977-observable-game-lag-on-a-computer-in-same-lan-larger-than-lag-from-internet/#comment-765844
• forums.kleientertainment.com/topic/63348-bug-report-massive-lag-and-rubberbanding/#comment-714427

And it's absolutely not an exhaustive list. None of this thread has real anwser for the lag problem (and some of them, no answer at all...)

Yes, I really insist, but Klei, just look, it is a really big problem, and a LOT of people complains about it. Some mods were even created to try to go around of this problem ! You have written a guide to help (It doen't help me, but nice try ^^).

So yes, Minimap HUD makes this problem bigger, but this mod is not the cause, it just reveal the problem faster. Yes, sometimes, this problem can happen because of a bad host, of bad connection, but trust me, there is here a real code problem that needs to be investigated.

I absolutely know that this problem can be really hard to solve, I understand that, maybe, to resolve it, you will have to do big things like rewriting your communication protocol. I know. I'm ok to wait, I'm ok to help if I can, but please, take this problem into account, and fix it.

Thanks.

(And sorry again).

(And sorry for my bad english too, it's not my native langage)

 

Edit : 05 july 2016 :

Spoiler

 

It's FIXED !

THANK YOU !

 

 

--

Onirian.

Save the soldier FPS.

[Asparagus]

4 minutes ago, Onirian said:

For memory, the lag happen each times I go into my (mega)base. I play in lan with two big pc (Core i7, and the host have 20Go of ram...).

Hi there

So, I do feel your pain as moving close to people's bases where they have a screen's worth of berry bushes, grass tufts, saplings + farms, drying racks, etc, causes me lag too...

I'm not certain about what really causes the massive FPS drop but I do believe that it's due to the fact that the game has to load too many network entities in one go... though with your rig, you should be good... how's your video card btw?

[Onirian]

Hi Asparagus ^^, and thanks for your answer !

My video card is a NVidia GeForce GTX 780 Ti which is not exactly a bad card ^_^. The problem doen't comes from here ^^.

Personnaly I think the lag problem comes from the communication protocol. Too many information are probably resent all the times, even if this information is not absolutely necessary. If I'm right (but only Klei (I love you !) can know it), this problem needs to rewrite lot of mechanims to be fixed, maybe adding cache or things like that... 

But I'm sure this problem is not a hardware problem. It's a code problem, and I'm also sure, considering the quantity of information needed by the client, that this problem can be fixed.

[Onirian]

Not even one complete day after my hopefull, vibrant and strong call, an other people is complaining about exactly the same thing.

Klei ? Please... ? Seriously, this problem is really major...

And if you fix it, I will shut up myself. Great deal isn't it ? ^_^.

 

 

[calculuschild]

I really wish this issue would finally be resolved as well.  It is pretty frustrating to see how many bug reports, forum topics, etc. have been created on the subject, and when the devs ask for logs or PC specs and get them, there are no further responses.

 

I'm especially worried about the topics that talk about lag on LAN, where ping should be for all intents at the very minimum possible.  I was once told by a dev that "There's not really much of anything we can do re network lag - if you're too far away network wise you'll never be able to kite (unless we get that speed-of-light-violating tech integrated)." And yet people still have game-breaking lag on LAN.  I've had that problem on LAN.  I keep seeing these half-joking, but sadly true comments all over the forums: Can't kite because of lag.  Couldn't light torch in time because of lag.  Ran into tentacles and couldn't get out because of lag.  Frozen in place when hounds come because of lag.  Forced to make minimal bases because anything bigger causes lag.  It's just not fun when that happens.

 

When it has been shown repeatedly that certain game entities (farms, walls, hounds, etc.) are directly causing lag when viewed on the screen, that tells me there is definitely something wrong in the networking code for those items.  Are farms being transmitted as separate location coordinates for each of their prefabs in every packet?  Shouldn't this be something that is sent one time when the farm is first created, and then the client just remembers there is a farm at (xx, yy) with rr rotation, until the server tells it that the structure is destroyed or on fire or whatever?  Such stationary entities should be sending hardly any network traffic since they have no AI and no movement, and yet the evidence suggests that merely having them present on the screen will cause the client to rubberband.

[Onirian]

25 minutes ago, calculuschild said:

I really wish this issue would finally be resolved as well.  It is pretty frustrating to see how many bug reports, forum topics, etc. have been created on the subject, and when the devs ask for logs or PC specs and get them, there are no further responses.

(...)

When it has been shown repeatedly that certain game entities (farms, walls, hounds, etc.) are directly causing lag when viewed on the screen, that tells me there is definitely something wrong in the networking code for those items.

I can't agree more, and I have provided all the log I can (and I'm totally ok to provide more if needed).

There is here a big issue that must be investigated. And the very frustrating thing is that the more you love the game, the more you play, the more you are affected by it.

I agree too with the fact that when the ping is too high there is effectively nothing to do. But here, there is a really big room of improvement before reaching network limitation. Not necessary easy to solve, but necessary to solve ^^.

[Onirian]

Hi, it's me again, you know, the boring guy which is always talking about lag ? ^_^.

So, before all, I will say "thanks" to Klei, to have fix a problem about farm, discovered by @DarkXero in this thread. It doen't resolve all the lag problem, but, it was a part of the problem and this part was fixed, and it's very cool ^^.

But this fps problem with megabase is still there, and is still a major problem for all user which are playing long games. So I will continue this thread (yes, until death... Mine, or bug death, we will see which one will die first ^^).

The goal of this thread, as I said before, is only to convince Klei (Yeah, I know you are reading me right now, and maybe you are smiling a little while reading me ;-p) that the lag problem is a very big problem that must be fixed. It will takes times, it will takes resources, but so many people complains about it...  Fixing it will makes us so happy ! ^_^.

Even if you say : this next three week, there will be absolutely no new content, because all our resources will be on the lag problem, all people will be happy ! (Ok, not all, but the most ^^).

 

Soooo, the thread of the day of other people than me complaining about lag (ok, I cheat a little, the first is about your server, not directly lag in lan, but optimizing works everywhere ^^).

 

Ps : a part of me is so sorry to be a soooo annoying guy... But I really love this game, and I really want to play with it without lag, or to be obligated to disconnect and reconnect all the times, and in fact, as a simple user, I do not have any other ways to be listened...

pps : of course and again, if I can help you with tests, logs, or anything, I would be very happy to do it.

ppps : I love you.

[GiddyGuy]

1 hour ago, Onirian said:

soooo annoying guy

I can't tell if someone called you annoying or you just want to be called annoying, anyway whatever suits you.  

I'm not a programmer(but know a little about it) but I'm pretty sure it'll take them a little bit to find out the problem and fix it up...then again they usually fix bug/glitchs really quick since they're very skilled at their jobs.  

And I'm pretty sure they do check the frequently forums, but do remember all good things come in due time good madam/and or sir.  

[Onirian]

36 minutes ago, GiddyGuy said:

And I'm pretty sure they do check the forums, but do remember all good things come in due time good madam/and or sir.  

Hope you're right ! ^^.

[calculuschild]

@Onirian, I'm glad you're making sure to keep this topic alive!  I've had my share of time being the new guy on software development teams and the amount of odd, inefficient things that can slip by an experienced team are just astounding; when all the testing is done on the powerful company computers it was easy for my coworkers not to notice the slowdowns and annoyances that regular users were having on their home machines; wasn't until I could take a fresh look at some of the old code, and with the perspective of somebody who had also been a customer before I was hired, that we cleared out a lot of those bugs, dead code, and simple inefficient logic, which drastically increased performance (4 times faster response).  I think we just need to keep "being annoying" as you say until finally someone takes a serious look at this major problem.  Hopefully some day we will have an actual Klei staff member working to fix these issues rather than waiting for DarkXero to do it for them again.

[PurpleHatEvolution]

i used to play this game very easily, like around the start of the beta and the introduction of rog added, later on after rog i just started getting giant internet problems, i couldn't play like i usually would, i couldn't go on any server, i would be forced to go on low populated  super low ping servers and still be reduced to lag. This basically what turned me away from DST. I used to play this game alot, turning into my 3rd most played game on steam, now i cant even enjoy it :L

Really hoping they fix whatever caused this giant internet issue so i can go back to my favorite game again.

[Onirian]

Hi Klei !

Yes, I know you love me, and you absolutely love my thread which remind you how much is important to solve this *ù£$^ù$*!$* lag problem, even if it's hard and sooo less fun than adding cool stuff in the game.

In attachment, a graphic created by @calculuschild from logs I do with the command TheSim:Profile() (recommanded by @rezecib) in this thread. First log created in the center of my base, one other far from it. I repost it, because this graph is so clear. The "update" method need a lot of improvement, and EntityManagerWallUpdate. After that, maybe other thing can be doing but try to walk step by step.

Serilously, we do the job no ? Your turn !

 

And because the real goal of this thread is to remind you, again and again (and again, and again (and again (until lag death))) that the fps / lag / rubberbanding problem is soooooo important (in fact, it's probably the most important problem of this game for now), and sooooo boring for sooooo many peoples, here is the posts of other peoples than me which are soooo sad because of lag.

Today, 3 more posts !

 

 

ps : yes, today I decide to add lot of "o" to the "sooooo" word. It's like that.

pps : Please Klei, listen to us, we are your lovely player.

ppps : I love you.

 

[Onirian]

Hello !

Today, it's not really a new topic, just a message in an existing topic, but the complain stay the same : it's just an other of your lovely players which just wants to play without lag. 

Oh, did I have ever say to you that this lag problem is a very very major problem affecting so many people ? A so big problem than it's more important to spent (lot of) time to fix it than to create new (unusable because of lag) stuff ?

Well. It's a major problem. A very very major problem. See how many people complains about it, again, and again, and again.

(Ok, I'm the more annoying of them, your're right, but other post will come, I know you you know it).

ps : I love you.

[Faintly Macabre]

So here's the thing

Development teams are not made up of people who all do the same things with equal proficiency. They're made up of people with particular jobs and specific proficiencies and skillsets. The guy who put together salt lick is probably not the guy who fixed farms, and he might not know a thing about keeping client performance smooth. Moreover, although there's definitely some simple optimization fixes to be made, they're generally more complicated because they're generally not as simple as a logical failure.

I want inexplicable FPS drops to go away as much as the next guy (if not moreso since it actually does effect me) but I know it's not as simple as we'd like it to be and will probably take some time. Please, don't be That Guy. It's not helpful.

[DarkXero]

@Onirian, check out:

and report back.

[Onirian]

Yup @DarkXero , I see that !

I also still have to test the -tick 60 option (I didn't have anytime to play the last few days, but I can't wait, so I will test that during my meal)

ps : Thanks Klei ! I love you ^^.

[Onirian]

13 hours ago, TheHalcyonOne said:

 Please, don't be That Guy. It's not helpful.

I think this message is for me, so I will reply (fast. I just wrote a big message but I just loose it, so you will have the short version).

I'm a computer scientist since 15 years old. I know how projects works, and a project like dst is all about priority (according to the fact it's just impossible to the team to do all they want, because of time). With this thread, I just try to increase the priority.

I give visibility to people complaning to show that the problem affect lot of people and to convince Klei to spend all necessary ressource to fix this problem. I never said it will be simple, but I say spending resources is a good idea because of prority.

And know Klei do an update with performance improvement ! Maybe planned since a long time, or maybe a little because of this thread, I don't know (and to be honest, I don't care, I just want to play without lag ^^).

According to my first test, it seems really better, I gain near 20fps !

So...

Thanks KLEI !

When I spawn at the center of my base, my fps is 45 (25 before), so it's not perfect, but it's playable ! And the hosting pc (the one which have the server + a client) is at 60 which is perfect !

In some place in my base, my fps can drop to 25/30 (where I have lot of farms) (10/20 before).

So, all are not resolved, but it seems really better !

There is still some improvement to fix the difference between the hosting pc (server+client) and the client pc (only client). I'm in lan with a ping under 5, so it's not the network.

Also, when I display the global map, fps still goes down to 30 which is showing another problem too.

I can provide new logs if anyones wants it.

 

[calculuschild]

Hey @Onirian.  This is good news!  I personally would like to see some new logs if you have them with your in base and far from base, so we can compare to the previous ones.  I'm curious to see which areas they were able to fix.  I'm glad to see they have put some good work into making the game smoother!

[Onirian]

Hey @calculuschild, If I have the time (which is not sure at all), I will post the new logs here tonight (and if I can't, it will be this week end). But the little tests I do sounds good ^^.

[Onirian]

Here is the log. So, I confirm the gain.

When I move the fps drop and when I stop moving the fps goes up again.

There is also a gain when the map is displayed.

I put log from 3 place, center of the base (to have a comparison point with previous logs), center of my farms (which is the most laggy point of my map) and far from my base, which is a place with no lag.

I also do log when map is displayed, far from my base and in the center of farm.

Enjoy @calculuschild, I'm very curious to see your result ! (And Klei, don't hesitate to look at it too ^^)

 

ps : I love you Klei, and thanks again (and continue the good job, one other update like this one and the lag will completely diseapear in lan !)

fps 30 - center of farms - map displayed - profile.json

fps 40 - center of farms - profile.json

fps 48 - base center - profile.json

fps 59 - far from base - profile.json

fps 60 - far from base - profile.json

[bizziboi]

@Onirian

Did you try running with the -threaded_render command-line option?

 

 

[Onirian]

Hi @bizziboi !

First of all, thanks for the recent optimization, it's better (not perfect, but really better) !

So, yes, all this tests were done with the -threaded_render option activated. In fact, activate this option does not really improve the performances (maybe it is slightly better, but nothing really visible). I also try the "-tick 60" option, but same result. I also try a mod which desactivated the collision for the wall, and it doesn't change anything too.

The fix, few weeks ago, about the farm memory leak makes things better, and the very last update make things better again.

On the server side (the pc which is hosting the server + a player), now, there is almost no more lag, but as you can see with the json log, on client side all is not fixed (I have just check my ping in game, It is moving between "0" and "3" ("1" the most of the time, hard to do better ^^).

If you want it / need it, I can provide you more logs or game files (or anything you want) to help you reproduce and fix the last problems remaining.

Thanks !

ps : oh, and of course, I love you ^^.

[bizziboi]

@Onirian

 

That's....odd. The profile seems to indicate the game is running without threaded rendering. Are you running the client with -threaded_render? How did you set the command line option?

 

[bizziboi]

@Onirian

Hmmmm, seems something got accidentally broken with that command-line option.

Could you go in the client.ini file and add

use_threaded_renderer = true

...in the [MISC] section?

 

[Onirian]

I set the command line option in steam > library > Don't starve Together > property > define launch option, and I have (copy/paste "-threaded_render -tick 60"

Here is my client_log.txt, in the first line, we can see :

Quote

[00:00:00]: Command Line Arguments: -threaded_render -tick 60

That why I think it was activated.

I'll do your new test right now, I give you the result in few minutes.

client_log.txt