Caves won't work in LAN

[Alex_77]

Hello Guys,

I'm having problems with caves on my dedicated server. Here is the story:

A few months ago I installed 2 dedicated server in my docker environment and all went fine (including caves). After a pause of gaming I now wanted to reinitialize this setup. A made a game update and started the server again. It all seemed good on the first side, but now when entering a cave I get a timeout.

As far as I can see from the logs, the shards are connected correctly. But I see an error message in the cave logfile, which says that the user migration failed (177). Is this a bug or am I doing something wrong?

I hope somebody can help me with this problem.

Alex

P.S. Here are the relevant parts of the log-files. If you need more information, please let me know it.

Master/server_log.txt

[00:00:29]: Sim paused
[00:00:29]: Gameserver logged on to Steam, assigned identity steamid:90134624042959874
[00:00:29]: [Steam] Authenticated host '76561198126831020'
[00:00:30]: There is no active event to validate against.
[00:00:30]: [Shard] Read save location file for (KU_hesNpoOX)
[00:00:35]: Resuming user: session/04EC7077AB48F51B/A7LEHMBSU9H1/0000000207
[00:00:35]: Could not find anim build FROMNUM
[00:00:35]: Could not find anim build FROMNUM
[00:00:35]: Could not find anim build FROMNUM
[00:00:35]: Spawning player at: [Load] (-15.54, 0.00, -172.86)
[00:00:35]: Sim unpaused
[00:00:37]: Best lobby region is aws/EU (ping 21)
[00:00:37]: Registering master server in EU lobby
[00:00:37]: [Shard] Slave Caves(586447616) connected: [LAN] 10.0.17.2
[00:00:38]: [Shard] Slave Caves(586447616) ready!
[00:00:38]: World 586447616 is now connected
[00:00:38]: Validating portal[9] <-> 586447616[9] (disabled)
[00:00:38]: Validating portal[7] <-> 586447616[7] (active)
[00:00:38]: Validating portal[4] <-> 586447616[4] (active)
[00:00:38]: Validating portal[3] <-> 586447616[3] (active)
[00:00:38]: Validating portal[1] <-> 586447616[1] (active)
[00:00:38]: Validating portal[2] <-> 586447616[2] (active)
[00:00:38]: Validating portal[5] <-> 586447616[5] (disabled)
[00:00:38]: Validating portal[10] <-> 586447616[10] (active)
[00:00:38]: Validating portal[8] <-> 586447616[8] (active)
[00:00:38]: Validating portal[6] <-> 586447616[6] (active)
[00:00:47]: Registering master server in EU lobby
[00:00:53]: [Steam] SendUserDisconnect for '76561198126831020'
[00:00:53]: Connection lost to 10.255.0.2|52656 <7485662598609974029>
[00:00:53]: [Shard] (KU_hesNpoOX) disconnected from [SHDMASTER](1)
[00:00:54]: Serializing user: session/04EC7077AB48F51B/A7LEHMBSU9H1/0000000207
[00:00:54]: Sim paused
[00:00:57]: Registering master server in EU lobby
[00:01:44]: New incoming connection 10.255.0.2|58580 <5696325330713552796>
[00:01:44]: Client connected from [LAN] 10.255.0.2|58580 <5696325330713552796>
[00:01:44]: ValidateGameSessionToken XXXXXXXX for <5696325330713552796>
[00:01:44]: Client authenticated: (KU_hesNpoOX) Zombie
[00:01:44]: [Steam] Authenticated host '76561198126831020'
[00:01:45]: There is no active event to validate against.
[00:01:45]: [Shard] Read save location file for (KU_hesNpoOX)
[00:01:45]: Registering master server in EU lobby
[00:01:53]: Resuming user: session/04EC7077AB48F51B/A7LEHMBSU9H1/0000000207
[00:01:53]: Could not find anim build FROMNUM
[00:01:53]: Could not find anim build FROMNUM
[00:01:53]: Could not find anim build FROMNUM
[00:01:53]: Spawning player at: [Load] (-15.54, 0.00, -172.86)
[00:01:53]: Sim unpaused
[00:01:55]: Registering master server in EU lobby
[00:02:00]: [(KU_hesNpoOX) Zombie] ReceiveRemoteExecute(c_save()) @(-8.69, -176.37)
[00:02:01]: Serializing user: session/04EC7077AB48F51B/A7LEHMBSU9H1/0000000207
[00:02:02]: Serializing world: session/04EC7077AB48F51B/0000000207
[00:02:13]: [(KU_hesNpoOX) Zombie] ReceiveRemoteExecute(c_spawn("cave_entrance")) @(-8.69, -176.37)
[00:02:13]: Validating portal[-1] <-> <nil>[0] (inactive)
[00:02:13]: Validating portal[11] <-> 586447616[11] (disabled)
[00:02:25]: [(KU_hesNpoOX) Zombie] ReceiveRemoteExecute(c_spawn("pickaxe")) @(-10.93, -173.32)
[00:02:31]: Validating portal[11] <-> 586447616[11] (active)
[00:02:33]: Activating portal[11] to 586447616
[00:02:33]: Serializing user: session/04EC7077AB48F51B/A7LEHMBSU9H1/0000000208
[00:02:33]: [Shard] Migration request: (KU_hesNpoOX) to Caves(586447616)
[00:02:33]: [Shard] Begin migration #1 for (KU_hesNpoOX)
[00:02:33]: [Shard] #1 [SHDMASTER](1) -> Caves(586447616)
[00:02:33]: [Shard] #1 <- session/04EC7077AB48F51B/A7LEHMBSU9H1/0000000208
[00:02:33]: CloseConnectionWithReason: ID_DST_SHARD_SILENT_DISCONNECT
[00:02:33]: Ignored request to resume during migration
[00:02:34]: [Steam] SendUserDisconnect for '76561198126831020'
[00:02:34]: [Shard] (KU_hesNpoOX) disconnected from [SHDMASTER](1)
[00:03:33]: [Shard] Migration for KU_hesNpoOX timed out.
[00:03:33]: [Shard] Cancelling pending migration #1 for (KU_hesNpoOX)
[00:03:33]: Sim paused
[00:03:33]: Registering master server in EU lobby

Caves/server_log.txt

[00:00:28]: Reset() returning
[00:00:28]: [Steam] SteamGameServer_Init(8766, 10998, 27016)
[00:00:28]: [Steam] SteamGameServer_Init success
[00:00:28]: [Shard] Connection to master failed. Waiting to reconnect...
[00:00:28]: Validating portal[6] <-> 1[6] (inactive)
[00:00:28]: Validating portal[7] <-> 1[7] (inactive)
[00:00:28]: Validating portal[2] <-> 1[2] (inactive)
[00:00:28]: Validating portal[10] <-> 1[10] (inactive)
[00:00:28]: Validating portal[3] <-> 1[3] (inactive)
[00:00:28]: Validating portal[8] <-> 1[8] (inactive)
[00:00:28]: Validating portal[5] <-> 1[5] (inactive)
[00:00:28]: Validating portal[4] <-> 1[4] (inactive)
[00:00:28]: Validating portal[1] <-> 1[1] (inactive)
[00:00:28]: Validating portal[9] <-> 1[9] (inactive)
[00:00:29]: Gameserver logged on to Steam, assigned identity steamid:90134624042942474
[00:00:33]: About to start a shard with these settings:
[00:00:33]:   ShardName: Caves
[00:00:33]:   ShardID: 586447616
[00:00:33]:   ShardRole: SLAVE
[00:00:33]:   MasterHost: dst_master
[00:00:33]:   MasterBind: (null)
[00:00:33]:   MasterPort: 10888
[00:00:33]: [Shard] Connecting to master...
[00:00:33]: [Shard] Sending slave information to master...
[00:00:38]: Overriding region to EU
[00:00:38]: Registering slave in EU lobby
[00:00:39]: [Shard] Slave is now ready!
[00:00:39]: World 1 is now connected
[00:00:39]: Validating portal[6] <-> 1[6] (active)
[00:00:39]: Validating portal[7] <-> 1[7] (active)
[00:00:39]: Validating portal[2] <-> 1[2] (active)
[00:00:39]: Validating portal[10] <-> 1[10] (active)
[00:00:39]: Validating portal[3] <-> 1[3] (active)
[00:00:39]: Validating portal[8] <-> 1[8] (active)
[00:00:39]: Validating portal[5] <-> 1[5] (active)
[00:00:39]: Validating portal[4] <-> 1[4] (active)
[00:00:39]: Validating portal[1] <-> 1[1] (active)
[00:00:39]: Validating portal[9] <-> 1[9] (active)
[00:00:40]: [Shard] Slave LUA is now ready!
[00:00:55]: Sim paused
[00:01:54]: Sim unpaused
[00:02:02]: Serializing world: session/DEFD1AE173744279/0000000207
[00:02:02]: Truncating to snapshot #207...
[00:02:02]:  - session/DEFD1AE173744279/0000000201
[00:02:02]:    1 file(s) removed
[00:02:34]: [Shard] #1 -> session/DEFD1AE173744279/A7LEHMBSU9H1/0000000208
[00:02:34]: [Shard] Received migration #1 data for (KU_hesNpoOX)
[00:03:34]: [Shard] Migration by user ('KU_hesNpoOX') failed (177).
[00:03:34]: Sim paused

 

[Zillvr]

Hello @Alex_77, welcome to the forums. I recall encountering the same issue with a different user with regards to "failed (177)". 

As mentioned in that thread I'm not sure if it would help with your situation either but I suggested that they whitelist DST on their Anti Virus program or disable Anti Virus program entirely and see if it fixes it. Fortunately for them, it fixed the issue. If that doesn't work for you, then it could be due to other reasons, but for now try that first.

What ever antivirus program you have, look for instructions on how to set up whitelisting the executables of DST, primarily:

• dontstarve_steam.exe
• dontstarve_dedicated_server_nullrenderer.exe

Both are found under Steam\steamapps\common\Don't Starve Together\bin. BUT IF you're running your dedicated servers through SteamCMD, then also whitelist the corresponding executables there too (Steamcmd\steamapps\common\Don't Starve Together Dedicated Server\bin).

[Alex_77]

Hi, thanks for your reply. I'm not using any special antivirus program, just the windows 10 integrated one. I tried to completely deactivate it, but the problem still exists. On Client side I have a connection timeout and in the server logs I can read the same error message.

There is one more thing, that I've forgotten to mention. I changed the host network structure some time ago and I thought that the error is maybe caused by a missconfiguration of the firewall or nat. So I checked the firewall and saw, that the udp ports 10998 and 10999 are open (like before). I continued and wanted to check, what packages arrive on this ports. And then I saw, that on the port of the cave (10998) no packages arrived, when I tried to go into a cave. Should I see at this point some traffic and is this maybe the problem?

Edit:

I think I'm getting closer, but I need to explain a lit more. As I said, my dst servers are running in my docker environment. So there is a virtual network with an ip range 10.0.20.xxx and two virtual host (Master and Caves). The Cave Server gets the IP 10.0.20.2 and I can see in the Master log, that the Cave is connected:

[00:00:38]: [Shard] Slave Caves(586447616) connected: [LAN] 10.0.20.2

I hope you understand the docker infrastructure, then you know, that this network is isolated from docker and will be only accessable via docker (and the physical host). Here comes the point, which I don't understand. When sniffing the udp packages send by my client when entering the cave, I see that the client tries to reach the endpoint 10.0.20.2:10998 (in my physical network). Of course this will not work, because 10.0.20.2 is the private IP of the cave host. I cannot understand where the client gets this private IP from. Instead the client has to connect to the public IP, like he does with the master host. Also I don't understand, why running this infrastructure worked before. Any ideas?

[Zillvr]

@Alex_77 considering that you are operating your servers in a unique way, I unfortunately do not have the necessary understanding to help you diagnose the issues you are having. However, I can direct you to a bit more information given by the devs that might help you with your investigations.

• Dedicated Server Settings Guide - cluster.ini and server.ini configuration information
• Understanding Shards and Migration Portals
• Klei Support Article: Network and Performance Troubleshooting Guide

Aside from that, we may have to wait on other users with better know-how than I do (I'm unfortunately only a forum moderator that has dabbled a bit with dedicated servers) OR if the devs could give better insight and should their availability line up. We'll just have to wait on that. Sorry for not being much help. Good luck with your investigations though. If ever I encounter any other possibilities, I'll let you know.

[Alex_77]

Thanks for your help again, but I already read the articles and they really helped me for understanding the communication strategies of the shades. I really hope that somebody with a good network understanding will have the time to look at my case.

Maybe I can summarize my problem in a more general way, which is better understandable. So I have a multi-machine setup with two hosts. Both are located in a private LAN and get the IPs 10.0.20.1 and 10.0.20.2. This LAN Can be accessed by an Gateway with a public IP which routes the udp port 10999 to the master host and the udp port 10998 to the caves host. Other ports are not reachable from outside. The communication between the hosts are LAN only.

Now when a client connects to the cave he uses the ip 10.0.20.2 and port 10998, which of course never will be successfull, because he uses the private IP of the host. So my primary question is, where did the client get the ip of the cave host from? I suggest the master host tells him this IP, because the cave host connected to him with this IP. This will be an explanation why this setup will not work. But I consider this behaviour as I bug. And then there must have been some code changes which have caused this behaviour, because this setup worked in the past.