Jump to content

Retrieving DST Server Data


Recommended Posts

Hey @nome, as requested I'm moving this to the public forums. I had a few more questions I'm hoping you would know the answers to. To bring others up to speed on what I'm doing. Basically I'm wanting to build my own version of https://dstserverlist.appspot.com/ except I want to enhance it a bit so I can do some analytics on the servers my friends and I host.

1. 

I can do a GET to https://s3.amazonaws.com/klei-lobby and it provides a list of lobbies. I can do a more specific GET such as https://s3.amazonaws.com/klei-lobby/US-Steam-noevent.json.gz and it provides a long list of servers. Is there a lobby that lists all of the servers?

Taking a look at the lobbies:

  • lobbyListings_all_1.json.gz
  • lobbyListings_all_2.json.gz
  • ...
  • lobbyListings_all_9.json.gz

They all return the same 5 servers. I don't want to iterate through all the lobbies and get a ton of duplicates so I'm hoping there's a lobby that just lists all of the servers.

2.
Is there some ID that I can use to track a server? Something that will remain the same until the cluster_token is changed? When doing a GET requests I receive fields such as __rowId, guid, session, steamid. All of these change when the server shuts down and restarts with the exception of session which changes on world regen. You had mentioned I could send UDP packets to get more information on specific servers (I’m assuming it will give me players, mods, etc). I'm wondering if there will also be a permanent ID I can use.

3.

Which ID's match up to which platform? I know steam = 1, but not sure about which ID xbox, ps4, etc use.

 

Thank you for the help :)

Link to comment
Share on other sites

  • Developer

I'm afraid you need to concatenate all the files in https://s3.amazonaws.com/klei-lobby/ to get the complete lobby listings.

lobbyListings_all_1.json.gz is for an incredibly old version of the game. Not sure who would still be playing that but people do weird things. :) Look at the version tag ("v":388775) to see what builds people are playing on.

What even is the same server? What if I take the same save and start it on two different processes? Is it still the same server if my survival map dies and we all start again with a need seed? As you figured out, __rowId maps to a single listing, session changes on regen. Obviously KU is reasonably stable (the user who is hosting the server). And you shouldn't discount the server name string, the combination of KU,ServerName is what I'd probably use.

You don't need to connect with UDP to get stuff like mods. Something more like this should do the trick:

curl -X POST -d '{"__token": "Blahblah", "__gameId": "DST", "query": {"__rowId":"b7f91091ffa161a81fb221bc1b8e2584"}}' https://lobby-china.kleientertainment.com/lobby/read | python -mjson.tool | less

(Server picked based on it being the first server in the first region in alphabetical order. :) )

You'll need a token to make that happen, you can use the one from your game but that'll eventually expire. Send me a private message with your KU and I'll build you one that lasts forever.
For the platform field:

const ( 
        PLATFORM_NONE   = 0x0000
        PLATFORM_STEAM  = 0x0001
        PLATFORM_PSN    = 0x0002
        PLATFORM_TGP    = 0x0004
        PLATFORM_QQGAME = 0x0008
        PLATFORM_XBLIVE = 0x0010
)
 

QQGAME is defunct (those users were given copies on TGP which became WeGame). Observant people might guess that 0x03 would be a server that could host both Steam and PS4 - fingers crossed for some day!

 

Note you can totally use a tool like charles or other HTTPS proxies, along with the https_proxy environment variable (which DST should respect) to spy on what DST is saying to the servers and reverse engineer it. And the UDP stuff is all unencrypted on PC, so tcpdump or wireshark work great there. But I'm also happy to help when I'm not too busy. Good luck, and let me know if you have any other questions!

Link to comment
Share on other sites

@X-lem

As per permission from nome there, this information is freely able to be posted.

If you exclude the rowId from the query and leave it blank, then it will try to spit back all of the servers as one blob.

Nome mentioned that it's undocumented behaviour that may vanish at any point in time, so mind that.

From my test you have 2 minutes to download the file, or on their end the session times you out but doesn't disclose that it stops.

 

This is a BaSH script I made a while ago that scrapes all of the major regions and spits them as json files for further processing:

#!/bin/bash

token='pcl-usc^KU_^DontStarveTogether^otherpartsoftokenhere'

servers=(us eu china sing)

for i in ${servers[*]}
do
  curl -k -X POST -d \
  '{"__gameId":"DontStarveTogether","__token":"'${token}'","query":{}}' \
  https://lobby-${i}.kleientertainment.com/lobby/read > \
  /data/dst_${i}.json
done

On that drive I have a mount point /data/ for things like this.  I use the '-k' field there to ignore any sort of cert errors since the data passed and received really isn't anything too sensitive that I'd lose time or money on if someone sniffed it out of the blue.

Further parsing is on you, since json has so many third party libraries to handle it for you these days.

 

On 2/8/2020 at 4:45 PM, nome said:

For the platform field:

const ( 
        PLATFORM_NONE   = 0x0000
        PLATFORM_STEAM  = 0x0001
        PLATFORM_PSN    = 0x0002
        PLATFORM_TGP    = 0x0004
        PLATFORM_QQGAME = 0x0008
        PLATFORM_XBLIVE = 0x0010
)
 

QQGAME is defunct (those users were given copies on TGP which became WeGame). Observant people might guess that 0x03 would be a server that could host both Steam and PS4 - fingers crossed for some day!

Quote

"host":"KU_j5dBuNMf"
"id":"3412915395","steamid":"90132542528669705","port":11007}},"clanonly":false,
"platform":19,
"mods":false,"name":"PS4 Germany 4 - Klei Official","pvp":false,"session":"65FEB6B5BA15CE2B","fo":false,"password":false,"guid":"11501185584297207609","maxconnections":6,"dedicated":true,"clienthosted":false,"connected":6,"mode":"survival","port":11006,"v":392022000

You know I was asking someone earlier today how many platforms Klei was going to support because this 19 seemed arbitrarily high.

Now I know the PS4 servers Klei hosts are superior in that they work for XBLIVE, PSN, and STEAM all at once..!

Link to comment
Share on other sites

39 minutes ago, X-lem said:

Hey @CarlZalph, tried using your bash script. It's printing out 


{"error":"WRONG_TOKEN_PURPOSE"}

in all the files. I'm just using one of my dedicated server tokens. Did you have to request a special one?

The token is a client's authentication token, which generally expires after 24-48 hours.  Ask nome for a permanent one if you want one.

I personally generate a new one when I want to probe things by using Steamwork's ISteamUser's GetAuthSessionTicket and then authenticating it with the Klei backend to generate a new session token.

It's the same token you can get in-game at the main menu via using the console: TheFrontEnd:GetAccountManager():GetToken()

Link to comment
Share on other sites

Archived

This topic is now archived and is closed to further replies.

Please be aware that the content of this thread may be outdated and no longer applicable.

×
  • Create New...