guide [GUIDE] A Good Way of Talking Between Shards

[rawii22]

Hey! If you've ever needed to send information from one shard to another, we're pretty sure we figured out how. I partially wanted to write all this down since it doesn't look like anyone has had an in-depth discussion on handling inter-shard communication. @AlbertoRomanach (my brother) and I were recently working on a mod that needed this ability and we finally sat down and figured it out. There's a link at the bottom. As a little teaser, it is possible to send commands from one shard to another...

This guide could help if you're making a mod that needs to deal with overworld and cave shards differently.

First of all, Klei is very nice with their mod utils, and none of this would be possible without the AddShardModRPCHandler function. I'm pretty sure we also saw an example of this function used in GemCore under gemscripts/tools/shardrpc.lua in case anyone wants to take a look, so say thank you to @zarklord_klei!

From A Mod

Here is a very simple implementation of an RPC that allows you to execute a Lua string on any target shard:

AddShardModRPCHandler(modname, "ShardInjection", function(shardId, namespace, code, injection)
    ------THIS BODY CAN BE CUSTOMIZED------
	-- Only run if the current shard is different from the shard that called the RPC. The *calling* shard's ID is held in shardId.
	if GLOBAL.TheShard:GetShardId() ~= tostring(shardId) then
		GLOBAL.ExecuteConsoleCommand(injection)
	end
    ------END------
end)

When you are using caves (or a dedicated server with caves), when one shard uses this RPC, the opposite shard will detect it and execute the contents of the RPC (which shown above is just a call to ExecuteConsoleCommand). If you remove the "GLOBAL.TheShard:GetShardId() ~= tostring(shardId)" check, then the RPC will run on ALL SHARDS. If you're somehow running a server with more than two shards, then you'll likely have to be a little more specific with the shardId check depending on what you're doing.

The three parameters "shardId, namespace, and code" we found in GemCore's shardrpc.lua, but we never used them. But we added a custom parameter "injection" which is where the Lua string is passed in. You'll see its usage a little lower.

Here is an example of its use in a mod file:

function testFunction(string)
  		--In mods, "modname" will automatically translate to the name of the directory the mod is in.
		SendModRPCToShard(GetShardModRPC(modname, "ShardInjection"), nil, nil, nil, "c_announce(\""..string.."\")")
end

For us normal cave users, the ShardInjection RPC will simply execute the string "c_announce(\""string"\")" from the opposite shard. If you're using more than two shards and you only want to target a single shard, I imagine you could theoretically change the first "nil" argument in the GetShardModRPC to be a more specific shardId, but I've never used more than two shards before, so you'll have to mess around with that. The Lua string can be literally anything you want, as long as the string can be executed from the console.

Remember, this RPC can be customized to do something other than code execution. You can add parameters as you please. If you simply need to retrieve or send info from other shards or do anything else, you can do that. In fact, data retrieval was what we mainly used the AddShardModRPCHandler for in the first place.

From the Terminal

~~~This next section talks about sending commands to another shard from the terminal, but a mod with this RPC must be enabled first.~~~

Now, for an even more interesting use of this, admins on a dedicated server can technically use this RPC from the console if they know what to do. They can execute a command on another shard. Clients without admin privileges, however, are not able to use the following command since they don't have access to the remote terminal. If they did, this would be a security issue since any random client would be able to hop onto a server with this RPC and execute Lua code injections. Fortunately, they can't! Unless you deliberately set that up, this should not be a problem.

Using the same RPC as before, this command can be used from the console (remotely) by admins:

SendModRPCToShard(GetShardModRPC("workshop-########", "ShardInjection"), nil, nil, nil, "Lua code here")

"workshop-########" must be the mod that has the ShardInjection RPC defined in it. If you're testing locally, then you can just change "workshop-########" to be the mod's directory name like "folderNameHere" (it must be in quotes). Also, you could theoretically set up a new console command from inside a mod that simply accepts the Lua string and executes this line internally. It would look very similar to the RPC I showed earlier, but instead of "c_announce(\""string"\")" you could just put the variable "string". You'd also have to make the function GLOBAL to make it accessible from the console.

Here's a console command that could go in a mod file:

function GLOBAL.testFunctionTwo(string)
		SendModRPCToShard(GetShardModRPC(modname, "ShardInjection"), nil, nil, nil, string)
end

You should be able to call it from the console like this: testFunctionTwo("AllPlayers[2].components.health:SetPercent(1)"). If called from the overworld, this command would refill the health of player 2 from the perspective of the cave shard. This also works in the other direction. For example, if there are 4 players, 2 in the overworld, and 2 in the caves, the command will refill whoever is in the AllPlayers[2] spot in the caves. However, this command will not work if you try to target players who are in the same shard as you.

Our mod shows a list of every player in the game (including caves) on the screen based on the server's version of AllPlayers. Using the same example, player 2 in caves would appear as player 4 on our list. We have a few console commands that are capable of detecting whether player 4 is in the caves. If the target player is in the caves, we have a function that both converts the number 4 into the proper player number and sends the command to the caves. But if player 4 is NOT in the caves, it can ALSO execute the console command on the same shard. You can see how my brother and I dealt with this toward the bottom of our modmain. You can also read more about it in the mod comments and in the readme for the mod.

In Conclusion:

Communication between shards doesn't have to be so much of a nightmare. Once you have it down, it's pretty easy to replicate and tailor to your needs. Like I said earlier, it's possible this could help with mods that need to deal with overworld and cave shards differently. Regardless, as I also mentioned earlier, Klei is very nice. I admire Don't Starve Together not only for its gameplay, but also for its amazing programming and team. I've learned a lot from its code, and I hope I can help people with what I've learned before I forget all of this !

Also, if anyone needs help setting up dedicated servers on Linux, I just recently figured that out (at least on ubuntu) so ask me about that if you need to.

Leave a comment in case any of you need more info! If you think I should discuss anything else in here, please let me know and I can add it so people can see it.

You can see the mod with which we used this method of communication right here. It was used in order to collect the list of players from both the overworld AND the caves and print them in order based on the server's version of AllPlayers:

Spoiler

Steam Workshop: https://steamcommunity.com/sharedfiles/filedetails/?id=2169859220

GitHub: https://github.com/rawii22/PlayerStatus

Sorry! I realized two seconds after I posted this that there was a tutorials and guides subforum 

Edited March 16, 2022 by rawii22

[rawii22]

As a teaser for a guide I might make later, I've figured out how to host more than two shards AND how to host them on completely different computers. I used the guides from the devs, but I've never seen it in action until now. So, if anyone is curious about that, let me know!

I also know how to set up dedicated servers in linux, so if anyone wants to do that for themselves, let me know and I'll see what I can do to help. (I highly recommend it since the game runs a million times faster in linux. I have a theory that Klei develops most of their games in linux for this reason, but also because Klei is always immediately ready for releases on linux based systems.)

(learning computer networking with DST is the best thing that has ever happened to me. Klei is amazing. They made these things possible even though they're not usually used in the main game...)

...