Understanding Shards and Migration Portals

[thasmo]

I'm not sure there is a command at all, at least I'm not aware of it.

[thomathy007]

Update: I figured out the problem! I am now up and running! The issue was I didn't realize I needed to launch the nullrenderer exe file through bat files.

 

I wasn't sure how to add multiple shards into the launch options, so I just put them in consecutively. Running the DST dedicated tool only opens one console and only the winter shard tried to connect to the master. The console starts looping and showing that it's trying to connect the Winter shard, but then says "[sHARD] Connecting to master..." "Connection to master failed. Waiting to reconnect..." "About to start a shard with these settings:" Then it shows the Winter shard and loops again.

Edited December 31, 2015 by thomathy007

[thomathy007]

@Ipsquiggle I would really like to have a constant season in one world. Will this be possible in the future? Can seasons be desynchronized? Is there a way I can bypass the OVERRIDE: setting season_start to winter at this time?

 

Also, the command that links portals to certain worlds only worked for me if the world id was in quotes like so:

 

c_find('cave_entrance').components.worldmigrator:SetDestinationWorld("2", true)

 

I think the OP should be corrected to show this. I had trouble with it for a while.

Edited December 31, 2015 by thomathy007

[thasmo]

A mod for auto-connecting sinkholes/portals would be slick! Wondering if someone's already working on this?

[UnderwearApp]

Let me see if I can shed some light here:

 

 

 

 

~huge snip~

 

Unfortunately managing the "received" portals isn't simple and we haven't put in a good way of doing this yet. Right now it just assumes that the IDs match on both sides, which is pretty gross. This is enforced in OnLoad, so if you want to overwrite it you have to do it (again, ideally in a mod) one frame after the game loads (e.g. using inst:DoTaskInTime(0, ...)).

 

Hope that answers some of your questions and gets you in the right direction... Just ask if you need more specific advice!

 

I was hoping to get some clarification on this. Does this mean every time I load (start up) my dedicated server the caves get reset by the auto-linking? I just want to make sure because it seemed to work well when playing earlier.

 

However, I also thought that some cave exits which I blocked by setting their destination world to something that doesn't exist were open again. If the destination world is false or nil, do these exits get linked somewhere on startup? If that is the case, will they overwrite ones I have already manually set up?

 

And while we're at it, is there an easy way to obtain the shard id of a server, ideally through some command I can put into a mod?

 

Thanks in advance!

 

Prof

Edited January 8, 2016 by ProfFarnsworth

[Nuke1]

I made mod for connecting the shards. It's available on workshop and DST forums. I included tutorial how to setup the mod. I'm using the mod and was able to connect 6 shards together by numerous portals.

Any feedback is welcomed.

[Nuke1]

Is there a way for a mod to get this world's shard_id?

I tried TheStard:GetShardId() but TheShard was nil. Did I forgot to do something before calling TheShard?

EDIT: nevermind, I figured it out

And is it possible for mods on each world to send some information between them?
 

Edited January 21, 2016 by Nuke1

[Ipsquiggle]

Hey everyone, I updated this post to use the up-to-date .ini files and clarified the example (I hope!). Please let me know if this needs any further clarification. Also @Nuke1, that mod is great, I linked it in the top post. Thanks!

[su2lin]

Hi, I need some help...

I am trying to connect 2 over-world and 1 cave together,  like 

But no matter how I set the portal, the cave world will always use all the connection to the master world, do you have any suggestions?

[Ipsquiggle]

10 hours ago, su2lin said:

But no matter how I set the portal, the cave world will always use all the connection to the master world, do you have any suggestions?

If you've already got your servers set up, I'd strongly recommend using Nuke1's tool for configuring the connections. But if you'd rather do it yourself, make sure that when you connect the portals, you disable the automatic connections by setting the second parameter to true:

:SetDestinationWorld(2, true)

If you forget to do this, then the connections will get reset next time the server loads. Also don't forget that you have to configure the portals from both ends: point the forest portals at the cave, and the cave portals back at the forest.

Does that help?

[su2lin]

On 2016/3/23 at 11:19 PM, Ipsquiggle said:

If you've already got your servers set up, I'd strongly recommend using Nuke1's tool for configuring the connections. But if you'd rather do it yourself, make sure that when you connect the portals, you disable the automatic connections by setting the second parameter to true:

:SetDestinationWorld(2, true)

If you forget to do this, then the connections will get reset next time the server loads. Also don't forget that you have to configure the portals from both ends: point the forest portals at the cave, and the cave portals back at the forest.

Does that help?

Great thanks! And two more questions: If I dont want to use Nuke1's mod tool (some part in China the network is not easy to connect klei's server to update mods), I need to start all the server world include the cave, then use the command to set the connection? While if I restart the server, will I need to set the connections again?

[Ipsquiggle]

5 hours ago, su2lin said:

Great thanks! And two more questions: If I dont want to use Nuke1's mod tool (some part in China the network is not easy to connect klei's server to update mods), I need to start all the server world include the cave, then use the command to set the connection? While if I restart the server, will I need to set the connections again?

Technically, the worlds don't all have to be running at the same time to set the connections. But it will probably easier if they are all running at the same time.

For making the servers reconnect to each other: Once the settings in the .ini are correct, they will remain connected after a restart.

For making the portals link to each other: So long as you do SetDestinationWorld(2, true) <-- (don't forget the true!) then they will remain connected after a restart.

[simplex]

@Ipsquiggle

Are there any Lua-side methods for inter-shard communication? The only thing I spotted as a possible vector of data transmission is the world_data parameter passed to Shard_UpdateWorldState in shardnetworking.lua, but from my limited experiments it's always been nil.

Edited March 29, 2016 by simplex
grammar

[Ipsquiggle]

@simplex They communicate from master to shard using netvars. shard_clock.lua is an example of this, it reads the time from the master server's clock, sends it to the pair shard_clock on shards via netvars, and then pushes the state onto the shard's clock. (This component is added to the "shard_network" prefab, whose :AddShardNetwork() causes it to sync to servers instead of clients.)

At the moment there is to communication from shard back to master.

[simplex]

1 minute ago, Ipsquiggle said:

At the moment there is to communication from shard back to master.

I presume this is why the game hasn't yet implemented a handshake when pairing portal ids? This was specifically what I was hoping to implement.

[Ipsquiggle]

1 minute ago, simplex said:

I presume this is why the game hasn't yet implemented a handshake when pairing portal ids? This was specifically what I was hoping to implement.

Correct. I was discussing this with Nuke a little. Since central configuration is pretty much required to get the number of portals to match in each world, I think it's plausible for the master to generate all connections, push the data to the shards, and they update or hide their portals as appropriate.

[simplex]

Just now, Ipsquiggle said:

Correct. I was discussing this with Nuke a little. Since central configuration is pretty much required to get the number of portals to match in each world, I think it's plausible for the master to generate all connections, push the data to the shards, and they update or hide their portals as appropriate.

But this would require the slaves to push its list of portal ids (and ideally metadata, such as a list of tags, to allow for mod-specific automatic pairing logic), correct? So this isn't currently possible without extending the engine/Lua API?

[Ipsquiggle]

Just now, simplex said:

But this would require the slaves to push its list of portal ids (and ideally metadata, such as a list of tags, to allow for mod-specific automatic pairing logic), correct? So this isn't currently possible without extending the engine/Lua API?

In an ideal world, yes.  In the meantime, a system like Nuke's tool does describe unambiguous pairings. You just have to make the distinctions when generating this list of worlds and pairs, rather than when pairing. "1,1" connects to "2,7" (and "2,7" connects to "1,1") or whatever. How do we know world 2 has a portal 7? We don't, that's the shard's problem now. It can create or reclaim a portal to make it so.

[simplex]

11 minutes ago, Ipsquiggle said:

In an ideal world, yes.  In the meantime, a system like Nuke's tool does describe unambiguous pairings. You just have to make the distinctions when generating this list of worlds and pairs, rather than when pairing. "1,1" connects to "2,7" (and "2,7" connects to "1,1") or whatever. How do we know world 2 has a portal 7? We don't, that's the shard's problem now. It can create or reclaim a portal to make it so.

But how can the master shard have sufficient information to pair 2 slaves? If one of the pairing's endpoint is the master, conceivably it could have sufficient information to generate the pairing itself and just push it to the slaves. But if both endpoints are slaves, then how can the master shard know the pairing should be made (other than from manual configuration or reading savedata)? Do the slave shards export any form of metadata to the master shard that may be used to instruct the pairing heuristic?

[Ipsquiggle]

24 minutes ago, simplex said:

But how can the master shard have sufficient information to pair 2 slaves? If one of the pairing's endpoint is the master, conceivably it could have sufficient information to generate the pairing itself and just push it to the slaves. But if both endpoints are slaves, then how can the master shard know the pairing should be made (other than from manual configuration or reading savedata)? Do the slave shards export any form of metadata to the master shard that may be used to instruct the pairing heuristic?

I think we're considering two different scenarios. In my thinking, only the master has enough information to pair two slaves. The multi-world topology is known at generation time (or as close as possible), just as the topology within a world is known. The master (or user configuration) contains the definition of how a shard should behave when it attaches itself, just as the data for worldgen flowed from the master (or user configuration). For example, in a diamond configuration, the master knows the diamond. There are pairings between two shards that the master isn't "involved in", but it nonetheless instructed those shards to link to each other.

I feel like you're thinking about a more ad-hoc, pseudo-peer-to-peer thing, where a shard can attach and announce it's potential, and the rest of the cluster absorbs it. While I'd absolutely love to see that, it definitely grinds against some of the assumptions we've made under the above scenario. Right now, the only metadata the master gets from shards is their world ID, which it could use as a key into the configuration.

[simplex]

Just now, Ipsquiggle said:

I think we're considering two different scenarios. In my thinking, only the master has enough information to pair two slaves. The multi-world topology is known at generation time (or as close as possible), just as the topology within a world is known. The master (or user configuration) contains the definition of how a shard should behave when it attaches itself, just as the data for worldgen flowed from the master (or user configuration). For example, in a diamond configuration, the master knows the diamond. There are pairings between two shards that the master isn't "involved in", but it nonetheless instructed those shards to link to each other.

I feel like you're thinking about a more ad-hoc, pseudo-peer-to-peer thing, where a shard can attach and announce it's potential, and the rest of the cluster absorbs it. While I'd absolutely love to see that, it definitely grinds against some of the assumptions we've made under the above scenario. Right now, the only metadata the master gets from shards is their world ID, which it could use as a key into the configuration.

I acknowledge that the hierarchy consists of a central, authoritative server, that dictates the network topology. I think our divergence is in that what I'm considering is extensions to the automatic pairing as performed right now for simple topologies (one master overworld, one slave cave). As far as I know, the master doesn't have access to data such as the slave's saveindex entries, or at least the preset it's running; without that, only knowing which shards there are and having a unique handle for each, portal configuration has very limited automation potential.

[simplex]

@Ipsquiggle

Nevermind. The world_data parameter of Shard_UpdateWorldState is, in fact, non-nil when a slave shard connects to the master shard, and it contains the metadata I was after (most importantly, the "actualpreset" entry). That should be enough to extend the automatic pairing of portals.

[Muche]

Just wondering what is world_network's :AddShardClient() relation to this.

[Oarc]

On 11/24/2015 at 9:23 AM, absimiliard said:

Does anyone know what happens if you try to link worlds with different modes?  (survival and wilderness for example)

 

I kind of like the idea of having a "hidden" world in wilderness that can't be reached except by going through a survival surface and a survival caves, or maybe a cave inside a cave -- the interesting bit being having a permanent world linked to two temporary worlds.  Having it difficult to reach/find would help somewhat with griefing, I hope.

Does anyone have an answer to this? Or any success with getting this working?

 

In addition, commands like "c_regenerateworld()" seem to affect all connected shards? I can see how that makes sense on one level, but I assume it means the above scenario isn't possible? If you wanted an endless world paired with a survival world that resets occasionally... seems like that would be difficult to handle.

Edited April 8, 2016 by Oarc
Adding another questions.

[Ipsquiggle]

@Oarc Right now the shard systems assumes that all connected servers form a "single world". This is intentional as that is the main design we want to support, and supporting other scenarios increases complexity. We do want to allow more varied clusters in the future but at the moment it's not what the system is built for, unfortunately.