Ipsquiggle

Understanding Shards and Migration Portals

62 posts in this topic

Update: I figured out the problem! I am now up and running! The issue was I didn't realize I needed to launch the nullrenderer exe file through bat files.

 

I wasn't sure how to add multiple shards into the launch options, so I just put them in consecutively. Running the DST dedicated tool only opens one console and only the winter shard tried to connect to the master. The console starts looping and showing that it's trying to connect the Winter shard, but then says "[sHARD] Connecting to master..." "Connection to master failed. Waiting to reconnect..." "About to start a shard with these settings:" Then it shows the Winter shard and loops again.

Edited by thomathy007

Share this post


Link to post
Share on other sites

@Ipsquiggle I would really like to have a constant season in one world. Will this be possible in the future? Can seasons be desynchronized? Is there a way I can bypass the OVERRIDE: setting season_start to winter at this time?

 

Also, the command that links portals to certain worlds only worked for me if the world id was in quotes like so:

 

c_find('cave_entrance').components.worldmigrator:SetDestinationWorld("2", true)

 

I think the OP should be corrected to show this. I had trouble with it for a while.

Edited by thomathy007
1 person likes this

Share this post


Link to post
Share on other sites

A mod for auto-connecting sinkholes/portals would be slick! Wondering if someone's already working on this?

Share this post


Link to post
Share on other sites

Let me see if I can shed some light here:

 

 

 

 

~huge snip~

 

Unfortunately managing the "received" portals isn't simple and we haven't put in a good way of doing this yet. Right now it just assumes that the IDs match on both sides, which is pretty gross. This is enforced in OnLoad, so if you want to overwrite it you have to do it (again, ideally in a mod) one frame after the game loads (e.g. using inst:DoTaskInTime(0, ...)).

 

Hope that answers some of your questions and gets you in the right direction... Just ask if you need more specific advice!

 

I was hoping to get some clarification on this. Does this mean every time I load (start up) my dedicated server the caves get reset by the auto-linking? I just want to make sure because it seemed to work well when playing earlier.

 

However, I also thought that some cave exits which I blocked by setting their destination world to something that doesn't exist were open again. If the destination world is false or nil, do these exits get linked somewhere on startup? If that is the case, will they overwrite ones I have already manually set up?

 

And while we're at it, is there an easy way to obtain the shard id of a server, ideally through some command I can put into a mod?

 

Thanks in advance!

 

Prof

Edited by ProfFarnsworth

Share this post


Link to post
Share on other sites

I made mod for connecting the shards. It's available on workshop and DST forums. I included tutorial how to setup the mod. I'm using the mod and was able to connect 6 shards together by numerous portals.

Any feedback is welcomed.

2 people like this

Share this post


Link to post
Share on other sites

Is there a way for a mod to get this world's shard_id?

I tried TheStard:GetShardId() but TheShard was nil. Did I forgot to do something before calling TheShard?

EDIT: nevermind, I figured it out

And is it possible for mods on each world to send some information between them?
 

Edited by Nuke1
2 people like this

Share this post


Link to post
Share on other sites

Hey everyone, I updated this post to use the up-to-date .ini files and clarified the example (I hope!). Please let me know if this needs any further clarification. Also @Nuke1, that mod is great, I linked it in the top post. Thanks!

2 people like this

Share this post


Link to post
Share on other sites

Hi, I need some help...

I am trying to connect 2 over-world and 1 cave together,  like 

56f223d587f69_.png.5d69a1cacba436e0c8819

But no matter how I set the portal, the cave world will always use all the connection to the master world, do you have any suggestions?

Share this post


Link to post
Share on other sites
10 hours ago, su2lin said:

But no matter how I set the portal, the cave world will always use all the connection to the master world, do you have any suggestions?

If you've already got your servers set up, I'd strongly recommend using Nuke1's tool for configuring the connections. But if you'd rather do it yourself, make sure that when you connect the portals, you disable the automatic connections by setting the second parameter to true:

:SetDestinationWorld(2, true)

If you forget to do this, then the connections will get reset next time the server loads. Also don't forget that you have to configure the portals from both ends: point the forest portals at the cave, and the cave portals back at the forest.

Does that help?

1 person likes this

Share this post


Link to post
Share on other sites
On 2016/3/23 at 11:19 PM, Ipsquiggle said:

If you've already got your servers set up, I'd strongly recommend using Nuke1's tool for configuring the connections. But if you'd rather do it yourself, make sure that when you connect the portals, you disable the automatic connections by setting the second parameter to true:

:SetDestinationWorld(2, true)

If you forget to do this, then the connections will get reset next time the server loads. Also don't forget that you have to configure the portals from both ends: point the forest portals at the cave, and the cave portals back at the forest.

Does that help?

Great thanks! And two more questions: If I dont want to use Nuke1's mod tool (some part in China the network is not easy to connect klei's server to update mods), I need to start all the server world include the cave, then use the command to set the connection? While if I restart the server, will I need to set the connections again?

Share this post


Link to post
Share on other sites
5 hours ago, su2lin said:

Great thanks! And two more questions: If I dont want to use Nuke1's mod tool (some part in China the network is not easy to connect klei's server to update mods), I need to start all the server world include the cave, then use the command to set the connection? While if I restart the server, will I need to set the connections again?

Technically, the worlds don't all have to be running at the same time to set the connections. But it will probably easier if they are all running at the same time.

For making the servers reconnect to each other: Once the settings in the .ini are correct, they will remain connected after a restart.

For making the portals link to each other: So long as you do SetDestinationWorld(2, true) <-- (don't forget the true!) then they will remain connected after a restart.

Share this post


Link to post
Share on other sites

@Ipsquiggle

Are there any Lua-side methods for inter-shard communication? The only thing I spotted as a possible vector of data transmission is the world_data parameter passed to Shard_UpdateWorldState in shardnetworking.lua, but from my limited experiments it's always been nil.

Edited by simplex
grammar

Share this post


Link to post
Share on other sites

@simplex They communicate from master to shard using netvars. shard_clock.lua is an example of this, it reads the time from the master server's clock, sends it to the pair shard_clock on shards via netvars, and then pushes the state onto the shard's clock. (This component is added to the "shard_network" prefab, whose :AddShardNetwork() causes it to sync to servers instead of clients.)

At the moment there is to communication from shard back to master.

Share this post


Link to post
Share on other sites
1 minute ago, Ipsquiggle said:

At the moment there is to communication from shard back to master.

I presume this is why the game hasn't yet implemented a handshake when pairing portal ids? This was specifically what I was hoping to implement.

Share this post


Link to post
Share on other sites
1 minute ago, simplex said:

I presume this is why the game hasn't yet implemented a handshake when pairing portal ids? This was specifically what I was hoping to implement.

Correct. I was discussing this with Nuke a little. Since central configuration is pretty much required to get the number of portals to match in each world, I think it's plausible for the master to generate all connections, push the data to the shards, and they update or hide their portals as appropriate.

Share this post


Link to post
Share on other sites
Just now, Ipsquiggle said:

Correct. I was discussing this with Nuke a little. Since central configuration is pretty much required to get the number of portals to match in each world, I think it's plausible for the master to generate all connections, push the data to the shards, and they update or hide their portals as appropriate.

But this would require the slaves to push its list of portal ids (and ideally metadata, such as a list of tags, to allow for mod-specific automatic pairing logic), correct? So this isn't currently possible without extending the engine/Lua API?

Share this post


Link to post
Share on other sites
Just now, simplex said:

But this would require the slaves to push its list of portal ids (and ideally metadata, such as a list of tags, to allow for mod-specific automatic pairing logic), correct? So this isn't currently possible without extending the engine/Lua API?

In an ideal world, yes. :) In the meantime, a system like Nuke's tool does describe unambiguous pairings. You just have to make the distinctions when generating this list of worlds and pairs, rather than when pairing. "1,1" connects to "2,7" (and "2,7" connects to "1,1") or whatever. How do we know world 2 has a portal 7? We don't, that's the shard's problem now. It can create or reclaim a portal to make it so.

Share this post


Link to post
Share on other sites
11 minutes ago, Ipsquiggle said:

In an ideal world, yes. :) In the meantime, a system like Nuke's tool does describe unambiguous pairings. You just have to make the distinctions when generating this list of worlds and pairs, rather than when pairing. "1,1" connects to "2,7" (and "2,7" connects to "1,1") or whatever. How do we know world 2 has a portal 7? We don't, that's the shard's problem now. It can create or reclaim a portal to make it so.

But how can the master shard have sufficient information to pair 2 slaves? If one of the pairing's endpoint is the master, conceivably it could have sufficient information to generate the pairing itself and just push it to the slaves. But if both endpoints are slaves, then how can the master shard know the pairing should be made (other than from manual configuration or reading savedata)? Do the slave shards export any form of metadata to the master shard that may be used to instruct the pairing heuristic?

Share this post


Link to post
Share on other sites
24 minutes ago, simplex said:

But how can the master shard have sufficient information to pair 2 slaves? If one of the pairing's endpoint is the master, conceivably it could have sufficient information to generate the pairing itself and just push it to the slaves. But if both endpoints are slaves, then how can the master shard know the pairing should be made (other than from manual configuration or reading savedata)? Do the slave shards export any form of metadata to the master shard that may be used to instruct the pairing heuristic?

I think we're considering two different scenarios. In my thinking, only the master has enough information to pair two slaves. The multi-world topology is known at generation time (or as close as possible), just as the topology within a world is known. The master (or user configuration) contains the definition of how a shard should behave when it attaches itself, just as the data for worldgen flowed from the master (or user configuration). For example, in a diamond configuration, the master knows the diamond. There are pairings between two shards that the master isn't "involved in", but it nonetheless instructed those shards to link to each other.

I feel like you're thinking about a more ad-hoc, pseudo-peer-to-peer thing, where a shard can attach and announce it's potential, and the rest of the cluster absorbs it. While I'd absolutely love to see that, it definitely grinds against some of the assumptions we've made under the above scenario. Right now, the only metadata the master gets from shards is their world ID, which it could use as a key into the configuration.

Share this post


Link to post
Share on other sites
Just now, Ipsquiggle said:

I think we're considering two different scenarios. In my thinking, only the master has enough information to pair two slaves. The multi-world topology is known at generation time (or as close as possible), just as the topology within a world is known. The master (or user configuration) contains the definition of how a shard should behave when it attaches itself, just as the data for worldgen flowed from the master (or user configuration). For example, in a diamond configuration, the master knows the diamond. There are pairings between two shards that the master isn't "involved in", but it nonetheless instructed those shards to link to each other.

I feel like you're thinking about a more ad-hoc, pseudo-peer-to-peer thing, where a shard can attach and announce it's potential, and the rest of the cluster absorbs it. While I'd absolutely love to see that, it definitely grinds against some of the assumptions we've made under the above scenario. Right now, the only metadata the master gets from shards is their world ID, which it could use as a key into the configuration.

I acknowledge that the hierarchy consists of a central, authoritative server, that dictates the network topology. I think our divergence is in that what I'm considering is extensions to the automatic pairing as performed right now for simple topologies (one master overworld, one slave cave). As far as I know, the master doesn't have access to data such as the slave's saveindex entries, or at least the preset it's running; without that, only knowing which shards there are and having a unique handle for each, portal configuration has very limited automation potential.

1 person likes this

Share this post


Link to post
Share on other sites

@Ipsquiggle

Nevermind. The world_data parameter of Shard_UpdateWorldState is, in fact, non-nil when a slave shard connects to the master shard, and it contains the metadata I was after (most importantly, the "actualpreset" entry). That should be enough to extend the automatic pairing of portals.

1 person likes this

Share this post


Link to post
Share on other sites

Just wondering what is world_network's :AddShardClient() relation to this.

Share this post


Link to post
Share on other sites
On 11/24/2015 at 9:23 AM, absimiliard said:

Does anyone know what happens if you try to link worlds with different modes?  (survival and wilderness for example)

 

I kind of like the idea of having a "hidden" world in wilderness that can't be reached except by going through a survival surface and a survival caves, or maybe a cave inside a cave -- the interesting bit being having a permanent world linked to two temporary worlds.  Having it difficult to reach/find would help somewhat with griefing, I hope.

Does anyone have an answer to this? Or any success with getting this working?

 

In addition, commands like "c_regenerateworld()" seem to affect all connected shards? I can see how that makes sense on one level, but I assume it means the above scenario isn't possible? If you wanted an endless world paired with a survival world that resets occasionally... seems like that would be difficult to handle.

Edited by Oarc
Adding another questions.

Share this post


Link to post
Share on other sites

@Oarc Right now the shard systems assumes that all connected servers form a "single world". This is intentional as that is the main design we want to support, and supporting other scenarios increases complexity. We do want to allow more varied clusters in the future but at the moment it's not what the system is built for, unfortunately.

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!


Register a new account

Sign in

Already have an account? Sign in here.


Sign In Now