Server-side Mod Management

[JohannesMP]

Disclaimer 

I'm assuming that Klei will remain fairly mum regarding multiplayer mods for now, as it's probably not the main focus of their development efforts at this time. I'm sure they would prefer that the core gameplay and network code is rock solid before adding mods to the mix. I totally get and support that.

 

However given how big a part user-made mods have played in the forming of this community, and the evolution of the singleplayer experience, it would be foolish to assume that mods will not be coming to Don't Starve Together.

 

With that out of the way...

 

 

Let's talk about Multiplayer Mod Management 

Obviously multiplayer mods that add anything other than the most basic of cosmetic changes will only work correctly if both the server as well as all connected clients have the required mod(s) installed. 

 

Many games that have multiplayer mods make this very difficult for players, by forcing each of them to manually download and install the mods, as well as having to enable or disable them when they want to switch servers. And you have to do this every time the server host decides to update to a new version of a mod. If you're lucky you'll find some sort of third party mod manager that can help you with this but even that is typically a pain to use, and does not integrate well with the server itself. Minecraft and its long-promised, yet still elusive Mod API is a prime example of this.

 

 

Ideally, I would like Don't Starve Together to be capable of doing the following:

 

1. The server contains a list of the mods required.

- One neat way that the Kerbal Multiplayer Mod handles this is by simply allowing the server admin to place all required mods into a folder, and then auto-generate the list with a simple console command.

2. When a client attempts to connect, it checks if it has all the required mods installed.

- The server would have the option of requiring a checksum of a given client mod, to ensure it is the correct version/untampered. This however would be optional, and probably only required on public, non-whitelisted servers.

- The option to both whitelist and blacklist mods would also be useful, for example some client-side texture mods might not require the server's approval.

3. If a mod is missing, the client automatically downloads and installs the correct version.

- How this could be implemented is debatable. The easiest, but probably also least secure/reliable would be for the server to simply keep zips of the mods, and send them as needed to any player that wants to connect.

- Alternatively, it could provide the client with a link that allows the user to download it themselves, preferably from within the client itself (with a simple 'download' button). For extra security this download could be limited to mods hosted on steam/klei's own system. links to other third-party mods could warn the player, but still allow them to use it if they are certain it's safe.

4. Once all the required mods are installed, the client can join the given server. If they disconnect, and connect to a different server the game should dynamically load all, if any mods required by that server. 

 

 

While the exact details of how mods are delivered to the client can be debated, the underlying idea that I hope the devs pick up on during development of Don't Starve Together, is this:

 

Managing multiplayer mods needs to be excessively user-friendly

 

If Klei manages to pull that off, it would truly set Don't Starve apart from all the other explore-y/survive-y/sandbox-y games with multiplayer out there, because I have yet to see a game that truly makes managing multiplayer mods a joy for both users and server admins. I still get hives thinking about how years ago I tried to walk some of my less computer-savy friends through installing various multiplayer mods for Minecraft.

[JohannesMP]

EDIT: Thanks SethR!

[SethR]

hmm, can't seem to find an 'edit' button... Ideally I would like to rename this topic to 'Server-side Mod Management', since the current title implies discussing ideas for actual server-side mods, not the actual management thereof.

 

Changed it for you

 

Thanks for the thoughtful post--this is definitely something we'll be looking at and looking for feedback on once the alpha is available.

[Sindex]

This was something I was also wondering about. How the expansion was going to work with mods. As well as those who have the RoG DLC and those who don't have it.

 

I agree a user friendly system should be top priority when the expansion gets to this point. Both Minecraft and Terraria have a user based community mod manger systems. The latter having a system where the mods download on join, but does not check if you just already have the mods installed. So it ends up installing the entire thing from scratch each time you join a modded server. Especially if you try to get other people to join you. It's doable but people can be rather jaded and lazy, where they won't even take the most basic steps. Torchlight 2 system is a bit better, but I think you still need to download the same mods the host has before you can join.  

[XCompWiz]

Sounds like a good thing.  Server's being "in charge" of the mods on the server is a good plan.

 

From the approaches I've seen, I'd have the mods as zip files themselves in a mod folder on the server side.  Then a basic checksum checking can be done on the client to ensure they have all of the mods and the right versions of the mods.  Any mismatches can be resolved by the server sending the zip file itself.

Basically this works like standard MMO data distribution.

 

The main difference this would require from how mods work currently is to keep mods in zip files rather than folders.  Not too difficult of a change, really.  I'd suggest handling both structures in general, though the server might require a zip file for the mods.

Of course the server could just send the folder, but the properties of being compressed and a single file are beneficial.  Checking against the folder is also a bit messier.

 

Where the client would keep these mods is up to implementation, of course.  Since there is already the fantastic system of enabled/disabled mods, I'd just stick them in the mods folder and enable them on a connection basis.

 

Note that I'd not bother with original source for downloads.  While this might seem odd, it's mostly to ensure that the mods are truly matching.  This woudl allow for server hosts to make minor edits to the mods (ideally with a name change) without having to make clients make the same changes.  It coudl also allow for server-specifc mods without a more full blow "publication."

Dev versions of mods, personal mods, private mods, and tiny mods would all benefit from such an approach.

[debugman18]

This could likely be done so long as the mods used are from the workshop. You could have the list, as described, then have each client check for those mods (provided they are installed through the workshop), and if they are not, then give the user the option to download them in order to connect.

[JohannesMP]

This could likely be done so long as the mods used are from the workshop. You could have the list, as described, then have each client check for those mods (provided they are installed through the workshop), and if they are not, then give the user the option to download them in order to connect.

 

While I agree that this could be one possible approach, as XCompWiz points out in his post, this would not account for dev builds, and private or altered mods.

 

Also what about the standalone PC builds of the game that don't use Steam? It would be better to have a general solution that works for all versions of the game.

[debugman18]

While I agree that this would be one solution, this would still be somewhat limiting and not ideal. As XCompWiz points out in his post, this would not account for dev builds, and private or altered mods.

 

Yes, his would be ideal, certainly. The only issue with his solution is the actual engine implementation.

 

We'll see what happens!

[Guest]

One idea I had on another thread with this same topic was to make mods go through a "multiplayer-certification" process, where you could submit your mods to Klei, they would go through the files, check them, give a playtest, and tell you what you need to fix.  When they're 100% satisfied with the mod, it gets a "Multiplayer Certified" status.

 

This would work similarly to how TF2 makes people have their mods "Gold Star Certified" before they can be implemented into the game.

[debugman18]

One idea I had on another thread with this same topic was to make mods go through a "multiplayer-certification" process, where you could submit your mods to Klei, they would go through the files, check them, give a playtest, and tell you what you need to fix.  When they're 100% satisfied with the mod, it gets a "Multiplayer Certified" status.

 

This would work similarly to how TF2 makes people have their mods "Gold Star Certified" before they can be implemented into the game.

 

That sounds like a lot of work that would steer their limited resources away from working on the mod api and the game itself.

 

There are a lot of mods for Don't Starve, the amount of time it would take to go through all of them... I shudder at the thought.

[Guest]

That sounds like a lot of work that would steer their limited resources away from working on the mod api and the game itself.

 

There are a lot of mods for Don't Starve, the amount of time it would take to go through all of them... I shudder at the thought.

 

This would be mostly for simple character mods.  I probably should've specified that.

[debugman18]

This would be mostly for simple character mods.  I probably should've specified that.

 

That doesn't change it too much, most of the mods out there are characters.

 

I think players should be able to pick and choose their own mods as they do now, with servers of course having a Requires/Banned mods list.

[Guest]

That doesn't change it too much, most of the mods out there are characters.

 

I think players should be able to pick and choose their own mods as they do now, with servers of course having a Requires/Banned mods list.

 

Well, either way, that was just my two cents.

 

I'm not opposed to a system like that, but I'd like to be able to use mods I've created on other servers, without having to make my own.

[debugman18]

Well, either way, that was just my two cents.

 

I'm not opposed to a system like that, but I'd like to be able to use mods I've created on other servers, without having to make my own.

 

Of course.

 

I think the best solution is a way to filter servers which use particular mods (or lack particular mods).

[Guest]

At the very least, we could contact the server admin and ask if they would add a certain mod to the list, I suppose.

 

Of course, I'll imagine every Tom, RICHARD, and Harry that makes a mod will want theirs added, and it'll be frustrating--especially on big and popular servers.

 

Edit: Yes, the shortened name for "Richard" was censored. XD

[debugman18]

At the very least, we could contact the server admin and ask if they would add a certain mod to the list, I suppose.

 

Of course, I'll imagine every Tom, RICHARD, and Harry that makes a mod will want theirs added, and it'll be frustrating--especially on big and popular servers.

 

Edit: Yes, the shortened name for "Richard" was censored. XD

 

I imagine so.

 

Of course, provided a mod is well-liked (and/or well-balanced), it would likely be on the server anyways.

[Guest]

I imagine so.

 

Of course, provided a mod is well-liked (and/or well-balanced), it would likely be on the server anyways.

 

I suppose that is true.

 

Alright, time to begin my quest to make Weston a household name!

[debugman18]

I suppose that is true.

 

Alright, time to begin my quest to make Weston a household name!

 

Well, I've already downloaded him, so you're off to a good start.

 

What I'm interested in, is whether or not a mod will need to be partially or totally rewritten to be multiplayer compatible, or if they have implemented the current mod-loading system into the multiplayer code.

[Guest]

Well, I've already downloaded him, so you're off to a good start.

 

What I'm interested in, is whether or not a mod will need to be partially or totally rewritten to be multiplayer compatible, or if they have implemented the current mod-loading system into the multiplayer code.

 

I never even thought about that.  I would assume that they could develop some sort of conversion method in the Multiplayer DLC itself if that was the case, or they could include some sort of way to include some tag that makes complatible with multiplayer.

 

I'm not really that experienced when it comes to things like this, so if I sound like a blithering idiot, just have pity on me and move on.

[JohannesMP]

Personally I don't think Klei needs to do any sort of moderation here. That is just adding a ton of friction to the process that would be unnecessary in almost every single case.

 

As long as a user can see the list of mods when they connect to a server, and are aware that they'll be playing on a modded server that should be enough.

 

Klei just needs to ensure that the mods are sandboxed correctly so that no personal/critical information can be accessed/modified by a mod.

[simplex]

The main difference this would require from how mods work currently is to keep mods in zip files rather than folders.  Not too difficult of a change, really.  I'd suggest handling both structures in general, though the server might require a zip file for the mods.

Of course the server could just send the folder, but the properties of being compressed and a single file are beneficial.  Checking against the folder is also a bit messier.

 

Keeping mods in zips would be extremely messy. It'd break the whole system being used for asset and script loading and overriding, as well as lose compatibility with any mod requiring read/write filesystem access to its folder. The first point could be circumvented by implementing virtual paths allowing use of zips as directories, with the form of access being transparently translated by the engine after some pattern matching, but it'd be messy.

Checksumming/transfering filesystem subtrees (i.e., "directories") will be quite straightforward if a proper library is used for this, such as rsync (whose underlying library, librsync, is licensed under the LGPL, and thus may be linked to a proprietary application). rsync is designed precisely for this, and transparently takes care of checksumming, on-the-fly compression (including delta compression, if using diff transfer), encoding to a network stream, etc. This is what Dropbox uses, for example.

 

What I'm interested in, is whether or not a mod will need to be partially or totally rewritten to be multiplayer compatible, or if they have implemented the current mod-loading system into the multiplayer code.

Mod code would likely not need to be changed, except for obvious details such as the use of GetPlayer. But this depends on how multiplayer will be handled on the Lua side: depending on how this will be done, additional checks may be required to avoid duplicating things (such as "spawn this entity at full moon") for each connected client.