DSTConfig - modoverrides.lua generation tool for servers

[denoflions 10]

Not sure if this sort of thing is allowed on these boards, but I've made a tool to parse mod's modinfo.lua and generate a modoverrides.lua with every option so you can tweak to your heart's content with less hassle.

It also labels each mod's name in the file next to its workshop id so you can easily ctrl-f around if you have a lot of mods.

Code is in the linked repo so folks can see there isn't any funny business.

https://github.com/INpureProjects/DSTConfig/releases

[Zillvr 1800]

Hey @denoflions welcome to the forums. I'm no luaj programmer but I personally think this is great. Thanks for sharing your work. With regards to whether this is the right subforum or not, I'm leaning towards you got it sort of right. This .jar file does involve Dedicated Servers and your work allows players to easily create modoverrides.lua files for their dedicated servers based on the current available mods they have in the mods folder for Don't Starve Dedicated Server folder.

This could probably be better off in the the Mods and Tools subforum though as this is technically a tool (as title implies) that players can use. You probably shouldn't worry about reposting this again since a moderator can easily come by and move this thread over there if they see it fit.

As a suggestion with the instructions in the github page, maybe add the lines "-right beside the jar file." to "After a few seconds a modoverrides.lua file should appear."

I hope more knowledgeable members of the community can come in and inspect your work as most members of the community who may utilize this more are probably not experienced with reading/inspecting code and may not be too inclined to jump in and use this right off the bat.

Anyways, thanks again for sharing this. Cheers.

[Asparagus 9138]

Adding to what @Zillvr said, I would like to mention @rezecib, @Mobbstar, @_Q_, to list a few modders/forumers who might take a gander that this.

[Mobbstar 14132]

Sorry, it what with the what for what? From what I understand, this generates a list of possible mod configurations for servers? I don't usually host a DST server, so if that is what it's for, I don't understand why @Asparagus summoned me.

[Asparagus 9138]

IDK... my eyes have been blurry lately... I just saw "mod" and automatically think of your badge O__O

ahh well...

[Muche 295]

The tool has some problems processing modinfo.lua.

[denoflions 10]

4 hours ago, Muche said:

The tool has some problems processing modinfo.lua.

Appears to be related to escaped characters within the name of the option. Not sure why in the world anyone would do that, but I'll see if I can work around that.

Released new version that can parse that. https://github.com/INpureProjects/DSTConfig/releases/tag/1.1

[Muche 295]

That list of option names is meant to be examples of problematic characters, it is in no way complete. A slightly more complete list could contain characters with codes 32-47, 58-64, 91-94, 96, 123-127. I agree that it is not usual (I have seen used only "!" and "."), but it has to be supported. Option names beginning with a number fall into this category as well. For example, scripts/dumper.lua:129 detects name category with ^[_%a][_%w]*$.

However, ignoring options with a space in the name is just plain wrong. Using them is not incorrect and it is used frequently.

It treats values of opt11 and opt12 as numbers even though they are strings in modinfo.

Value of opt15 has lesser precision. I haven't tested what precision DST's lua engine supports, so I'm not sure if this has an effect.

It doesn't support mod names with non-ASCII UTF8 characters. This doesn't create invalid modoverrides, it just doesn't help distinguish mods if someone has many mods with only e.g. asian characters.

[denoflions 10]

48 minutes ago, Muche said:

That list of option names is meant to be examples of problematic characters, it is in no way complete. A slightly more complete list could contain characters with codes 32-47, 58-64, 91-94, 96, 123-127. I agree that it is not usual (I have seen used only "!" and "."), but it has to be supported. Option names beginning with a number fall into this category as well. For example, scripts/dumper.lua:129 detects name category with ^[_%a][_%w]*$.

However, ignoring options with a space in the name is just plain wrong. Using them is not incorrect and it is used frequently.

It treats values of opt11 and opt12 as numbers even though they are strings in modinfo.

Value of opt15 has lesser precision. I haven't tested what precision DST's lua engine supports, so I'm not sure if this has an effect.

It doesn't support mod names with non-ASCII UTF8 characters. This doesn't create invalid modoverrides, it just doesn't help distinguish mods if someone has many mods with only e.g. asian characters.

Good points. The space ignoring hard rule was from before it supported generating options using the brackets and quotation marks method (i.e. ["option"] = "something", instead of option = "something") as when I wrote the original code I was following exactly to the specs listed on the wiki. I'll remove that and make it handle them appropriately.

Thinking about it... is there any real reason in lua to not just use the [""] method of setting table variables? That would simplify the code and probably remove an edge case or two.

The option being treated as a number instead of a string might be a weakness of lua itself. After I load the modinfo I fetch the variable in question and issue an isnumber check. I assume this returns true in this case because lua isn't strongly typed and "10" is technically a valid number in lua that can have arithmetic operations applied to it. I'll have to think on how to solve this.

For example:

-- This will print No!
if (10 == "10") then
    logger:lua("Yes!")
else
    logger:lua("No!")
end

local test = "10" + 1

-- This will print Yes!
if (test == 11) then
    logger:lua("Yes!")
else
    logger:lua("No!")
end

-- Gotta love lua.

Doing research now to see if I can handle this on the lua side somehow.

Losing precision on opt15 must be a LuaJ thing. I'm dumping the value I'm returned from the lua environment exactly as is.

I'm not sure about the non-ASCII UTF8 characters problem. Happen to have a file containing asian characters so I can experiment with it?

[Muche 295]

12 minutes ago, denoflions said:

Thinking about it... is there any real reason in lua to not just use the [""] method of setting table variables? That would simplify the code and probably remove an edge case or two.

I'm not sure about the non-ASCII UTF8 characters problem. Happen to have a file containing asian characters so I can experiment with it?

The ["name"] method of access is indeed superior to .name. Humans are just sometimes lazy typing those quotes and brackets.

The name in the provided modinfo is "Test mod áĽз". While it doesn't contain asian characters, it does contain some non-ASCII UTF8 characters. Currently it produces "Test mod ?" on my machine.

[denoflions 10]

3 minutes ago, Muche said:

The ["name"] method of access is indeed superior to .name. Humans are just sometimes lazy typing those quotes and brackets.

The name in the provided modinfo is "Test mod áĽз". While it doesn't contain asian characters, it does contain some non-ASCII UTF8 characters. Currently it produces "Test mod ?" on my machine.

On my machine I see it producing the correct result for that modinfo's name. Interesting. I'll see about finding a way to force proper encoding just to be on the safe side.

[Muche 295]

java -Dfile.encoding=UTF-8 -jar DSTConfig-v1.1.jar fixed that for me.

 

[denoflions 10]

1 hour ago, Muche said:

java -Dfile.encoding=UTF-8 -jar DSTConfig-v1.1.jar fixed that for me.

 

Give 1.2 a shot. The output isn't formatted anymore for now, but I do believe the options will have their type respected now. No more "10" getting ninja-converted to 10.

[Muche 295]

1.2 crashes if it finds a folder with no modinfo.lua inside it, or modinfo.lua doesn't assign configuration_options.

[denoflions 10]

10 hours ago, Muche said:

1.2 crashes if it finds a folder with no modinfo.lua inside it, or modinfo.lua doesn't assign configuration_options.

The first one probably, but the second shouldn't happen. If the options table is nil it should just generate a blank one and move on. Maybe its that something expected within the table doesn't exist. Welp, time to add more nil checks.

edit: Released 1.3 that should hopefully fix any nil crashes.

[Muche 295]

1.3 still crashes when there is no modmain.lua with

Exception in thread "main" org.luaj.vm2.LuaError: script:17 vm error: groovy.lang.GroovyRuntimeException: Ambiguous method overloading for method info.inpureprojects.DSTConfig.LuaManager#load.
Cannot resolve which method to invoke for [null] due to overlapping prototypes between:
	[class java.io.File]
	[class java.io.InputStream]
	[class java.lang.String]
	at org.luaj.vm2.lib.jse.JavaMethod.invokeMethod(Unknown Source)
	at org.luaj.vm2.lib.jse.JavaMethod$Overload.invokeBestMethod(Unknown Source)
	at org.luaj.vm2.lib.jse.JavaMethod$Overload.call(Unknown Source)
	at org.luaj.vm2.LuaClosure.execute(Unknown Source)
	at org.luaj.vm2.LuaClosure.onInvoke(Unknown Source)
	at org.luaj.vm2.LuaClosure.invoke(Unknown Source)
	at org.luaj.vm2.LuaValue.invoke(Unknown Source)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
	at java.lang.reflect.Method.invoke(Unknown Source)
	at org.codehaus.groovy.runtime.callsite.PojoMetaMethodSite$PojoCachedMethodSite.invoke(PojoMetaMethodSite.java:192)
	at org.codehaus.groovy.runtime.callsite.PojoMetaMethodSite.call(PojoMetaMethodSite.java:56)
	at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48)
	at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:113)
	at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:125)
	at info.inpureprojects.DSTConfig.LuaManager.invoke(LuaManager.groovy:25)
	at info.inpureprojects.DSTConfig.LuaManager$invoke$2.call(Unknown Source)
	at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48)
	at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:113)
	at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:133)
	at info.inpureprojects.DSTConfig.Main.<init>(Main.groovy:37)
	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
	at sun.reflect.NativeConstructorAccessorImpl.newInstance(Unknown Source)
	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(Unknown Source)
	at java.lang.reflect.Constructor.newInstance(Unknown Source)
	at org.codehaus.groovy.reflection.CachedConstructor.invoke(CachedConstructor.java:80)
	at org.codehaus.groovy.runtime.callsite.ConstructorSite$ConstructorSiteNoUnwrapNoCoerce.callConstructor(ConstructorSite.java:105)
	at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCallConstructor(CallSiteArray.java:60)
	at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callConstructor(AbstractCallSite.java:235)
	at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callConstructor(AbstractCallSite.java:247)
	at info.inpureprojects.DSTConfig.Main.main(Main.groovy:16)
Caused by: groovy.lang.GroovyRuntimeException: Ambiguous method overloading for method info.inpureprojects.DSTConfig.LuaManager#load.
Cannot resolve which method to invoke for [null] due to overlapping prototypes between:
	[class java.io.File]
	[class java.io.InputStream]
	[class java.lang.String]
	at groovy.lang.MetaClassImpl.chooseMostSpecificParams(MetaClassImpl.java:3241)
	at groovy.lang.MetaClassImpl.chooseMethodInternal(MetaClassImpl.java:3194)
	at groovy.lang.MetaClassImpl.chooseMethod(MetaClassImpl.java:3137)
	at groovy.lang.MetaClassImpl.getMethodWithCachingInternal(MetaClassImpl.java:1328)
	at groovy.lang.MetaClassImpl.createPogoCallCurrentSite(MetaClassImpl.java:3418)
	at org.codehaus.groovy.runtime.callsite.CallSiteArray.createCallCurrentSite(CallSiteArray.java:111)
	at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCallCurrent(CallSiteArray.java:52)
	at info.inpureprojects.DSTConfig.LuaManager$load$1.callCurrent(Unknown Source)
	at info.inpureprojects.DSTConfig.LuaManager.load(LuaManager.groovy:33)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
	at java.lang.reflect.Method.invoke(Unknown Source)
	... 32 more

 

[denoflions 10]

9 hours ago, Muche said:

1.3 still crashes when there is no modmain.lua with

Exception in thread "main" org.luaj.vm2.LuaError: script:17 vm error: groovy.lang.GroovyRuntimeException: Ambiguous method overloading for method info.inpureprojects.DSTConfig.LuaManager#load.
Cannot resolve which method to invoke for [null] due to overlapping prototypes between:
	[class java.io.File]
	[class java.io.InputStream]
	[class java.lang.String]
	at org.luaj.vm2.lib.jse.JavaMethod.invokeMethod(Unknown Source)
	at org.luaj.vm2.lib.jse.JavaMethod$Overload.invokeBestMethod(Unknown Source)
	at org.luaj.vm2.lib.jse.JavaMethod$Overload.call(Unknown Source)
	at org.luaj.vm2.LuaClosure.execute(Unknown Source)
	at org.luaj.vm2.LuaClosure.onInvoke(Unknown Source)
	at org.luaj.vm2.LuaClosure.invoke(Unknown Source)
	at org.luaj.vm2.LuaValue.invoke(Unknown Source)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
	at java.lang.reflect.Method.invoke(Unknown Source)
	at org.codehaus.groovy.runtime.callsite.PojoMetaMethodSite$PojoCachedMethodSite.invoke(PojoMetaMethodSite.java:192)
	at org.codehaus.groovy.runtime.callsite.PojoMetaMethodSite.call(PojoMetaMethodSite.java:56)
	at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48)
	at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:113)
	at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:125)
	at info.inpureprojects.DSTConfig.LuaManager.invoke(LuaManager.groovy:25)
	at info.inpureprojects.DSTConfig.LuaManager$invoke$2.call(Unknown Source)
	at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48)
	at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:113)
	at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:133)
	at info.inpureprojects.DSTConfig.Main.<init>(Main.groovy:37)
	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
	at sun.reflect.NativeConstructorAccessorImpl.newInstance(Unknown Source)
	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(Unknown Source)
	at java.lang.reflect.Constructor.newInstance(Unknown Source)
	at org.codehaus.groovy.reflection.CachedConstructor.invoke(CachedConstructor.java:80)
	at org.codehaus.groovy.runtime.callsite.ConstructorSite$ConstructorSiteNoUnwrapNoCoerce.callConstructor(ConstructorSite.java:105)
	at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCallConstructor(CallSiteArray.java:60)
	at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callConstructor(AbstractCallSite.java:235)
	at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callConstructor(AbstractCallSite.java:247)
	at info.inpureprojects.DSTConfig.Main.main(Main.groovy:16)
Caused by: groovy.lang.GroovyRuntimeException: Ambiguous method overloading for method info.inpureprojects.DSTConfig.LuaManager#load.
Cannot resolve which method to invoke for [null] due to overlapping prototypes between:
	[class java.io.File]
	[class java.io.InputStream]
	[class java.lang.String]
	at groovy.lang.MetaClassImpl.chooseMostSpecificParams(MetaClassImpl.java:3241)
	at groovy.lang.MetaClassImpl.chooseMethodInternal(MetaClassImpl.java:3194)
	at groovy.lang.MetaClassImpl.chooseMethod(MetaClassImpl.java:3137)
	at groovy.lang.MetaClassImpl.getMethodWithCachingInternal(MetaClassImpl.java:1328)
	at groovy.lang.MetaClassImpl.createPogoCallCurrentSite(MetaClassImpl.java:3418)
	at org.codehaus.groovy.runtime.callsite.CallSiteArray.createCallCurrentSite(CallSiteArray.java:111)
	at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCallCurrent(CallSiteArray.java:52)
	at info.inpureprojects.DSTConfig.LuaManager$load$1.callCurrent(Unknown Source)
	at info.inpureprojects.DSTConfig.LuaManager.load(LuaManager.groovy:33)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
	at java.lang.reflect.Method.invoke(Unknown Source)
	... 32 more

 

Ahh, I see what is happening. It can't decide which load method to use since there is a null. Making the method names unique should fix that up I think.

1.4 should have that fixed.

[Muche 295]

If modmain doesn't assign name or configuration_options has unexpected value (while still being valid lua value) will cause it to crash. While those values make it invalid mod, it would be nice if it just printed a message and moved on instead of crashing.

Feature request: It would be nice if the tool did not simply overwrite existing modoverrides but updated it, i.e. read values from it and add new options and mods, but left existing values as they are set.

[denoflions 10]

7 hours ago, Muche said:

If modmain doesn't assign name or configuration_options has unexpected value (while still being valid lua value) will cause it to crash. While those values make it invalid mod, it would be nice if it just printed a message and moved on instead of crashing.

Feature request: It would be nice if the tool did not simply overwrite existing modoverrides but updated it, i.e. read values from it and add new options and mods, but left existing values as they are set.

Gotcha. I'll work on both of these tomorrow.

[denoflions 10]

1.5 is out.

• Each modinfo is checked for a name variable and if defined makes sure the configuration_options is actually a table.
• If an existing modoverrides.lua is present it will read this first and keep any matching values on a per-mod basis (so if two mods use the same variable names it won't get confused).

[NoJoke 4]

Greetings,

As a network engineer (involves understanding most programming languages), it took me some time to review the coding and I personally didn't see anything malicious for those who are having doubts. However, just like any other tool created by the community use at your own risk.

@denoflions Do you plan on cleaning up the code a bit? It's a little clustered which can cause problems in the program itself along with conflicting errors which does explain some of the issues listed above. I'm not saying it's bad, infact I'm impressed that you whipped this up in a short amount of time, good job! 

Keep up the good work, thanks for releasing the source aswell to the public so they can see for themselves that it's clean. You could turn this into what's called an "A.I.O" (All In One) release which people can edit all the major files as in admin lists, server configs, and so forth if you'd like to get advanced. This would make preconfigured dedicated servers much easier for the new users looking to host a server that's preconfigured or just in general. 

[denoflions 10]

20 minutes ago, NoJoke said:

Greetings,

As a network engineer (involves understanding most programming languages), it took me some time to review the coding and I personally didn't see anything malicious for those who are having doubts. However, just like any other tool created by the community use at your own risk.

@denoflions Do you plan on cleaning up the code a bit? It's a little clustered which can cause problems in the program itself along with conflicting errors which does explain some of the issues listed above. I'm not saying it's bad, infact I'm impressed that you whipped this up in a short amount of time, good job! 

Keep up the good work, thanks for releasing the source aswell to the public so they can see for themselves that it's clean. You could turn this into what's called an "A.I.O" (All In One) release which people can edit all the major files as in admin lists, server configs, and so forth if you'd like to get advanced. This would make preconfigured dedicated servers much easier for the new users looking to host a server that's preconfigured or just in general. 

The Lua is a bit of a mess, yes. The distinct lack of a 'continue' in lua made the main loop into a train wreck. It could probably be rewritten cleaner. I might shunt more of the processing back into the groovy side and just do the option parsing on the lua side to make it less nested.

I'm actually working on a web app that manages dedicated servers. It will take the current process of setting up a server and change it into 'download jar, run it, enjoy'. It will have a browser based interface for managing your mods, admins, bans, world backups, and whatever else is necessary. DSTConfig will remain a separate thing, but my new app will take advantage of it and probably have it bundled with it.

[NoJoke 4]

11 minutes ago, denoflions said:

The Lua is a bit of a mess, yes. The distinct lack of a 'continue' in lua made the main loop into a train wreck. It could probably be rewritten cleaner. I might shunt more of the processing back into the groovy side and just do the option parsing on the lua side to make it less nested.

I'm actually working on a web app that manages dedicated servers. It will take the current process of setting up a server and change it into 'download jar, run it, enjoy'. It will have a browser based interface for managing your mods, admins, bans, world backups, and whatever else is necessary. DSTConfig will remain a separate thing, but my new app will take advantage of it and probably have it bundled with it.

Your web app idea is great! Especially for ones that get confused and/or frustrated at setting one up or even resetting the server completely due to server-side complications for preconfigs. 

I'd even pay for such a thing if it's ease of access for server administration. Problem is some preconfig hosts have the same thing available, so my question to you is what would be the difference between a preconfig dedi's control panel and the web app you'd like to design? Maybe I'm reading this wrong, but it does sound like you want to start up your own reselling service for preconfigured dedicated servers. Correct me if I'm wrong, just very interested in your idea is all.

[denoflions 10]

13 minutes ago, NoJoke said:

Your web app idea is great! Especially for ones that get confused and/or frustrated at setting one up or even resetting the server completely due to server-side complications for preconfigs. 

I'd even pay for such a thing if it's ease of access for server administration. Problem is some preconfig hosts have the same thing available, so my question to you is what would be the difference between a preconfig dedi's control panel and the web app you'd like to design? Maybe I'm reading this wrong, but it does sound like you want to start up your own reselling service for preconfigured dedicated servers. Correct me if I'm wrong, just very interested in your idea is all.

I'm sure something like it already exists, but its usually part of a server rental. This is an open source manager, so you can run it on your own pc, home server, vps, whatever. It exists currently as a Java webapp, so anyone with an existing Tomcat server can deploy it and go. It will have an embedded Jetty version for those who don't do Tomcat that is just an executable jar.

I'm not intending to sell anything. I'm just an open source guy making software for myself and sharing it with those who might find it useful themselves.

[Muche 295]

When a mod is disabled in modoverrides.lua, the updated modoverrides.lua will have it enabled.

Regarding indentation, this looks like a simple solution: http://pastebin.com/CJcm95tb.

@JoeW, for some reason, the patch triggered forum security solution (CloudFlare Ray ID: 278a9346e0bd0589), even though it was in the code block.