Jump to content

World Gen Data and Mod Changes


Recommended Posts

  • Developer

Hey modders!

 

There has been some refactoring and cleanup of the level data and worldgen code. Unfortunately, this broke several worldgen-related mods. However, in the process I've also tried improving the way worldgen mods are done.

 

Here's the upshot: If you have a mod that runs modworldgenmain.lua, please update it according to the instructions below.

 

 

1. Level data format. If your mod adds or modifies levels, read this!

 

Spoiler

 

a. The version of the data is now 2. If you look at our levels, they have version=2. This will help with compatibility in the future. As you update your levels using these instructions, don't forget to add the new version number!


b. Levels have been broken into two parts, a "location" which specifies the basic properties of a world, for consistency, and a "level", which specifies overrides and is basically equivalent to a preset. So for example, the game has two "locations" right now, "forest" and "cave".
 

All levels (added with AddLevel()) must now specify a location. Please see our level definitions in scripts/map/levels/forest.lua for examples.


c. The overrides table is now key-value pairs, instead of tuples. So what used to be this:


overrides = {
   {"carrots", "more"},
   {"bearger", "never"},
},

...is now this:


overrides = {
   carrots = "more",
   bearger = "never",
},

d. It hopefully won't affect you, but map/levels/survival.lua was renamed to map/levels/forest.lua as well. For all of the above changes, look at map/levels/forest.lua and map/locations.lua to see these changes in action.

 

 

2. New Mod API methods, and some changes. If your mod modifies any world data -- levels, task sets, rooms, etc, read this!

Spoiler

 

a. Modworldgenmain.lua now gets run from the server creation screen. This means Levels and Task Sets get loaded in the World Customization tab! This means that if you add a cool new configuration using AddLevel and/or AddTaskSet, you can select it from the preset list on the World tab in the client! For dedicated servers, levels can still be selected by using the "preset" entry in worldgenoverride.lua (but some notes on that below).

b. AddTaskSetPreInit(tasksetname, fn) / AddTaskSetPreInitAny(fn) -- this should have been there all along, it was an oversight. Now you can modify existing task sets the same way as other worldgen elements.

c. AddLocation(locationdata) -- Basically the same as AddLevel, but creates location data instead. In the short term you probably won't use this as AddLevel should do everything you want, but we've put this in now so we don't forget later when it becomes more important. :)

d. AddStartLocation(startlocationname, data) -- This allows new start setpieces to be chosen for a level (with the start_location override) and selected from the frontend. An example of this is the Darkness start which has a firepit next to the multiplayer portal.

e. Important: I recommend you don't use require() in modmain.lua or modworldgenmain.lua for running .lua files that are part of your mod. Use modimport() instead. If you use modimport(), it lets the imported files run in the mod environment so that the API is available. Basically, if you modimport() a file from modmain, it means you can use all the same API and environment that modmain has.

This helps us ensure that your mod is cleanly loaded and unloaded, and makes it easier for us to put in backwards compatibility. This is especially important due to some changes in this update. That means that this:


   require("map/levels/mylevels")

...should become this:


   modimport("scripts/map/levels/mylevels")

f. terrain.rooms should no longer be accessed by mods. Please use AddRoomPreInit instead if you want to modify the contents of an existing room. So this:


   local terrain = require("map/terrain")
   terrain.rooms.Marsh.contents.distributeprefabs.berrybush = 0.1

...becomes this:


   AddRoomPreInit("Marsh", function(room) room.contents.distributeprefabs.berrybush = 0.1 end)

g. Custom game modes can now specify a level_type which controls what presets are valid for that game mod. This helps ensure that game modes are run on valid levels, while being expandable by other mods. So put this in your modsettings.ini:


game_modes =
{
    {
        name = "pigfeed",
        label = "PigFeed",
        description = "Feed the pig king the most in each round to get points!",
        settings =
        {
            ghost_sanity_drain = false,
            portal_rez = true,
            level_type = "LEVELTYPE_PIGFEED", -- NEW!
        }
    }
}

and then in your modworldgenmain you define the level for that level type


AddLevel( "LEVELTYPE_PIGFEED", {
    id = "PIGFEED",
    name = "Pigfeed preset",
    desc = "Pigfeed preset description!",
    location = "forest",
    version = 2,
    overrides={
        flint = "often",
        grass = "often",
    },
})

This will cause PIGFEED to be the only available preset when the PigFeed game mode is selected.

 

3. worldgenoverride.lua changes. If you use worldgenoverride.lua, read this!

Spoiler

 

The changes should be backwards compatible, but be advised the format for this file has changed a bit.

a. "actualpreset" is no more, there is now only "preset", and it should always behave as expected. (If you didn't need this, don't worry about it.)

b. The various override categories have been merged into a single "overrides" table. So what was once this:


   unprepared = {
       berrybush = "always",
   },
   misc = {
       autumn = "noseason",
   },

...is now this:


   overrides = {
       berrybush = "always",
       autumn = "noseason",
   },

As well, we've added some extra logging to help you determine if the contents of your worldgenoverride.lua are correct.

 

 

4. EnableModError. If you are serious about your modding, read this!

Spoiler

 

a. A new setting has been added to modsettings.lua. Now if you have the entry:


EnableModError()

It will cause the game to more strict about mod data and access. So far this only affects level loading as that's what I was working on, but over time we'll extend this to more areas.

The idea is to make it easier for you to upgrade your code, and to use correct methods of accessing and modifying the game data. If this is turned off, the errors will print to the log, but if it's turned on then the game will crash, giving you a full call stack.

For normal users it will be turned off, so if you do make a mistake, it shouldn't affect them where possible!

b. You may wish to use this in your own code too, to help with development. There are two methods you can use, modassert() and moderror().

modassert(test, message) will print the message if the test fails, or crash the game if EnableModError() is set. This is the same as a normal assert if you have used one before, except that it will not crash for most users. Example:


modassert(some_var ~= nil, "some_var must not be nil!")

moderror(message) will either print the message, or crash the game if EnableModError() is set.

In both these cases, because execution can continue, you'll probably want to return afterwards, or perform some kind of default action. Basically treat it like a print(). If the game state is unrecoverable, then use a regular assert() or error().

 

 

 

 


So now that all the instructions are out of the way, here's what you can do to help:

Please give this a try!

Try running your mods unmodified and let me know what kinds of errors and problems you run into. Not everything can be made backwards compatible, but I will try to make it backwards compatible where I can.

Try updating your mods to use the new APIs. Do they work as you expect? Any issues?

Also, if you want any help upgrading your mods before this goes live, please ask here! I upgraded numerous mods in the process of testing these changes and I've gotten pretty good at it now. :)

Thanks so much for your feedback!

  • Like 17
  • Thanks 1
Link to comment
Share on other sites

With Turfed! mod enabled, I got the crash:

Spoiler

[00:01:56]: Attempting to send resume request
[00:01:56]: Resuming user: session/13E46A72FF4D3397/KU_fakeuser_/0000000127
[00:01:57]: [string "scripts/entityscript.lua"]:492: calling 'HasTag' on bad self (string expected, got nil)
LUA ERROR stack traceback:
=[C]:-1 in (method) HasTag (C) <-1--1>
scripts/entityscript.lua:492 in (method) HasTag (Lua) <491-493>
   self (valid:true) =
      DynamicShadow = DynamicShadow (6BB1F6A0)
      inlimbo = false
      GetMoistureRateScale = function - scripts/prefabs/player_common.lua:96
      ghostenabled = true
      EnableMovementPrediction = function - scripts/prefabs/player_common.lua:447
      player_classified = 108079 - player_classified (valid:true)
      playercolour = table: 17B882C0
      jointask = PERIODIC 108078: 0.000000
      IsOverheating = function - scripts/prefabs/player_common.lua:66
      Light = Light (6BB1F6E0)
      Network = Network (6BB1F7A0)
      OnRemoveEntity = function - scripts/prefabs/player_common.lua:578
      GetMoisture = function - scripts/prefabs/player_common.lua:76
      pendingtasks = table: 1533DBD8
      LightWatcher = LightWatcher (6BB1F700)
      spawntime = 0.033333335071802
      SetGhostMode = function - scripts/prefabs/player_common.lua:486
      Transform = Transform (6BB1F5E0)
      actionreplica = table: 1533D548
      event_listening = table: 1533E218
      actioncomponents = table: 1533D890
      lower_components_shadow = table: 1533D5C0
      GetMaxMoisture = function - scripts/prefabs/player_common.lua:86
      modactioncomponents = table: 6BC33A10
      entity = Entity (713AAD30)
      userid = 
      CanUseTouchStone = function - scripts/prefabs/player_common.lua:36
      updatecomponents = table: 6BD40EE8
      event_listeners = table: 1533E0B0
      AttachClassified = function - scripts/prefabs/player_common.lua:567
      persists = false
      IsActionsVisible = function - scripts/prefabs/player_common.lua:1175
      MiniMapEntity = MiniMapEntity (6BB1F6C0)
      IsFreezing = function - scripts/prefabs/player_common.lua:56
      GUID = 108078
      DetachClassified = function - scripts/prefabs/player_common.lua:573
      ShakeCamera = function - scripts/prefabs/player_common.lua:1229
      AnimState = AnimState (6BB1F680)
      Physics = Physics (6BB1FCA0)
      replica = table: 1533D8B8
      SoundEmitter = SoundEmitter (6BB1F620)
      components = table: 1533D7C8
      GetTemperature = function - scripts/prefabs/player_common.lua:46
   tag = nil
scripts/components/builder.lua:52 in (field) _ctor (Lua) <26-56>
   self =
      bonus_tech_level = 0
      buffered_builds = table: 6BC2A960
      inst = 108078 -  (valid:true)
      exclude_tags = table: 6BC2AA00
      recipes = table: 6BC2A528
      _ = table: 6BC2A898
      accessible_tech_trees = table: 6BC2ACA8
   inst = 108078 -  (valid:true)
   k = Turf
   v = table: 165D6648
scripts/class.lua:181 in (local) cmp (Lua) <171-184>
   class_tbl = table: 6BC29D80
   arg = nil
   obj = table: 6BC2A8E8
scripts/entityscript.lua:520 in (method) AddComponent (Lua) <509-531>
   self (valid:true) =
      DynamicShadow = DynamicShadow (6BB1F6A0)
      inlimbo = false
      GetMoistureRateScale = function - scripts/prefabs/player_common.lua:96
      ghostenabled = true
      EnableMovementPrediction = function - scripts/prefabs/player_common.lua:447
      player_classified = 108079 - player_classified (valid:true)
      playercolour = table: 17B882C0
      jointask = PERIODIC 108078: 0.000000
      IsOverheating = function - scripts/prefabs/player_common.lua:66
      Light = Light (6BB1F6E0)
      Network = Network (6BB1F7A0)
      OnRemoveEntity = function - scripts/prefabs/player_common.lua:578
      GetMoisture = function - scripts/prefabs/player_common.lua:76
      pendingtasks = table: 1533DBD8
      LightWatcher = LightWatcher (6BB1F700)
      spawntime = 0.033333335071802
      SetGhostMode = function - scripts/prefabs/player_common.lua:486
      Transform = Transform (6BB1F5E0)
      actionreplica = table: 1533D548
      event_listening = table: 1533E218
      actioncomponents = table: 1533D890
      lower_components_shadow = table: 1533D5C0
      GetMaxMoisture = function - scripts/prefabs/pla
[00:01:57]: [string "scripts/entityscript.lua"]:492: calling 'HasTag' on bad self (string expected, got nil)
LUA ERROR stack traceback:
    =[C]:-1 in (method) HasTag (C) <-1--1>
    scripts/entityscript.lua:492 in (method) HasTag (Lua) <491-493>
    scripts/components/builder.lua:52 in (field) _ctor (Lua) <26-56>
    scripts/class.lua:181 in (local) cmp (Lua) <171-184>
    scripts/entityscript.lua:520 in (method) AddComponent (Lua) <509-531>
    scripts/prefabs/player_common.lua:1662 in (field) fn (Lua) <1464-1773>
    scripts/mainfunctions.lua:148 in () ? (Lua) <137-179>
    =[C]:-1 in (method) SendResumeRequestToServer (C) <-1--1>
    scripts/prefabs/world_network.lua:26 in (field) fn (Lua) <19-30>
    scripts/scheduler.lua:194 in (method) OnTick (Lua) <168-225>
    scripts/scheduler.lua:406 in (global) RunScheduler (Lua) <404-412>
    scripts/update.lua:166 in () ? (Lua) <150-223>
	
[00:01:57]: Failed to instantiate prefab in: OnResumeRequestLoadComplete
[00:01:57]: ReceiveResumeNotification

 

due to components/builder's

    self.exclude_tags = { "INLIMBO" }
    for k, v in pairs(CUSTOM_RECIPETABS) do
        if not inst:HasTag(v.owner_tag) then
            table.insert(self.exclude_tags, v.owner_tag)
        end
    end

It seems this expects all custom recipetabs to have a non-nil owner_tag.

 

Turfed mod's tile adder (that adds them into GLOBAL.GROUND) also assumes that it is running for the first time. FrontEndMod loader can run Turfed's modworldgenmain.lua multiple times if the mod is enabled in more save slots, with changes to GLOBAL.GROUND intact, so that assumption leads to an assert. That could be solved by lessening the assert into if+print though.
Just making sure this assumption is no longer valid.

 

Link to comment
Share on other sites

  • Developer

Cool, thanks for pointing these things out (I'll look at that ModDebugPrint suggestion as well!)

Yes, I'm not super satisfied with loading modworldgenmain in the frontend and that may change a bit in the future, but for now I'd assume that has to load multiple times cleanly. Part of the answer would probably be to put in an AddTile API that takes care of the business that tile_adder is doing...

Link to comment
Share on other sites

Cool. I'll take a closer look into this as soon as I can.

By the way, if anyone prefers testing via dedicated servers/steamcmd, this is done by appending '-beta worlddata -betapassword worlddatatesting'  to the '+app_update 343050' portion of steamcmd's invocation line. I also incorporated easy access to public/private betas in my Linux wrapper script for dedicated servers.

Link to comment
Share on other sites

@Ipsquiggle

On each topic, saving #2 for last because it's a minor pet peeve of mine.

 

1. Level data format.

This is great. In order to get U&A/wicker compatible with both DS and DST, what I had been doing is adopt a custom level data representation format, a superset of both the DS and DST formats, which then got normalized into a temporary internal format (which already required a location to be set), which finally got turned into the respective game's level data format. And this temporary internal format was... exactly DST's version 2 format. So all this change does is save me some work in the DST case.
 

3. worldgenoverride.lua changes.

These are nice, making worldgenoverride's format more consistent with level data representation. Not much else to be said.

 

4. EnableModError.

I love the intention behind this, but I have some objections to its implementation. Firstly, moderror and modassert should behave like their stdlib counterparts: error and assert.

Namely, moderror should take a second optional parameter, level, which is passed to the stdlib error function to indicate the stack level where the error occurred (the current implementation will always point to modutil.lua:79 as the source of the error). If this value is nil, then it should default to 1. Then, only if it is non-zero, one should be added to it (since the moderror function itself adds a stack level to the call stack). This is the code for what I suggest:

function moderror(message, level)
    local modname = (global('env') and env.modname) or ModManager.currentlyloadingmod or "unknown mod"
    local message = string.format("MOD ERROR: %s: %s", ModInfoname(modname), tostring(message))
    if KnownModIndex:IsModErrorEnabled() then
        level = level or 1
        if level ~= 0 then
            level = level + 1
        end
        return error(message, level)
    else
        print(message)
        -- I'm just making it explicit that we should return nothing in this case.
        -- A true 'nothing', such as below, makes select("#", moderror(stuff)) return zero.
        return
    end
end

In similar spirit, modassert's second parameter should default to some string if nil (not necessarily "assertion failed!", though that's a fine pick). But more importantly, modassert should return its first argument if an error was not raised. A very common use of assert is in 'var = assert(this_must_never_be_nil_or_false())'. This is the code for what I suggest:

function modassert(test, message)
    if not test then
        return moderror(message or "assertion failed!", 2)
    else
        return test
    end
end

 

Secondly, there should be a modinfo flag that forces mod error enabling. Defensive programming is a perfectly valid (and personal favorite) style of programming, with strict errors making it much easier to detect and fix bugs from user reports, whereas stealth log prints lead to hard to find bugs and users confused about why something isn't working correctly. If you'd like to add a modsettings option which force disables mod errors, that's fine, but the mod creator should have the say on whether the default will be strict or permissive mode.

 

2. New Mod API methods, and some changes.

This is not the first time we've had this discussion, but I don't think it's feasible to suggest modimport can replace require. Using require expresses a functional dependency tree, with files being loaded at most once and having a return value. On the other hand, modimport does no caching and discards the return values of the loaded chunk, forcing the use of stateful constructs (such as the setting of variables in the mod environment to track duplicate loading or "returning" values).

I have, for years now, used custom require-like functionality which loads files like require but in a specified environment, which while not the mod environment itself, imports most of it, with specific care to import what's needed to play nice with the game's mod error handling system (that is, it includes 'env', 'modname', 'MODROOT', 'modinfo', etc.). If the new mod error handling system requires anything else from the mod environment, let me know and it'll be there, but I won't use modimport when it's a dofile that discards return values.

Edited by simplex
improved moderror's return if error checking is disabled
Link to comment
Share on other sites

@Ipsquiggle

Could you add a preinit to world savedata? Run over the savedata parameter of PopulateWorld (gamelogic.lua) before it gets passed into PopulateWorld? I used to do this by patching the PopulateWorld function itself (which used to be global), but now that it's local I haven't found a good way to do it (it's hard to get stable references to it as an upvalue).

Link to comment
Share on other sites

  • Developer

@simplex Thanks for the great feedback! Our stacktrace doesn't really respect the level param of error so I didn't bother, but I can put that in easily enough. I also didn't know that assert returned a value... learn something new every day. :)

Specifically concerning modimport: I agree that the require/modimport thing is a little bogus and at some point I'd really like to put in something more sensible and easier for everyone involved. However, my observation of most mods is that when they are require'ing another script in their mod from modmain, they're usually looking for textual inclusion (i.e. "and also do that") rather than setting up a module. In addition, I have noticed a number of bad patterns that arise from people using require instead of modimport when they don't understand the difference.

For skilled programmers who do understand the difference, I leave it to you to choose how you want to fling your code. :) We still do provide GLOBAL, after all. But for people who haven't yet managed to understand the difference, I suggest that modimport is what they should use, most of the time. I'll update the top post to clarify what I mean there a little bit.

  • Like 1
Link to comment
Share on other sites

  • Developer

We just pushed a new version to the worlddata branch, version 172739.

Notable changes include lots of crash fixes for starting worlds with no save data, old save data, old dedicated servers, etc., as well as better handling of Game Modes throughout all aspects of the game. Also, the modassert and initprint fixes have been put in, thanks!

  • Like 1
Link to comment
Share on other sites

7 hours ago, Ipsquiggle said:

@simplex Thanks for the great feedback! Our stacktrace doesn't really respect the level param of error so I didn't bother, but I can put that in easily enough. I also didn't know that assert returned a value... learn something new every day. :)

Lua's default debug.traceback() also doesn't use the stack level number passed to error (though if called on its own it does accept its own starting level parameter). This level number is only used for the error header (i.e., the first, non-indented line), which is generated by error itself before passing it as the first argument to debug.traceback (or whichever was configured as the error handler). The game does respect error's second argument in the same way "vanilla Lua 5.1" does.

For example, running this program under the standard Lua 5.1 interpreter

function f()
    error("test", 2)
end

function g()
    f()
end

g()

produces the following as output:

lua5.1: errortest.lua:6: test
stack traceback:
	[C]: in function 'error'
	errortest.lua:2: in function 'f'
	errortest.lua:6: in function 'g'
	errortest.lua:9: in main chunk
	[C]: ?

 

7 hours ago, Ipsquiggle said:

Specifically concerning modimport: I agree that the require/modimport thing is a little bogus and at some point I'd really like to put in something more sensible and easier for everyone involved. However, my observation of most mods is that when they are require'ing another script in their mod from modmain, they're usually looking for textual inclusion (i.e. "and also do that") rather than setting up a module. In addition, I have noticed a number of bad patterns that arise from people using require instead of modimport when they don't understand the difference.

For skilled programmers who do understand the difference, I leave it to you to choose how you want to fling your code. :) We still do provide GLOBAL, after all. But for people who haven't yet managed to understand the difference, I suggest that modimport is what they should use, most of the time. I'll update the top post to clarify what I mean there a little bit.

Well, as I mentioned this is just a "minor pet peeve" of mine. What I mean by this oxymoron is that, while this isn't super relevant, the shortcomings of modimport compared to the standard Lua utilities have always greatly irked me. But I agree making it clear what are the differences between modimport and require would be the ideal scenario. Too often already we see threads with modders asking how they can get mod configuration data from outside of modmain; understanding environments and how they may be manipulated is definitely an advanced knowledge of Lua, and one required (no pun intended) if one wishes to truly drop modimport.

Edited by simplex
ooops, I had only renamed partially the test script's name in the traceback
  • Like 1
Link to comment
Share on other sites

  • Developer
1 hour ago, Ipsquiggle said:

We just pushed a new version to the worlddata branch, version 172739.

Notable changes include lots of crash fixes for starting worlds with no save data, old save data, old dedicated servers, etc., as well as better handling of Game Modes throughout all aspects of the game. Also, the modassert and initprint fixes have been put in, thanks!

One aspect of this is that the game mode defined in your modinfo can now specify the level type (preset), as follows

game_modes =
{
	{
		name = "pigfeed",
		label = "PigFeed",
		description = "Feed the pig king the most in each round to get points!",
		settings =
		{
			ghost_sanity_drain = false,
			portal_rez = true,
			level_type = "LEVELTYPE_PIGFEED",
		}
	}
}

and then in your modworldgenmain you define the level for that level type

AddLevel( "LEVELTYPE_PIGFEED", {
		id = "PIGFEED",
		name = "Pigfeed preset",
		desc = "Pigfeed preset description!",
        location = "forest",
        version = 2,
        overrides={
			flint = "often",
			grass = "often",
        },
})


In the client, the presets list is now filtered based on the level type defined by the game mode selected, and for dedicated servers, currently the first preset is chosen based on the level type/game mode.

There's currently a bug where a mod game mode that doesn't define a level type and crashes, but that will be resolved next build.

Link to comment
Share on other sites

On 4/5/2016 at 3:27 PM, Ipsquiggle said:

Cool, thanks for pointing these things out (I'll look at that ModDebugPrint suggestion as well!)

Yes, I'm not super satisfied with loading modworldgenmain in the frontend and that may change a bit in the future, but for now I'd assume that has to load multiple times cleanly. Part of the answer would probably be to put in an AddTile API that takes care of the business that tile_adder is doing...

Already sent this to @PeterA but, since you are looking at adding the AddTile feature, here is a working version of it. You will probably need to change/modify it, but it was working before this update.

moddable_tiles.zip

Note: I used it as a basis for creating this mod: http://steamcommunity.com/sharedfiles/filedetails/?id=580851939

Edited by Kzisor
Link to comment
Share on other sites

On 4/5/2016 at 5:27 PM, Ipsquiggle said:

Yes, I'm not super satisfied with loading modworldgenmain in the frontend and that may change a bit in the future, but for now I'd assume that has to load multiple times cleanly. Part of the answer would probably be to put in an AddTile API that takes care of the business that tile_adder is doing...

Sorry, but I couldn't resist: U&A has no problem with this precisely because it uses require instead of modimport (the modworldgen.lua is just a stub doing the require'ing) :P.

 

@Kzisor

Is this mod no longer working for you? The actual code is the file tile_adder.lua, which when modimported adds a AddTile function to the mod environment (without overriding any game files and such). The rest of the mod is just sample usage of it.

This is still what U&A successfully uses in both DS and DST. It's possible I changed the version within wicker/U&A since that mod was posted, though, so if that implementation no longer works I should be able to extract the current one from wicker/U&A and reupload it.

Link to comment
Share on other sites

10 hours ago, Kzisor said:

Already sent this to @PeterA but, since you are looking at adding the AddTile feature, here is a working version of it. You will probably need to change/modify it, but it was working before this update.

moddable_tiles.zip

Note: I used it as a basis for creating this mod: http://steamcommunity.com/sharedfiles/filedetails/?id=580851939

I wonder, how is it defining the order of the tile layers? By that I mean, for example carpet is on top of wood flooring which is on top of grass.
It seems to me the layer order is defined by the order in which MapLayerManager:CreateRenderLayer(...) Map:AddRenderLayer(handle) calls are made.

The original does ipairs(groundtiles.ground), i.e. sequential as they are specified in the table, yours does pairs(groundtiles.ground), which I believe is not specified, i.e. possibly random.

Edited by Muche
renderlayer order facilitator
Link to comment
Share on other sites

@Ipsquiggle

If you do implement official AddTile support, the major concern (and the only thing that can't really be done from mods themselves) is ensuring numerical stability of tile ids. Suppose mods A and B were enabled at worldgen, with both adding custom tiles and A loading before B. Now suppose, an unspecified time later, A is updated and now further adds a new tile. If the new tile ids are just handled consecutively to each AddTile call performed by mods, this means B's tile ids have now been shifted by 1 to the right; but now loading a world from before A's update will have B's first custom tile mapped into A's new tile, breaking the save.

This is why in my tile_adder.lua implementation I require custom tile ids to be set manually, with a not so reasonable request that they differ from any id used by other mods adding tiles.

Edited by simplex
grammar
  • Like 1
Link to comment
Share on other sites

@Ipsquiggle

One possible solution for the custom tile id stability concern I pointed out above is the following scheme:

Each mod would have its own tile id range space, starting from 1 (or 0, but I mean this is Lua). The game would reserve a flat range of tile ids for all mods (say, all integers greater than or equal to 64 and strictly less than GROUND.UNDERGROUND, though it'd be nice if the range could be infinite).

The game, after loading modworldgenmain.lua from each mod, would build a mod tile id translation table, let's call it 'mods_custom_tiles', mapping each modname (that is, the mod directory name) to its array of real tile ids; the indices in this array would be the mod-space tile ids for its custom tiles, and each associated value would be the corresponding real tile id. Whenever a world saves, this lookup table would be reversed, let's call the result world_tile_overrides, with world_tile_overrides mapping the real id of each custom tile to a pair {modname, modspace_tile_id}. This table would be constructed in the following way:

function GetWorldTileOverrides(mods_custom_tiles)
    local world_tile_overrides = {}
    for modname, custom_tiles in pairs(mods_custom_tiles) do
        for mod_id, real_id in pairs(custom_tiles) do
            if world_tile_overrides[real_id] ~= nil then
                error("There is a logic error in AddTile, the same real id should never be handed out twice.")
            end
            world_tile_overrides[real_id] = {modname, mod_id}
        end
    end
    return world_tile_overrides
end

It is important that this table be re-generated whenever a world saves (i.e., whenever world savedata is generated and serialized), and not just once when it's baked in worldgen.

Then world_tile_overrides would be saved in the world data of each level. Whenever world savedata is loaded, for each tile 't_id' in the map data map them through the following:

--[[/*
-- 't_id' stands for a tile id loaded from the map savedata.
--
-- 'world_tile_overrides' comes from the map savedata.
--
-- 'mods_custom_tiles' was built from the *current game instance*, right after
-- mods finished loading.
--
--
-- The purpose of this function is to perform a dual lookup on tile ids,
-- translating them if needed under the premise that it is the '{modname,
-- mod_ids}' pairs that should remain fixed.
--*/]]
function MigrateTileId(t_id, world_tile_overrides, mods_custom_tiles)
    --// We check if there is any custom tile override in the savedata.
    local t_override = world_tile_overrides[t_id]
    if t_override then
        --[[/*
        -- If there is, then we use the mods_custom_tiles from the *current*
        -- game state.
        --*/]]
        local modname, mod_id = t_override[1], t_override[2]
        --[[/*
        -- Here we get the lookup table for a specific mod, mapping its mod
        -- tile ids to the global, real ones. If this table or any of the
        -- needed subentries are nil, then we leave the tile id alone. This
        -- should prevent breaking savedata if a mod adding custom tiles is
        -- disabled.
        --
        -- If I recall correctly, the game renders invalid tile ids as
        -- GROUND.GRASS.
        --*/]]
        local lookup = mods_custom_tiles[modname]
        if lookup ~= nil then
            local new_t_id = lookup[mod_id]
            if new_t_id ~= nil then
                t_id = new_t_id
            end
        end
    end
    return t_id
end

 

Unless I'm missing something, this should fully ensure id stability.

 

EDIT: Modspace tile ids could also be modder-given strings ids, instead of consecutive numbers. This would make the code more robust, because it wouldn't break a mod that reorders its AddTile call chain or removes an element from it. The code above supports this without change.

Edited by simplex
suggested strings as mod tile ids
Link to comment
Share on other sites

1 hour ago, simplex said:

Sorry, but I couldn't resist: U&A has no problem with this precisely because it uses require instead of modimport (the modworldgen.lua is just a stub doing the require'ing) :P.

 

@Kzisor

Is this mod no longer working for you? The actual code is the file tile_adder.lua, which when modimported adds a AddTile function to the mod environment (without overriding any game files and such). The rest of the mod is just sample usage of it.

This is still what U&A successfully uses in both DS and DST. It's possible I changed the version within wicker/U&A since that mod was posted, though, so if that implementation no longer works I should be able to extract the current one from wicker/U&A and reupload it.

Does not take loading order of mods into account for Client/Server. The code I posted does take than into account as well as removing all traces of the now not used "walls"; this opens up the possibility for tiles to be infinite whereas before you could only have roughly 30ish modded tiles before you ran into issues.

1 hour ago, Muche said:

I wonder, how is it defining the order of the tile layers? By that I mean, for example carpet is on top of wood flooring which is on top of grass.
It seems to me the layer order is defined by the order in which MapLayerManager:CreateRenderLayer(...) calls are made.

The original does ipairs(groundtiles.ground), i.e. sequential as they are specified in the table, yours does pairs(groundtiles.ground), which I believe is not specified, i.e. possibly random.

The mod version was changed to be completely mod friendly from a mod aspect, sorry it's been like 6-8 months since I created it so I have no clue how it works at this moment in time. I know I wasn't modifying the order of the tile layers, I simply made changes to allow more tiles to be added due to the restriction as noted above, 30ish was the rough limit of tiles which could be modded into the game at the time. The mod I posted bypasses that limit, allowing an infinite to be added; but differently than the original code in the .zip file.

  • Like 1
Link to comment
Share on other sites

  • Developer

Just posted another update to the worlddata beta, v. 172848, which fixes a couple more crashes, and adds an AddStartLocation mod function.

Yes, the tile thing is tricky, hence it not getting shoved in yet. Thanks for both those utilities, I'll have a browse through them and see whether or not this is something I can get in in the short term.

  • Like 2
Link to comment
Share on other sites

During browsing the various save slots in the host game menu, the values for configuration options loaded for a particular mod seem to be coming from general modconfiguration*_CLIENT, not that save slot's.

I'm not sure how important this is, in that how many mods use configuration options to affect stuff in modworldgenmain.lua that could be shown in the UI.

Link to comment
Share on other sites

  • Developer

@Muche I tested this and saw that once a slot had savedata, when you selected that slot, it's mod configs would come from the savedata. (It's layered on top of the client data I believe, so you may see load messages in the log in all cases.) If you're moving between empty slots then it should use the client value, or the last viewed value if going to a slot for the first time. It's a little weird, but in the important case (having the correct saved config on an existing save slot) it appears to be correct. Have you noticed otherwise?

Link to comment
Share on other sites

@Ipsquiggle

I just remembered a request: could you allow for some override options in levels being hidden from its customizationtab in server creation? For example, U&A requires rain to be set to "always"; it doesn't actually rain in U&A, this is just an element of how we represent the static mechanic and make it relate to vanilla prefabs. Changing rain to anything other than "always" will in fact break the mod, so having that be a spinner in customizationtab is really bad.

The issue is, now that level overrides are in a key-value dictionary instead of an array of tuples, the simplest solution (adding support for an optional 'hideinfrontend = true' entry in the tuples themselves) is no longer available, so a custom scheme is necessary. Maybe an optional 'hiddenoverrides' array of override key names added to level data, which is checked by customizationtab to exclude certain override options from the frontend, but ignored by everything else in the game?

Edited by simplex
typo
Link to comment
Share on other sites

  • Developer

Interesting, I'll have to think on that... In the meantime you can use an AddLevelPreInitAny, and just force that override value at that point. 

In general, I think mod interaction with override options could be improved a lot...

Link to comment
Share on other sites

4 minutes ago, Ipsquiggle said:

Interesting, I'll have to think on that... In the meantime you can use an AddLevelPreInitAny, and just force that override value at that point. 

In general, I think mod interaction with override options could be improved a lot...

We use custom world and world network entities, so adding a preinit wouldn't even be necessary. There are certainly workarounds, but the major concern is user transparency: having a worldgen option which does nothing (because there's code overriding the override) feels off and glitchy.

I'd say this is a low priority request, but a relatively simple one (due to not requiring affecting other game subsystems) to be worth it.

Edited by simplex
Changed my angle; user experience is the main concern.
Link to comment
Share on other sites

@Ipsquiggle, while you're looking into AddTile feature, please also look into making custom tiles be usable in custom Set Pieces.

Tutorial Example:

Spoiler

Tiled Tutorial:

Inside tiled we must create a new map, here is the settings I used.

Ba3tCaX.png

 

Next we had to create a new tile set, here is the settings I used.

rUy3o3W.png

 

After creating the tile set, add some images to the tile set.

sGMqTfO.png

 

Once you have images in the tile set, simply select an image, right click and select properties.

nhCwIn8.png

 

Add a new property with the name "name", "id", or "tile_name" as the new code will pick up all these names.

1Ea1Jwk.png

 

As the value, put the name of the ground tile you want to use.

JjSw2TQ.png

 

Finally simply use them to create the layout as you normally would.

uqVr6bE.png

Code:

Spoiler

local ParseNestedKey -- must define it first so we can recurse
ParseNestedKey = function(obj, key, value)
    if #key == 1 then
        obj[key[1]] = value
        return
    else
        local key_head = key[1]
        if key_head == nil then
            return
        end
         
        local key_tail = {}
        for i,k in ipairs(key) do
            if i > 1 then table.insert(key_tail, k) end
        end
        if obj[key_head] == nil then
            obj[key_head] = {}
        end
        ParseNestedKey(obj[key_head], key_tail, value)
    end
end
require("constants")
local default_tiles = {}

for k, v in pairs(GROUND)do
   table.insert(default_tiles, v)
end
 
local function LoadLayout( src, additionalProps )
 
    -- Load layout into a variable.
    local staticlayout = require( src )
 
    -- Table which contains the tiles for the layout.
    -- Required to be nil for a check later on in the code.
    local tiles = nil
 
    -- Get tiles from imported layout.
    if staticlayout.tilesets ~= nil then
 
        -- Iterate through the tilesets to obtain all the tiles used.
        -- Note: There should ever only be one tile set being used, a custom or the default.
        for _, tileset in pairs( staticlayout.tilesets ) do
 
            -- Make sure the tiles are not nil.
            if tileset.tiles ~= nil then
 
                -- Iterate through the tiles of the tileset.
                for _, tile in ipairs( tileset.tiles ) do
 
                    -- Make sure the properties table is not nil.
                    if tile.properties ~= nil then
 
                        -- Get property for tile from 3 available options.
                        local tile_name = tile.properties["name"] or tile.properties["id"] or tile.properties["tile_name"] or nil
 
                        if tile_name ~= nil then
 
                            -- Required for a check later on in the code.
                            if tiles == nil then
                                 
                                tiles = { }
 
                            end
 
                            -- Insert it into the tiles table for the layout.
                            table.insert( tiles, GROUND[ string.upper( tile_name ) ] )
 
                        end
 
                    end
 
                end
 
            end
 
        end
 
    end
     
    -- Allows us to support 16 and 64 wide grids from tiled.
    local tilefactor = math.ceil(64/staticlayout.tilewidth)
 
    -- Base layout creation.
    local layout = additionalProps or {}
     
    -- Type of layout we are loading.
    layout.type = LAYOUT.STATIC
 
    -- Scale of the layout.
    layout.scale = 1
 
    -- See ..\Don't Starve Mod Tools\tiles\dont_starve\tiles.png for default tiles.
    layout.ground_types = nil
    layout.ground = {}
 
    -- Uses custom tiles from layout file if found. Otherwise we use default selection.
    if tiles ~= nil then
 
        layout.ground_types = tiles
 
    else
 
        layout.ground_types = default_tiles
 
    end
 
    -- See \tools\tiled\dont_starve\objecttypes.xml for objects
    layout.layout = {}
 
    for _, layer in ipairs( staticlayout.layers ) do
 
        -- The background tile layer must be named BG_TILES.
        if layer.type == "tilelayer" and layer.name == "BG_TILES" then
 
            -- This has not been refactored from the original as this seems to be the only way to accomplish this task.
            local val_per_row = layer.width * (tilefactor-1)
            local i = val_per_row
 
            while i < #layer.data do       
                local data = {}    
                local j = 1
                while j < layer.width and i+j < #layer.data do
                    table.insert(data, layer.data[i+j])
                    j = j + tilefactor
                end
                table.insert(layout.ground, data)    
                i = i + val_per_row + layer.width
            end
 
        -- The forground objects must be named FG_OBJECTS
        elseif layer.type == "objectgroup" and layer.name == "FG_OBJECTS" then
 
            local x_eq = 64.0 - ( staticlayout.width / tilefactor ) / 2
            local y_eq = 64.0 - ( staticlayout.height / tilefactor ) / 2
 
            -- Iterate through the objects list.
            for _, obj in ipairs( layer.objects ) do
 
                if layout.layout[ obj.type ] == nil then
 
                    layout.layout[ obj.type ] = { }
 
                end
 
                -- TODO: Check the object properties for other options to substitute here.
                local x = obj.x + obj.width / 2
                x = x / x_eq
 
                local y = obj.y + obj.height / 2
                y = y / y_eq
 
                local width = obj.width / 64.0
                local height = obj.height / 64.0
 
                local properties = {}
 
                if obj.properties then
 
                    for k, v in pairs( obj.properties ) do
 
                        local key = k:split(".")
 
                        local number_v = tonumber( v )
 
                        if v == "true" or v == "false" then
 
                            ParseNestedKey( properties, key, v == "true" )
 
                        else
 
                            ParseNestedKey( properties, key, number_v or v )
 
                        end
 
                    end
 
                end
 
                table.insert( layout.layout[ obj.type ], { x = x, y = y, properties = properties, width = width, height = height } )
 
            end
 
            if layout.initfn then
 
                 
                layout.initfn( layout.layout )
 
            end
 
        end
 
    end
 
    return layout
 
end

 

I sent @PeterA this code before the forum revamp, but unfortunately the code is not in a state to actually post here so I ripped the code from my Additional Set Pieces mod which actually added the capability. It's a simple change, just need to modify the static_layout.lua file to look for properties (as noted in the above code).

The code above also fixed the issue of having to store tiles in multiple places to get the same affect by looking at the GROUND constant instead of a local table like it originally did.

Regards,

Kzisor/Ysovuka

Edited by Kzisor
Grammar.....horrible....tired.....must sleep.....
  • Like 2
Link to comment
Share on other sites

@Ipsquiggle, while I am still up I might as well throw this into the fray.

I personally have loathed the size of the DST world because they were always too small, you could always explore it all within a single year. My computer is fast enough to handle a 1024x1024 size map and from time to time I've hard coded it for world generation and loved it. I know people have been asking for larger maps for a long time.

It would be nice to at least have the ability to put 'Custom' in the world generation overrides and manually set it to the size you want it to, specifically through the world generation overrides file.

  • Like 1
Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
 Share

×
  • Create New...