Shard RPC Gem API

[Serpens]

@ZarklordCurrently I fail to reproduce the initial bug without even adding this overridesblocker code
If I remember right the bug was: When enabling gem api and then generating a world, this first world will have default worldsettings instead of the worldsettings from eg. adventure mod. For every next world, while gem api is still active, this did not happen.
So it had sth to do with enabling the api gem.

But now I fail to reproduce it, I enable first the adventure+teleportato mod and after that your gem api. But the worldsettings are fine after generating the world.
Since you have more inside, how can I savely repdroduce the old behaviour?

And after that:
Should I put the gemrun in modmain or modworldgenmain, you write modmain above, but overrides seems to fit better in modworldgenmain?
And the "overrides" from that gemrun function, must look like this:

overrides={ 
                task_set = "cave_default",
                start_location = "caves",
                layout_mode = "RestrictNodesByKey", -- default for cave
                wormhole_prefab = "tentacle_pillar",
                roads = "never",
                -- .....
                }

?  And does it also mean I dont have to use AddTaskSetPreInitAny to change the overrides, because gem api will do this?
So should I define such an overrides list and IF gem api is active, I will run gemrun and then there is no need to change the overrides within AddTaskSetPreInitAny, but if gem api is not active, I have to change overrides within AddTaskSetPreInitAny?

[zarklord_klei]

22 minutes ago, Serpens said:

@ZarklordCurrently I fail to reproduce the initial bug without even adding this overridesblocker code
If I remember right the bug was: When enabling gem api and then generating a world, this first world will have default worldsettings instead of the worldsettings from eg. adventure mod. For every next world, while gem api is still active, this did not happen.
So it had sth to do with enabling the api gem.

But now I fail to reproduce it, I enable first the adventure+teleportato mod and after that your gem api. But the worldsettings are fine after generating the world.
Since you have more inside, how can I savely repdroduce the old behaviour?

And after that:
Should I put the gemrun in modmain or modworldgenmain, you write modmain above, but overrides seems to fit better in modworldgenmain?
And the "overrides" from that gemrun function, must look like this:

overrides={ 
                task_set = "cave_default",
                start_location = "caves",
                layout_mode = "RestrictNodesByKey", -- default for cave
                wormhole_prefab = "tentacle_pillar",
                roads = "never",
                -- .....
                }

?  And does it also mean I dont have to use AddTaskSetPreInitAny to change the overrides, because gem api will do this?
So should I define such an overrides list and IF gem api is active, I will run gemrun and then there is no need to change the overrides within AddTaskSetPreInitAny, but if gem api is not active, I have to change overrides within AddTaskSetPreInitAny?

First question, are you doing this on a world with caves, or without?, cause from my testing, if done without caves, leveldataoverride isn't generated, and therefore it wouldn't update your settings, but when ran with my mod your settings would update the world gen options setting, deleting the original world settings.
as for where you call that function, using your Teleportato mod as an example, you would call the function just after the table where you defined your overrides was, so for the table defined at line 307 of modworldgenmain, after you finish defining that you would do something like this:

if GemCoreIsEnabled then --make sure Gem Core exists, this would need to be something you do, I haven't defined anything for this.
	gemrun("overridesblocker", tasksetdata.overrides, modname, {"list of overrides to prevent updating"})
end

blocked overrides are stored inside the world save file, so upon regenerating the world they get cleared, this means two things:
you only ever need to set blocked overrides once.
you don't ever need to worry about clearing them.

EDIT: I messed up saying modmain instead of modworldgenmain, that was me speaking, really about if your inside the mod environment instead of the global environment.

Edited September 26, 2019 by Zarklord

[Serpens]

30 minutes ago, Zarklord said:

First question, are you doing this on a world with caves, or without, cause from my testing, if done without caves, it wouldn't be needed.
as for where you call that function, using your Teleportato mod as an example, you would call the function just after the table where you defined your overrides was, so for the table defined at line 307 of modworldgenmain, after you finish defining that you would do something like this:

if GemCoreIsEnabled then --make sure Gem Core exists, this would need to be something you do, I haven't defined anything for this.
	gemrun("overridesblocker", tasksetdata.overrides, modname, {"list of overrides to prevent updating"})
end

blocked overrides are stored inside the world save file, so upon regenerating the world they get cleared, this means two things:
you only ever need to set blocked overrides once.
you don't ever need to worry about clearing them.

tried it first without caves and now with caves, but same result... So I guess I just have to hope the problem is solved after adding gemrun, although I would like to test properly if I made everything correct

So I put that gemrun line now into that line. Now I wonder:
Shouldn't the overrides_to_block include the same values like overrides (except that it is an array)? So I dont get the point of that list currently, because of course I dont want any of my overrides to be overwritten, when should I want this?

And you mentioned earlier:

On 8.9.2019 at 6:49 PM, Zarklord said:

and also a option that allows you to protect individual options, and let worldsettings control the rest.

is this overrides_to_block ? If so, how to use it?
Imagine I want all the overrides you see in teleportato, but now I dont want to change the "day"  myself, but want the world to use the worldsettings set up by the user. So I wont include the day within tasksetdata.overrides of course, but it seems it does not matter if I add the "day" to overrides_to_block or not, the user chosen worldsetting is never used.

Edited September 26, 2019 by Serpens

[zarklord_klei]

3 minutes ago, Serpens said:

tried it first without caves and now with caves, but same result... So I guess I just have to hope the problem is solved after adding gemrun, although I would like to test properly if I made everything correct

So I put that gemrun line now into that line. Now I wonder:
Shouldn't the overrides_to_block include the same values like overrides (except that it is an array)? So I dont get the point of that list currently, because of course I dont want any of my overrides to be overwritten, when should I want this?

Well, the log file will print all the overrides that got blocked and by what mod, you can check to see if its working(if your values got printed, then it blocked those from getting updated by leveldataoverrides/worldgenoverrides).

8 minutes ago, Serpens said:

is this overrides_to_block ? If so, how to use it?
Imagine I want to all the overrides you see in teleportato, but now I dont want to change the "day"  myself, but want the world to use the worldsettings set up by the user. So I wont include the day within tasksetdata.overrides of course, but it seems it does not matter if I add the "day" to overrides_to_block or not, the user chosen worldsetting is never used.

yes; one option, is to shallowcopy(see this for an explanation of what a shallowcopy is) the overrides table, and then after it gets updated by your mod, do a comparison of changed values and table.insert the names of them into the overrides_to_block table, and pass that.

[Serpens]

3 minutes ago, Zarklord said:

yes; one option, is to shallowcopy(see this for an explanation of what a shallowcopy is) the overrides table, and then after it gets updated by your mod, do a comparison of changed values and table.insert the names of them into the overrides_to_block table, and pass that.

could you please show me complete sample code for this? if I do print(tasksetdata.overrides) within AddTaskSetPreInitAny before I change it, I will get "nil" as far as I know. So how am I able to compare anything?

And that does not explain what exactly overrides_to_block is doing. If I could get overrides values from before my mod is changing them, I would neither need gem api nor a shallowcopy ?!

Edited September 26, 2019 by Serpens

[zarklord_klei]

7 minutes ago, Serpens said:

could you please show me complete sample code for this? if I do print(tasksetdata.overrides) within AddTaskSetPreInitAny before I change it, I will get "nil" as far as I know. So how am I able to compare anything?

And that does not explain what exactly overrides_to_block is doing. If I could get overrides values from before my mod is changing them, I would neither need gem api nor a shallowcopy ?!

use AddLevelPreInitAny instead of AddTaskSetPreInitAny, and tell me if you get the same issue, (you shouldn't need to change anything inside your preinit, just change the method), let me know if that fixed those issues.

[Serpens]

53 minutes ago, Zarklord said:

use AddLevelPreInitAny instead of AddTaskSetPreInitAny, and tell me if you get the same issue, (you shouldn't need to change anything inside your preinit, just change the method), let me know if that fixed those issues.

I always wondered what the difference between those two is, but found none. Yes, indeed with LevelPreInit I can acces the worldsettings chosen by the user. So I can simply change all the tasksetdata.overrides one by one, instead of overwriting the whole overrides table to keep the user worldsettings for the ones I do not change
Instead I could also use your complicated method with shallowcopy and overrides_to_block, while overwriting the whole overrides table. The result would be the same, right?

But to come back to the inital problem that API Gem might overwrite the overrides of my mod:
I simply should make overrides_to_block to include the same names like overrides (the overrides table my mod is setting up) and then I'm fine. Or am I wrong?

I just found out that the "Batch Enabler" which should only work for client mods, also executes modworldgenmain of my server (or at least all_clients_require) mods, resulting in major problems...
What is a easy way to check within modworldgenmain, if it is run because the world is generated now, and not because it was just activated or manipulated by another mod? I have to add such a security check to my mods, so the code only executes when it is supposed to be executed (eg. the whole POSITIONS stuff from my teleportato mod)
 

Edited September 27, 2019 by Serpens

[zarklord_klei]

42 minutes ago, Serpens said:

I always wondered what the difference between those two is, but found none. Yes, indeed with LevelPreInit I can acces the worldsettings chosen by the user. So I can simply change all the tasksetdata.overrides one by one, instead of overwriting the whole overrides table to keep the user worldsettings for the ones I do not change
Instead I could also use your complicated method with shallowcopy and overrides_to_block, while overwriting the whole overrides table. The result would be the same, right?

if you look at Level:ChooseTasks() in map/level/lua@line87, you can see the difference between them.

my recommendation is doing something like this:

AddLevelPreInitAny(function(level)
    local _overrides = GLOBAL.shallowcopy(level.overrides)
    --apply your custom overrides
    local overrides_to_block = {}
    for k, v in pairs(level.overrides) do
      	--original blockoverrides updateanyways are 3 things that could be in this table that aren't actually overrides.
    	if k ~= "original" and k ~= "blockoverrides" and k ~= "updateanyways" and _overrides[k] ~= v then
        	table.insert(overrides_to_block, k)
        end
    end
    if GemCoreIsEnabled then
    	GLOBAL.gemrun("overridesblocker", level.overrides, modname, overrides_to_block)
    end
end)

1 hour ago, Serpens said:

But to come back to the inital problem that API Gem might overwrite the overrides of my mod:
I simply should make overrides_to_block to include the same names like overrides (the overrides table my mod is setting up) and then I'm fine. Or am I wrong?

as long as you properly call overridesblocker, gem core will never overwrite your overrides.

1 hour ago, Serpens said:

I just found out that the "Batch Enabler" which should only work for client mods, also executes modworldgenmain of my server (or at least all_clients_require) mods, resulting in major problems...
What is a easy way to check within modworldgenmain, if it is run because the world is generated now, and not because it was just activated or manipulated by another mod? I have to add such a security check to my mods, so the code only executes when it is supposed to be executed (eg. the whole POSITIONS stuff from my teleportato mod)

I don't actually see how that's possible, in order for Batch Enabler to enable/disable your mod, client_only_mod would have to be true, which for Teleportato mod is false.

[Serpens]

ok, thank you for the sample code.

28 minutes ago, Zarklord said:

I don't actually see how that's possible, in order for Batch Enabler to enable/disable your mod, client_only_mod would have to be true, which for Teleportato mod is false.

you are right, I thought it was that mod, because the problem disappeared after I disabled it. but it only disappeared because I restartet the game. In fact it is your API Gem Mod.
Do this to reproduce:
1) Enable the current version of teleportato and adventure mod and also enable your api gem
2) Start the world (default world without caves)
3) quit the world, but dont quit the game.
4) delete the worldslot, to be ready to config a new world.
5) enable only the teleportato mod, no other mod.
6) open the console, you will see a crashlog, because the modworldgenmain from my teleportato mod got executed WITH "next(WORLDS)" be true, although that should be impossible since adventure mod is not active, so WORLDS should be empty). The crash happens because of line 75 of modworldgenmain, because it can not split the string W.positions. But in fact, right now, this is not a string it is a table, that is why it is crashing. If you print the content of the W.positions table you will receive the content of WORLDS , although W.positions is WORLDS[x].positions. 

So to sum it up:
1) although the API Gem mod is no longer enabled, it is still affecting the mods.
2) for whatever reason the tables get messed up, like described above.

Although you receive this crash log when enabling teleportato, as far as I know everything that needs to work still works (because modworldgenmain from teleportato is allowed to crash during modmenu, since its code does not need to run at that time)

EDIT:
tested again without your api mod ever active, and the same happens.
So it is not your mod doing this, but the game?! @Zarklord (would like to trigger an update at your side, so you see the edit, before wasting to much time :D)

Edited September 27, 2019 by Serpens

[zarklord_klei]

10 minutes ago, Serpens said:

ok, thank you for the sample code.

you are right, I thought it was that mod, because the problem disappeared after I disabled it. but it only disappeared because I restartet the game. In fact it is your API Gem Mod.
Do this to reproduce:
1) Enable the current version of teleportato and adventure mod and also enable your api gem
2) Start the world (default world without caves)
3) quit the world, but dont quit the game.
4) delete the worldslot.
5) enable only the teleportato mod, no other mod.
6) open the console, you will see a crashlog, because the modworldgenmain from my teleportato mod got executed WITH "next(WORLDS)" be true, although that should be impossible since adventure mod is not active, so WORLDS should be empty). The crash happens because of line 75 of modworldgenmain, because it can not split the string W.positions. But in fact, right now, this is not a string it is a table, that is why it is crashing. If you print the content of the W.positions table you will receive the content of WORLDS , although W.positions is WORLDS[x].positions. 

So to sum it up:
1) although the API Gem mod is no longer enabled, it is still affecting the mods.
2) for whatever reason the tables get messed up, like described above.

Although you receive this crash log when enabling teleportato, as far as I know everything that needs to work still works (because modworldgenmain from teleportato is allowed to crash during modmenu, since its code does not need to run at that time)

I can simplify the steps to get that crash, for you:
enable Adventure Mod and Teleportato, disable both and re-enable Teleporato(you might need to enable/disable it a couple times for you to get the crash), if you want to prevent that kinda of thing from happening(and you don't do anything on the frontend) you can add a simple check at the top of the file to prevent execution on the server creation screen:

if GLOBAL.rawget(GLOBAL, "TheFrontEnd") and GLOBAL.rawget(GLOBAL, "IsInFrontEnd") and GLOBAL.IsInFrontEnd() then return end

this will prevent any code below it from executing on the server creation screen(frontend)

[Serpens]

2 minutes ago, Zarklord said:

 

if GLOBAL.rawget(GLOBAL, "TheFrontEnd") and GLOBAL.rawget(GLOBAL, "IsInFrontEnd") and GLOBAL.IsInFrontEnd() then return end

this will prevent any code below it from executing on the server creation screen(frontend)

yes, that is what I was looking for, thank you very much. I also edited my post above, but no clue how to trigger a message at your side so you see the edit... =/

Edited September 27, 2019 by Serpens

[zarklord_klei]

Just now, Serpens said:

yes, that is what I was looking for, thank you very much. I also edited my post above, but no clue how to trigger a message at your side so you see the edit... =/

Its fine, I'm happy to help.

[Serpens]

On 27.9.2019 at 4:14 AM, Zarklord said:

 

AddLevelPreInitAny(function(level)
    local _overrides = GLOBAL.shallowcopy(level.overrides)
    --apply your custom overrides
    local overrides_to_block = {}
    for k, v in pairs(level.overrides) do
      	--original blockoverrides updateanyways are 3 things that could be in this table that aren't actually overrides.
    	if k ~= "original" and k ~= "blockoverrides" and k ~= "updateanyways" and _overrides[k] ~= v then
        	table.insert(overrides_to_block, k)
        end
    end
    if GemCoreIsEnabled then
    	GLOBAL.gemrun("overridesblocker", level.overrides, modname, overrides_to_block)
    end
end)

 

I forgot to ask how to check if gemapi is active within modworldgenmain? The code you gave me seems not to work, because of "TheNet" does not exist yet.

[zarklord_klei]

12 minutes ago, Serpens said:

I forgot to ask how to check if gemapi is active within modworldgenmain? The code you gave me seems not to work, because of "TheNet" does not exist yet.

TheNet should always exist, did you try putting GLOBAL before it?

[Serpens]

13 minutes ago, Zarklord said:

TheNet should always exist, did you try putting GLOBAL before it?

yes

print(GLOBAL.TheNet)

Quote

 variable 'TheNet' is not declared

within modworldgenmain outside and even tried inside of AddLevelPreInitAny.

But although I added

if GLOBAL.rawget(GLOBAL, "TheFrontEnd") and GLOBAL.rawget(GLOBAL, "IsInFrontEnd") and GLOBAL.IsInFrontEnd() then return end 

at the top of modworldgenmain, it is still called 2 times in total (when generating the world). The first time TheNet is defined, but the second time it is not.

edit:
I confirmed it with a new empty mod, only containing:

print("HHIIEERR")
print(GLOBAL.rawget(GLOBAL, "TheNet"))

in modworldgenmain.
It gets printed 3 times (without blocking the Frontend) and the third time TheNet is nil.

Edited September 30, 2019 by Serpens

[zarklord_klei]

oh right, I forgot that during worldgen TheNet is nil.
 

local function IsModLoaded(modname)
    return GLOBAL.KnownModIndex:IsModEnabled(modname) or GLOBAL.KnownModIndex:IsModForceEnabled(modname)
end
--usage:
if IsModLoaded("workshop-1378549454") then --gemcore
  --do gem core things
end