Extremely low mod loading times

[UL7RA]

So, before the patch the mods were loading at a reasonable time, but now, after the beta relese of Reign of Giants (which I obviously bought), the loading time is extremely long. It just takes ages to register all the prefabs.

F***! I meant EXTREMELY LONG! Why isn't there an edit button?!

[JanH]

Hey @UL7RA, we've had a few people report this issue. It looks like it's happening because you're able to toggle DLC on and off on the fly. We're looking into this and can hopefully be able to optimize the process; thanks for sending us your report! 

[squeek]

Hey @UL7RA, we've had a few people report this issue. It looks like it's happening because you're able to toggle DLC on and off on the fly. We're looking into this and can hopefully be able to optimize the process; thanks for sending us your report!

The culprit is the added lines to mods.lua (lines 254-256 and 263-276). I can't really figure out what they are intended to do, but the loop is identical to the one below it (so its registering the prefabs twice for some reason), and then it calls HandleDLCAssetSwitch which reloads all of the game's prefabs. This is done twice for every mod loaded (once for the 'mod' prefab and then once for all the registered mod prefabs).

Code of HandleDLCAssetSwitch (dlcsupport.lua):

function HandleDLCAssetSwitch( set )    for i,v in ipairs(set) do        HandlePrefabSwitch(v)    end    -- and reload everything    for i,file in ipairs(PREFABFILES) do	LoadPrefabFile("prefabs/"..file)    end	end

Here's a verbose log illustrating the problem (with 3 mods enabled): http://privatepaste.com/601d9eb940

[SethR]

Yep, that's exactly what's happening. Nice detective work @squeek! The reason for the duplicate loops is that in order to not have duplicates of an asset loaded (which crashes the game), we need to iterate over the things we will load and unload them and then iterate over them again to load them. It's because we're allowing players with the DLC to toggle it on and off on the fly. Since it has modified versions (but with the same name) of  a lot of assets and resources in the vanilla game, we initially did the safe thing and unload and load everything. Tomorrow we're planning on putting out a patch that will clean that whole process up quite a bit. Hopefully it'll get load times back closer to what they were!

[squeek]

Yep, that's exactly what's happening. Nice detective work @squeek! The reason for the duplicate loops is that in order to not have duplicates of an asset loaded (which crashes the game), we need to iterate over the things we will load and unload them and then iterate over them again to load them. It's because we're allowing players with the DLC to toggle it on and off on the fly. Since it has modified versions (but with the same name) of  a lot of assets and resources in the vanilla game, we initially did the safe thing and unload and load everything. Tomorrow we're planning on putting out a patch that will clean that whole process up quite a bit. Hopefully it'll get load times back closer to what they were!

Not possible to treat the DLC as a mod (that loads before the 'real' mods)? Seems like mod loading does all those things already.

Also, keep in mind that this new DLC prefab loading/unloading affects a common technique used in modding to add a component with save data to all player prefabs: http://forums.kleientertainment.com/topic/29599-code-tips-and-tricks/?p=422699 (the workaround would be to iterate GetActiveCharacterList() and add PrefabPostInits to each entry, but that has potential issues with mod load priority [if your mod loads before the custom character mod, the custom character would not be in MODCHARACTERLIST while your modmain.lua is running, I believe])

[SethR]

It's not possible to treat DLC as a mod--there is stuff we need to do on the backend to ensure that only players who have bought it can play it. The mod loading does a lot of similar stuff, but in doing this overhaul for the DLC we actually discovered that the mod system wasn't doing it in the cleanest way possible, so we're trying to address that. We've got a patch coming out tomorrow that affects this stuff.

 

I'm not sure if there's much we can do about the mod loading order and how mods interact with each other based on that. Obviously we'd like to support mods as much as possible, but at some point when using multiple mods, things can get pretty crazy and we get behaviors we can't anticipate. But if you have ideas on how to address this, I'm all ears! And please let us know if you're still having issues after tomorrow's patch.

[squeek]

It's not possible to treat DLC as a mod--there is stuff we need to do on the backend to ensure that only players who have bought it can play it. The mod loading does a lot of similar stuff, but in doing this overhaul for the DLC we actually discovered that the mod system wasn't doing it in the cleanest way possible, so we're trying to address that. We've got a patch coming out tomorrow that affects this stuff.

 

I'm not sure if there's much we can do about the mod loading order and how mods interact with each other based on that. Obviously we'd like to support mods as much as possible, but at some point when using multiple mods, things can get pretty crazy and we get behaviors we can't anticipate. But if you have ideas on how to address this, I'm all ears! And please let us know if you're still having issues after tomorrow's patch.

Well the ideal (and simple) fix in this particular case would be to add a real AddPlayerPostInit function that gets called whenever a player prefab is created, since the SimPostInit is called too late. Here's a quick example implementation:

--// in mainfunctions.lua(115): function SpawnPrefabFromSim	if prefab then		local inst = prefab.fn(TheSim)		if inst ~= nil then			inst:SetPrefabName(inst.prefab or name)			for k,mod in pairs(prefab.modfns) do				mod(inst)			end--// begin added lines			if inst:HasTag("player") then				for k,playerpostinit in pairs(ModManager:GetPostInitFns("PlayerPostInit")) do					playerpostinit(inst)				end			end--// end added lines						return inst.entity:GetGUID()		else			print( "Failed to spawn", name )			return -1		end	end--// in modutil.lua(59): function InsertPostInitFunctions	env.postinitfns.PlayerPostInit = {}	env.AddPlayerPostInit = function(fn)		initprint("AddPlayerPostInit")		table.insert(env.postinitfns.PlayerPostInit, fn)	end

[squeek]

And please let us know if you're still having issues after tomorrow's patch.

Just wanted to let you know that the patch improved things tremendously (as you probably already knew).

[SethR]

Just wanted to let you know that the patch improved things tremendously (as you probably already knew).

 

Excellent news! In return, I wanted to let you know that we're hearing you loud and clear about the mod loading order stuff and general mod support (I saw your post over in the mod and tools forum). Naturally we're pretty busy with DLC but hopefully we can address some of the issues before too long. I'll have a chat with @bizziboi about it tomorrow. As always, thanks for your patience and understanding

[iWitch]

good news.

 

i hope your dlc will be able to get on steam as soon as possible.

 

want to see it even on beta.

[squeek]

Excellent news! In return, I wanted to let you know that we're hearing you loud and clear about the mod loading order stuff and general mod support (I saw your post over in the mod and tools forum). Naturally we're pretty busy with DLC but hopefully we can address some of the issues before too long. I'll have a chat with @bizziboi about it tomorrow. As always, thanks for your patience and understanding

Appreciate it. The mod loading order stuff is really not something I'm very concerned about. The real issue (and the only reason that workaround I linked to is necessary) is more about the lack of a proper catch-all PlayerPostInit function, since the SimPostInit (the only PostInit that has the player as its parameter) gets called after save data has been loaded. The code I posted above is the ideal solution and it's been needed for as long as the modding API has existed. I'd like to hear whether or not you intend to add it to the game, or if not, why not.

I really am confused by these sorts of responses to specific requests with (small, easily reviewable) code attached. I'm used to them by now, but I'm also used to it going ignored for months/indefinitely afterwards.

[SethR]

Appreciate it. The mod loading order stuff is really not something I'm very concerned about. The real issue (and the only reason that workaround I linked to is necessary) is more about the lack of a proper catch-all PlayerPostInit function, since the SimPostInit (the only PostInit that has the player as its parameter) gets called after save data has been loaded. The code I posted above is the ideal solution and it's been needed for as long as the modding API has existed. I'd like to hear whether or not you intend to add it to the game, or if not, why not.

I really am confused by these sorts of responses to specific requests with (small, easily reviewable) code attached. I'm used to them by now, but I'm also used to it going ignored for months/indefinitely afterwards.

 

Hey @squeek. I saw that you read @Bigfoot's post over in the mods forum, which did a better job explaining where we're at with mods than I would . In any case, because I'm not especially familiar with that aspect of the modding system, today I summoned @lpsquiggle to help me look into the PlayerPostInit function idea.

 

Here's what we came up with: It absolutely makes sense to have a generic post init for the player character, regardless of the particular prefab used as the player character.  In gamelogic.lua, on line 418, the playercharacter prefab gets spawned and stored in the wilson variable. Rather than checking every single spawned instance of a prefab to see if is a player prefab, we saw an opportunity to grab the player prefab after it gets created, while we still have a reference to it, but before the save data for the player prefab gets created on line 536 and do the PlayerPostInit function there (i.e. on line 535).

 

If you see any issues with that, let us know. Otherwise, I'll try to get it into an update as soon as I can manage. Thanks!

[squeek]

Hey @squeek. I saw that you read @Bigfoot's post over in the mods forum, which did a better job explaining where we're at with mods than I would . In any case, because I'm not especially familiar with that aspect of the modding system, today I summoned @lpsquiggle to help me look into the PlayerPostInit function idea.

 

Here's what we came up with: It absolutely makes sense to have a generic post init for the player character, regardless of the particular prefab used as the player character.  In gamelogic.lua, on line 418, the playercharacter prefab gets spawned and stored in the wilson variable. Rather than checking every single spawned instance of a prefab to see if is a player prefab, we saw an opportunity to grab the player prefab after it gets created, while we still have a reference to it, but before the save data for the player prefab gets created on line 536 and do the PlayerPostInit function there (i.e. on line 535).

 

If you see any issues with that, let us know. Otherwise, I'll try to get it into an update as soon as I can manage. Thanks!

That sounds like a good solution assuming that there will only ever be one player prefab that is spawned. However, with mods like Two player (local) co-op and Multiplayer Local CooP: 3 players shared screen that wouldn't always be the case, but it's up to you which approach you want to take.

Thank you for looking into it and for the clear and specific response.

[SethR]

Ah, good point! I'll definitely take that into account when I do get around to this. For now, this tab will stay open as a reminder to myself to hook this up.

[squeek]

Ah, good point! I'll definitely take that into account when I do get around to this. For now, this tab will stay open as a reminder to myself to hook this up.

Hey, I can't seem to PM you so I just wanted to make sure to point you in the direction of this thread where I've compiled a list of mod-related suggestions/improvements. Hope it helps.

[SethR]

Cheers, thanks @squeek!

[SethR]

Hey @squeek, just following up on this--wanted to make sure what I've gone and implemented would suit your needs. We ended up going for a global prefab post init function, rather than a player specific one. This way, you can set a post init function will get run for prefabs as they spawn (and you can check for the player tag inside of the function). While it's less apparent that it can be used as a player post init (as compared to a function literally called that, hehe), it's more flexible in that you can make a post init for all prefabs with a monster tag, or anything along those lines.

 

Here's what it looks like:

 

mainfunctions.lua

for k,mod in pairs(prefab.modfns) do	mod(inst)endfor k,globalprefabpostinit in pairs(ModManager:GetPostInitFns("GlobalPrefabPostInit")) do    globalprefabpostinit(inst)end

modutil.lua

env.postinitfns.GlobalPrefabPostInit = {}env.AddGlobalPrefabPostInit = function(fn)	initprint("AddGlobalPrefabPostInit")	table.insert(env.postinitfns.GlobalPrefabPostInit, fn)end

So, yeah, let me know if that seems like it'll suit your needs--want to get it right the first time!

[squeek]

SethR, that'd certainly work for my purposes, since I know why it should be used, but it'd definitely be less clear to new/inexperienced modders that it should be used over AddSimPostInit (which most people use currently as a player post init because its callback parameter is the player) for player post inits.

Up to you if that's a concern, but it'd certainly be nice, for clarity's sake, to include a wrapper function actually called AddPlayerPostInit that does the player tag check automatically. Something like:

 

env.AddPlayerPostInit = function(fn)    env.AddGlobalPrefabPostInit(function(inst)         if inst and inst:HasTag("player") then fn(inst) end     end)end

[SethR]

Ah, that's a good idea! I'm trying to avoid going down a rabbit hole of potentially having specific post inits for a variety of different types of prefab, but I can see my way to including that one (especially since I'm putting a comment above it to make it more obvious that it is an illustrative example of how to use the global prefab post init). Cheers.

[simplex]

@SethR

This post init hook would be a welcome addition. In fact, I implemented it in Up and Away a few months ago (exactly how you suggested it) and it's proven to be very useful.

My only recommendation is to call it AddPrefabPostInitAny instead of AddGlobalPrefabPostInit. That's in order to keep uniformity with the naming conventions adopted by the mod API (AddLevelPreInitAny comes to mind). Calling it AddGlobalPrefabPostInit might also cause confusion, since it could be referring to post inits added to the global entity:

main.lua:

	--- GLOBAL ENTITY ---	TheGlobalInstance = CreateEntity()	TheGlobalInstance.entity:SetCanSleep( false )	TheGlobalInstance.entity:AddTransform()

[SethR]

@SethR

This post init hook would be a welcome addition. In fact, I implemented it in Up and Away a few months ago (exactly how you suggested it) and it's proven to be very useful.

My only recommendation is to call it AddPrefabPostInitAny instead of AddGlobalPrefabPostInit. That's in order to keep uniformity with the naming conventions adopted by the mod API (AddLevelPreInitAny comes to mind). Calling it AddGlobalPrefabPostInit might also cause confusion, since it could be referring to post inits added to the global entity:

main.lua:

	--- GLOBAL ENTITY ---	TheGlobalInstance = CreateEntity()	TheGlobalInstance.entity:SetCanSleep( false )	TheGlobalInstance.entity:AddTransform()

 

Fair point about the naming convention! Updated:

env.postinitfns.PrefabPostInitAny = {}env.AddPrefabPostInitAny = function(fn)	initprint("AddPrefabPostInitAny")	table.insert(env.postinitfns.PrefabPostInitAny, fn)end