How to debug net_vars hell?

[Maris]

Hello all.

I have a few interesting lines of code in my mod Show Me:

	local MAIN_VAR_NAME = 'net_ShowMe_chest';
	local NETVAR_NAME = 'ShowMe_chest';
	local EVENT_NAME = 'ShowMe_chest_dirty';

I use these constants in the code. But recently I decided to change values a little. And the hell began...

Case 1:

	local MAIN_VAR_NAME = 'net_ShowMe_chest1';
	local NETVAR_NAME = 'ShowMe_chest1';
	local EVENT_NAME = 'ShowMe_chest_dirty1';

Looks fine, but I have many mods installed and somehow it causes a crash (mouseover on Crock Pot and may be other actions):

[string "scripts/componentactions.lua':535: attempt to inder a nil value
LUA ERROR stack traceback:
scripts/componentactions.lua:1535 in (method) CollectActions (Lua) <1521-1547>
scripts/components/playeractionpicker.lua:102 in (method) GetSceneActions (Lua) <99-124>
scripts/components/playeractionpicker.lua:249 in (method) GetLeftClickAction (Lua) <202-273>7
scripts/components/playeractionpicker.lua:388 in (method) DoGetMouseActions (Lua) <338-392>
scripts/components/playecontroller.lua:1895 in (method) OnUpdate (Lua) <1758 2177>
scripts/update.lua:202 in () ? (Lua) <159-238>

or

[00:04:34]: [string "scripts/componentactions.lua"]:1535: attempt to index a nil value
LUA ERROR stack traceback:
scripts/componentactions.lua:1535 in (method) CollectActions (Lua) <1521-1547>
   self (valid:true) =

People was sending me their crash reports and I was not able to understand what's wrong.

It was very long quest to figure out that exact these changes are the reason of the crash. And I was surprised. But OK, I changed it a bit more and the crash has left:

	local MAIN_VAR_NAME = 'net_ShowMe_chst';
	local NETVAR_NAME = 'ShowMe_chst';
	local EVENT_NAME = 'ShowMe_chst_dirty';

I still can't belive that such fix can help. But this is real.

Also I noticed that my feature (highlighted chests) stop working for Crock Pot. So this not real fix.

Later crash came back for these valuse.

Step to reproduce:

1) Install mods: Show Me (change vars to these valuse), Smarter Crock Pot, Quick Drop.
2) Start a game with caves.
3) Put a berrie in a Crock Pot.
4) Mouse over the Crock Pot.

I'm not sure if you can reproduce crash too. But it worked for me on my computer.

It was the same crash with action components. 

I decided that's easy to find a place with error step by step. So I started from simplest mod - Quick Drop. I removed many pieces of code and each time I started the game and got crash again. Then I removed ALL the code from Quick Drop and... got crash again.

What?

Really.

I started thinking that I'm going crazy.

Let's take a closer look. There is an EMPTY mod that causes the crash. Let's check again.

1) If I turn the mod off, there is no crash (with mods Show Me + Smarter Crock Pot). If I turn mod on, crash appears.

2) Okay, may be I mixed files or mods. I added to modmain.lua a line print("Hello from Star!") - and I found it both in the server log and in the client! So exact empty mod causing the crash.

3) Okay, may be I'm missing something... Secret symbols in modinfo or whatever. I removed ALL files and subfolders and created modinfo and modmain from scratch (keeping only version number to prevent updating). But everything other was not the same, even mod name. But mod still caused the crash. 

4) Okay, I disabled mod; Quit the DST; Renamed folder of the mod. Started the DST. Enabled the mod (with the same name but in another folder). And now don't cause the crash. Finally! But what is was? The crash depends on folder name? Really?

Note that AddComponentAction is not used in all of these three mods. So it's very strange why crash is about components.

I decided to play with var names again and it helped:

	local MAIN_VAR_NAME = 'net_ShowMe_cht';
	local NETVAR_NAME = 'ShowMe_cht';
	local EVENT_NAME = 'ShowMe_cht_dirty';

No! No no no no... Not again! I don't believe that! Why it helped? How it may help?

I just renamed some constants in the mod.

Looks quite easy. If you got a crash, rename a constant. That's it! Really?

But the issue has not gone. I know that it's still there. Because my feature is still not working for Crock Pot. My feature should highlight Crock Pot if there is berrie in Crock Pot AND in cursor of the player.

Let's make a trick. Rename the constants to initial values:

	local MAIN_VAR_NAME = 'net_ShowMe_chest';
	local NETVAR_NAME = 'ShowMe_chest';
	local EVENT_NAME = 'ShowMe_chest_dirty';

Tada! Now all works fine. My feature works again. No crash, no conflicts (possibly).

But can someone explain to me what is going on here?
Why simple renaming of netvars cause such issues?

Sadly, this is not the end of story because I keep receiving messages about crashes from other people. I think such bugs should be explained so other modders can solve them.

I hope you can help me. May be it's time to summon devs, I'm not sure.

[Serpens]

It may be a conficlt with other files put into the game files/modconfigs/client_save and so on.
That would explain why regardless of changes it crashes and after a restart, it suddenly does not crash.
So to better debug it, you have to restart the whole game every single time before testing. At least that is my guess.

Same for the netvars thing, maybe they are saved elsewhere and if they suddenly have a different name, the old ones arent used and therefore dont cause a crash. 

That is no solutio to your problem, but may explain this behaviour, although I have no knowledge about it, just a guess.

Edited August 24, 2019 by Serpens

[zarklord_klei]

3 minutes ago, Serpens said:

It may be a conflict with other files put into the game files/modconfigs/client_save and so on.
That would explain why regardless of changes it crashes and after a restart, it suddenly does not crash.
So to better debug it, you have to restart the whole game every single time before testing. At least that is my guess.

Same for the netvars thing, maybe they are saved elsewhere and if they suddenly have a different name, the old ones aren't used and therefore don't cause a crash. 

That is no solution to your problem, but may explain this behavior, although I have no knowledge about it, just a guess.

I change netvar names all the time, never have a problem, somewhere your getting a network sync issue, but I'm not finding it, YET.

[Maris]

13 minutes ago, Serpens said:

you have to restart the whole game every single time before testing

I did so.

[Serpens]

ok, I I'm sure if Zarklord is working on it, it is only a matter of time until he finds the solution Great

[Maris]

5 minutes ago, Serpens said:

if Zarklord is working on it, it is only a matter of time until he finds the solution

I hope so!

But I'm not sure if he can reproduce it. It mystically appears and disappears while I change the code.

[zarklord_klei]

Just now, Maris said:

I hope so!

But I'm not sure if he can reproduce it. It mystically appears and disappears while I change the code.

I have a state where it can reproduce the crash, so I'm all good their.

[zarklord_klei]

well, this makes zero sense, I can confirm that everything has been said is true, but for the life of me I have zero clues why changing the netvar name kills just the cookpots, chests still work fine, which... also makes zero sense.
I know why you get the ComponentAction crash, which is that an entity failed to deserialize its net data(though usually thats printed in the log, but I digress), as for why it fails, I have no freaking clue. probably should ask a dev whats going on.

[PeterA]

Sounds like you've got a naming conflict. I'm not super familiar with the inner working of the netvars at the moment, but it's possible that you have a string hash collision with your names. Renaming them until the problem goes away should suffice.

This happens from time to time. feet_wickerbottom_ice had to be renamed to feet_wickerbottom_ice1 because it has a hash collision with some other textures in the game.

[CarlZalph]

37 minutes ago, PeterA said:

Sounds like you've got a naming conflict. I'm not super familiar with the inner working of the netvars at the moment, but it's possible that you have a string hash collision with your names. Renaming them until the problem goes away should suffice.

This happens from time to time. feet_wickerbottom_ice had to be renamed to feet_wickerbottom_ice1 because it has a hash collision with some other textures in the game.

Hmm.

When you get some spare time, could you check if the hash that's used the same as LUA's string hash function found in LUA 5.1.4's lstring.c?

https://stevedonovan.github.io/lua-5.1.4/lstring.c.html#luaS_newlstr

[PeterA]

21 hours ago, CarlZalph said:

When you get some spare time, could you check if the hash that's used the same as LUA's string hash function found in LUA 5.1.4's lstring.c?

No, it's not the same hash function. Here's our current string hash function.

uint32_t Util::Hash(const char *str)
{
	const char * striter = str;
	uint32_t hash = 0;
	int c;
	while ((c = *striter++) != 0)
	{
		hash = tolower(c) + (hash << 6) + (hash << 16) - hash;
	}
	return hash;
}

There used to be a bunch of different hash functions in our codebase (as is typical with older code bases), but I believe we've cleaned them up and only use this one. Lua has it's own internal hashing though, we haven't modified that as far as I'm aware.
 

[zarklord_klei]

5 hours ago, PeterA said:

No, it's not the same hash function. Here's our current string hash function.

uint32_t Util::Hash(const char *str)
{
	const char * striter = str;
	uint32_t hash = 0;
	int c;
	while ((c = *striter++) != 0)
	{
		hash = tolower(c) + (hash << 6) + (hash << 16) - hash;
	}
	return hash;
}

There used to be a bunch of different hash functions in our codebase (as is typical with older code bases), but I believe we've cleaned them up and only use this one. Lua has it's own internal hashing though, we haven't modified that as far as I'm aware.
 

calling hash("string") inside the global env in lua calls this right?

Edited August 28, 2019 by Zarklord

[Maris]

40 minutes ago, Zarklord said:

calling hash("string")

http://cpp.sh/7hh2x

On 27.08.2019 at 1:39 AM, PeterA said:

Sounds like you've got a naming conflict

I don't think so, because I tried different names.

For example:

local NETVAR_NAME = 'ShowMh_xz1d'; -- hash = 3921860124 (very high), causes crash

If I change one or two characters closer to the end of the name, the hash will also be high. And there will also be a crash.

But:

local NETVAR_NAME = 'a1'; -- hash = 6363152 (very small), all ok, even feature works.

It seems like a higher hash could cause a crash.

Still testing with those three mods.

[zarklord_klei]

18 hours ago, Maris said:

http://cpp.sh/7hh2x

I don't think so, because I tried different names.

For example:

local NETVAR_NAME = 'ShowMh_xz1d'; -- hash = 3921860124 (very high), causes crash

If I change one or two characters closer to the end of the name, the hash will also be high. And there will also be a crash.

But:

local NETVAR_NAME = 'a1'; -- hash = 6363152 (very small), all ok, even feature works.

It seems like a higher hash could cause a crash.

Still testing with those three mods.

he didn't mean internal naming conflict, he meant external, IE in another netvar.

[CarlZalph]

1 hour ago, Zarklord said:

he didn't mean internal naming conflict, he meant external, IE in another netvar.

From the skin name having a conflict with another texture implies that all networked strings go through this function and are put into one lookup table, 'netvar' or just a plain networked string.

I suppose a way to test this is to grab all strings from the lua files and run them through the hash function and see which ones conflicts with "feet_wickerbottom_ice".  Assuming the collision still exists~

[PeterA]

I believe the collision is with other netvars in the same entity. The collision I had with the skinname was in the texture manager if I recall.

21 hours ago, Zarklord said:

calling hash("string") inside the global env in lua calls this right?

I believe that's correct.

[zarklord_klei]

1 hour ago, PeterA said:

I believe the collision is with other netvars in the same entity. The collision I had with the skinname was in the texture manager if I recall.

I believe that's correct.

I actually verified this myself, but forgot to say so, thanks anyway!