Question for the Devs: Sharing the Results of Decompiling SimDLL?

[Zistack]

In another thread, I expressed interest in decompiling SimDLL in an attempt to really understand fluid and gas mechanics, and other people on the forums showed some real interest.  In particular, one user who has professional experience with reverse-engineering has privately offered to help me.  Along with my own experience with C/C++ and compilers, I suspect that it will only be a matter of time before SimDLL's secrets are spilled.

What I want to discuss is primarily what the long-term results of this effort might look like.  For various reasons, I'd like to be able to share the results of this work with the ONI community.  Besides being a handy reference for the exact simulation behaviors in the game, knowing how things are put together can also have value for modders.  In principle, if all of the behaviors were known and the interfaces were documented, a mod could distribute a modified SimDLL with different or new behaviors.  This effort could also allow the ONI community to assist in tracking down bugs and proposing fixes for them.

The issue, I think, is that I don't know how Klei feels about a public distribution of decompiled source code for parts of their game.  There's no real need to distribute decompiled source code for the C# code as it generally comes out of the decompiler in human-readable form with no additional effort.  This isn't the case with SimDLL, which isn't human-readable without significant manual effort that most people don't have the requisite skills to perform.  The only way that most users would be able to get anything meaningful out of it is by relying on the efforts of others.

So, my question to Klei is: What level of sharing are you comfortable with?  The optimal case for the purpose of collaboration would be to make the repository containing the decompiliation effort public, so that anybody could contribute or access the information if they wanted to.  The primary issue is that it would also make a version of SimDLL's source code public.  The decompilation repository could be private, but then people have to ask to be contributors, and modders don't have free access to the information that is represented by the source code.  All the information that outsiders have would have to pass through someone who has access to the code.  I am unsure what other compromises could be made, but am open to ideas.

[mathmanican]

This seems to explicitly be described as not allowed in their terms of service. I'm not a lawyer though, and I have no clue what @JoeW would say.

Spoiler

1. LIMITED LICENCE
Subject to this Agreement, we grant you a limited, non-exclusive, non-transferable, non-sublicensable, limited right and licence to access and use the Game for your personal non-commercial gameplay.
You may download information, images, artwork, text, video, audio, pictures and other materials from the Game to the extent necessary to access and use the Game for the purposes of this Agreement.  However, you must not otherwise copy, reproduce, republish, post, transmit, display, perform, distribute, modify, translate, reverse-engineer, reverse-compile or decompile, disassemble or create any derivative work from any of those materials or any other portion of the Game without our prior written approval.
We, and our licensors, reserve all rights not specifically granted to you in this Agreement.

 

[Zistack]

The key phrase there is "without prior written approval".  They've already granted that for decompiling the C# code, which is the only reason that modding thus far isn't against the TOS.

[Gurgel]

45 minutes ago, Zistack said:

The key phrase there is "without prior written approval".  They've already granted that for decompiling the C# code, which is the only reason that modding thus far isn't against the TOS.

Agreed. You need to ask them and refrain from doing it until they have made a statement about that to you. Well, under EU law at least you are probably allowed to do it, but not to share it. In any case, the right thing to do is to ask.

[Cairath]

On 1/22/2021 at 12:03 AM, Zistack said:

They've already granted that for decompiling the C# code, which is the only reason that modding thus far isn't against the TOS.

Nobody has gotten permission to post entire decompiled assembly and when we see people in the modding discord who are doing that we advise them to take it down. There hasn't been any official "you can post decompiled sources" anywhere. I wouldn't expect that for legal reasons. They'd be opening themselves to people abusing that if it was ever officially permitted.

[melquiades]

I will ask you because of my sheer ignorance in the subject.

What problem would pose for moders and the general public to know the source code?

Is there any posibility of theft by another game developer?

I know that Rimworld even had a decompiler made specifically for it. Anybody can decompile that game.

[Zistack]

@Cairath: To be clear, you're just saying that they haven't stated anything about posting decompiled sources outside of the TOS, right?  Has anyone actually asked this question and gotten a response from the devs one way or the other before?

@melquiades: I think the concern is largely theft.  The scenario would be that I share the human-readable decompiled source of SimDLL, and then someone comes along and uses it as a reference for producing a similar library with similar functionality, and then they produce a similar game using that library.  They avoid doing a lot of the problem-solving work to figure out how to implement such functionality, but their library is technically a distinct implementation which Klei doesn't have any rights over, so they can profit at Klei's expense.  Except that most of the game isn't in SimDLL, and even if it was, It would actually be impractical for a bad actor to use this for any meaningful gain, since to steal Klei's ideas legally would require essentially just as much effort as coming up with them from scratch would.  In practice, a bad actor would just copy the code directly if they were gonna bother stealing anything, and that would be straight-up illegal no matter what (plus there are already easier ways to do that).  I also don't expect that there are any particularly valuable trade secrets hiding in SimDLL, or I wouldn't even consider this project.

Suppose that I instead provided some sort of database to use with ghidra that would, together with the source code, allow someone to produce something that is at least vaguely human-readable without much effort?  Then I would not be directly distributing source code, at least.

Still, this wouldn't really be adequate in the case of modding.  The ideal scenario for mods would be for there to be an open-source implementation of SimDLL that modders could just change.  Even with annotations, you won't get source code out of Ghidra that is easy to modify.  I could theoretically produce one using what I've learned by decompiling SimDLL, but doing so would require Klei's approval - hence the thread.

[gabberworld]

my question is why you need mod SimDLL, as noone needed this before why suddenly you need that?

[Zistack]

If you wanted to change the way that state transitions worked, you would need to mod SimDLL.  If you wanted to implement mixed gasses or fluids, you would need to mod SimDLL.  If you wanted to change how germs worked or make them more complicated, you'd probably need to mod SimDLL.

If you wanted to write a mod like Minecraft's optifine, you'd almost certainly need to mod SimDLL.

And, of course, if you wanted to write an unofficial patch for all the simulation bugs, you'd need to mod SimDLL - though this is a situation we'd like to avoid, if possible.

[FIXBUGFIXBUGFIX]

13 hours ago, gabberworld said:

my question is why you need mod SimDLL, as noone needed this before why suddenly you need that?

I needed it. I spent 3 months on finding a way to mix different gas with different molar mass, and finally I made a rubbish. I don't want to modify simdll, but if I know how this black box works, I won't waste so much time.

But I know that asking Klei to share simdll code is impossible and stupid, so I never mentioned it.

 

BTW, I need to raise a question. Assuming Klei share the code of Simdll, then how to make a mod? Writing a new Simdll and uploading it is a bad idea. If two modders upload two branches of Simdll, the only one mod will make effect.

We need a way to patch code, not rewrite.

[gabberworld]

2 minutes ago, FIXBUGFIXBUGFIX said:

I needed it. I spent 3 months on finding a way to mix different gas with different molar mass, and finally I made a rubbish. I don't want to modify simdll, but if I know how this black box works, I won't waste so much time.

But I know that asking Klei to share simdll code is impossible and stupid, so I never mentioned it.

 

BTW, I need to raise a question. Assuming Klei share the code of Simdll, then how to make a mod? Writing a new Simdll and uploading it is a bad idea. If two modders upload two branches of Simdll, the only one mod will make effect.

We need a way to patch code, not rewrite.

 

there is methods like its done inside  harmony self but it may take soo much your time that i think myself it not worth time.

also when they change Simdll you need start over. mod inside c# its much easier than go inside c++

 

[Zistack]

12 hours ago, FIXBUGFIXBUGFIX said:

BTW, I need to raise a question. Assuming Klei share the code of Simdll, then how to make a mod? Writing a new Simdll and uploading it is a bad idea. If two modders upload two branches of Simdll, the only one mod will make effect.

We need a way to patch code, not rewrite.

That is a really good question.  I have thought about this, but I'm not totally sure what the right answer is.  My guess is that we might need a re-implementation of the library to use as a base for modders that is designed specifically to allow multiple mods to co-exist.  They would mod the re-implementation rather than the original.

I presume that SimDLL exists because their C++ code runs significantly faster than their C# code, and they're using it as a performance optimization.  If I had more information about their use (or lack thereof) of concurrency in the game, it would be easier for me to estimate the viability of replacing SimDLL with an equivalent implementation in C#, allowing for modding via Harmony.  If they're not using concurrency all that much, then it might actually be reasonable to add it in, giving the game enough of a performance boost that we can afford the penalty incurred by using a slower language for the core simulation code.

Right now, I don't have that information, and I seriously wonder about the viability of replacing it with a C# library.  There might be some tricks that could (in C/C++) be done by playing games with dynamic loading and function pointer assignment (we could put a relatively nice interface on this), which could get you method replacement capabilities sortof similar to that of Harmony.

[Cairath]

On 1/30/2021 at 9:33 PM, Zistack said:

To be clear, you're just saying that they haven't stated anything about posting decompiled sources outside of the TOS, right?  Has anyone actually asked this question and gotten a response from the devs one way or the other before?

That was a comment to your statement: "The key phrase there is "without prior written approval".  They've already granted that for decompiling the C# code, which is the only reason that modding thus far isn't against the TOS."

We haven't received any written "it's okay to publish decompiled code online". We obviously need to decompile it for modding, and nobody's going to go after people for posting snippets in relation to modding. Nobody asked whether we can post the entire code online, we've always assumed that to be not okay.

Replacing simdll with c# lib would probably hurt performance a lot, and allowing people to mod that would most likely have tragic effects - most modders are absolute newbies, and modifying such critical code would lead to people unknowingly installing mods that butcher their performance even further.

[he77789]

Couldn't we reverse engineer SimDLL's interface from how the C# part calls it, then write an unofficial C# replacement?

They could have a setting tucked away that tells you about the possible performance drop when you change it.

[RageLeague]

I thought the TOS is just standard anti-piracy stuff for software. This statement probably exist in every software TOS. It probably just means you can't just decompile everything and share it to others so they don't have to buy oni. If you only decompile what you need like the fluid dynamics you might be ok?

I am also not a lawyer, so I don't know what I'm talking about.

[gabberworld]

you are allowed make mods, like you see developers even support that by added mod support to the game, but you not allowed share they code in public. soo if you need mod Simdll for your own needs then go ahead

[he77789]

2 hours ago, gabberworld said:

developers even support that by added mod support to the game

The current mod support doesn't include SimDLL afaik.

[gabberworld]

2 hours ago, he77789 said:

The current mod support doesn't include SimDLL afaik.

in support i mean they added harmony what makes easier  inject/hook dlls to game, including the simple mod button at main screen with mod menu.

not every game makes that. if you talk about c# what they use then that is not support, its unity default code, you can mod almost any game this way what made in unity

anyway what it means they accept that we peek they code when make mods but you still not allowed share they code at public

[cpy]

If you can make better sim.dll, i'd love to see better physics. Modded physics, sign me up!