So, ToolsNotIncluded is gone for good...

[Commander Steps]

EDIT: On hold right now due to Cairath announcement of TNI coming back.

---

What's next?

I deeply miss that invaluable service, and as of now, there appears to be no adequate replacement. Based on my readings on Reddit, Cairath has declined to share even the seed database, making it impossible for anyone to seamlessly take over and continue the service.

Motivated by this, I delved into researching what it would take to construct a replacement, focusing primarily on features essential for discovering desired world traits and geysers – aspects that hold particular significance for me.

Regrettably, Cairath has not made any part of the project open source. However, RoboPhred's Duplicity (Open Source, MIT licensed) still functions with current save game files, allowing for the parsing of save games to build a new database.

Establishing a service to collect save games poses no challenge. This time, I would design it as a public archive under a public domain license, ensuring the community's collaborative efforts are never lost, as I've read happened with the loss of 50k seeds.

Creating a web app for searching the database, similar to toolsnotincluded.net, should be a straightforward task.

Now, let's address the challenges I foresee:

• I haven't yet identified where the world traits are stored in the file. Can someone provide a hint?
• A "map miner" mod is missing. The accompanying mod for ToolsNotIncluded created new maps, revealing and saving them to mine the seeds. Doing this manually is tedious. I don't currently see such a mod, and I'm unsure how to create it. Does one exist, and is it relatively easy to develop?
• Geysers must first be revealed or will be missing in the save game. As far as I can see, the "gameObjects" only list geysers that have been revealed. Is there a way to access them without revelation? Does the file contain information allowing us to calculate all geyser positions or at least identify the types of geysers present?
• Only starting planet geysers are listed. I switched to other planets in the DLC and saved, but the oni-save-parser consistently reveals only the starting planet's geysers. Does the oni-save-parser overlook something? From my observation, everything is interpreted without "simdata." Does anyone know what it contains?

I am willing to invest my spare time in resurrecting a seed browser for the community. I can develop a seed browser app, parse the seeds and set up a database.
However, I lack experience in creating ONI mods and am unfamiliar with reverse engineering the game to uncover the missing elements in the save game.

All my work will be licensed under the GNU Affero General Public License to prevent another incident where the community loses a valuable tool, as witnessed with ToolsNotIncluded.

What are your thoughts? Should we endeavor to create a new seed browser?

[SGT_Imalas]

1 hour ago, Commander Steps said:

Cairath has declined to share even the seed database

the data was lost, there is nothing left to share

1 hour ago, Commander Steps said:

A "map miner" mod [...] Does one exist

the TNI collection mod still exists and it is open source: https://steamcommunity.com/sharedfiles/filedetails/?id=1825374565

1 hour ago, Commander Steps said:

I haven't yet identified where the world traits are stored in the file. Can someone provide a hint?

they are attached to the worldcontainer gameobject for each asteroid, m_worldTraitIds and m_storyTraitIds

1 hour ago, Commander Steps said:

Geysers must first be revealed or will be missing in the save game.[...] Does the file contain information allowing us to calculate all geyser positions or at least identify the types of geysers present?

geysers that are not yet revealed have a placeholder sitting in their place that spawns them on reveal and deletes itself, geysers that are not preselected but instead fully random use a "GeyserGeneric" placeholder. these are picked on reveal based on the "geyserGeneric"-placeholders x and y position and the world seed

 

1 hour ago, Commander Steps said:

parsing of save games to build a new database

BIG issue here:

mods can affect worldgen and you have no way of tracing what mods were present at worldgen.

WGSM casually breaks any seed compatibility because it simply replaces most parts of the worldgen and you have no way of detecting it.

Any mod that adds worldtraits will shuffle the trait list with that, resulting in an altered worldgen due to different seeds getting selected instead.

positions of pois could have been moved via debug or mod, or there could have been some added/removed via sandbox tools (you cant detect that either if the person choose to disable sandbox afterwards)

 

[Commander Steps]

 

21 minutes ago, SGT_Imalas said:

the data was lost, there is nothing left to share

Without any backups? That's a pity and a bit hard to believe to be honest.
I read about this:

However, whether Cairath truly lacks any data and backups or simply chooses not to share, it may not be of consequence. We find ourselves without access to it. Our only recourse now is to derive lessons from this experience and endeavor to establish a public database the next time.

21 minutes ago, SGT_Imalas said:

the TNI collection mod still exists and it is open source: https://steamcommunity.com/sharedfiles/filedetails/?id=1825374565

 

It appears that it is not open source. Although the ONI-Mods repository hosts numerous mods, this particular one is not among them.

However, that's not what I meant. The linked mod pertains to manually initiating a map and uploading it—an approach aimed at gathering community maps.
I am in search of the tool that is not publicly available and is only referenced:

 

Quote

Advanced users can also contribute using an automatic mod, "map miner", available on the tool's website.

I haven't utilized it myself, but my understanding is that it's an automated tool designed to generate savegames for parsing purposes.
Unfortunately, it is no longer accessible.

29 minutes ago, SGT_Imalas said:

they are attached to the worldcontainer gameobject for each asteroid, m_worldTraitIds and m_storyTraitIds

 

Thank you, I will take a look.

29 minutes ago, SGT_Imalas said:

geysers that are not yet revealed have a placeholder sitting in their place that spawns them on reveal and deletes itself, geysers that are not preselected but instead fully random use a "GeyserGeneric" placeholder. these are picked on reveal based on the "geyserGeneric"-placeholders x and y position and the world seed

Alright, without disclosing the map, the types of geysers remain unknown. Therefore, it seems that the TNI map uploading tool needs to perform a map reveal before initiating the upload. However, this crucial step is not explicitly mentioned in the instructions.

I still wonder how the TNI seed browser managed to scan the whole cluster. The information must be part of the save game, but RoboPhred did not manage to extract it as far as I can tell. And I don't expect Cairath to share the savegame parser logic used by TNI.

31 minutes ago, SGT_Imalas said:

BIG issue here:

mods can affect worldgen and you have no way of tracing what mods were present at worldgen.

WGSM casually breaks any seed compatibility because it simply replaces most parts of the worldgen and you have no way of detecting it.

Any mod that adds worldtraits will shuffle the trait list with that, resulting in an altered worldgen due to different seeds getting selected instead.

positions of pois could have been moved via debug or mod, or there could have been some added/removed via sandbox tools (you cant detect that either if the person choose to disable sandbox afterwards)

Yes, it's indeed a challenge, but it's a problem that has been addressed in the past, as outlined in the Contribution page instructions. Community contributions have historically proven effective, given the existence of a substantial database.

As for malicious intent, it's difficult to determine how many individuals knowingly submitted harmful save games to troll others. However, I presume that with good faith, successful contributions were feasible, as evidenced by previous instances

Do I miss something?

What is "WGSM"?

I'm interested in hearing your candid opinion:
Do you believe that replacing a tool like the TNI seed browser isn't worth the effort?

 

[SGT_Imalas]

wgsm is an old mod by ony that in theory allows modifying world traits.

like a lot of onys mods, it introduces bugs and issues in the process, namely, due to poor implementation, it completely breaks world gen, resulting in seeds generating different map layouts when the mod is installed

the problem with that is that a lot of invalid/wrong data is potentially introduced without harmful intent, just by the fact that person unknowingly broke seed compatibility by having the mod installed

3 hours ago, Commander Steps said:

However, that's not what I meant. The linked mod pertains to manually initiating a map and uploading it—an approach aimed at gathering community maps.

both mods use the same base functions, one could easily automate the process based on the manual mod.

I browsed that mods code recently, I just assumed it was on github-it wasnt, I decompiled the mod from the workshop, so thats atleast an option

As for 'if its worth the effort'

-

not sure yet. the 'big features' it had a monopoly on was Geyser selection, and asteroid preview everything else is pretty much covered by mods nowadays.

even geysers can be adjusted with customizeGeyser by fumihiko, but simply preselecting it is a lot more easy to use.

for players that dont want to hassle with mods, it might be worth the effort, especially with a publicly available database.

 

[Commander Steps]

1 hour ago, SGT_Imalas said:

both mods use the same base functions, one could easily automate the process based on the manual mod.

I browsed that mods code recently, I just assumed it was on github-it wasnt, I decompiled the mod from the workshop, so thats atleast an option

I'm unsure about the process of decompiling a mod and the utility of the outcome.

If you've examined the code, do you have any insights into where it might be hosted?

Assuming we're unable to locate the source code, how should we approach a mod that initiates new games automatically, reveals the entire map, places a duplicate on each planet (potentially needed to cover all asteroids, as far as I know), saves the game under its seed name and repeats the process? Is this a complex task or can a skilled modder accomplish it relatively quickly?

It's conceivable that individuals who are meticulous about having only this specific mod installed might use such a tool to contribute data.

Furthermore, there may already be individuals with well-established savegames from their initial contributions, and we could simply consolidate them in a centralized public space. Perhaps you have some of these saves yourself?
 

1 hour ago, SGT_Imalas said:

not sure yet. the 'big features' it had a monopoly on was Geyser selection

I'm seeking to replace only that specific element and the world traits. I won't have the capacity to recreate the entire thing during my free time. It's a substantial application that was also supported through Patreon.
 

1 hour ago, SGT_Imalas said:

even geysers can be adjusted with customizeGeyser by fumihiko, but simply preselecting it is a lot more easy to use.

Sure, I prefer playing without any mods or the sandbox mode. My approach involves exploring naturally occurring and interesting combinations within the game.

 

 

[Commander Steps]

I'm thrilled to report some positive developments here!

Upon delving deeper into the save game, I discovered that all the necessary information is already present in the initial savegame that is automatically generated. This means that the map miner mod has a much simpler task to automate.

There exists a game object named "SaveGame", housing a behavior named "WorldGenSpawner". This behavior falls under the type "UserDefined", signifying it as a Map/Dictionary. Among its keys are, among others, "id", "location_x", and "location_y".

Examples:

GeyserGeneric_hot_hydrogen at x=96 y=24
GeyserGeneric_salt_water at x=241 y=42
GeyserGeneric_slush_water at x=270 y=36
GeyserGeneric_filthy_water at x=243 y=72
GeyserGeneric_hot_steam at x=259 y=32
GeyserGeneric_methane at x=375 y=66
GeyserGeneric_molten_gold at x=362 y=55
GeyserGeneric_molten_cobalt at x=409 y=38
GeyserGeneric_molten_cobalt at x=448 y=25
[...]

As evident from the high coordinates, these encompass all geysers across all asteroids.

The list is reset when the objects are revealed, necessitating a fresh savegame. This clarifies the requirement in the Contribution manual for ToolsNotIncluded, which specifies a savegame no older than 60 seconds.

The bounds for asteroids are also in the WorldContainer behaviour as a combination of worldSize & worldOffset.

Now I only need a way to quickly get a lot of save games to process.

[Commander Steps]

13 hours ago, SGT_Imalas said:

geysers that are not yet revealed have a placeholder sitting in their place that spawns them on reveal and deletes itself, geysers that are not preselected but instead fully random use a "GeyserGeneric" placeholder. these are picked on reveal based on the "geyserGeneric"-placeholders x and y position and the world seed

Alright, I understand now.

I spent the entire day working with four save game files to test my parser, and they were perfectly populated from the very first save game, as demonstrated above.

Upon testing with a fifth file, I encountered the following:

[...]
GeyserGeneric_steam, locationX=221, locationY=86
GeyserGeneric_chlorine_gas, locationX=76, locationY=199
GeyserGeneric_methane, locationX=113, locationY=253
GeyserGeneric, locationX=40, locationY=214
GeyserGeneric, locationX=94, locationY=256
GeyserGeneric, locationX=23, locationY=250
[...]

It appears that the determination of whether all GeyserGeneric instances are decided is somewhat random, which is quite frustrating.

Do you know if the map upload build created by Cairath addressed this issue or was there a possibility that certain maps couldn't be uploaded?

Having access to the source code of the original TNI map sharing mod at this point would be beneficial.
 

[SGT_Imalas]

2 hours ago, Commander Steps said:

It appears that the determination of whether all GeyserGeneric instances are decided is somewhat random, which is quite frustrating.

"these are picked on reveal based on the "geyserGeneric"-placeholders x and y position and the world seed"

this is what I meant with that; the "GeyserGeneric" is the placeholder for a full random geyser (opposed to those that have a name, ie. "GeyserGeneric_steam"  - those are predetermined in the worldgen file). On reveal, it takes a list of all available geysers that are marked as "can appear as generic geyser" and picks one at random. that random is seeded, based on a combination of x and y coordinates and worldseed (hence calculatable).

the randoms initializer is

new KRandom(globalWorldSeed + posX + posY).Next(0, configs.Count)

KRandom is a klei random class thats accessible in the source code,
"configs" is the list of potential generic geyser types.
these are (in that order):

generic geysers in base game:

steam
hot_steam
hot_water
slush_water
filthy_water
slush_salt_water
salt_water
small_volcano
big_volcano
liquid_co2
hot_co2
hot_hydrogen
hot_po2
slimy_po2
chlorine_gas
methane
molten_copper
molten_iron
molten_gold
oil_drip

generic geysers in dlc:

steam
hot_steam
hot_water
slush_water
filthy_water
slush_salt_water
salt_water
small_volcano
big_volcano
liquid_co2
hot_co2
hot_hydrogen
hot_po2
slimy_po2
chlorine_gas
methane
molten_copper
molten_iron
molten_gold
molten_aluminum
molten_cobalt
oil_drip
liquid_sulfur

 

[Commander Steps]

Okay, got ya. I find this design choice a bit strange that some are random, but not really since it's dependend on the seed.

If I import the DLLs, can I utilize the KRandom from C# code? Unless the algorithm is open source or well-documented, enabling me to port it to Kotlin, this may not be a viable solution.

So, I presume I still require a map miner mod that reveals the entire map to ensure the initialization of all geysers. Is that correct?

I'm still uncertain if this is programmatically achievable and whether someone would take on the task of crafting such a solution.

[SGT_Imalas]

1 hour ago, Commander Steps said:

If I import the DLLs, can I utilize the KRandom from C# code?

yes.

and no, you dont have to reveal the map, as the WorldGenSpawner data is there for the beginning, you just have to calculate what type these fully random geysers would become on reveal (by mirroring the logic thats used by the game which I mentioned above)

since we work with raw, decompiled gamecode here, there is no documentation, but the implementation of the class isnt that complex and it shouldnt be that difficult to translate the logic to other languages

[Commander Steps]

I just followed the modding tutorial published by Cairath on GitHub to make my first Hello World mod to check if the KRandom might happen to have the same implementation/result as JAVA Random... and of course it's different.

7 minutes ago, SGT_Imalas said:

since we work with raw, decompiled gamecode here, there is no documentation, but the implementation of the class isnt that complex and it shouldnt be that difficult to translate the logic to other languages

As I'm absolutely not familiar with Visual Studio and the the decompiling stuff, do you think you could reveal the implementation of KRandom() for me?

I hope it's less complex than writing a mod to reveal everything - which would indeed be the lesser attractive solution to this.

[SGT_Imalas]

sharing raw source code is probably not the best thing to do on a public forum

you can hit f12 with the keyboardcursor somewhere inside a class call (if you use VS2022) to get to the implementation

so you write "new KRandom()" somewhere inside your mod, click somewhere within bounds of the classname so the cursor sits inside and then hit f12

if you are on visual studio 2022, it will decompile the referenced dll and open up a new tab that contains the full KRandom class

[DevJerry]

Hey y'all!

I'd love to help out, either maintaining or building a new TNI.

Never wrote a mod but I had c# in school.

Anyways I'm eager to dive in and get my hands dirt.

A bit about myself I'm a tech lead with 14+ years of engineering experience.

Tech stack I'm most familiar with is php, js/ts, python, java & go. Most familiar DevOps stack is aws, gcp docker, tf & github.

So what happened @Cairath ? You still working on this?

I get that time is precious and it takes a lot to maintain a project like that, are you willing to share the code ? Or alternatively a small writeup about the inner workings?

In any case thank you for all your efforts, would love to see it continue!

I'll be checking out a few mods to get more familiar with harmony's usage.

Would love a high level overview of how TNI actl worked under the hood. If anyone could jump in or has more info @SGT_Imalas?

Best

Jerry

 

 

[SGT_Imalas]

Cairath said on the oni discord that TNI is coming back online at the end of january.

She also never made any of the code public, but in the end, atleast on the data side of things, its "just" a lot of world information in a database that has been collected by running a lot of worldgen with a (closed source) collection mod to then collect data about that seed.

TNI website then had that database at its disposal and ran queries based on user input.

I wrote a small mod that can autonomously collect similar data from worldgen, its just not hooked up to any server and instead writes it to json files (you can find the source of that here: https://github.com/Sgt-Imalas/Sgt_Imalas-Oni-Mods/tree/master/_WorldGenStateCapture)

so if TNI would be down for good, the data collection side of a successor could be done by that mod (allthough that should probably be updated to a) use a server api instead of json dumps and b) store the world biome info as curves instead of a bitmap)

if you want to try the current iteration of it: _WorldGenStateCapture_dev.zip
 

[Commander Steps]

@DevJerry Excited to learn that you're interested in contributing!

I've been developing a replacement service following the information shared by SGT_Imalas about TNI's potential return.

My current progress involves creating an ONI save game parser in Kotlin to extract necessary information from savegames.

However, I've encountered two challenges: Savegames are approximately 1 MB each, potentially leading to a substantial network load, and they lack details on geyser emit rates and active periods. While the savegames indicate the geyser locations before being revealed, they don't provide specifics on emit rates and activity periods. These values might be calculable based on the seed, though the formula remains unknown to me.

Moving forward, it seems more practical to enhance the WorldGenStateCapture mod, enabling it to run automatically and submit JSON data to a web API (POST request). This approach is more efficient than collecting and uploading gigabytes of savefiles through a web form for analysis.

I've transformed my progress into an online save game viewer, accessible at https://stefan-oltmann.de/oni-save-parser/.
The source code is available on GitHub and linked at the bottom of the page.

(In fact, I was already one step further by verifying the save game's validity (such as being less than 60 seconds old, at cycle 0, and free of installed mods), and subsequently uploading the extracted information into a Supabase database. However, at that point, I had not yet developed a GUI for the Seed Browsers frontend.)

At this point, I'm adopting a wait-and-see approach regarding TNI's return. If it happens, no immediate action is necessary.
However, if TNI doesn't return, we have a solid foundation to resume work on the replacement project.

[AYes]

Hi! Any news regarding the project?

[SGT_Imalas]

Cairath plans on getting the site back online towards the end of january

[AYes]

On 1/23/2024 at 11:11 PM, SGT_Imalas said:

Cairath plans on getting the site back online towards the end of january

17 February

[Commander Steps]

On 2/17/2024 at 12:18 PM, AYes said:

17 February

Welcome to the world of software development where everything takes forever.

I wait for the final death note until I take any action. As long as she is willing to bring TNI back I will wait.

[Jorge Junior]

@SGT_Imalas Can't you do this? ToolsIncluded.

[SGT_Imalas]

I can in theory, but I wont

On 2/24/2024 at 11:48 AM, Commander Steps said:

As long as she is willing to bring TNI back I will wait.

^

[AYes]

On 2/25/2024 at 4:09 PM, SGT_Imalas said:

I can in theory, but I wont

On 2/24/2024 at 1:48 PM, Commander Steps said:

The process has dragged on and threatens to become eternal!

[Commander Steps]

8 hours ago, AYes said:

The process has dragged on and threatens to become eternal!

Software development.

I guess it's not her priority number one.

[AYes]

On 3/6/2024 at 8:29 PM, Commander Steps said:

Software development.

I guess it's not her priority number one.

I don’t like it…

[Commander Steps]

1 hour ago, AYes said:

I don’t like it…

You can go on the Discord and ask her about the progress.