Best of both worlds: less RAM and less lag?

[Wumpus the 19th]

i am multiple things that means i really don't like screwing with things like this.

i don't know what this "win" folder is, nor can i find where in god's name is a bin in my folders.

i keep on going down klei->don'tstarvetogether and i don't see where the bin is.

it's very likely this is a process most people could do in five minutes, but i don't know anything about this and am already paranoid about god damn anything that touches my computer.

[hyiltiz]

Huh, so WINMM.dll is overloading stuff from %system32% I geuss? 

34 minutes ago, PaintDream said:

In fact you needn't to make a backup of original binaries files (but still recommend to backup your save slots). DontStarveLuaJIT does not replace any existing binary files

Your uninstallation steps are correct and it can be much simpler: just rename the WINMM.dll with aother name if you want to disable/uninstall it.

 

[Wumpus the 19th]

i know i'm asking a lot of questions but...okay i don't really have an excuse for that part

i found where the bin thing was, although the way file explorer displayed it was different slightly, and pasted in the win folder from the download.

it says you have to type into the dev console print(_VERSION), and after making sure that was what i typed in, the bottom of the dev log just says lua 5.1

is that a good thing? please by the name of every greek god shown in percy jackson that's a good thing.

[hyiltiz]

5 minutes ago, Wumpus the 19th said:

i know i'm asking a lot of questions but...okay i don't really have an excuse for that part

i found where the bin thing was, although the way file explorer displayed it was different slightly, and pasted in the win folder from the download.

it says you have to type into the dev console print(_VERSION), and after making sure that was what i typed in, the bottom of the dev log just says lua 5.1

is that a good thing? please by the name of every greek god shown in percy jackson that's a good thing.

Wonderful! Glad to see non-programmer players can also install it. Feel free to point out any troubles you had, since other players may also stumble upon them.

It should say LuaJIT, not lua 5.1 if your installation succeeded. I am guessing you may have put the folder "release" or "win" along with its contents into that "bin" folder. Instead, only put the *contents* (the three .dll files) of that "release" folder into "bin". 

[Wumpus the 19th]

5 minutes ago, hyiltiz said:

Wonderful! Glad to see non-programmer players can also install it. Feel free to point out any troubles you had, since other players may also stumble upon them.

It should say LuaJIT, not lua 5.1 if your installation succeeded. I am guessing you may have put the folder "release" or "win" along with its contents into that "bin" folder. Instead, only put the *contents* (the three .dll files) of that "release" folder into "bin". 

 

first of all, you are giving me way too much credit saying that i was able to install it all on my own. 

second of all, i placed all the folder's contents individually into the folder, punched in the command, and now it spout out the luajit thing, and flat out said paint dream in the line of code.

either it's some virus that's slowly but surely infecting my game, or it's working like a charm, one that i had no clue how to use until you helped me

as thanks for helping me, here's a really cute pokemon video

Spoiler

 

 

[zVince]

I didn't quite understand. Is that already accessible to us? 

[Yuuko]

3 hours ago, Charlie Dark said:

Hi there person with a wonderful profile picture, and thanks for letting us know about this!, And I would recommend that you close any browsers while playing... especially Chrome! I believe it uses way too much RAM.

So you're saying running a twitch stream in chrome while in a discord voice chat and playing DST might be why I'm lagging...?

I sure hope this guy's mod fixes this for me >_<

[OMGkawaiiAngel]

3 minutes ago, Shosuko said:

So you're saying running a twitch stream in chrome while in a discord voice chat and playing DST might be why I'm lagging...?

I sure hope this guy's mod fixes this for me >_<

The lag is "based" on how much RAM you have, and on your internet connection as well...

[CarlZalph]

1 hour ago, hyiltiz said:

Huh, so WINMM.dll is overloading stuff from %system32% I geuss?

By default, Windows will automatically use any dll next to an executable in a folder before looking to core system files.

I personally use DInput8.dll for when I want to modify things, as it is really easy to create a proxy dll for as it has a few functions to contend with, and a lot of games use DirectX.  A proxy dll has all of the same function names as the original, and usually calls the core system's functions, but it also calls its custom code along with it and has full access to the parent process' memory.  Further, a dll can be thought of as a box of functions where the host process may call them after the dll loads.

This ultimately lets a user 'mod' a process by editing the parent process' memory to do work via things called signature scanning and detouring (finding and replacing functions).

 

Since DST has no anticheat, this is perfectly fine to do.  I would not recommend doing this in games that use anticheats as any sort of proxy dll or dll injection method is easily flagged and more than likely would result in account suspensions or terminations.

[hyiltiz]

12 minutes ago, zVince said:

I didn't quite understand. Is that already accessible to us? 

Yes if you are on Windows or Linux. macOS folks seem to need to wait a bit, or buy more expensive Apple product (as is always the case to solve Apple problems)

[zVince]

49 minutes ago, hyiltiz said:

Wonderful! Glad to see non-programmer players can also install it. Feel free to point out any troubles you had, since other players may also stumble upon them.

Just move those three files to this directory?
C:\Program Files (x86)\Steam\steamapps\common\Don't Starve Together\bin

[hyiltiz]

For Windows, 

4 minutes ago, zVince said:

Just move those three files to this directory?
C:\Program Files (x86)\Steam\steamapps\common\Don't Starve Together\bin

Yes.

[hyiltiz]

43 minutes ago, Portmanteau said:

Would there be any reasons as to why Klei would not want to bring this into the game natively? At the moment this looks like it would just be a straight upgrade so I'm curious.

If this really does what it claims to for little to no disbenefit I'd love it if Klei took the time to implement this into DS and DST, might fix a lot of the performance issues with old computers and Hamlet I'd imagine.

EDIT: Also another question; would DS mod developers need to do anything differently to account for players using this? If so, what would be the easiest way to check for if a player is using this add-on or not so one could keep compatibility for all users?

TL;DR Yes, Klei can (legally and practically) and hopefully would just incorporate this.

Longer answer

Klei can (legally and practically) and hopefully would just incorporate this. Until then, the best we the players can do is to test this out rigorously by playing our normal games with this enabled and see whether/how it would fail. I've learned that it has been undergoing tests in the Chinese community and received many fixes throughout the last 4 years. If it gains enough tests and regarded as "stable" by the players, its developer and finally Klei, we may get this as a major performance upgrade to our beloved game.

Klei haven't used something like this, aka ditch the old lua5.1 and use LuaJIT, for the following two reasons I am *guessing* (ask them for the real reasons):

- It is hard. As the author of the "mod" @PaintDream points out in the original post (link at the top of this thread), simply swapping out the lua5.1 for luajit5.1 didn't work as there were a few dark ugly corners that needed to be fixed, even though LuaJIT is claimed to be fully compatible with lua5.1.

- Unpredictability. Deciding to ditch the "engine" that runs the whole game for a different "engine" that claims full compatibility brings development unpredictability. Say a Klei dev tested this LuaJIT thing out (a long time ago since it was out there since 2010). Some stuff probably didn't work out and they didn't pursue further as there is no telling how deep that rabbit hole would go, while we the players are impatiently waiting for something tangible. Development and research is costly, and it may not result in a benefit in the end. Even if it did, there may be unforeseen/untested cases where it may fail and introduce new bugs, which Klei would hate to see. So they couldn't release such an upgrade without extensive testing.

What about our mods?

For mod authors, there is nothing they'll need to do. This "mod" already tries to fix a few common programming mistakes mod authors have done which resulted in issues and fixed it before the game loads those mods with issues. Most of the common mods now work just fine with no changes at all.

As for recommended practices for mod authors, I think @PaintDream can specify better than I can. What I recall is that: a) mod authors should use proper string formatting and not something buggy as \umbrella, which in programming languages mean the unicode "mbrella", which then doesn't exist and would result in luajit since it is more strict than lua5.1 in unicode handling; b) using an undefined variable called arg, which can be fixed by defining it first before using it. Both of these issues have already been fixed in this "mod".

 

[Toros]

I’ve been aware of this project for years and I’m really happy to see it coming to a stable place.

It’s hard to argue against even the conservative performance improvements though.  Pretty fantastic work.

[WhatTheCabbage]

Looks real good. Do I need to enter the command each time I launch DST or am I good for now

[hyiltiz]

17 minutes ago, TopNoob said:

Looks real good. Do I need to enter the command each time I launch DST or am I good for now

No, that command is just there to show you that it worked. Once it works, there is no need do anything else. Just play your game and enjoy!

[WhatTheCabbage]

Ah, thanks. Looking forward to see the results 

[PaintDream]

6 hours ago, CarlZalph said:

By default, Windows will automatically use any dll next to an executable in a folder before looking to core system files.

I personally use DInput8.dll for when I want to modify things, as it is really easy to create a proxy dll for as it has a few functions to contend with, and a lot of games use DirectX.  A proxy dll has all of the same function names as the original, and usually calls the core system's functions, but it also calls its custom code along with it and has full access to the parent process' memory.  Further, a dll can be thought of as a box of functions where the host process may call them after the dll loads.

This ultimately lets a user 'mod' a process by editing the parent process' memory to do work via things called signature scanning and detouring (finding and replacing functions).

 

Since DST has no anticheat, this is perfectly fine to do.  I would not recommend doing this in games that use anticheats as any sort of proxy dll or dll injection method is easily flagged and more than likely would result in account suspensions or terminations.

I used DINPUT8.DLL years ago since it is simple for forwarding original apis. But it was not default-loaded by dedicated version of game so I changed to WINMM.DLL.

In fact, this mod doesn't work on Don't Starve published at WeGame platform, since the platform does some anticheat modifications and I have no plan to deal with it. For me, the biggest contribution of this mod is NOT how to inject a dll to game but the modifications for LuaJIT that removes incompatibilities between LuaJIT and Lua, which could be much valuable than hacking techniques for all of us, especially for Klei.

[hyiltiz]

For anyone playing the game in Linux, here are the steps. It turns out this "mod" not only works for the Dedicated Server in Linux, but also for the game itself! Pretty amazing!

(DISCLAIMER: I just found this and haven't tested out extensively. I found the game disabled all the mods, and clicking "Mods" button in game to open the "Mods" screen would crash the game). 

Just tried a workaround and it worked (game starts, and salutes Mike
Pall) for the Linux game itself in Debian testing using
`lib32readline8`. Here are the steps:

1.  Download the `.zip` from Release and copy the three `lib*.so` files
    into `[game]/bin/lib32/`
2.  Install the available `lib32readline` library (first do
    `apt search lib32readline` then install the one available with
    `sudo apt install lib32readline7` or `sudo apt install libreadline8`
    depending on which one is available
3.  Mask as `libreadline.so.6` with
    `sudo ln -s /lib32/libreadline.so.7 /lib/i386-linux-gnu/libreadline.so.6`
    (note that 7 could be 8 if 8 was available and installed in the
    previous step)
4.  Rename `dontstarve_steam` to `dontstarve_steam.bin`
5.  Create a bash script called `dontstarve_steam`:

        #!/bin/sh
        LD_PRELOAD=./lib32/libpreload.so ./dontstarve_steam.bin

6.  Make that script executable with `chmod +x ./dontstarve_steam`
7.  Run the game as normal and enjoy!

 

[Cunning fox]

On 5/19/2020 at 8:07 AM, CarlZalph said:

If by garbage you mean really good, then yes.

LUA is quite a powerhouse for being able to do it all while having integration with C.

It also is a lot easier to have mods interact with the base code since all of the source code is there for you to poke at and manipulate, and with the simple syntax and no memory management on the programmer's part makes it a quick to pick up language.

 

If you want to see how to not make a modding interface, look no further than Bethesda's engine for any of their titles.  Clunky software to make things in, records overriding other mods on things that a mod didn't even change because the data is copied as a chunk rather than an edit, etc.  Once again the modders picked up their slack by making tools to help alleviate the pain that is installing and managing mod lists for any of their games.  Things like BOSS/LOOT to help sort, Wyre Bash to help merge, and TESEdit for manually going in for those got-to-have interaction fixes.  I shudder to think of how many issues those modders went through to make the content they did.

Compare that to Klei's where I've really yet to see any major mod conflict when the mods are done in good practices such as not direct file replacements.

But Lua is pretty basic, isn't it? It has very limited number of features, compairing to other languages. 
Also, I think they chose Lua because they were not planning on making a big game, like DST is. But I may be wrong

[PaintDream]

37 minutes ago, Cunning fox said:

But Lua is pretty basic, isn't it? It has very limited number of features, compairing to other languages. 
Also, I think they chose Lua because they were not planning on making a big game, like DST is. But I may be wrong

Not really. Lua is one of the best script languages for game, since:

1. It's small, ANSI C portable and easy for integrating with C.

2. Its performance is higher than nearly all other script languages without jit.

3. It provides powerful language mechanism such as metatable and coroutine, allowing programmers to write more flexible and asynchronized code.

In game developing, script language is designed to describe a world within a sandbox. Most other languages, such as python, javascript, coupled with too many heavy functions that should be disabled to ensure sandbox safety. 

[PaintDream]

9 hours ago, hyiltiz said:

For anyone playing the game in Linux, here are the steps. It turns out this "mod" not only works for the Dedicated Server in Linux, but also for the game itself! Pretty amazing!

(DISCLAIMER: I just found this and haven't tested out extensively. I found the game disabled all the mods, and clicking "Mods" button in game to open the "Mods" screen would crash the game). 

Just tried a workaround and it worked (game starts, and salutes Mike
Pall) for the Linux game itself in Debian testing using
`lib32readline8`. Here are the steps:

1.  Download the `.zip` from Release and copy the three `lib*.so` files
    into `[game]/bin/lib32/`
2.  Install the available `lib32readline` library (first do
    `apt search lib32readline` then install the one available with
    `sudo apt install lib32readline7` or `sudo apt install libreadline8`
    depending on which one is available
3.  Mask as `libreadline.so.6` with
    `sudo ln -s /lib32/libreadline.so.7 /lib/i386-linux-gnu/libreadline.so.6`
    (note that 7 could be 8 if 8 was available and installed in the
    previous step)
4.  Rename `dontstarve_steam` to `dontstarve_steam.bin`
5.  Create a bash script called `dontstarve_steam`:

        #!/bin/sh
        LD_PRELOAD=./lib32/libpreload.so ./dontstarve_steam.bin

6.  Make that script executable with `chmod +x ./dontstarve_steam`
7.  Run the game as normal and enjoy!

 

Linux DST version fixed. Please download the latest release at: https://github.com/paintdream/DontStarveLuaJIT/releases/tag/2020.5.21

Linux DS remains broken. It seems that machine code matching goes totally wrong.

[zVince]

Hey. Do you have any graph or performance comparison between versions?  

[hyiltiz]

21 minutes ago, zVince said:

Hey. Do you have any graph or performance comparison between versions?  

Hmm, just tested out both the original and the "modded" games in an old Linux computer. Doesn't seem to be much different. If anything, the "modded" one is heavily dropping more frames. I tested at the spot in my messy base where frame drops are most significant (in the middle of farms, drying racks, chests and stuff, around the flingo).

Lua:

LuaJIT:

My specs:

 

4 hours ago, PaintDream said:

Linux DST version fixed. Please download the latest release at: https://github.com/paintdream/DontStarveLuaJIT/releases/tag/2020.5.21

Linux DS remains broken. It seems that machine code matching goes totally wrong.

Just tested out, and Linux DST version works with Mods, i.e. game starts and still keeps the mods as before, and clicking "Mods" button in the game loads the Mods screen.

[PaintDream]

7 hours ago, hyiltiz said:

Hmm, just tested out both the original and the "modded" games in an old Linux computer. Doesn't seem to be much different. If anything, the "modded" one is heavily dropping more frames. I tested at the spot in my messy base where frame drops are most significant (in the middle of farms, drying racks, chests and stuff, around the flingo).

Lua:

LuaJIT:

My specs:

 

Just tested out, and Linux DST version works with Mods, i.e. game starts and still keeps the mods as before, and clicking "Mods" button in the game loads the Mods screen.

I can't see your images since their links are unavailable in China.
How much RAM are installed on your old linux computer? LuaJIT consumes much more RAM and it is possible that the RAM is insufficient and had to be exchanged from swap partition.
Jit compiler mostly consumes more RAM for generating and caching jitted functions to reduce CPU overload significantly. If your CPU is not bottleneck for game, it will not benefit a lot from jit.

For me, an empty DST world with jit consumes 900MB memory while non-jit version consumes 850MB.
.
BTW, There is another issue about framedrops: #8 in Chinese. It is so strange that it appears only on linux. I guess this framedrop is mainly caused by garbage collection which could stop the world at this issue.

Here's my comparison:

Hardware: Old linux computer purchased on 2012, i7-3610QM (4C8T) + 16 GB DDR3 RAM + Intel HD 4000. My RAM is sufficient for game running and no swap partitions are configured on my linux (Debian 8.5, 64bit).

Lua version (FPS: 48-50, CPU full load on one core, Don't starve is not multi-threaded optimized at all):

LuaJIT version (FPS: 57-58, CPU full load)

It benefits a lot. With the same full CPU load, luajit can execute 57-58 frames per second while lua version can only do 48-50 frames.

Once paused by mod "Global Pause", it goes to 60fps for both version.