Jump to content

[Tutorial] How To Create A Custom Shader


Hornete
 Share

Recommended Posts


Hello and welcome to my tutorial on how to make your own custom shader for DST! This sort of knowledge is limited to very few people in the community so I thought to make a tutorial to make this knowledge available for everyone who wants to make a custom shader for their mods!

What’s a Shader?
Originally shaders were used to shade 3D models in video games. Today they're used to shape, change color, size and etc on textures, used in post processing and so on. They're used all the time in DST and I can show you some examples!

swTR_sM19-w4Us5aV8R31uCQJdA3rI8mlOdRR97i288E_TqtE0jGcUVG0CgLNeLSS3BepXCSFngJY687xU-Rptr2VlYQWBGRR2tjlZ3Jj0tB32CiDHrJukw2nV0d7vE1uddyM7Qt-Fm4gbNV3i_pjJUw9XU6ro8lo5wh9rtOKkM8CLsW9VndwJDf7lGswvWnWwnY_2AuGYXy9fUlyqr9-PONIWXmYHZJkQM8SCEp5VDLhezVAK9ENj96Nt2Mb81DM3Uh019sUE2amcu8

One common shader is the “bloom” shader. In the first image here is a roaring campfire where the fire has the “bloom” shader applied that greatly brightens the fire, the 2nd picture has a campfire without the “bloom” shader. The “bloom” shader is generally used to make things bright and shiny! They’re used everywhere in the game. They’re used for the tiles you stand on, the ocean, the UI, spider webbing, haunted effect, colour cubes and much much more!

(Bloom On)
YpQpYGq2GmMuXooPSjxh2EGFq8u1I1lusnbj6DK40md4UcdA7mnLIxl7TJScYuH6K_C5nkxNJt7VECWxUjfiwOh67s3gohl5Qxfi5y9-z7mHyyIi5sgNtClnfmNAVKA8q8kx5N_b


(Bloom Off)

oBBStJsj5W-xyqq8vhqYElnYhIpCDQ4E65eyhtVJR5RsYSVs_4_FwlnHzqDSJ9za9BZHTjZcr9YcC6-DKEIrB9gSFRRrUS5_xfVY0pN051sks9i-9rnxPNfv_q98Ry_8zkWldNwv

 

So what do I need to make a custom Shader?
 

-To make a shader file(.ksh), you’ll need a vertex shader(.vs) file and a pixel/fragment shader(.ps) file
Quick Explanation on Vertex and Pixel Shaders

Spoiler

The Vertex Shader determines the traits (texture coords, position) of a vertex, and the Pixel Shader determines the traits(alpha value, colors) of a Pixel. Both of these shaders are extremely important to making our finished and compiled shader. The explanation I gave was extremely simplified.

https://en.m.wikipedia.org/wiki/Shader#:~:text=Shaders are simple programs that,alpha value

I absolutely suggest reading up on them in this wiki article on Vertex Shaders and Pixel/Fragment Shaders. There's also many videos and resources out there explaining what these shaders do if you'd like to find and watch them.

-Some basic knowledge of GLSL(OpenGL Shading Language)
GLSL is the main language we’ll be using to make our custom shaders, I recommend you get some basic understanding and knowledge of the language but if you understand other languages fairly well(such as LUA, which is used for DST programming most of the time) then you should be able to figure out GLSL as you go along

An important resource will be this website here! 

https://thebookofshaders.com/
https://thebookofshaders.com/edit.php

This website has a lot of knowledge on shaders and even has an editor for making fragment shaders! Some shaders being showcased are clickable and will lead you to an editor with the code that make up that shader. Almost every shader you see on this website can be applied in DST itself, so take a gander around and see what shaders you like.


Programming your Shader

So let’s get to programming your shaders! In both the vertex shader and pixel shader files!
uHHOuLhd0wov-huj3ZE33dYXVLp8rnHyJQDCWnOlNs9dlFAdg1alwJhEUESz--2cqYeb5NQdrpj8_PJHp_5Mt2b-TxItP7X02mP0pKm2AEPd_bOm9n-4a_xejYM938lEM7jVqAfs
Create two files like these wherever you’d like. (We will be moving these files later however) You can name them whatever you’d like. But I've named them “examplePixelShader” and “exampleVertexShader” in my case. Be sure the PixelShader file has .ps has its extension, and the VertexShader has .vs as its extension.

Now open up your VertexShader.vs file and copy paste this code in.
 

uniform mat4 MatrixP;
uniform mat4 MatrixV;
uniform mat4 MatrixW;

attribute vec4 POS2D_UV;

varying vec3 PS_TEXCOORD;

void main()
{
    vec3 POSITION = vec3(POS2D_UV.xy, 0);
    float samplerIndex = floor(POS2D_UV.z/2.0);
    vec3 TEXCOORD0 = vec3(POS2D_UV.z - 2.0*samplerIndex, POS2D_UV.w, samplerIndex);

    mat4 mtxPVW = MatrixP * MatrixV * MatrixW;
    gl_Position = mtxPVW * vec4( POSITION.xyz, 1.0 );

    PS_TEXCOORD = TEXCOORD0;
}

Why do we put this code in? Well this code is necessary for making sure our PixelShader will only apply to the texture/entity we apply it to. What the code is doing is loading in some variables the game gives us (POS2D_UV, PS_TEXCOORD, MatrixP, MatrixV, etc) and we’re doing calculations with those variables to give the game the positions(gl_Position variable). And now the PixelShader will only apply to pixels in that position.

If we just did something like

gl_Position = vec4(1.0)

And set that to the gl_Position variable, our Pixel Shader will run for every pixel on the screen which isn’t what we’d like.

As for the code you put in your PixelShader? Well, it can be anything really! But you can find many examples in the bookofshaders website I linked earlier. For my example i’ll be using the code here!

yK1GSr3EQPeJuC6Ci8YZ3q4Jix6kyao0x760RnCVBSGo4qwMdFw1kxbl-zs1UYLOb8sfuBCvuWiNnWAKfft0IRDMoA-BRFyh13CWh7zsUWqyn76wSRDYXnOJmbKBJLkXpkYvdaN6
Found in: https://thebookofshaders.com/06/

This pixel shader here will switch the color of our entity/texture from yellow to blue and back constantly. You can follow along and plop the code of the example i'm using, or plop your own code in.

LQxr6iGf5Y16WGyS29FlE7gC532z_SIT5Yd5xWoscqZF5Dp07VWjQtJwCRU9OcyRh9QRcRIn1KsVYoSl-Q7ga3mVG09qT2rNtBGxuQd4Q1yieF19TmYI7wYrEFr4Y_HWMUekK3sR
Just like that! But before we continue, there’s something very important we need to do for my example. (This might not apply to the code you're using)

Do you see that “u_time” variable there? Well we need to replace it! Unfortunately the game does not provide us with a u_time variable to use. However! That’s because there’s another variable that’s used to measure time.

uniform vec4 TIMEPARAMS;

We can use the TIMEPARAMS variable instead! Go ahead and replace the

uniform float u_time;

line with the “uniform vec4 TIMEPARAMS;” line up above.

The TIMEPARAMS variable is used in the game's own shader files(anim.ksh) and that’s where I found it. Now go ahead and replace all instances of “u_time” in your code with “TIMEPARAMS.x”

For example, from this

xTalE-7yhs5F304OMu6bTEKAncaVsEyvZX3ljqMyTEiWo7yMz3IUGsXf8JgBa36wRVkf_tNuEoCOAIB1hp_TBizuEzTcbfBExADLfVtc-tQrb75z83bgMwM-N-pxakTTZpO7t_Y9
To this!
q1PRCWobi2sUcTLUVmQUgWtwwXIiPUUTEb1gz1Pjr2iOcZ1kOUUabQ64a-ymQLsBZufbrCrFI6Hbz-GyTU5q3-grRJhdRcf9DoX7Ng-Dbgc3nljfMEgSNeP-mqGpUYnNb8-wLHv8

Of course if you don’t have the “u_time” variable anywhere in your code you can ignore this. If your pixel shader does something that's unchanging and constant then you probably don’t need a time variable to use.

Compiling Our Shader

Now we should be done programming our VertexShader and PixelShader files. It’s time for the fun part of compiling our files into the actual “ksh” file which is the actual shader we’ll be loading in-game.
Copy your files, and head on over to this directory here

“C:\Program Files (x86)\Steam\steamapps\common\Don't Starve Mod Tools\mod_tools\tools\bin”

And paste your files here. You’ll notice there’s an executable called “ShaderCompiler.exe” which will be the program we use to compile our shader. Open your command prompt (Windows Key + R, or search Command Prompt in your computer search bar)

5G8N7_6HI4-jHmpDsb-rnsfKicdVxQ35HsS4iRU-2txjbr0LbU1wJedEaLEyOeDFTBIpTp1m_A8rGpLVRkgPEQRTTmUJGKR2lT9NKypm4TFgxLtWKEmhXty_bPIXJoALEws_ZMDF
Type in “cd”, put a space and then paste the directory I gave you earlier and press Enter. Now you’re able to do the compiling as you’re in the directory with the ShaderCompiler.

ShaderCompiler.exe -little shadername exampleVertexShader.vs examplePixelShader.ps shadername.ksh -oglsl

Paste this line into your command prompt like this. Make sure to replace instances of “shadername” with what you’d like to name your shader. And replace “exampleVertexShader” and “examplePixelShader” with the names you have given them. And then you can execute the command by pressing Enter.
Q0sYPnEpz-FNXJXB1DqEQZ4EWPJGoq0Q1EBnF9KpUPccqtHR-3xsxkx4wnj_e0QHuxcL64c32S16PLYUvsD-aSUdDzqsoxjKW760zynsG5gzs5YO3Bm5uTFUYW4sWDmvH354AyR4

If your files were prepared correctly and there are no errors, you should get something like this right after in your command prompt.
4qDZipZtvNaQsijpV-tqklTK4Od0s272cCJNbevssuppB68y-JaQnso-psIwvfhBfQvuqhYS_Ei3tWwiEB_hcyrQtTTln11nfiFfx9xHZS9GdleNrQqfWJcEhLL_jLgYiHXldhHQ

If an error did pop up. Let me know and i'll try to help you out!

But anyways, if you did it correctly, this means the shader compiled correctly! You should now find a file named “shadername.ksh” or whatever you set the name of your shader to be, in the directory.

6xowJUI28qD110J6DJOYpNxvYFBrUQ6T5DmZ5OqKnlvGoEzmopmJb-xPco2DIgvdCTWjI1iJoxNqde1XqpOPH9UBNOeanPWdaVCvUlSusdTfkdQN-2NWCwiEn1-P3qqzys023QQ1
Yup, there it is! Grab this file, and head on over to your mod directory
OVkQ9OIj3Qssk1uqeVsFbqV2WTmKtdvjL7uxaUZRmv_wUyBU1Iz1-HnD9tDxHwBahWywpOvM1xkBy3xHg3Rv2pWscRwxPBAXBaNQi7XCjGu720bW29YAI5suCVo5eVhURpFQbBib
Make a new folder named “shaders” in your mod directory., place/paste your new shader file in there.

Head on over to your “modmain.lua” and load the shader file in the Assets table like this.

HpLRXairc154ii5bt6dUw1pSEM3yvugGzdHUJ5LNlVEeteba9YuJpdqTtY7hSbEWK1sk_U2Vl3raGblzvgPyNqN6nmnxDfRQOARsJlHJp00TgiZ0jQplrXYqBFk_W1LVB9fDdYGi
You’ll probably already have an Asset table filled up so just plop the 

Asset(“SHADER”, shaders/shadername.ksh”)

 line in your assets table, and be sure to rename accordingly.

It’s finally time to test in-game, will your custom shader work? Let’s find out! Create a world and enable your mod. Once you’re in, copy and paste this command and hover over any entity in the game. (Be sure to replace “shadername” with the actual name of your shader)

c_select().AnimState:SetBloomEffectHandle(resolvefilepath(“shaders/shadername.ksh”))

p23sBwtZi7qQLqrZ-xNpTIrVLQ2KDtZDgD21GMxyDBQ51gMi9Am5bb5Jn0bXxn3xhM4pS7lCVyF6BeACEhZ3VIqS7-XD0vErTa9gZFu1oD8YfdZF57_pbDEjc5Kkj_8hGLNwHFMF

And Congratulations you (hopefully) did it! You may use this shader on any entity/texture you’d like at any time. Have fun with this new knowledge and understanding of shaders and I hope us Modders can make some wicked stuff with this!

Notes:

Spoiler

You may crash when trying to apply your custom shader and that’s somewhat normal. Unfortunately it can be difficult to debug the crash and what happened when you applied your custom shader. It’s not as easy as regular modding and debugging where you can just go to your client/server log and see what happened. Crashes could happen for a variety of reasons, the main ones being syntax errors and nil/null variables. As you saw in this tutorial the u_time variable isn’t used in DST, there may be some other cases such as that. If you’re having trouble with getting your custom shader working crash-free, PLEASE comment down below and let me know and I’ll try to help you find the problem.

All of the game's own Shader files can be found in this directory here.


C:\Program Files (x86)\Steam\steamapps\common\Don't Starve Together\data\databundles\shaders.zip\shaders

I absolutely reccommend exploring the games shaders and seeing how they did things. so take a look at the files.









 

Edited by Hornete
Added info where game shaders are
  • Like 6
  • Thanks 3
Link to comment
Share on other sites

2 hours ago, IronHunter said:

How does one extract shader code from existing shader files?
As that would be really helpful as reference material.

Aah! I can't believe I forgot to mention how to find the existing shader code haha.

You can find all of the game's shader files in this directory.

C:\Program Files (x86)\Steam\steamapps\common\Don't Starve Together\data\databundles\shaders.zip\shaders

1089925077_Screenshot(215).thumb.png.b76463f57a79c305b1094d7cbec7bc19.png

You should see something like this. 
image.thumb.png.90aaedecd0d0d6c3f6478f85debf2359.png

The first half-ish of the file is the vertex shader.
 


image.thumb.png.9f65cd8b33a51c276c7a7cc28f5e7dd9.png
And the 2nd half-ish of the file is the pixel shader!(As indicated by "anim.ps" at the top there

Hope that helps

 

3 hours ago, Cunning fox said:

Will it work for UI? I'd want to make something similar to summer shader that distorts the image, will it work for images too?

Yup! It should work for ui. You can check out the overheat shader to see how they did the vertex and pixel shaders for that.
image.png.b128c5a9b99c283f992832729689394d.png

Looking at the heatover widget the shader is applied to the image with SetEffect.

And something cool that widgets over entities when using shaders, you can actually pass parameters for the shader from the lua side.
image.thumb.png.6cfa1870114bfd0b25b626a570b6af31.png
You can see SetEffectParams being used and being passed effectTime, effectSize, and effectFrequency variables.


image.png

And I believe those become these AMPLITUDE, FREQUENCY and VELOCITY variables. It's too bad AnimState doesn't have a SetEffectParams function too, but thats a cool thing you can do for UI.
  

Edit:
image.thumb.png.30fb122a3c080c1989718d7ecd5265e4.png
You can also use this SetAlphaRange function to set alpha values and that becomes these variables in the overheat shader
image.png.9e94f4cdacd9ed9d7710b5824892d671.png 
  

 

 

Edited by Hornete
  • Thanks 3
Link to comment
Share on other sites

Any idea of how one would port a shader from DS to DST and how to set the shader's variables?
I am using these to add one and it seems to work fine by which I mean it adds the shader and it can be enabled but it crashes when enabled due to what I assume is it missing it's varaibles.

AddModShadersInit(VisionBlurBuild)
AddModShadersSortAndEnable(VisionBlurSortAndEnable)

Looking in postprocesseffects.lua I found where it builds and enables shaders and all that and it looks like it sets up variables like this, but I don't understand how it works and it isn't exactly helping.

local function VisionBlurBuild()
	local BLUR_PARAMS = PostProcessor:AddUniformVariable("BLUR_PARAMS", 4)
	PostProcessorEffects.VisionBlur = PostProcessor:AddPostProcessEffect(resolvefilepath("shaders/postprocess_blur.ksh"))
	PostProcessor:SetEffectUniformVariables(PostProcessorEffects.VisionBlur, BLUR_PARAMS)
end

The part that annoys me most about this issue is that I keep seeing it reference "PostProcessor" But I can't find any "PostProcessor.lua" or anything else that might contain the code of it in DS or DST and without having the whole picture of the variables are set and used I can't figure out how to set them for myself.

Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
 Share

×
×
  • Create New...