Jump to content

New Post Processing System API


zarklord_klei
 Share

Recommended Posts

  • Developer

In the newest Return of Them beta, we have added some changes to our post processing system, the most important of these is the fact that mods can now add custom post processing filters.

This is not an explanation of how to actually write post processing shaders, google will be your friend for that, this is just an explanation of how the Lua API links up to the OpenGL Shading Language, or GLSL for short.

Inside the spoiler is the list of all the calls defined in the engine and what arguments they take and what return values they give.

Spoiler

SetPostProcessingEnabled(bool enabled)

Enables or Disables all post processing effects.
Returns nothing.


AddTextureSampler(string texture_path)

Adds a new texture sampler.
Returns a TextureID used to reference this sampler.
Arguments:

  • texture_path - the path to the texture it should use.

SetTextureSampler(TextureID textureID, string texture_path)

Updates the texture samplers texture for the given texture.
Returns nothing.
Arguments:

  • textureID - the texture to modify.
  • texture_path - the new texture for the sampler.

SetTextureSamplerState(TextureID textureID, WRAP_MODE wrap_mode)

Sets the UV mode of this texture.
Returns nothing.
Arguments:

  • textureID - the texture to modify.
  • wrap_mode - one of the values in the WRAP_MODE table defined in constants.lua.

SetTextureSamplerFilter(TextureID textureID, FILTER_MODE mag_filter, FILTER_MODE min_filter, MIP_FILTER_MODE mip_filter)

Sets the Filter modes of this texture.
Returns nothing.
Arguments:

  • textureID - the texture to modify.
  • mag_filter - one of the values in the FILTER_MODE table defined in constants.lua.
  • min_filter - one of the values in the FILTER_MODE table defined in constants.lua.
  • mip_filter - one of the values in the MIP_FILTER_MODE table defined in constants.lua.

AddUniformVariable(string uniform_name, int size)

Creates a new uniform variable for shader effects to access, this variable is accessible via uniform uniform_type uniform_name; in your shader code, where uniform_type and uniform_name are as explained below.
Returns a UniformID used to reference this uniform variable.
Arguments:

  • uniform_name - the name of the uniform variable, as accessible from glsl.
  • size - the size of the uniform variable, must be a value between 1-4, if the value is 1 uniform_type is float, 2 is vec2, 3 is vec3, and 4 is vec4.

SetUniformVariable(UniformID uniformID, ...)

Sets the values of the uniform variable that the shader can read.
Returns nothing.
Arguments:

  • uniformID - the uniform variable to set.
  • ... - a number of arguments between 1 and size, where size is the value used in AddUniformVariable, if the argument is nil, the value is left unaffected, otherwise the value is replaced by the number passed in.

AddSamplerEffect(string effect_path, SamplerSizes sampler_size, number sampler_size_x, number sampler_size_y, SamplerColourMode sampler_colour_mode, SamplerEffectBase sampler_base, ...)

Adds a new sampler effect for post process effects to use.
Returns a SamplerEffectID used to reference this effect.
Arguments:

  • effect_path - the path to the shader effect file to use for this effect.
  • sampler_size - one of the values in the SamplerSizes table defined in constants.lua.
  • sampler_size_x - if the value of sampler_size is SamplerSizes.Relative the samplers x dimension would be resolution*sampler_size_x if its SamplerSizes.Static then the samplers x dimension just be sampler_size_x.
  • sampler_size_y - the same as for x, but the y dimension instead of x dimension.
  • sampler_colour_mode - one of the values in the SamplerColourMode table defined in constants.lua.
  • sampler_base - the type of the base effect stored in SAMPLER[0], one of the values in the SamplerEffectBase table defined in constants.lua.
  • ... - extra arguments based on the value of sampler_base.
    • SamplerEffectBase.PostProcessSampler - has no extra arguments.
    • SamplerEffectBase.BloomSampler - has no extra arguments.
    • SamplerEffectBase.Shader - takes a SamplerEffectID to define the source shader.
    • SamplerEffectBase.Texture - takes a TextureID to define the source texture.
    • SamplerEffectBase.Smoke - has no extra arguments.

SetSamplerEffectState(SamplerEffectID effectID, WRAP_MODE wrap_mode)

Sets the UV mode of this sampler effect.
Returns nothing.
Arguments:

  • effectID - the sampler to modify.
  • wrap_mode - one of the values in the WRAP_MODE table defined in constants.lua.

SetSamplerEffectFilter(SamplerEffectID effectID, FILTER_MODE mag_filter, FILTER_MODE min_filter, MIP_FILTER_MODE mip_filter)

Sets the Filter modes of this sampler effect.
Returns nothing.
Arguments:

  • effectID - the sampler to modify.
  • mag_filter - one of the values in the FILTER_MODE table defined in constants.lua.
  • min_filter - one of the values in the FILTER_MODE table defined in constants.lua.
  • mip_filter - one of the values in the MIP_FILTER_MODE table defined in constants.lua.

AddPostProcessEffect(string effect_path)

Adds a new post process effect that can modify the output of the screen, SAMPLER[0] contains the current PostProcessSampler for all post process effects.
Returns a PostProcessEffectID used to reference this effect.
Arguments:

  • effect_path - the path to the shader effect file to use for this effect.

SetEffectUniformVariables(SamplerEffectID/PostProcessEffectID effect_id, ...)

Enables the given sampler or post process effect to use the supplied uniform variables.
Returns nothing
Arguments:

  • effect_id - the sampler or post process effect to enable uniform variables on.
  • ... - a list of uniform variables ids to give it access to.

AddSampler(SamplerEffectID/PostProcessEffectID effect_id, SamplerEffectBase sampler_base, ...)

    Adds a new sampler effect for post process effects to use, the sampler is stored in SAMPLER[X] where X is the number of times AddSampler has been called plus 1 for the given effect.
    Returns a SamplerEffectID used to reference this effect.
    Arguments:

    • effect_id - the sampler or post process effect to add a sampler on.
    • sampler_base - the type of the base effect, one of the values in the SamplerEffectBase table defined in constants.lua.
    • ... - extra arguments based on the value of sampler_base.
      • SamplerEffectBase.PostProcessSampler - has no extra arguments.
      • SamplerEffectBase.BloomSampler - has no extra arguments.
      • SamplerEffectBase.Shader - takes a SamplerEffectID to define the source shader.
      • SamplerEffectBase.Texture - takes a TextureID to define the source texture.
      • SamplerEffectBase.Smoke - has no extra arguments.
    
    EnablePostProcessEffect(PostProcessEffectID effect_id, bool enabled)

    Enables or Disables a post process effect, the effect must be sorted before it can be enabled.
    Returns true if it was successfully enabled, false otherwise.
    Arguments:

    • effect_id - the post process effect to enable.
    • enabled - whether to enable or disable the effect.
    
    SetBasePostProcessEffect(PostProcessEffectID effect_id)

    Sets the base post process for sorting, can only be called once.
    Returns nothing.
    Arguments:

    • effect_id - the post process effect to set as the base effect.
    
    SetPostProcessEffectBefore(PostProcessEffectID src_effect_id, PostProcessEffectID trg_effect_id)

    Sorts the source effect before the target effect.
    Returns true if successfully sorted, false otherwise.
    Arguments:

    • src_effect_id - the source post process effect to sort.
    • trg_effect_id - the target post process effect to sort.
    
    SetPostProcessEffectAfter(PostProcessEffectID src_effect_id, PostProcessEffectID trg_effect_id)

      Sorts the source effect after the target effect.
      Returns true if successfully sorted, false otherwise.
      Arguments:

      • src_effect_id - the source post process effect to sort.
      • trg_effect_id - the target post process effect to sort.
      
      SetColourCubeSamplerEffect(SamplerEffectID effect_id)

      Tells the engine which sampler effect creates the finished colour cube effect.
      Returns nothing.
      Arguments:

      • effect_id - the sampler effect that contains the finished colour cube data.
      
      SetBloomSamplerParams(SamplerSizes sampler_size, number sampler_size_x, number sampler_size_y, SamplerColourMode sampler_colour_mode)

      Sets the bloom sampler size parameters.
      Returns nothing.
      Arguments:

      • sampler_size - one of the values in the SamplerSizes table defined in constants.lua.
      • sampler_size_x - if the value of sampler_size is SamplerSizes.Relative the samplers x dimension would be resolution*sampler_size_x if its SamplerSizes.Static then the samplers x dimension just be sampler_size_x.
      • sampler_size_y - the same as for x, but the y dimension instead of x dimension.
      • sampler_colour_mode - one of the values in the SamplerColourMode table defined in constants.lua.

      The following is an explanation of how are post processing is setup for a couple of our shaders, meant to help explain how the API works.

      Colour Cubes

      This is probably the most complicated effect, so lets break it down.

          local IDENTITY_COLOURCUBE = "images/colour_cubes/identity_colourcube.tex"
          TexSamplers.CC0_SOURCE = PostProcessor:AddTextureSampler(IDENTITY_COLOURCUBE)
          PostProcessor:SetTextureSamplerState(TexSamplers.CC0_SOURCE, WRAP_MODE.CLAMP_TO_EDGE)
          PostProcessor:SetTextureSamplerFilter(TexSamplers.CC0_SOURCE, FILTER_MODE.POINT, FILTER_MODE.POINT, MIP_FILTER_MODE.NONE)
          TexSamplers.CC0_DEST = PostProcessor:AddTextureSampler(IDENTITY_COLOURCUBE)
          PostProcessor:SetTextureSamplerState(TexSamplers.CC0_DEST, WRAP_MODE.CLAMP_TO_EDGE)
          PostProcessor:SetTextureSamplerFilter(TexSamplers.CC0_DEST, FILTER_MODE.POINT, FILTER_MODE.POINT, MIP_FILTER_MODE.NONE)
      
          TexSamplers.CC1_SOURCE = PostProcessor:AddTextureSampler(IDENTITY_COLOURCUBE)
          PostProcessor:SetTextureSamplerState(TexSamplers.CC1_SOURCE, WRAP_MODE.CLAMP_TO_EDGE)
          PostProcessor:SetTextureSamplerFilter(TexSamplers.CC1_SOURCE, FILTER_MODE.POINT, FILTER_MODE.POINT, MIP_FILTER_MODE.NONE)
          TexSamplers.CC1_DEST = PostProcessor:AddTextureSampler(IDENTITY_COLOURCUBE)
          PostProcessor:SetTextureSamplerState(TexSamplers.CC1_DEST, WRAP_MODE.CLAMP_TO_EDGE)
          PostProcessor:SetTextureSamplerFilter(TexSamplers.CC1_DEST, FILTER_MODE.POINT, FILTER_MODE.POINT, MIP_FILTER_MODE.NONE)
      
          TexSamplers.CC2_SOURCE = PostProcessor:AddTextureSampler(IDENTITY_COLOURCUBE)
          PostProcessor:SetTextureSamplerState(TexSamplers.CC2_SOURCE, WRAP_MODE.CLAMP_TO_EDGE)
          PostProcessor:SetTextureSamplerFilter(TexSamplers.CC2_SOURCE, FILTER_MODE.POINT, FILTER_MODE.POINT, MIP_FILTER_MODE.NONE)
          TexSamplers.CC2_DEST = PostProcessor:AddTextureSampler(IDENTITY_COLOURCUBE)
          PostProcessor:SetTextureSamplerState(TexSamplers.CC2_DEST, WRAP_MODE.CLAMP_TO_EDGE)
          PostProcessor:SetTextureSamplerFilter(TexSamplers.CC2_DEST, FILTER_MODE.POINT, FILTER_MODE.POINT, MIP_FILTER_MODE.NONE)

      This code creates 6 texture samplers, CC0_SOURCE, CC0_DEST, CC1_SOURCE, CC1_DEST, CC2_SOURCE, and CC2_DEST, each of these samplers are initialized with the identity colour cube, and then the state and filter modes of the texture samplers are set.

          UniformVariables.CC_LERP_PARAMS = PostProcessor:AddUniformVariable("CC_LERP_PARAMS", 3)
          UniformVariables.CC_LAYER_PARAMS = PostProcessor:AddUniformVariable("CC_LAYER_PARAMS", 2)
          UniformVariables.INTENSITY_MODIFIER = PostProcessor:AddUniformVariable("INTENSITY_MODIFIER", 1)
          PostProcessor:SetUniformVariable(UniformVariables.INTENSITY_MODIFIER, 1)

      This creates 3 different uniform variables, CC_LERP_PARAMS, CC_LAYER_PARAMS, and INTENSITY_MODIFIER, these would be accessed in the shader like so:

      uniform vec3 CC_LERP_PARAMS;
      uniform vec2 CC_LAYER_PARAMS
      uniform float INTENSITY_MODIFIER;

      Then it sets the value of the intensity modifier uniform to have the value of 1.

          SamplerEffects.CombineColourCubes = PostProcessor:AddSamplerEffect("shaders/combine_colour_cubes.ksh", SamplerSizes.Static, 1024, 32, SamplerColourMode.RGB, SamplerEffectBase.Texture, TexSamplers.CC0_SOURCE) --SAMPLER[0]
          PostProcessor:AddSampler(SamplerEffects.CombineColourCubes, SamplerEffectBase.Texture, TexSamplers.CC0_DEST) --SAMPLER[1]
          PostProcessor:AddSampler(SamplerEffects.CombineColourCubes, SamplerEffectBase.Texture, TexSamplers.CC1_SOURCE) --SAMPLER[2]
          PostProcessor:AddSampler(SamplerEffects.CombineColourCubes, SamplerEffectBase.Texture, TexSamplers.CC1_DEST) --SAMPLER[3]
          PostProcessor:AddSampler(SamplerEffects.CombineColourCubes, SamplerEffectBase.Texture, TexSamplers.CC2_SOURCE) --SAMPLER[4]
          PostProcessor:AddSampler(SamplerEffects.CombineColourCubes, SamplerEffectBase.Texture, TexSamplers.CC2_DEST) --SAMPLER[5]
          PostProcessor:SetEffectUniformVariables(SamplerEffects.CombineColourCubes, UniformVariables.CC_LERP_PARAMS, UniformVariables.CC_LAYER_PARAMS, UniformVariables.INTENSITY_MODIFIER)
          PostProcessor:SetSamplerEffectState(SamplerEffects.CombineColourCubes, WRAP_MODE.CLAMP_TO_EDGE)
          PostProcessor:SetSamplerEffectFilter(SamplerEffects.CombineColourCubes, FILTER_MODE.LINEAR, FILTER_MODE.POINT, MIP_FILTER_MODE.NONE)
          PostProcessor:SetColourCubeSamplerEffect(SamplerEffects.CombineColourCubes)

      This code creates a sampler effect with a static sampler size of 1024x32, that only uses RGB and has the texture CC0_Source stored in SAMPLER[0].
      Then it adds 5 more samplers, 1 for each of the other 5 textures created above.
      Then we enable the 3 uniform variables that we created to be used on this sampler effect.
      Then we set the state and filter of this sampler effect.
      Then we tell the engine that this is the colour cube sampler effect.

          PostProcessorEffects.ColourCube = PostProcessor:AddPostProcessEffect("shaders/postprocess_colourcube.ksh")
          PostProcessor:AddSampler(PostProcessorEffects.ColourCube, SamplerEffectBase.Shader, SamplerEffects.CombineColourCubes)

      Finally, we create the post process effect, and attach the sampler we created above to get stored in SAMPLER[1], so that we can apply the colour cube to the rendered scene, stored in SAMPLER[0].

      Bloom

          PostProcessor:SetBloomSamplerParams(SamplerSizes.Relative, 0.25, 0.25, SamplerColourMode.RGB)

      Sets the bloom sampler params to screen res*0.25 and the color mode to RGB only.

          SamplerEffects.BlurH = PostProcessor:AddSamplerEffect("shaders/blurh.ksh", SamplerSizes.Relative, 0.25, 0.25, SamplerColourMode.RGB, SamplerEffectBase.BloomSampler)
          PostProcessor:SetEffectUniformVariables(SamplerEffects.BlurH, UniformVariables.SAMPLER_PARAMS)
      
          SamplerEffects.BlurV = PostProcessor:AddSamplerEffect("shaders/blurv.ksh", SamplerSizes.Relative, 0.25, 0.25, SamplerColourMode.RGB, SamplerEffectBase.Shader, SamplerEffects.BlurH)
          PostProcessor:SetEffectUniformVariables(SamplerEffects.BlurV, UniformVariables.SAMPLER_PARAMS)
      
          PostProcessor:SetSamplerEffectFilter(SamplerEffects.BlurV, FILTER_MODE.LINEAR, FILTER_MODE.LINEAR, MIP_FILTER_MODE.NONE)

      This creates the gaussian blur effect commonly used for bloom effects.
      First we blur the horizontal axis, we create a sampler effect with the screen params as screen res*0.25 and the color mode to RGB only, and set the bloom sampler as the base effect.
      Then we enable the SAMPLER_PARAMS uniform variable, which is a special uniform variable that contains the following 4 values, {sampler_size_x, sampler_size_y, 1 / sampler_size_x, 1 / sampler_size_y} sampler_size_x and sampler_size_y are the values after taking into account the screen resolution if you used SamplerSizes.Relative.
      Then we blur the vertical axis setting up a nearly identical shader, but using the horizontal blur shader as a source instead of the bloom sampler.
      We also set the filter of the vertical blur shader.

          PostProcessorEffects.Bloom = PostProcessor:AddPostProcessEffect("shaders/postprocess_bloom.ksh")
          PostProcessor:AddSampler(PostProcessorEffects.Bloom, SamplerEffectBase.Shader, SamplerEffects.BlurV)

      Then we create the post process effect, and attach the blurred bloom sampler as SAMPLER[1], so that we can apply it to the rendered scene stored in SAMPLER[0].
      The thing to note is how this is rendered, starting with the BloomSampler which is rendered with the BlurH SamplerEffect which is Rendered with the BlurV SamplerEffect, which is finally attached to the PostProcessEffect.

      Final Notes

      We have two new API functions for mods to use:

      AddModShadersInit(fn)

      You should put all your shader creation code code inside a function you pass to this.

      AddModShadersSortAndEnable(fn)

      You should do all your sorting and enabling calls inside a function you pass to this.

      All file paths should be enclosed by a:

      resolvefilepath("path/to/your/file")

      so that the game can properly locate your shaders and textures.

      If you want to see how all of our shader effects are setup, you can look at scripts/postprocesseffects.lua.
      if you want to see an example mod with a couple custom shader effects, you can look at this steam workshop mod here.

      If anything is unclear you can ask below and I'll do my best to answer it.

      Happy Modding.

      • Like 8
      • Thanks 3
      • Happy Hazard 1
      • Big Ups 1
      • Potato Cup 1
      Link to comment
      Share on other sites

      I used the function PostProcessor:AddTextureSampler() in this way:

      local texpath = "images/noise_texture.tex"
      table.insert(Assets,Asset("IMAGE"texpath))
      AddModShadersInit(function()
      /...code.../
          local PostProcessor = GLOBAL.PostProcessor
          TexSamplers.noise_tex=PostProcessor:AddTextureSampler(texpath)--when this code is annotated the game can start properly
      /....code..../
      end)

      then the game crashed when i load a save

      and the log says 

      "Assert failure 'mTex != INVALID_RESOURCE_HANDLE' at ..\source\game\render\PostProcessor.cpp(459): Trace follows..."

      I have tried something like "resolvefilepath" or "softresolvefilepath" or just using the absolute path but none of them works.

      What's the proper form for  the parameter accepted by the function postprocessor:AddTextureSampler() or is there something else wrong?

      Link to comment
      Share on other sites

       

      7 hours ago, infiniteob said:

      I used the function PostProcessor:AddTextureSampler() in this way:

      local texpath = "images/noise_texture.tex"
      table.insert(Assets,Asset("IMAGE"texpath))
      AddModShadersInit(function()
      /...code.../
          local PostProcessor = GLOBAL.PostProcessor
          TexSamplers.noise_tex=PostProcessor:AddTextureSampler(texpath)--when this code is annotated the game can start properly
      /....code..../
      end)

      then the game crashed when i load a save

      and the log says 

      "Assert failure 'mTex != INVALID_RESOURCE_HANDLE' at ..\source\game\render\PostProcessor.cpp(459): Trace follows..."

      I have tried something like "resolvefilepath" or "softresolvefilepath" or just using the absolute path but none of them works.

      What's the proper form for  the parameter accepted by the function postprocessor:AddTextureSampler() or is there something else wrong?

      It seems there is a bug in the api.

      I found that I can use PostProcessor:AddTextureSampler(mod_path) in the console but not in AddModShadersInit(fn).

      Maybe you can set the texpath as a game tex file and change it to your mod tex file later on.

      -- modmain environment
      
      local texpath = MODROOT.."images/texture/some_mod_file.tex"
      local init = "images/overlays_lunacy.tex"
      
      AddModShadersInit(function()
      	local id = PostProcessor:AddTextureSampler(init) 
      	scheduler:ExecuteInTime(0, function() PostProcessor:SetTextureSampler(id, texpath) end) -- delay one frame to set the texpath
      	-----
      end)

      Well, this code doesn't crash the game, but not sure if it works properly....

      Edited by LaoWang27
      Link to comment
      Share on other sites

      • Developer
      On 10/8/2021 at 10:38 PM, infiniteob said:

      I used the function PostProcessor:AddTextureSampler() in this way:

      local texpath = "images/noise_texture.tex"
      table.insert(Assets,Asset("IMAGE"texpath))
      AddModShadersInit(function()
      /...code.../
          local PostProcessor = GLOBAL.PostProcessor
          TexSamplers.noise_tex=PostProcessor:AddTextureSampler(texpath)--when this code is annotated the game can start properly
      /....code..../
      end)

      then the game crashed when i load a save

      and the log says 

      "Assert failure 'mTex != INVALID_RESOURCE_HANDLE' at ..\source\game\render\PostProcessor.cpp(459): Trace follows..."

      I have tried something like "resolvefilepath" or "softresolvefilepath" or just using the absolute path but none of them works.

      What's the proper form for  the parameter accepted by the function postprocessor:AddTextureSampler() or is there something else wrong?

      As said in the main post:

      On 4/19/2021 at 3:03 PM, zarklord_klei said:

      All file paths should be enclosed by a:

      
      resolvefilepath("path/to/your/file")
      • Thanks 2
      Link to comment
      Share on other sites

      Create an account or sign in to comment

      You need to be a member in order to leave a comment

      Create an account

      Sign up for a new account in our community. It's easy!

      Register a new account

      Sign in

      Already have an account? Sign in here.

      Sign In Now
       Share

      ×
      • Create New...