Painterly Shader with GLSL

Besides all the regular fancy shaders you would normally find (reflections, normal mapping...) the demo features one advanced fullscreen shader to produce the painterly effect. Its different from the usual cel-shading techniques because it doesn't rely on silhouettes or a different lighting model to produce the effect. Instead, the shader utilizes a brush pass ( the scene re-rendered with a brush texture applied to everything) and the depth pass, alongsde the main color pass to distort the image in a specific way. The brush pass is used to define the magnitude of the distortion, and some math applied to the depth pass to define the direction of the distortion.


Its useful to note that the main color pass is rendered completely normally, the lighting is baked from Maya into the textures, and shaders are applied normally for reflections and normal mapping.Below are screenshots of the passes with a brief explanation of how it works.



1. The brush pass. The color value decides the distortion magnitude. The green channel is used as alpha (when an object is rendered green or white in the brush pass, the corresponding pixels are masked out from the result.






2. The main color pass. Some objects are rendered separately if I don't want the  painterly effect to apply to them.




3. Your standard depth pass. Most of the shader math is done with this.





4. The result. The distortion magnitude has to remain limited otherwise some strange artifacts appear where there is the most depth variation.



Code:


//Painterly Effects Shader for StarChild v1.0 by Majd Akar
//You may use this code whatever way you want, a credit would be nice
//but not necessary. A full description of the shader can be found at
//http://majdakar.blogspot.com/2011/10/techy-stuff-shader.html


//Description
//The aim of this shader is to distort a normally rendered framebuffer to 
//give a painterly effect. The distortion is driven by 2 things, the brush pass
//and the depth pass. The brush pass is the whole scene rendered again with 
//a brush texture applied to everything. For every pixel, we look at the depth
//of its neighbouring regions, and then we sample the actual framebuffer with 
//a UV value offset with the brush pass value (like a smudge) in the opposite 
//direction of the greatest depth difference. We do this because artifacts appear
//at the borders of objects, so we always smudge inwards towards the object's outline.
  
uniform sampler2D ColorPass;
uniform sampler2D BrushPass;
uniform sampler2D DepthPass;

varying vec2 vTexCoord;
uniform float mode;

void main (void)  
{  
vec3 brushValue; 
vec3 bTex =  texture2D(BrushPass, vTexCoord).rgb;

//Depth Values for current pixel and distored pixel
//---------------------------------------------------------------------------------------------------
float d_mag = 0.005; //Magnitude of distortion
//Apply 1.0 - smoothstep to depth values to map them to the visible 0-1 range
//this is a fast and simple way to remap the depth buffer values
//Get the depth value of the current pixel we're shading
float depthVal = 1.0 - smoothstep( 0.998, 1.0, texture2D(DepthPass, vTexCoord).x);
//Get the depth values of the neighbouring pixels in an x

      float depthVal_offset1    = 1.0 - smoothstep( 0.998, 1.0, texture2D(DepthPass, vTexCoord +  vec2( -d_mag, -d_mag)).x);
float depthVal_offset2   = 1.0 - smoothstep( 0.998, 1.0, texture2D(DepthPass, vTexCoord + vec2(  d_mag, -d_mag)).x);
float depthVal_offset3   = 1.0 - smoothstep( 0.998, 1.0, texture2D(DepthPass, vTexCoord + vec2( -d_mag,  d_mag)).x);
float depthVal_offset4   = 1.0 - smoothstep( 0.998, 1.0, texture2D(DepthPass, vTexCoord + vec2(  d_mag,  d_mag)).x);
    
    //Find the difference between the neighbours' depth and our current pixel's depth
    float d_diff1 = depthVal_offset1 - depthVal;
    float d_diff2 = depthVal_offset2 - depthVal;
    float d_diff3 = depthVal_offset3 - depthVal;
    float d_diff4 = depthVal_offset4 - depthVal;
    
    //Find the maximum difference
    float depthMax = max( max( max( depthVal_offset4, depthVal_offset3 ), depthVal_offset2 ), depthVal_offset1 );
    
vec3 color_offset;

//Smudge more if object is distant, like DOF but painterly
d_mag = 0.008*(1.0-1.5*depthVal);
//Which neighbour has the maximum depth difference, 
//get the color value from the opposite direction , and smudge
//multiply the distort vector by the brush texture to get the paint bands.  
if ( depthVal_offset4 == depthMax )
{
brushValue =  texture2D(BrushPass, vTexCoord - vec2(-d_mag,-d_mag) ).rgb; 
color_offset = texture2D(ColorPass, vTexCoord + (-0.5+brushValue.r)*vec2( -d_mag, -d_mag)).rgb;
}
else if ( depthVal_offset3 == depthMax )
{
brushValue =  texture2D(BrushPass, vTexCoord - vec2( d_mag,-d_mag) ).rgb; 
color_offset = texture2D(ColorPass, vTexCoord + (-0.5+brushValue.r)*vec2(  d_mag, -d_mag)).rgb;
}
else if ( depthVal_offset2 == depthMax )
{
brushValue =  texture2D(BrushPass, vTexCoord - vec2( -d_mag, d_mag) ).rgb; 
color_offset = texture2D(ColorPass, vTexCoord + (-0.5+brushValue.r)*vec2(  -d_mag,  d_mag)).rgb;
}
else if ( depthVal_offset1  == depthMax )
{
brushValue =  texture2D(BrushPass, vTexCoord - vec2( d_mag, d_mag) ).rgb; 
color_offset = texture2D(ColorPass, vTexCoord + (-0.5+brushValue.r)*vec2(  d_mag,  d_mag)).rgb;
}
//Bump ----------------------------------------------------------------------

vec3 myLight = vec3(0.35,0.659,0.47);
float bumpValue = 1.8*dot( myLight, vec3( bTex.r, bTex.r, bTex.r ) );
bumpValue = clamp( bumpValue, 0.5, 1.0 );
//------------------------------------------------------------------------------
//Vignette --------------------------------------------------------------------
float dist = distance(vTexCoord.xy, vec2(0.5,0.5));
float a = smoothstep(0.9, 0.1, dist);
float b = 0.4 *a;
//------------------------------------------------------------------------------
// Color Correction--Levelling ----------------------------------------------
float mr = smoothstep( 0.0, (0.95), color_offset.r );
float mg = smoothstep( 0.0, (0.95), color_offset.g );
float mb = smoothstep( 0.0, (0.95), color_offset.b );
//------------------------------------------------------------------------------

     vec4 FinalResult = vec4( a*(vec3(mr,mg,mb) - (0.1*bumpValue) + 0.05*vec3(bTex.r,bTex.r,bTex.r) ), (1.0-bTex.g));
if ( mode == 1 ) //Normal Rendering
gl_FragColor = FinalResult;
if ( mode == 2 ) //Dimmed for menus
gl_FragColor = vec4(  0.4*vec3(FinalResult.r ,  FinalResult.g,  FinalResult.b), 1.0);
if ( mode == 3 )    //Red-ish for loosing health
gl_FragColor = vec4(  FinalResult.r ,  FinalResult.g*0.25,  FinalResult.b*0.25, 1.0);
}    



2 comments:

  1. The d_diff values are never used in your code. Is that an error? Shouldn't these values be compared with the max depth instead of the depthVal_Offsets?

    ReplyDelete
  2. Hello Jeremie. You are right. depthVal_Offsets were not used. Excuse me if I'm a little hazy on the details (this is an old project), but what we are doing is essentially finding the the neighbour thats the furthest away from the camera. If I'm not mistaken, you can remove the d_diff stuff and the shader will run normally. Are you having problems replicating the results?

    ReplyDelete