Wirehead Studios

Wirehead Modifications => General Development ('Laced Neptune') => Topic started by: Phoenix on 2015-11-06, 00:02



Title: How Shaders Work and Discussion
Post by: Phoenix on 2015-11-06, 00:02
First, grab the shader manual.  It's located HERE. (http://www.wireheadstudios.org/phoenix/editing/shader_manual.zip)  Just unzip the file in a folder somewhere then open the Index.html file.

First, what exactly is a shader?  It's a script file that tells the renderer to do something special with a texture.  It can affect the transparency and blending of textures with other textures, animate multiple textures over time across a surface, or affect the texture coordinates to make a texture rotate, wobble, expand and contract, or scroll in a direction.  It also controls how the engine lights the texture so that textures can glow, pulsate, or flicker in intensity.

Which textures get shaders?  Technically all of them.  If a texture is referenced without an explicitly assigned shader script the engine uses a built-in default.  Generally this maps the texture 1:1 to the texture coordinates, keeps it in a static position, and lights it based on the level's lightmap.  If you need a different effect, such as a transparent or glowing bit, you need to write an explicit shader script.

Where are shader files located?  In the "scripts" folder.  All shaders files are loaded from there, and all shader files are an unformatted .txt that is renamed to .shader.

It should be noted that there are two different groups of modders that are going to be working with the shader scripts.  One is mappers, the other is modelers.  Mappers will use shader keywords and functions that usually do not apply to models.  For example, every keyword that starts with q3map_ or surfaceparm is going to be used by a mapper.  You would not need a lava or fog parameter on a weapon or player model, but you would need the lava parameter on a map with a lava pit that you want to cook someone in.  A modeler might need a glowing texture on a plasma gun, while the lava pit will also need to glow.  Both the mapper and modeler would want to be using the rgbGen stage specific keyword to make their respective textures glow.

So that being said, the best way to look at how shaders work is to find an existing shader and dissect what it does.  Let's look at the Q3 plasma gun's effects.  In your baseq3 folder you'll find a models.shader file.  Open that in your favorite text editor and take a look at the bottom few entries in the file.  You'll find this section of code, which covers our aforementioned plasma gun:

Code:
models/weapons2/plasma/plasma_glo
{
    
{
map models/weapons2/plasma/plasma_glo.tga
                tcmod rotate 33
                tcMod scroll .7 1
                rgbGen identity
}
        {
map textures/effects/tinfxb.tga
tcGen environment
                tcMod scroll .01 .02
blendfunc GL_ONE GL_ONE
rgbGen lightingDiffuse
}
}

So what's all this mean?  The first entry is the shader name.  This can be an explicit name, or just the name of a texture path.  In this case the texture path is used, but notice the entry does NOT have .tga or .jpg at the end.  The shader name must NEVER end in an extension or it will not be applied to the texture.

Our shader name in this case is:
models/weapons2/plasma/plasma_glo

Below the texture is a { bracket.  This starts the "General Keyword" portion of the texture.  If a texture were to be double-sided, you would see "cull disable" or "cull none".  This would apply to grates or other see-through surfaces that can be visible from the back.  If the texture were to bulge, like the organic parts of The Place of Many Deaths, you'd see a deformVertexes statement.  This particular shader has no general keywords, so this section is blank.

Next is another { bracket.  This starts the FIRST STAGE of the shader.  This stage is the most important.  If r_vertexlight is set to "1" in Quake 3 (vertex lighting enabled in place of lightmap) then this is the ONLY shader stage that will be processed.  BE SURE to test your shaders with vertex lighting!  If you do not, it could look very bad for some players.  If you have multiple textures that need to draw on the same surface sometimes it's better to duplicate a model part and use separate single-stage shaders for each part so they draw the same in both lighting modes.

Anyway, each shader stage operates in order and will consist of stage specific keywords.  The first keyword will ALWAYS be the texture reference, beginning with either map, clampmap, or animmap, followed by the texture path.  This will be the FULL texture path, including the extension.  It is important to note that Q3 will always try to load textures in a specific order.  If you end your texture path in .tga, it will look first for a .tga file, then a .jpg if it cannot find it.  If you end your texture path in .jpg, it will look for the .jpg but not look for a .tga.  This is important when dealing with 32-bit .tga's that might have an alpha channel.  If you have a .jpg version of the same file and you've converted it to .tga and added an alpha channel you'll want to make sure you're referencing the right texture in the shader.  I've found the best practice is to keep your .pk3 clean and not have duplicate files in multiple formats, and to always use .tga in the texture path keyword regardless of whether I'm loading a .jpg or .tga.

Our path keyword in this case is as follows:

map models/weapons2/plasma/plasma_glo.tga

Next we have stage specific keywords:
tcmod rotate 33

This is telling the renderer to modify the texture coordinates by rotating them 33 degrees a second clockwise.  This rotates the texture COORDINATES, clockwise, NOT THE TEXTURE!  The texture will move in the opposite direction of the coordinates, so if your texture rotates the wrong way, use a negative value.

The next keyword is:
tcMod scroll .7 1

This is telling it to scroll the texture coordinates 0.7 times horizontally every second, and 1 time vertically every second.

These effects are cumulative on the texture coordinates, so as they are rotating they're scrolling as well.  This is what makes the plasma appear to ebb and flow inside the gun.

The next keyword is:
rgbGen identity

The rgbGen keyword tells the renderer how to produce the Red, Green, and Blue channels for the texture.  In this case the word "identity" is used.  All lightmaps in the game work to darken textures, so this is telling the renderer to ignore the level lighting and light the texture fullbright all the time.  Now the plasma will glow in dark areas.

Following this is another bracket, this time }.  This indicates the end of the FIRST STAGE of the shader.

Below this bracket is another { bracket.  This inidicates the start of the SECOND STAGE of the shader.  Not all shaders have more than one stage, but our example here does so let's look at it.

Again, we see a texture path:
map textures/effects/tinfxb.tga

This is pretty self-explanatory at this point.  Below that, we find a keyword:
tcGen environment

That's new.  tcGen is like rgbGen, except that instead of handling how the Red, Blue, and Green color channels are created, this handles how the Texture Coordinates (tc) are generated.  In this case it's using the word "environment".  This is what's called "environment mapping".  It's a fancy way of saying that the texture coordinates are going to move around depending on the angle you're looking at something.  This is used primarily on things like glass or reflective surfaces that you want to give some kind of shine or sheen to.  In this case it makes the plasma look like it's held behind glass.

The next keyword:
tcMod scroll .01 .02

This makes the texture move very slowly along the X and Y axes.

The next keyword:
blendfunc GL_ONE GL_ONE

Another new one.  Blendfunc handles how textures can blend into each other.  It's what creates transparency interactions.  This can get kind of confusing and complicated at times.  There's a whole list of explicit modes in the shader manual, but here's a simpler way to explain it:

texture multiplied by (keyword) + framebuffer multiplied by (keyword).

The texture is what you stated in the "map" stage - in this case tinfxb.jpg.  The framebuffer consists of whatever is behind the texture.  GL_ONE is just that - "one".  So by this math you get:

texture * 1 + framebuffer * 1 = result,
or
texture + framebuffer = result.

This is ADDITIVE blending.  That means the texture's colors will be added to whatever it's blending with.  For example, if one pixel in the texture has RGB values of 10, 20, 30, and what's behind it has RGB values of 156, 212, and 0, the result will be 156+10, 212+20, and 0 + 30, which yields 166, 232, and 30.  This will always result in something getting brighter.  Another way to write this would be:

blendFunc add

Typically this is used for energy effects or fire where you want a bright glow from the texture.  I won't get into other blendmodes here, but if anyone has questions about them outside of the shader manual's explanation I'll provide them on a case-by-case basis.

Back to our keywords, the next one is as follows:
rgbGen lightingDiffuse

This is the default rgbGen mode that's applied to textures throughout Q3.  It tells the renderer to draw the lightning from the map's lightmap and any nearby dynamic lights.

Next we see another } bracket.  This ends the SECOND STAGE of the shader.

Last, there's yet another } bracket.  This ends the shader.  This MUST follow the closing } bracket of the last shader stage.  Every { must have a corresponding }, or it will break the shaders and you'll see a complaint in Q3's console.

Something to keep in mind about shader names vs stage-specific texture calls.  A stage-specific texture path can reference any texture, but every shader NAME must be unique.  You can have ten different shaders that reference models/weapons2/plasma/plasma_glo.tga in a stage.  You CANNOT have ten shaders named models/weapons2/plasma/plasma_glo.  This is where sometimes it's better to name a shader without a path.  An example of this would be in the gfx.shader file.  Look for this section:

Code:
console
{
nopicmip
nomipmaps
    
        {
map gfx/misc/console01.tga
                blendFunc GL_ONE GL_ZERO
                tcMod scroll .02  0
                tcmod scale 2 1
}
        {
                map gfx/misc/console02.tga
                //map textures/sfx/firegorre3.tga
                blendFunc add
                tcMod turb 0 .1 0 .1
                tcMod scale 2 1
                tcmod scroll 0.2  .1
}

}

You can see the shader name, in this case "console", is not a path.  This can help prevent problems when re-using a texture in different shaders in order to applying different effects to that same texture.

Ok, so now we know how a shader works, but how do we use the shader?  That depends on what you're doing.  If you're a mapper you'll be making references in Radiant.  If you're making a weapon or item model you'll have to assign the shader or texture path to the specific md3 component.  In the case of the plasma gun, first we have to locate the model.  It's located in models/weapons2/plasma in pak0.pk3.  Opening the .md3 file for viewing (I'm using Npherno's md3 compiler), we'll see the model consists of the following parts:

w_plasma01
w_plasma02
w_plasma03
w_plasma04
tag_flash
tag_weapon

Each component has a corresponding texture link with the exception of the tags, as follows:

w_plasma01  models/weapons2/plasma/plasma.tga
w_plasma02  models/weapons2/plasma/plasma_glass.tga
w_plasma03  models/weapons2/plasma/plasma_glo.tga
w_plasma04  models/weapons2/plasma/plasma_glo.tga

You'll see that plasma_glo.tga again on w_plasma03 and w_plasma04.  Notice the .tga is present in the MODEL file, even though the SHADER NAME omits it.  THIS IS IMPORTANT!  Q3 will look FIRST for a shader, THEN look for the texture without the shader.  If the .tga was left off of models/weapons2/plasma/plasma.tga on w_plasma01 then it would not load the texture at all because plasma.tga does NOT have a shader.  ALWAYS use the .tga when using a path as a SHADER NAME.  If you're NOT using a path as the SHADER NAME, such as with the "console" shader, then DO NOT put ".tga" or ".jpg" on the texture path in the model.

So for .md3 objects, If it's a path, it gets .tga.  If it's not a path, it does not get .tga.
For .shader files, if it's a SHADER NAME, it does not get a .tga.  If it's a texture path (map, clampmap, animmap), it gets a .tga.  Right?  Moving on.

The "tag_flash" and "tag_weapon" in the weapon model are placement tags.  These are a special triangle used to position and link specific model parts together.  The tag hierarchy works like this in Q3:

weapon_flash.md3 positions on tag_flash of weapon.md3
weapon_barrel.md3 positions on tag_barrel of weapon.md3 (if it has a spinning barrel)
weapon.md3 positions on tag_weapon of upper.md3

head.md3 positions on tag_head of upper.md3
upper.md3 positions on tag_torso of lower.md3
lower.md3 is positioned by game code.

You sometimes find duplicate tags in models.  The "tag_weapon" in a gun model does absolutely nothing.  It's leftover from the exporting of the model.  The same goes with the "tag_torso" in upper.md3, and "tag_head" in head.md3.  The origin 0,0,0 location of the head.md3 will ALWAYS position on the tag_head on the upper.md3, and the 0,0,0 location of the upper.md3 will ALWAYS position on the tag_torso position in lower.md3.  The above hierarchy is EXACTLY how Q3 positions player and weapon models, period.  Item models, such as health packs, powerups, etc, do not use tags.  They position strictly by the 0,0,0 origin of the model and are placed by game code.

Ok, so we know how a weapon model gets textured... what about player skins?  Player models work differently.  They use .skin files.  Let's look at grunt, located in models/players/grunt.  We'll find the following .skin files:

head_blue.skin
head_default.skin
head_red.skin
head_stripe.skin
lower_blue.skin
lower_default.skin
lower_red.skin
lower_stripe.skin
upper_blue.skin
upper_default.skin
upper_red.skin
upper_stripe.skin

Notice there's one .skin for each .md3 part per skin name.  We have four skins:  blue, default, red, and stripe.  Let's examine the "default" skin set.  We'll look at head_default.skin:

tag_head,
h_head,models/players/grunt/grunt_h.tga

This tells the engine a few things, pointing out which tags are in the file, and h_head is the actual .md3 component name of the single .md3 part in head.md3.  If you examine the head.md3 you'll find the component is named h_head.  In player models the md3 components MUST be named explicitly, and the component name in the .skin file has to point to this component name.  Weapon and item models do not have this concern because their texture paths are explicit, but player models use .skin files so the game engine has to know what model component to look at.  There's our texture path as well.

So let's look at the upper_default.skin file.  We have:

tag_head,
tag_weapon,
u_torso,models/players/grunt/grunt.tga
tag_torso,
u_larm,models/players/grunt/grunt.tga
u_rarm,models/players/grunt/grunt.tga

In this model there's more than one .md3 component.  Grunt's upper.md3 consists of three parts:  u_torso, u_larm, and u_rarm.  If we examine the model file we'll see three separate objects with these names.  Again, note the .skin file is referencing these .md3 object names and assigning a specific texture path to them.

Ok, let's look at head_stripe.skin:

tag_head,
h_head,models/players/grunt/stripe_h.tga

Ok, got a texture path, but wait a second:  Stripes eyes glow.  How does that work?  Go into shaders/models.shader and we find this:

Code:
models/players/grunt/stripe_h
{      
 
       {
       //map textures/effects/tinfx.tga
                map textures/sfx/electricgrade3.tga
//tcmod scale 6 6
                //tcmod rotate 350
                tcmod scroll 0 -.5
//tcGen environment
blendfunc GL_ONE GL_ZERO
                rgbGen identity
        }
        {
map models/players/grunt/stripe_h.tga
                blendfunc gl_src_alpha gl_one_minus_src_alpha
rgbGen lightingDiffuse
}
      
}

Hey, look at that!  Stripe's head texture has a shader!  Now we see where his eyes get their special effects from.  Also note that some statements in the shader are preceeded by // marks.  The // tells the shader to ignore that line and not run that code.  Typically this is done when debugging a shader to turn on and off specific functions without having to delete the text, or to leave notes or "remarks" as they're called about what you're doing.  It's OK to leave these in the shader script as the engine skips over them when loading the shader code.

So now we know how shaders work for simple item and weapon models as well as player models.  Hopefully this information is helpful for anyone wanting to delve into Q3's shader logic.  Any questions about this please post below and Phoenix will be happy to help.


Title: Re: How Shaders Work and Discussion
Post by: leilei on 2015-11-06, 00:57
The "detail" key is handy for making additional non-essential shader stages that could be disabled by r_detailTextures 0, like say, a shiny environment map on top of the model just to add a touch of gleam, or maybe some tiled grain with a modulated blend (which was its intended purpose - to break up the bilinear filtering)

Q3A only has one or two shaders total that use the key IIRC, so turning off detail textures won't affect much of the general game appearance.  


Title: Re: How Shaders Work and Discussion
Post by: Phoenix on 2015-11-06, 01:23
Castle's Place of Two Deaths for Generations does use detail textures as well.  I'm not sure how widely they've been used by other mappers.