Darkplaces material system

Revision as of 20:10, 22 July 2012

Introduction

Darkplaces material system is created to put more direct control over the surfaces qualities of textures into the hands of designers and artists.

This article will contain a parts of Quake 3 Shader Manual by By Paul Jaquays and Brian Hook adapted for Darkplaces engine.

What is a Material?

Materials are short text scripts that define the properties of a surface as it appears and functions in a game world (or compatible editing tool). By convention, the documents that contain these scripts usually has the same name as the texture set which contains the textures being modified (e.g; textures_inn, models_mapobjects_crypt, etc,). Several specific script documents have also been created to handle special cases, like liquids, environments and special effects.

For Darkplaces, material scripts are located in 'Path_To_The_Game/gamedir/scripts'.

A Darkplaces material file consists of a series of surface attribute and rendering instructions formatted within braces ("{" and "}"). Below you can see a simple example of syntax and format for a single process, including the Q3MAP keywords or "Surface Parameters", which follow the first bracket and a single bracketed "stage":

// material definition
textures/crypt/stone
{
   // parameters
   qer_editorImage radiant/textures/crypt/stone
   q3map_textureImage radiant/textures/crypt/stone
   dpglossexponentmod 0.5
   surfaceparm stone
   // base stage
   {
       map textures/crypt/stone // engine will search fo textures/mymaterial.tga, textures/mymaterial.jpeg
   }
   // lighting stage
   {
       map $lightmap
   }
}

Material names & Conventions

The first line is the material name. Material names can be up to 63 characters long. The names are often a mirror of a pathname to a .tga file without the extension or basedir (/Blood Omnicide/kain in our case), but they do not need to be. Materials that are only going to be referenced by the game code, not modeling tools, often are just a single world, like "collision" or "dpbihclip". Materials that are used on characters or other polygon models need to mirror a .tga file, which allows the modelers to build with normal textures, then have the special effects show up when the model is loaded into the game. Materials that are placed on surfaces in the map editor commonly mirror a .tga file, but the "qer_editorimage" material parameter can force the editor to use an arbitrary image for display.

Material pathnames have a case sensitivity issue - on windows, they aren't case sensitive, but on unix they are. Try to always use lowercase for filenames, and always use forward slashes "/" for directory separators.

Inspired by Quake 3 but different

Quake 3 shader scripts was an inspiration for darkplaces material system and visual look of material scripts is very very close to quake 3. So if you are familiar with Id Tech 3 shaders system, you will be right home with material script syntax. Therefore Darkplaces material system was written independently and there is some serious differences:

Aiming at realtime lightning

Any shader keywords that is not compatible with realtime lightning was removed. Other ones have sligtly changed behavior.

No render passes

Darkplaces material render pipeline can have 1 to 3 fixed usage stages:

• No draw: 0 stages
• Unlit texture: 1 stage
• Lit texture: 2 stages
• Lit texture with terrain blending: 3 stages

All other Id Tech 3 stage configuration such as skies are not supported.

Design Notes: To simulate multilayer materials, should make several copies of surface with different material atatched (for map surfaces can use Q3map2's cloneShader keyword)

Many new keywords

Darkplaces introduces many of it's own keywords that not exist in Id Tech 3.

Q3 Texture = Darkplaces skinframe

Id tech 3 only need color info for texture. Darkplaces engine using normalmap, specularity map, glow texture and other textures that store additional info. All that texture files toggether (color + normalmap + glossmap + glow etc.) forms a skinframe.

Read more at Texture article.

Different speed costs

Darkplaces's engine rendering pipeline is very different from Id Tech 3. By using pixel and vertex shaders, it introduces new speed cost parameters that was not exist in Idtech 3:

• shader instructions - amount of vertex and pixel shader instructions generated for particular material (depends of effects used)
• texture lookups - amount of texture lookups in pixel shader, additional textures in a skinframe, parallax mapping increase amount of texture lookups
• use of VBO - non-animated models render faster with VBO, some shader keywords (such as tcMod turbulent, deformVertexes) disables VBO
• use of Lighting - realtime light is slower on blended (add more draws of surface, one per light) or offset mapped surfaces (offset mapping is processed again for each light)

Material is 'Shader'

Since Darkplaces uses tools with Id Tech 3 roots, materials is often spelled as 'shaders' there. To not be confused, this terms are identical in q3 tools:

• Shader = Material
• Shader name = Surface shader = Material name = Surface material
• Shader script = Material script
• Shader keywords = Material keywords
• etc

Material Types

The keywords that affect materials are divided into three classes. The first class of keywords are global parameters. Some global parameters ( "surfaceparms" And all "q3map_" keywords) are processed by Q3MAP2 and change physical attributes of the surface that uses the material. These attributes can affect the player. To see changes in these parameters one must recompile the map.

Second class are keywords parsed by engine server part (surfaceparms, some other keywords). They are changes physical properties of surface and can affact gameplay.

The remaining global keywords, and all Stage Specific Keywords are processed by the renderer. They are appearance changes only and have no effect on game play or game mechanics. Changes to any of these attributes will take effect as soon as the game goes to another level or vid_restarts (type command vid_restart in the game console).

Material keywords are not case sensitive.

IMPORTANT NOTE: some of the material commands may be order dependent, so it's good practice to place all global material commands (keywords defined in this section) at the very beginning of the material and to place material stages at the end (see various examples).

Key Concepts

Ideally, a designer or artist who is manipulating textures with material files has a basic understanding of wave forms and knows about mixing colored light (high school physics sort of stuff). If not, there are some concepts you need to have a grasp on to make materials work for you.

Surface Effects vs. Content Effects vs. Deformation Effects

Materials not only modify the visible aspect of textures on a geometry brush, curve or mesh model, but they can also have an effect on both the content, "shape" and apparent movement of those things. A surface effect does nothing to modify the shape or content of the brush. Surface effects include glows, transparencies and rgb (red, green, blue) value changes. Content materials affect the way the brush operates in the game world. Examples include water, nonsolid, and detail. Deformation effects change the actual shape of the affected brush or curve, and may make it appear to move.

Power Has a Price

The material script gives the designer, artist and programmer a great deal of easily accessible power over the appearance of and potential special effects that may be applied to surfaces in the game world. But it is power that comes with a price tag attached, and the cost is measured in performance speed. Many OpenGL 2.0 effects attached with material keywords (such as water shader, refraction cubemap, offset mapping) makes renderer to do more calculations, which take more CPU/GPU time and make game slower. Water shader surface will draw a world 2 times to get reflective and refractive image, refraction cubemap will add one more texture sampler, parallax mapping will make more texture lookups (from 3 for traditional offset mapping to 14-15 for relief mapping). Blended surfaces will be drawn twice if lit by one or more realtime lights (this means their triangle count will have double effect on r_speeds).

RGB Color

RGB means "Red, Green, Blue". Mixing red, green and blue light in differing intensities creates the colors in computers and television monitors. This is called additive color (as opposed to the mixing of pigments in paint or colored ink in the printing process, which is subtractive color). In Darkplaces engine and most higher-end computer art programs (and the color selector in Windows), the intensities of the individual Red, Green and Blue components are expressed as number values. When mixed together on a screen, number values of equal intensity in each component color create a completely neutral (gray) color. The lower the number value (towards 0), the darker the shade. The higher the value, the lighter the shade or the more saturated the color until it reaches a maximum value of 255 (in the art programs). All colors possible on the computer can be expressed as a formula of three numbblack is 0 0 0. The value for ers. The value for complete complete white is 255 255 255. However, the Darkplaces graphics engine requires that the color range be "normalized" into a range between 0.0 and 1.0.

NOTE: often you can see RGBA abbreviation which stands for 'RGB + alpha', this is RGB with one more channel - transparency channel. 32-bit images is RGBA, while 24-bit is RGB.

Normalization: a scale of 0 to 1

The mathematics in Darkplaces engine use a scale of 0.0 to 1.0 instead of 0 to 255. Most computer art programs that can express RGB values as numbers use the 0 to 255 scale. To convert numbers, divide each of the art program's values for the component colors by 255. The resulting three values are your Darkplaces formula for that color component. The same holds true for texture coordinates.

Texture Sizes

Texture files are measured in pixels (picture elements). Textures are measured in powers of 2, with 16 x16 pixels being the smallest (typically) texture in use. Most will be larger. Textures need not be square, so long as both dimensions are powers of 2. Examples include: 32x256, 16x32, 128x16.

Measurements

The measurements used in the materials are in either game units, color units, or texture coordinates.

Game unit

A game unit is used by deformations to specify sizes relative to the world. In Blood Omnicide 16 units equals one foot, 48 units equals to one meter. The default texture scale used by the NetRadiant map editor results in two texels for each game unit, but that can be freely changed and perturbated with material scripts.

Color units

Colors scale the values generated by the texture units to produce lighting effects. A value of 0.0 will be completely black, and a value of 1.0 will leave the texture unchanged. Colors are sometimes specified with a single value to be used across all red, green, and blue channels, or sometimes as separate values for each channel.

Texture coordinates

This is the normalized (see above) dimensions of the original texture image. A full texture, regardless of its original size in texels, has a normalized measurement of 1.0 x 1.0. For normal repeating textures, it is possible to have value greater than 1.0 or less than 0.0, resulting in repeating of the texture. The coordinates are usually assigned by the level editor or modeling tools, but you still need to be aware of this for scrolling or turbulent movement of the texture at runtime.

Waveform Functions

Some of the material rendering functions use waveforms to modulate measurements over time. Where appropriate, additional information is provided with wave modulated keyword functions to describe the effect of a particular waveform on that process. Currently there are five waveforms in use in material scripts:

• sin: sine wave, a regular smoothly flowing wave ranging from -1 to 1.
• triangle: triangle is a wave with a sharp ascent and a sharp decay, ranging from 0 to 1. It will make a choppy looking wave forms.
• square: a square wave simply switches from -1 to 1 with no in-between.
• sawtooth: in the sawtooth wave, the ascent is like a triangle wave from 0 to 1, but the decay cuts off sharply back to 0.
• inversesawtooth: this is the reverse of the sawtooth... instant ascent to the peak value (1), then a triangle wave descent to the valley value (0). The phase on this goes from 1.0 to 0.0 instead of 0.0 to 1.0. This wave is particularly useful for additive cross-fades.

Waveforms all have the following properties:

base

Where the wave form begins. Amplitude is measured from this base value.

amplitude

This is the height of the wave created, measured from the base. You will probably need to test and tweak this value to get it correct for each new shader stage. The greater the amplitude, the higher the wave peaks and the deeper the valleys.

phase

This is a normalized value between 0.0 and 1.0. Changing phase to a non-zero value affects the point on the wave at which the wave form initially begins to be plotted. Example: In Sin or Triangle wave, a phase of 0.25 means it begins one fourth (25%) of the way along the curve, or more simply put, it begins at the peak of the wave. A phase of 0.5 would begin at the point the wave re-crosses the base line. A phase of 0.75 would be at the lowest point of the valley. If only one wave form is being used in material, a phase shift will probably not be noticed and phase should have a value of zero (0). However, including two or more stages of the same process in a single material, but with the phases shifted can be used to create interesting visual effects. Phase changes can also be used when you have two uses of the same effect near each other, and you don't want them to be synchronized. You would write a separate material for each, changing only the phase value.

freq

Frequency. This value is expressed as repetitions or cycles of the wave per second. A value of 1 would cycle once per second. A value of 10 would cycle 10 times per second. A value of 0.1 would cycle once every 10 seconds.

General Shader Keywords

surfaceparm

This is, by the fact, most widely used and most tricky material keyword. Many of these keywords are only used in common materials.

All surfaceparm keywords are preceded by the word surfaceparm as follows: surfaceparm nonsolid or surfaceparm trans. Each surfaceparm have corresponding q3map_surfaceparmname keyword (q3map2_nonsolid, q3map2_trans etc.) which is completely ignored by the engine. Surfaceparm keywords change the physical nature of the materials and the brushes that are marked with them. Changing any of these values will likely require the map to be re-compiled. These are global and affect the entire shader.

surfaceparm nonsolid

Surface does not block the movement of entities in the game world. This affects the content of a brush. This still make q3map2 generate collision surfaces.

Used by: engine (server & client collision), q3map2 (bsp phase).

surfaceparm water

Assigns to the texture the game properties set for water. This affects both the surface and the content of a brush.

Used by: engine (server & client BIH collision), q3map2 (bsp phase).

surfaceparm lava

Same as for water, but generating contents and surfaceflags of lava.

Used by: engine (server & client BIH collision), q3map2 (bsp phase).

surfaceparm slime

Same as for water, but generating contents and surfaceflags of slime. In Blood Omnicide, slime is swamp.

Used by: engine (server & client BIH collision), q3map2 (bsp phase).

surfaceparm nodraw

Prevents Q3map2 from generating drawsurfaces for this material.

Used by: engine (server & client BIH collision), q3map2 (bsp phase).

surfaceparm nomarks

Prevents engine from spawning decals for this surface.

Design Notes:Use this on any surface with a deformVertexes keyword. Otherwise, the marks will appear on the unmodified surface location of the texture with the surface wriggles and squirms through the marks.

cull none / cull disable

Every surface of a polygon has two sides, a front and a back. Typically, we only see the front side. For example, a solid block you only show the front side. In some situations we see both (grates, screens etc.).

To "cull" means to remove. This keyword allows to disable side culling, so both sides of the surfaces would be visible.

NOTE: Realtime lightning and lightmaps dont work well with twosided surfaces, caused back side to be lit same as front. To avoid that, should create two one-sided surfaces in game.

Design Note: When making things like grates and screens with brushes, put the texture with the cull none property on one face only. On the other faces, use a non-drawing texture.

deformVertexes

This function performs a general deformation on the surface's vertexes, changing the actual shape of the surface before drawing the material. You can stack multiple deformVertexes commands to modify positions in more complex ways, making an object move in two dimensions, for instance.

deformVertexes <wave> <div> <func> <base> <amplitude> <phase> <freq>

Designed for water surfaces, modifying the values differently at each point. It accepts the standard wave functions. The "div" parameter is used to control the wave "spread" - a value equal to the tesselatino distance (see q3map_tessSize, a subdivision size, in game units, used for the material when seen in the game world).

deformVertexes normal <div> <func> <base> <amplitude ~0.1-~0.5> <frequency ~1.0-~4.0>

This deformation affects the normals of a vertex without actually moving it, which will effect later material options like lighting (especially specular) and environment mapping (see stage specific keywords for tcGen environment). If the materialdon't use normals in any of their calculations, there will be no visible effect.

IMPORTANT NOTE: In Darkplaces, offset mapping uses vertex normals for the calculations. deformVertexes normal most likely will mess it up.

Design Notes: Putting values of 0.1 t o 0.5 in Amplitude and 1.0 to 4.0 in the Frequency can produce some satisfying results.

deformVertexes bulge <bulgeWidth> <bulgeHeight> <bulgeSpeed>

This forces a bulge to move along the given S and T directions. Designed for use on curved pipes.

deformVertexes move <x> <y> <z> <func> <base> <amplitude> <phase> <freq>

This keyword is used to make a brush, curve patch or model appear to move together as a unit. The <x> <y> and <z> values are the distance and direction in game units the object appears to move relative to it's point of origin in the map.

The product of the function modifies the values x, y, and z. Therefore, if you have an amplitude of 5 and an x value of 2, the object will travel 10 units from its point of origin along the x axis. This results in a total of 20 units of motion along the x axis, since the amplitude is the variation both above and below the base.

It must be noted that an object made with this shader does not actually change position, it only appears to.

Design Notes: If an object is made up of surfaces with different shaders, all must have matching deformVertexes move values or the object will appear to tear itself apart.

DeformVertexes autosprite

This function can be used to make any given triangle quad (pair of triangles that form a square rectangle) automatically behave like a sprite without having to make it a separate entity. This means that the "sprite" on which the texture is placed will rotate to always appear at right angles to the player's view as a sprite would. Any four-sided brush side, flat patch, or pair of triangles in an .md3 model can have the autosprite effect on it. The brush face containing a texture with this shader keyword must be square.

Design Note: This is best used on objects that would appear the same regardless of viewing angle. An example might be a glowing light flare.

DeformVertexes autosprite2

Is a slightly modified "sprite" that only rotates around the middle of its longest axis. This allows you to make a pillar of fire that you can walk around, or an energy beam stretched across the room.

Specific parameter definitions for deform keywords

div

This is roughly defined as the size of the waves that occur. It is measured in game units. Smaller values create a greater density of smaller wave forms occurring in a given area. Larger values create a lesser density of waves, or otherwise put, the appearance of larger waves. To look correct this value should closely correspond to the value (in pixels) set for tessSize (tessellation size) of the texture. A value of 100.0 is a good default value (which means your tessSize should be close to that for things to look "wavelike").

func

This is the type of wave form being created. Sin stands for sine wave, a regular smoothly flowing wave. Triangle is a wave with a sharp ascent and a sharp decay. It will make a choppy looking wave forms. A square wave is simply on or off for the period of the frequency with no in between. The sawtooth wave has the ascent of a triangle wave, but has the decay cut off sharply like a square wave. An inversesawtooth wave reverses this.

base

This is the distance, in game units that the apparent surface of the texture is displaced from the actual surface of the brush as placed in the editor. A positive value appears above the brush surface. A negative value appears below the brush surface. An example of this is the Quad effect, which essentially is a shell with a positive base value to stand it away from the model surface and a 0 (zero) value for amplitude.

amplitude

The distance that the deformation moves away from the base value. See Wave Forms in the introduction for a description of amplitude.

phase

See Wave Forms in the introduction for a description of phase.

frequency

See Wave Forms in the introduction for a description of frequency/

Design Note: The <div> and <amplitude> parameters, when used in conjunction with liquid volumes like water should take into consideration how much the water will be moving. A large ocean area would have have massive swells (big div values) that rose and fell dramatically (big amplitude values). While a small, quiet pool may move very little.

Q3Map2 Specific Keywords

All of q3map2 specific keywords are processed during map compile. If this keywords changes, recompiling the map is required for the changes to take effect.

General use keywords

q3map_globalTexture

This keyword disables texture coordinates optimization for texture projection on brusshes (optimization tries to keep texture coordinates closer to 0-1 range), making them to be related to the world center, not brush. This is useful when applying tcMod scale to several adjastent brushes, which can be wrong if not using this parameter.

q3map_tessSize <amount>

Controls the tessellation size (how finely a surface is chopped up in to triangles), in game units, of the surface. This is only applicable to solid brushes, not curves, and is generally only used on surfaces that are flagged with the deformVertexes keyword.

IMPORTANT NOTE:Abuse of this can create a huge number of triangles.

q3map_backShader <shadername>

This allows a brush to use a different shader on the back side of surface. By way of example, this would allow a water brush (or other) surfaces to have a different appearance when seen from the inside. 'Design Notes: q3map_backShader is a better alternative to cull none because it is consistent with lighting. Drawback are that it produces more geometry data (larger loading times, more memory required), which may be abusing if surface is used many times.

Lightmap keywords

q3map_sun <red> <green> <blue> <intensity> <degrees> <elevation>
This keyword in a sky shader will create the illusion of light cast into a map by a single, infinitely distance light source (sun, moon, hellish fire, etc.). This is only processed during the lighting phase of  q3map.
<red> <green> <blue> Color is described by three normalized rgb values. Color will be normalized to a 0.0 to 1.0 range, so it doesn't matter what range you use.
<intensity> is the brightness of the generated light. A value of 100 is a fairly bright sun. The intensity of the light falls off with angle but not distance.
<degrees> is the angle relative to the directions on the map file. A setting of 0 degrees equals east. 90 is north, 180 is west and 270 is south.
<elevation> is the distance, measured in degrees from the horizon (z value of zero in the map file). An elevation of 0 is sunrise/sunset. An elevation of 90 is noon
DESIGN NOTE: Sky shaders should probably still have a q3map_surfacelight value. The "sun" gives a strong directional light, but doesn't necessarily give the fill light needed to soften and illuminate shadows. Skies with clouds should probably have a weaker q3map_sun value and a higher q3map_surfacelight value. Heavy clouds diffuse light and weaken shadows. The opposite is true of a cloudless or nearly cloudless sky. In such cases, the "sun" or "moon" will cast stronger, shadows that have a greater degree of contrast.
Design Trick: Not certain what color formula you want to use for the sun's light? Try this. Create a light entity. Use the Q3Radiant editor's color selection tools to pick a color. The light's _color key's value will be the normalized RGB formula. Copy it from the value line in the editor (CTRL+c) and paste it into your shader.

q3map_surfaceLight <light value>
The texture gives off light equal to the <light value> set for it. The relative surface area of the texture in the world affects the actual amount of light that appears to be radiated. To give off what appears to be the same amount of light, a smaller texture must be significantly brighter than a larger texture. Unless the qer_lightimage keyword is used to select a different source for the texture's light color information, the color of the light will be the averaged color of the texture.

q3map_lightimage <texturepath/texturename>
The keyword q3map_lightimage generates lighting from the average color of the TGA image specified by the q3map_lightimage.
The keyword sequence for generating light on a q3map_surfacelight should be ordered as follows:
1) q3map_lightimage ; (the texture providing the light and the color of the light)
2) qer_editorimage ; (the editor-only image used to select the source map for the texture)
3) the average color of the light emitted from the shader is calculated from the lightimage.)

q3map_lightsubdivide <value>
This allows the user to define how large, or small to make the subdivisions (triangles) in a textured surface, particularly aimed at light-emitting textures like skies. It defaults to 120 game units, but can be made larger (256 or 512) for sky boxes or smaller for light surfaces at the bottoms of cracks. This can be a dominant factor in processing time for q3map lighting.