Technical Art #1 Render Pipeline

#TechnicalArtist #GameDesign #ComputerGraphics #GameEngine

This note writes about the graphics rendering and render pipeline in computer graphics.

Pipeline overview

The above diagram shows a typical overview of a render pipeline. There may be different names for each phase, and we will go through some most important stages of a TBDR(Tile Based Deferred Rendering).

Primitives Assembly

1. Preparing the data of vertices for render in the memory.

2. Sending these data to the GPU.

3. The most basic primitives are triangles.

4. Even though we need 3 points for saving the 3 vertices of a triangle, many of them are actually repetitive (triangles share vertices or edges). Thus, usually we use an indexed list to store the distinguished vertices, and maintain another buffer or list to save the order of the triangles.

5. For example, without indexing we need to save 6 points for the geometry. However, with indexing we need only 4 points. We can just refer to different triangles by using index triples, for example (0,1,2) to indicate the triangle (v0,v1,v2).

Vertex Shader

1. Vertex shaders process a bunch of computation related to the vertices.

Geometry Shader

1. Filters some computations made by the former step.

2. Can generate new primitives. (Potentially for generating hair, water surface, etc)

Stream Output

1. Further process the results we get from geometry shader.

2. Ensures correct render of the new primitives generated by the former step.

Clipping

1. Filters those vertices that are not visible (in the vision clipping plane or the vision pyramid).

2. Specifically in this step we refer to the clipping that "clip" the vertices that are not within the boundary of the screen space.

Screen Mapping

1. Map the 3D information to the 2D screen coordinates.

Triangle Assembly/Rasterization

1. Translate the vector triangles to rasterized triangles.

Pre-Z optimization

1. Write in the depth information (z-value).

2. Compute the occlusion, transparency, etc.

Fragment Shader

1. Compute some after-effects, for example, lighting, fog, shadow...etc.

2. Programmable.

Merge

1. Merge the information we got by the step.

2. Some additional computation, for example, stencil test, depth tests... etc.

---

The above describes some basic information for TBDR. There are also some other operations that manipulate with the vertices, for example, vertex animation. This refer to the animation that records vertex information in keyframes, and compute the middle frames with interpolation.

We also have some computer-aid animations supported by cosine and sine functions. These may be common in creating waves/wind/leaves etc.

---

Texture Mapping

The most frequently processed information is texture mapping. We have a 2D texture, or a sprite, and we want to wrap that sprite to a 3D model. Therefore, we need a scheme for telling where to put a specific point on the 2D texture.

There are some common addressing schemes: wrapping, mirroring, clamping, and bordering.

- wrapping: simply pave the texture.

- mirroring: the texture of odd indices will mirror the texture of even indices.

- clamping: trivially take the boundary pixels when out of boundary.

- bordering: trivially take another selected color when out of boundary.

Texture Filtering

Mosaics on the texture could be annoying. Some filter/anti-aliasing schemes will be necessary.

There are some common schemes: nearest, linear/bilinear/trilinear, mipmap.

- nearest: trivially take the nearest pixel for interpolation.

- linear: take the nearest bunch of pixels and make linear computation for interpolation.

- mipmap: store several levels of depth, and adaptively select the correct mipmap texture to apply.

---

Depth

Finally the problem of depth rendering. Since overlapping do exist in the reality, we cannot trivially define that object A is in front of object B. Therefore, we apply the z-buffer algorithm.

z-buffer algorithm

1. Initialize the z-buffer zbuf.
2. Initialize the color buffer cbuf.
3. If (primitive p is being written in cbuf):
      If (pixel not occupied):
         - write depth(p) into zbuf.
      Else:
         # pixel occupied by primitive q
         If depth(p) > depth(q):
            - Discard p.
         Else:
            - write p into cbuf and write depth(q) into zbuf.

Limited situation: we can't apply z-buffer algorithm to those without depth information. In that case, we use the traditional occlusion algorithm, which compute the final shading with alpha values.

---

Reference:

1. Indexed Triangles, https://www.google.com/url?sa=i&url=http%3A%2F%2Fwww.opengl-tutorial.org%2Fintermediate-tutorials%2Ftutorial-9-vbo-indexing%2F&psig=AOvVaw2LOCmMCBQJRcAQNKErKRs2&ust=1617750214739000&source=images&cd=vfe&ved=2ahUKEwj_zKyom-jvAhVWzIsBHejyA9AQr4kDegUIARCjAQ