This note mainly covers an overview of Metal and the structure of the simplest Metal application.
Rendering Pipeline
Metal's rendering pipeline can be summarized in four steps:
- Initialize Metal
- Load the model
- Set up the render pipeline
- Render
Like many other graphics APIs, although this process is concise and clear, each step requires many low-level instructions to complete. In this simplest application, we will not introduce too many features of each part, but only observe the general pipeline process from a high-level perspective.
Metal needs to be compiled through Xcode. We use the new macOS Blank Playground in Xcode for this demonstration. (File > New > Playground > macOS > Blank)
Window
Window management mainly uses Apple's official MetalKit library. To view our rendering results in real-time using the assistant editor in Playground, we use these two imports:
Plain Textimport PlaygroundSupport
import MetalKit
Window creation is based on the MTKView class in MetalKit.
Plain Text// MTKView requires a Metal-capable GPU device, so the first step is to declare a device constant
guard let device = MTLCreateSystemDefaultDevice() else {
fatalError()
}
// Set window size
let frame = CGRect(x: 0, y: 0, width: 600, height: 600)
// Declare MTKView
let view = MTKView(frame: frame, device: device)
// Clear background color
view.clearColor = MTLClearColorMake(red: 1, green: 1, blue: 1, alpha: 1)
Models loaded in Metal are also called primitives (图元). Here we first use MDLMesh to create a sphere instead of loading a mesh from an external file. Later we will also discuss how to load an external model file.
Plain Text// For a primitive, we need an allocator to manage its memory
let allocator = MTKMeshBufferAllocator(device: device)
// Use MDLMesh to create a sphere mesh
let mdlMesh = MDLMesh(sphereWithExtent: [0.25, 0.25, 0.25],
segments: [100, 100],
inwardNormals: false,
geometryType: .triangles,
allocator: allocator)
// Convert it to MTKMesh so that Metal can use it
let mesh = try MTKMesh(mesh: mdlMesh, device: device)
Shader Setup
We will introduce the details of Metal's render pipeline in the next note, but in fact the render pipelines of different APIs are largely similar. For Metal, the two most important parts are still the vertex shader and fragment shader. Apple officially names them Vertex Function and Fragment Function. We will use these official terms here.
In formal projects, they are written in .metal files using Metal Shading Language (MSL). MSL is a subset of C++ and follows C++ basic syntax. Here, we first use the approach of reading code strings to write the most basic vertex and fragment functions.
Plain Textlet shader = """
#include <metal_stdlib>
using namespace metal;
struct VertexIn {
float4 position [[attribute(0)]];
};
vertex float4 vert(const VertexIn vertex_in [[stage_in]]) {
return vertex_in.position;
}
fragment float4 frag() {
return float4(1,0,0,1);
}
"""
// Set up shaders
let library = try device.makeLibrary(source: shader, options: nil)
let vert = library.makeFunction(name: "vert")
let frag = library.makeFunction(name: "frag")
Render State Machine Setup
Like other APIs, Metal also adopts a state machine strategy. The characteristic of a state machine is that before the state changes, all information in the device can be assumed not to change, which allows the GPU to work more efficiently. Metal needs to inform the render pipeline of the information it needs to know through a medium called a descriptor (描述符).
The render state machine descriptor needs to have the following information and can be set with the following code:
Plain Text// State machine descriptor
let pipelineDescriptor = MTLRenderPipelineDescriptor()
// Describe the color read order as r/g/b/a, each color described by 8-bit unsigned integer
pipelineDescriptor.colorAttachments[0].pixelFormat = .rgba8Uint
// Describe vertex function
pipelineDescriptor.vertexFunction = vert
// Describe fragment function
pipelineDescriptor.fragmentFunction = frag
// Describe vertex descriptor (we will introduce this concept later)
pipelineDescriptor.vertexDescriptor = MTKMetalVertexDescriptorFromModelIO(mesh.vertexDescriptor)
// Create render state
let rpState = try device.makeRenderPipelineState(descriptor: pipelineDescriptor)
The resulting rpState is called a Render Pipeline State Object.
Render Command Encapsulation
Each frame contains multiple commands we send to the GPU. These commands need to be encapsulated through a Render Command Encoder. Then, the Render Command Buffer encapsulates multiple command encoders. Finally, the Render Command Queue encapsulates multiple render command buffers in order.
In terms of hierarchy:
- The Render Command Queue (RCQ) is created once and manages the execution order of all render command buffers. The Render Command Buffer (RCB) holds render commands. The Render Command Encoder (RCE) holds render commands for each frame. Each frame is generated.
Therefore, we first declare a render command queue as the basis for command execution.
Plain Textguard let commandQueue = device.makeCommandQueue() else {
fatalError()
}
Then, declare a command buffer and command encoder.
Plain Textguard
// Create render command buffer
let commandBuffer = commandQueue.makeCommandBuffer(),
// Create render pass descriptor
let renderPassDescriptor = view.currentRenderPassDescriptor,
// Create render command encoder, requires pass descriptor
let renderEncoder = commandBuffer.makeRenderCommandEncoder(descriptor: renderPassDescriptor)
else { fatalError() }
Here, the concept of a Render Pass appears. Each render can execute multiple render passes, each with its own responsibility—for example, some are responsible for rendering shadows, some for rendering color, some for rendering reflections. We combine the render passes to produce the final pixel color on screen. The render pass descriptor's role is to inform the render destination, such as whether we are rendering to a texture, or whether we need to preserve the texture during the render pass, etc.
On the render command encoder, we input our commands to the GPU. Here, we set the render state and the vertex buffer of the mesh we need to render.
Plain Text// Set vertex state
renderEncoder.setRenderPipelineState(rpState)
// Set vertex buffer
renderEncoder.setVertexBuffer(mesh.vertexBuffers[0].buffer, offset: 0, index: 0)
Submesh
Sometimes our Mesh may consist of multiple Submeshes, because artists may assign different materials to different submeshes when creating the model. In this simplest example, there is only one submesh because we only have a simple sphere. So, the step to set up the submesh is also simple.
Plain Textguard let submesh = mesh.submeshes.first else { fatalError() }
Sending Render Commands
Finally, we need to send the render command Draw Call to the GPU. We do this through the drawIndexedPrimitives(: : : : : ) function in the encoder class.
Plain TextrenderEncoder.drawIndexedPrimitives(type: .triangle,
indexCount: submesh.indexCount,
indexType: submesh.indexType,
indexBuffer: submesh.indexBuffer.buffer,
indexBufferOffset: 0)
Then, we end our render command input and finalize our render command buffer.
Plain Text// End encoding
renderEncoder.endEncoding()
guard let drawable = view.currentDrawable else {
fatalError()
}
// Finalize render command buffer
commandBuffer.present(drawable)
commandBuffer.commit()
// Render on Playground screen
PlaygroundPage.current.liveView = view
References:
- Metal by Tutorials, Beginning Game Engine Development With Metal, by Caroline Begbie & Marius Horga
- Using Metal to Draw a View's Contents,https://developer.apple.com/documentation/metal/using_metal_to_draw_a_view_s_contents
- Using a Render Pipeline to Render Primitives，https://developer.apple.com/documentation/metal/using_a_render_pipeline_to_render_primitives