Metal #16 Animation

This note covers skeletal animation in Metal, from keyframe interpolation to GPU skinning.

Why transforms are stored in three parts

A transform matrix can be interpolated, but linearly interpolating matrices is wrong. A value halfway between two rotation matrices is not a rotation matrix — it introduces scale and shear, which shows up as a model collapsing strangely in the middle of an animation.

So animation data always splits a transform into three parts, stored and interpolated separately:

Swiftstruct Transform {
    var translation: SIMD3<Float>
    var rotation:    simd_quatf       // quaternion
    var scale:       SIMD3<Float>
}

• Translation and scale interpolate linearly.
• Rotation interpolates as a quaternion, with spherical linear interpolation (slerp).

Quaternions are the key piece. Euler angles suffer gimbal lock and their interpolation path depends on rotation order; rotation matrices cannot be interpolated at all. A quaternion represents a rotation with four numbers, and interpolating along the shortest arc on the sphere is exactly the "most natural rotation transition" we want.

Swiftfunc lerp(_ a: Transform, _ b: Transform, _ t: Float) -> Transform {
    Transform(
        translation: mix(a.translation, b.translation, t: t),
        rotation:    simd_slerp(a.rotation, b.rotation, t),
        scale:       mix(a.scale, b.scale, t: t)
    )
}

simd_slerp handles the double cover problem for you: q and −q represent the same rotation but interpolate along completely different paths, one the short arc and one the long way round. If you implement it yourself, check the sign of the dot product and negate when needed.

Skeletons and skinning

A skeleton is a tree of transforms. Each bone has a parent, and its world transform is the parent's world transform times its own local transform:

Swiftfunc computeWorldTransforms(_ skeleton: Skeleton, _ pose: [Transform]) -> [float4x4] {
    var world = [float4x4](repeating: .identity, count: pose.count)
    for i in 0..<pose.count {
        let local = float4x4(pose[i])
        let parent = skeleton.parentIndex[i]
        world[i] = parent < 0 ? local : world[parent] * local
        // precondition: bones stored in topological order, so parent index < child index
    }
    return world
}

That comment matters. Storing bones in topological order turns this into a single linear pass with no recursion, and makes it parallelizable. Do the sort at import time.

The skinning matrix is what finally reaches the shader:

Plain TextskinMatrix[i] = worldTransform[i] * inverseBindPose[i]

inverseBindPose is the inverse of the bone's world transform in the bind pose. Its job is to bring a vertex from model space into the bone's local space first, so that the bone's current world transform can then place it correctly. Omit it and the vertex receives the transform twice over.

GPU skinning

Each vertex carries up to four bone indices and weights:

Ctypedef struct {
    vector_float3 position;
    vector_float3 normal;
    vector_float2 uv;
    vector_ushort4 jointIndices;   // ushort is plenty, and halves the space
    vector_float4  jointWeights;
} SkinnedVertex;

Linear blend skinning (LBS) in the vertex function:

MSLvertex VertexOut skinned_vertex(SkinnedVertex in [[stage_in]],
                                constant float4x4 *skinMatrices [[buffer(12)]],
                                constant Uniforms &u [[buffer(11)]])
{
    float4x4 skin =
        skinMatrices[in.jointIndices.x] * in.jointWeights.x +
        skinMatrices[in.jointIndices.y] * in.jointWeights.y +
        skinMatrices[in.jointIndices.z] * in.jointWeights.z +
        skinMatrices[in.jointIndices.w] * in.jointWeights.w;

    float4 posOS = skin * float4(in.position, 1.0);
    float3 nrmOS = (skin * float4(in.normal, 0.0)).xyz;   // w = 0: unaffected by translation

    VertexOut out;
    out.position = u.modelViewProjection * posOS;
    out.normalWS = normalize(u.normalMatrix * nrmOS);
    return out;
}

Weights must be normalized to sum to 1, or the model inflates or shrinks during animation. Checking this at export is cheaper than compensating in the shader.

LBS has a well-known defect: the candy-wrapper artefact. When one bone rotates close to 180° relative to another — an extreme wrist or elbow twist — the linear blend of the two transforms collapses the intermediate vertices towards the axis, and the limb looks like a twisted sweet wrapper. The remedies are dual quaternion skinning (DQS), which interpolates rotations rather than matrices, or the more practical route of adding intermediate twist bones to distribute the rotation.

Vertex function or compute?

The version above skins inside the vertex function, which recomputes it for every pass. A character rendered three times — main pass, shadow pass, depth prepass — is skinned three times.

The alternative is to skin once in a compute shader, write the result into an output buffer, and have every later pass consume the already-skinned vertices:

MSLkernel void skin_vertices(device const SkinnedVertex *in [[buffer(0)]],
                          device Vertex *out [[buffer(1)]],
                          constant float4x4 *skinMatrices [[buffer(2)]],
                          uint id [[thread_position_in_grid]])
{
    float4x4 skin = blendMatrices(in[id], skinMatrices);
    out[id].position = (skin * float4(in[id].position, 1)).xyz;
    out[id].normal   = (skin * float4(in[id].normal, 0)).xyz;
    out[id].uv       = in[id].uv;
}

Which to choose?

• The character is rendered once or twice → the vertex function is simpler and saves an intermediate buffer's memory and bandwidth.
• The character is rendered three or more times, or something else needs the skinned vertices (collision, cloth, updating a ray tracing BLAS) → compute skinning.

Ray tracing is the decisive case: the acceleration structure needs skinned world-space vertices, and those have to genuinely exist in a buffer. A vertex function's temporary result is not available to anything else.

Sampling keyframes and blending

Keyframes are usually not evenly spaced, so sampling needs a binary search for the interval containing the current time:

Swiftfunc sample(_ channel: AnimationChannel, at time: Float) -> Transform {
    var lo = 0, hi = channel.times.count - 1
    while lo + 1 < hi {
        let mid = (lo + hi) / 2
        if channel.times[mid] <= time { lo = mid } else { hi = mid }
    }
    let t = (time - channel.times[lo]) / (channel.times[hi] - channel.times[lo])
    return lerp(channel.values[lo], channel.values[hi], t)
}

Cache the previous frame's index and search linearly from there next frame — it usually hits within a step or two, which is far cheaper than a binary search, because animation time advances monotonically.

Animation blending interpolates two poses by weight, which is exactly why poses are kept as arrays of Transform rather than arrays of matrices:

Swiftfunc blend(_ a: [Transform], _ b: [Transform], _ t: Float) -> [Transform] {
    zip(a, b).map { lerp($0, $1, t) }
}

A walk-to-run transition, or a layered blend of an upper body aiming while the lower body moves, is built on this operation. Compute world transforms and skinning matrices only after blending — never the other way round, because matrices cannot be interpolated, which is the point this note opened with.

The next note begins the ray tracing chapters.