Here’s a twenty-one second demo of an open field at a density of 0.4 (more on what that means later). For this scene, I used a terrain heightmap generated from simplex noise.

Introduction

The 2021 Advanced Graphics Summit talk Procedural Grass in ‘Ghost of Tsushima’ presents an efficient and beautiful implementation of procedural grass that supports the game’s artistic vision while running within the PS4’s performance budget.

My goal was to preserve the feel and appearance of the game’s procedural grass without the more sophisticated artist authored assets and wind system. Instead, I use a scrolling Perlin noise texture for wind strength and orientation samples, a cellular noise texture to simulate clumping of nearby blade meshes, rounding of normals in the vertex shader, and albedo interpolation based on the height of the fragment with respect to the blade mesh. To tie it all together, a custom lighting pass simulates ambient bounces and occlusion, as well as blending of blade colors in the distance.

The approach comes down to three main tricks:

• Use GPU Instancing to draw chunks of grass blades together, avoiding millions of separate draw calls.
• Use moving noise patterns for wind and clusters of similar values to simulate grass clumping.
• Use color and lighting to make flat blade meshes look rounded and give the field depth with ambient occlusion.

Drawing Millions of Blades

The world map is divided into tiles of a specified tile size. Using a density parameter $d \in (0,1]$, we can prepare $\mathrm{row}^2$ evenly spaced grass blade meshes in a grid formation, where $\mathrm{row} = \mathrm{size}_t \times 10d$. Here, $\mathrm{size}_t$ is the side length of a square tile and $\mathrm{row}$ is the number of blades along each side. This gives us an easy way to modify how many grass blades we want in each tile by shifting $d$. Each tile is instantiated using Godot’s MultiMesh class, which draws all its grass blade meshes together. This matters a lot for performance, as we’d otherwise need a separate draw call for each of the millions of grass blades.

The simple UV map lets us use the $uv$ coordinates as the relative width and height of the grass blade in the shaders later on.

Making the Grass Move and Clump

Using a perlin noise texture and a wind speed, we can create the illusion of unique grass blades which appear to flow and bend with the wind. We can use another noise texture to simulate grass clumping, a natural phenomenon where nearby blades of grass will exhibit similar properties. Without it, we’d get grass of random heights all over the place which looks unnatural. I implement both of these things using a vertex and a fragment shader.

Bending in the Wind

To emulate wind we can “scroll” a wind texture across the field of grass. We effectively map the texture to the entire grid by always sampling relative to each blade’s position in the shader. To achieve scrolling, we add a time offset to the sampling position. This essentially sweeps the same wind strength values across a direction, achieving the illusion of a continuous gust of wind.

I use Perlin noise for the scrolling wind texture because its values change smoothly: nearby values are similar. This means neighboring blades get similar wind strength and direction, so the grass moves together instead of each blade jittering as we scroll the wind texture. Feel free to experiment with other kinds of noise. I generated the texture with Godot’s FastNoiseLite library. I enabled domain warping for a more interesting wind pattern and made the texture seamless to avoid strange patterns when scrolling across the edges.

I sample the Perlin noise texture at the same position for the wind’s target orientation and rotation strength, then at another position for bending. The bending sample scrolls faster and uses a height-based time offset, so points farther along the blade sample a bit ahead in time, emulating turbulence.

Giving Each Blade Some Variation

Individual blades are different from each other, but remain similar to nearby blades. I combine a value unique to each blade with a value from the clump texture, then use the result for width and height variation. By using a mix of the unique and clump values, we get groups of blades with similar heights and orientations (clump factor) that are still slightly different from each other.

I use cellular noise for the grass clump texture. Nearby values cluster together, with visible boundaries between groups of similar values. I generated this texture with FastNoiseLite too, using the default settings.

Making Flat Blades Look Rounded

The grass blades are simple, flat meshes. We can artificially round them without adding extra geometry by adjusting the angle of normals close to the edge of the blade. By pointing these further outwards we get the illusion of curvature on the flat mesh.

Coloring the Grass

I interpolate between a base color and a tip color to color every blade of grass. I also color distant fragments closer to the average color to simulate blending at far distances.

Without distance blending

With distance blending

Giving the Field Depth

To tie it all together, a custom lighting pass simulates ambient bounces and occlusion. I darken the base of the grass based on the field’s density, which gives the field more depth.

Results

The full Godot implementation shown in the video runs at 60-230 FPS on my test computer, depending on the density. This measurement is for the Godot scene, rather than the simplified browser demo. Individual blades are different from each other but remain similar to nearby blades, and the scrolling Perlin noise texture does a pretty good job of emulating wind.

Conclusion

I’m happy with how closely this captures the feel of Ghost of Tsushima’s grass. There’s still more I’d like to try, including the game’s specular lighting and shadows, and its bezier curve approach to bending individual blades.

You might notice that blades being viewed from their side look thin and not that great. The original Tsushima implementation rotates blades close to the camera to hide this problem. I’d like to work on this next if I do put it in a game.

For that, I’d also need to add performance optimizations like view frustum culling and LOD switching (this person implemented it).

Source Code and References

Source code for the project.

1. Procedural Grass in ‘Ghost of Tsushima’
2. Texture Generation using Random Noise
3. Similar implementation of this emulation
4. How do Major Video Games Render Grass?
5. Modern Foliage Rendering

Acknowledgment

A lot of the implementation details came from 2Retr0’s approach to the same idea. This is mostly a simplified version with reduced scope. Their version is also much better looking.

I’d like to thank the maintainers and contributors of the Godot game engine for providing an excellent platform on which to experiment with computer graphics.

I’d also like to thank the maintainers and contributors of the Blender 3D software for providing an excellent platform on which to create models and specify UV mappings.

Here’s a small overview of what I learned.

Modeling Curves and Surfaces

In Assignment 1, I implemented cubic Bézier and B-Spline curves, conversion between their control-point representations, and bicubic tensor product patches. This gave me a better understanding of non-polygonal representations for surfaces. Since the patches are continuous and differentiable, we can easily get the normal for lighting as well.

Attaching a Mesh to a Skeleton

Assignment 2 covered hierarchical transforms and skeletal subspace deformation, also called linear blend skinning. I built a node hierarchy and attached a mesh using per-vertex joint weights. Rotating a parent joint carries its descendants with it, while the weights let a vertex respond to several joints at once.

The approach is to first move a vertex from the character’s bind pose into a joint’s bind-pose coordinate frame, then move it back using that joint’s current transform. I also passed accumulated transforms down the scene tree when collecting objects for rendering, avoiding repeated walks up the hierarchy. I believe this is very similar (if not identical) to how videogames handle object rendering trees.

glm::mat4x4 &bind_pose_joint_to_character_inverse_ = Bs_[j];
glm::mat4x4 &joint_to_character = joint_to_character_matrices[j];
glm::vec4 position_contribution = (attachment_weights_[i][j] *
joint_to_character * bind_pose_joint_to_character_inverse_ * bind_pose_position);

new_positions->at(i) += glm::vec3(position_contribution);

Simulating Motion with Particles and Springs

For Assignment 3, I implemented forward Euler, explicit trapezoidal, and fourth-order Runge–Kutta integration, then used them for circular motion, a spring-connected pendulum, and wireframe cloth. The particle system combines gravity, drag, and spring forces. The integrator then advances its positions and velocities over time.

The cloth is a grid with structural springs connecting neighbors, shear springs connecting diagonals, and flex springs spanning two grid edges. These resist stretching, diagonal collapse, and bending. This was more of a programming exercise, but it was still cool to see how to make a cloth simulation.

Rendering with Rays

We were tasked with building a CPU ray tracer. I implemented sphere, plane, and triangle intersections, Phong lighting, shadow rays, and recursive reflections. Rays are transformed into object space for intersection, while normals return to world space using the inverse transpose of the object’s transform.

if(bounces > 0) {
  // ray bounces
  const glm::vec3 ray_bounce_origin    = hit_position + EPSILON * eye_reflection;
  const glm::vec3 ray_bounce_direction = eye_reflection;
  const Ray       ray_bounce(ray_bounce_origin, ray_bounce_direction);
  HitRecord bounce_record = HitRecord();
  color += material_specular_color * TraceRay(ray_bounce, bounces - 1, bounce_record);
}

Casting Shadows in OpenGL

Assignment 5 involved texture mapping and shadows from a directional light. I rendered the scene from the light’s viewpoint into a depth texture, then sampled that texture while shading the camera’s view. If a fragment is farther from the light than the stored depth, another surface is blocking it.

This involved framebuffer objects, texture sampling, coordinate transforms, and GLSL. A small depth bias helps prevent a surface from incorrectly shadowing itself because of numerical error.

vec3 proj_coords = (world_to_light_ndc_matrix * vec4(world_position, 1.0)).xyz;
proj_coords = proj_coords * 0.5 + 0.5;
float this_depth     = proj_coords.z;
float occluder_depth = texture(shadow_texture, proj_coords.xy).r;

float bias = 0.005;
if(occluder_depth < this_depth - bias) {
    return vec3(0.0, 0.0, 0.0);
}

These assignments preceded my final project, emulating Ghost of Tsushima’s procedural grass in Godot.

Updating a singular boid’s position requires the normalization of several vectors, which presents the need to develop a resource-efficient and low-latency module for vector normalizations. To acquire the three-dimensional position of the target, we implemented stereo vision using a pair of FPGAs — each with their own camera — which communicate with each other through the SPI protocol. Using the target’s positional information, we computed the positions of the boids every frame in three dimensions and overlaid them on the camera footage via a double frame buffer.

The double frame buffer also presented a unique challenge since, due to overlaying the boids over the camera feed, there was a limited amount of additional pixels/DRAM requests that could be made every frame, amounting to roughly 25% of the screen’s area.

For this class project, we committed to having multiple boids follow the target, moving every frame and being rendered at their computed positions. To demonstrate proper depth calculation, we modified every boid’s size based on its depth. The implemented boid update module includes only the target-following force; alignment, cohesion, and separation remain future work.

Contributions

This was a final project for MIT’s 6.2050 Digital Systems Laboratory, built with Sreeja Thipireddy. I was responsible for the implementation of the normalizer, boid manager, and boid update modules. Sreeja was responsible for the implementation of the inter-FPGA communication protocol, depth calculation module, double frame buffer, and modifications to the traffic generator. The renderer module was implemented in collaboration.

Vector Normalization

To compute the necessary vectors in the boid computations, it is necessary to normalize vectors. To normalize a vector, it’s sufficient to multiply the vector’s components by the reciprocal of the vector’s magnitude.

For a vector $\mathbf{v} = (v_x, v_y, v_z)$, we first compute its squared magnitude:

This lets us compute one reciprocal square root and reuse it for all three components.

Initial Approximation

We can compute an initial approximation to the reciprocal of the square root via the position of the number’s most significant bit.

Using $x \approx 2^b$, we obtain the initial approximation

For a nonzero integer $x$, the position $b$ of its most significant bit is

I implemented a custom combinational module with a tree-based approach which computes the count of leading zeroes. The result for any n-bit number can be derived from the result of its left and right halves. If the upper half of the number contains a 1 bit, the result comes from that half. Otherwise, the result is the width of the upper half plus the leading-zero count of the lower half. This gives us a combinational tree for finding the CLZ. The implementation starts with 32 two-bit pairs: 00 has two leading zeroes, 01 has one, and a pair starting with 1 has none. Five merge layers combine these counts into the result for the full 64-bit input. In clz_merge, the highest bit indicates that a whole block is zero; the remaining bits encode its leading-zero count.

module clz_merge #(
    parameter WIDTH
) (
    input wire [WIDTH-1:0] left,
    input wire [WIDTH-1:0] right,

    output logic [WIDTH:0] out
);

    logic both_full, left_not_full;
    assign both_full = left[WIDTH-1] & right[WIDTH-1];
    assign left_not_full = left[WIDTH-1] == 1'b0;

    logic [WIDTH-2:0] right_chopped;
    assign right_chopped = right[WIDTH-2:0];

    logic [WIDTH:0] result;

    always_comb begin
        if(both_full) begin
            result = {1'b1, {WIDTH{1'b0}}};
        end else if(left_not_full) begin
            result = {1'b0, left};
        end else begin
            result = {2'b01, right_chopped};
        end
    end

    assign out = result;

endmodule

module clz_64 (
    input wire [63:0] data,
    output logic [6:0] out
);
    
    logic [1:0] bit_pair [31:0];
    logic [6:0] assembly_layer [5:0] [31:0];

    genvar j;
    generate
        for (j = 0; j < 32; j++) begin
            assign bit_pair[j] = data[2*j+1:2*j];

            assign assembly_layer[0][j] =
                (bit_pair[j] == 2'b00) ? 6'b000010 :
                (bit_pair[j] == 2'b01) ? 6'b000001 :
                                         6'b000000 ;
        end
    endgenerate

    genvar layer, i;
    generate
        for (layer = 1; layer < 6; layer++) begin
            for (i = 0; i < (32 >> (layer - 1)); i += 2) begin
                clz_merge #(
                    .WIDTH(layer + 1)
                ) m_clz_merge (
                    .left(assembly_layer[layer - 1][i + 1][layer:0]),
                    .right(assembly_layer[layer - 1][i][layer:0]),
                    .out(assembly_layer[layer][i >> 1][layer+1:0])
                );
            end
        end
    endgenerate

    assign out = assembly_layer[5][0];

endmodule

The squared components of the vector are shifted right by FP_SHIFT before being added, so $x$ still has FP_SHIFT fractional bits. If $F = \mathrm{FP\_SHIFT}$ and $X$ is the stored squared magnitude, its real value is $X/2^F$. The stored reciprocal-square-root estimate therefore needs the scale factor

The code uses e = (3*FP_SHIFT - b) >> 1 for this exponent. It zero-pads the squared magnitude to 64 bits before counting leading zeroes, then uses b = 63 - len_sqr_clz to find its most significant bit. The clocked assignment y_pipe[2] <= (1 << e) << FP_SHIFT gives the estimate an extra FP_SHIFT bits, which are removed by y_pipe_shifted before it is used in the next multiplication.

clz_64 m_clz_64 (
        .data({{(64-DOT_PRODUCT_WIDTH){1'b0}},len_sqr_pipe[1]}),
        .out(len_sqr_clz)
    ); // 1

    assign b = 63 - len_sqr_clz; // 1
    assign e = (3*FP_SHIFT - b) >> 1; // 1

Newton-Raphson Refinement

To improve this initial guess, I computed several iterations of a Newton-Raphson approximation scheme. Through a C simulation I found that 4 iterations was enough to achieve visually satisfactory results.

For an estimate $y_n$ of $1/\sqrt{x}$, the iteration is

At the desired value, $xy_n^2 = 1$, so the correction factor is one. Each iteration employs only multiplication and substraction operations. In fixed-point arithmetic, the products also need to be shifted to account for the fractional bits. I split each iteration across three clock cycles: square the current estimate, multiply by the squared magnitude, and apply the correction. THREE_HALFS stores 1.5 in fixed-point format. The expression half_ay2_pipe[i] >> (2 * FP_SHIFT + 1) accounts for the product’s extra fractional bits and divides it by two. The generated stages implement the equivalent form $y_{n+1} = y_n(1.5 - xy_n^2/2)$.

genvar pipe_idx;
    generate
        for(pipe_idx = 3; pipe_idx < 3 + 3*ITERATIONS; pipe_idx += 3) begin
            always_ff @( posedge clk ) begin
                y2_pipe[pipe_idx] <= y_pipe_shifted[pipe_idx-1] * y_pipe_shifted[pipe_idx-1]; // 3*i
                // y2_pipe[pipe_idx+1] <= y2_pipe[pipe_idx];
                // y2_pipe[pipe_idx+2] <= y2_pipe[pipe_idx+1];

                // half_ay2_pipe[pipe_idx] <= half_ay2_pipe[pipe_idx-1];
                half_ay2_pipe[pipe_idx+1] <= (len_sqr_pipe[pipe_idx] * y2_pipe[pipe_idx]); // 3*i + 1
                // half_ay2_pipe[pipe_idx+2] <= half_ay2_pipe[pipe_idx+1];

                y_pipe[pipe_idx] <= y_pipe[pipe_idx-1];
                y_pipe[pipe_idx+1] <= y_pipe[pipe_idx];
                y_pipe[pipe_idx+2] <= ($signed(y_pipe_shifted[pipe_idx+1]) * ($signed({1'b0, THREE_HALFS}) - $signed({1'b0, half_ay2_pipe_shifted[pipe_idx+1]}))); // 3*i + 2
            end
        end
    endgenerate

Precision and Pipeline Cost

In the evaluation described in our final report, using an 11-bit integer range and an FP_SHIFT of 12, the computed vectors were within 10% of the desired vectors, which was enough accuracy for our application. This is an observed result for the evaluated inputs rather than a guarantee for every fixed-point input; sufficiently small vectors can have their squared components truncated to zero.

When synthesized for an 11-bit integer range and 12 bits of precision, the normalizer consumes 48 DSP slices. The latency of the normalizer is $3 + 3I$, where $I$ is the number of iterations used for the inverse square root solver. While it is fully pipelined, all multiplications are assumed to take one cycle, which limits the range and precision of the input vectors due to the volume of operations required between frames. Too high a latency and we would not be able to finish computing updates for all boids.

Boid Computations

Every boid is associated with a position and velocity vector. Every frame, we add the velocity vector multiplied by a small $\Delta t$ to its position. The intended full behavioral model calculates the acceleration of the boid based on:

1. Normalized offset to the target’s position. Used in computing the target force.
2. Average direction of other boids in the perception radius. Also known as the average flock heading. Used in computing the alignment force.
3. Normalized offset to the average position of other boids in the perception radius. Also known as the flock center. Used in computing the cohesion force.
4. Average of the negative offset to the position of other boids. Also known as the separation heading. In essence, this force makes sure that boids don’t fly over each other. Used in computing the separation force.

In the full model, each of the above results is computed into a steering force by subtracting the boid’s velocity from the influencing vector. These results are then clamped to have a maximum magnitude equal to a constant maximum steering force. These contributions to the acceleration are then summed and added to the boid’s velocity.

The velocity’s magnitude is then clamped between a constant minimum and maximum speed. This new velocity is then used next frame to update the boid’s position.

Boid Update

The boid_update module computes the boid’s new velocity, which is passed back to the boid_manager module. In the submitted implementation, it only implemented the first of the four steering forces, which is responsible for leading the boid towards the target. To do this, it implements an FSM comprised of the following states:

1. IDLE
2. NORMALIZE_OFFSET
3. GET_STEER
4. NORMALIZE_STEER_TARGET
5. CLAMP_STEER
6. GET_ACCELERATION
7. GET_VELOCITY
8. NORMALIZE_VELOCITY
9. CLAMP_VELOCITY

Of relevance here is that, to clamp a vector between given minimum and maximum magnitudes, it is necessary to normalize it beforehand. Due to resource limitations, it was impossible to instantiate a separate normalizer module for each of these operations, so instead we opted for a blocking implementation during which the boid_update module changes the inputs to its normalizer submodule and waits for results.

To implement the rest of the steering forces, it would then be necessary to add stages for the computation of the necessary data as well as the necessary normalization stages before each steering force computation and subsequent clamping of the output value.

Our final report recorded a 51-cycle latency for boid_update and estimated at least 19,000 boid updates per frame on the pixel clock, including boid_manager overhead. In the supplied source version, the FSM takes 56 cycles after accepting an update request, before manager overhead: three 16-cycle normalization states, three two-cycle steering or clamping states, and two single-cycle states. The report’s throughput estimate therefore needs to be recalculated for this version.

Boid Manager

The boid_manager module responsible for passing the necessary parameters to boid_update for every boid implements an FSM comprised of the following states:

1. IDLE
2. POS_UPDATE
3. POS_UPDATE2
4. UPDATE_START
5. UPDATE_WAIT

During the IDLE state, the module awaits the new_frame signal before saving the currently inputted target coordinates and transitioning to the POS_UPDATE state. In this state, the module updates all boid positions by adding their respective velocities scaled by a small positive number. Then, during POS_UPDATE2, all positions are boundary-checked and wrapped around the screen if necessary.

During UPDATE_START, the manager asserts the data_in_valid signal to the boid_update module and then transitions to the waiting state, where the current boid index is incremented after the boid_update module’s busy signal goes low. This continues until all boids’ velocities have been updated, at which point the module returns to the IDLE state.

Design Evaluation

Double Frame Buffer and Rendering Bottleneck

The objective of the submitted implementation was to be able to render as many boids as possible and have them change sizes appropriately. Writing camera pixels into the frame buffer consumes most of the frame time, and the system cannot write an amount of boid pixels that surpasses the amount of cycles in the remaining frame time. This amount of pixels is roughly equal to the amount of cycles in the v-sync period.

This upper bounds the size and quantity of the boids by the amount of pixels for each boid when they’re closest to the screen. In the configuration evaluated in our final report, we found that we could comfortably render 16 boids as 96 × 96 squares — or 147,456 pixels — whilst remaining under the frame time constraint. This measurement differs from the supplied renderer’s current settings, which produce nominal square sizes of 4–49 pixels before eight-pixel width packing. This was the design’s bottleneck and would be the first area to be improved upon in the next iteration as both the depth calculation and boid updates can handle increased throughput.

Additionally, we could cut off the additional pixels sent via the eight-pixel packing by modifying the strobe signal to the DDR3 controller to only write pixels included in the square region. This would remove the current rendering artifact where box widths change only in increments of eight.

Communication Protocol and Depth Calculation

Since the communication protocol only needs to communicate 22 bits for the center of mass information every frame regardless of any changes to the rest of the design, it would not need any changes or improvements. Similarly, the depth calculation module deals with known constants and is mostly a wrapper for a fixed-point divide. Even if a different camera were to be used, it would only require constant values and/or bit widths to be changed in this module.

Vector Normalization and Boid Updates

The biggest source of resource allocation in the design comes from the normalizer instanced by the boid_update module. Forty-eight of the sixty DSP slices in the module come from the normalizer instance. Most of these come from the pipelined Newton-Raphson iterations, which instance multiplications between numbers with result bit widths of up to $4\,\mathrm{FP\_RANGE} + 3\,\mathrm{FP\_SHIFT} + 2$. This results in timing and resource utilization issues for higher integer ranges and precision widths.

This problem could be solved by splitting each multiplication into different steps — each of them fitting into a single DSP slice. This would drastically reduce resource utilization and improve throughput at the cost of increased latency. Increased latency, however, directly influences the amount of boids that can be updated every frame due to the boid manager’s blocking implementation of updates and the choice to instance only one normalizer for boid_update.

Project source (MIT GitHub)