[{"content":"For this project, I tried to recreate the feel of Ghost of Tsushima\u0026rsquo;s procedural grass in Godot, following their 2021 Advanced Graphics Summit talk, Procedural Grass in \u0026lsquo;Ghost of Tsushima\u0026rsquo;. This was my final project for MIT\u0026rsquo;s 6.4400 Computer Graphics course.\nHere\u0026rsquo;s a twenty-one second demo of an open field at a density of 0.4 (more on what that means later). For this scene, I used a terrain heightmap generated from simplex noise.\nIntroduction The 2021 Advanced Graphics Summit talk Procedural Grass in \u0026lsquo;Ghost of Tsushima\u0026rsquo; presents an efficient and beautiful implementation of procedural grass that supports the game\u0026rsquo;s artistic vision while running within the PS4\u0026rsquo;s performance budget.\nMy goal was to preserve the feel and appearance of the game\u0026rsquo;s procedural grass without the more sophisticated artist authored assets and wind system. Instead, I use a scrolling Perlin noise texture for wind strength and orientation samples, a cellular noise texture to simulate clumping of nearby blade meshes, rounding of normals in the vertex shader, and albedo interpolation based on the height of the fragment with respect to the blade mesh. To tie it all together, a custom lighting pass simulates ambient bounces and occlusion, as well as blending of blade colors in the distance.\nThe approach comes down to three main tricks:\nUse GPU Instancing to draw chunks of grass blades together, avoiding millions of separate draw calls. Use moving noise patterns for wind and clusters of similar values to simulate grass clumping. Use color and lighting to make flat blade meshes look rounded and give the field depth with ambient occlusion. How Ghost of Tsushima\u0026#39;s system works The game world is tiled, with each tile containing an instance of multiple grass blade meshes via GPU instancing, using a single draw call to instantiate all instances of the blade mesh within the tile. Every blade of grass bends with respect to an artist authored bezier curve and has a unique random offset to its position, orientation, scale, and bend. Then, the wind strength and direction at the blade\u0026rsquo;s position is sampled from the game\u0026rsquo;s wind system to modify the blade\u0026rsquo;s bend and orientation. They also implemented a \u0026ldquo;clumping\u0026rdquo; system where nearby grass blades belonging to the same \u0026ldquo;clump\u0026rdquo; obtain similar random properties for all of the above parameters. Each grass blade \u0026ldquo;type\u0026rdquo; is artist authored and includes separate vein and stem albedo, gloss, and normal textures which round the edges of the grass blade to give it a more realistic appearance. For performance, the blade meshes are assigned one of two LOD models and are view frustum culled.\nGodot setup I built this in Godot 4.5, using its graphics pipeline and FastNoiseLite library. The setup is pretty simple: a light source and camera attached to the root node, a script, and a material asset for the grass shader.\nDrawing Millions of Blades The world map is divided into tiles of a specified tile size. Using a density parameter d∈(0,1]d \\in (0,1], we can prepare row2\\mathrm{row}^2 evenly spaced grass blade meshes in a grid formation, where row=sizet×10d \\mathrm{row} = \\mathrm{size}_t \\times 10d . Here, sizet\\mathrm{size}_t is the side length of a square tile and row\\mathrm{row} is the number of blades along each side. This gives us an easy way to modify how many grass blades we want in each tile by shifting dd. Each tile is instantiated using Godot\u0026rsquo;s MultiMesh class, which draws all its grass blade meshes together. This matters a lot for performance, as we\u0026rsquo;d otherwise need a separate draw call for each of the millions of grass blades.\nWhat are meshes, GPU instancing, and draw calls? A mesh is the shape of a 3D object, built from points (vertices) connected into triangles. Here, it\u0026rsquo;s a single blade of grass. A draw call is a command telling the graphics processor to draw something. GPU instancing uses one single draw call to draw the same shape many times with different positions, sizes, and rotations.\nOne simple blade is repeated across the field. On the right, the UV map tells the shader where a point sits on the blade.\nThe simple UV map lets us use the uvuv coordinates as the relative width and height of the grass blade in the shaders later on.\nMaking the Grass Move and Clump Using a perlin noise texture and a wind speed, we can create the illusion of unique grass blades which appear to flow and bend with the wind. We can use another noise texture to simulate grass clumping, a natural phenomenon where nearby blades of grass will exhibit similar properties. Without it, we\u0026rsquo;d get grass of random heights all over the place which looks unnatural. I implement both of these things using a vertex and a fragment shader.\nWhat are vertex and fragment shaders? A shader is a small program that runs on the graphics processor. A vertex shader moves the points that make up each blade, so it handles bending and rotation. A fragment shader decides what color the blade\u0026rsquo;s surface should be at each visible point.\nWhat is a noise texture? A noise texture is a texture with randomized values. Each kind of noise texture has useful properties which vary depending on the generation technique. Making every pixel a random (0, 1] value is often referred to as static or white noise, for example.\nBending in the Wind To emulate wind we can \u0026ldquo;scroll\u0026rdquo; a wind texture across the field of grass. We effectively map the texture to the entire grid by always sampling relative to each blade\u0026rsquo;s position in the shader. To achieve scrolling, we add a time offset to the sampling position. This essentially sweeps the same wind strength values across a direction, achieving the illusion of a continuous gust of wind.\nI use Perlin noise for the scrolling wind texture because its values change smoothly: nearby values are similar. This means neighboring blades get similar wind strength and direction, so the grass moves together instead of each blade jittering as we scroll the wind texture. Feel free to experiment with other kinds of noise. I generated the texture with Godot\u0026rsquo;s FastNoiseLite library. I enabled domain warping for a more interesting wind pattern and made the texture seamless to avoid strange patterns when scrolling across the edges.\nI sample the Perlin noise texture at the same position for the wind\u0026rsquo;s target orientation and rotation strength, then at another position for bending. The bending sample scrolls faster and uses a height-based time offset, so points farther along the blade sample a bit ahead in time, emulating turbulence.\nOne tile · shared wind samples Wind texture · dots mark the same blade positions Scroll wind texture Move the slider: nearby blades sample similar values and bend together.\nA simplified tile of grass sampling a seamless Perlin wind texture. Use the slider to scroll the texture and watch the blades respond. This simulation was generated from the project source code using AI.\nTechnical details: wind, rotation, and bending I implement all of this in the vertex shader.\nI sample the Perlin noise texture (or \u0026ldquo;wind texture\u0026rdquo;) at coordinates s0s_0, using elapsed time tt in seconds, wind speed wsw_s (default 1.0), and the blade\u0026rsquo;s ground-plane position posworld\\mathrm{pos}_{\\mathrm{world}} (world-space x and z). The constant cdc_d scales the sampling coordinates:\ns0=cd(ws(t,t)+posworldws) s_0 = c_d\\left(w_s(t,t)+\\frac{\\mathrm{pos}_{\\mathrm{world}}}{w_s}\\right) I multiply the sampled value by π/2\\pi/2 to get the wind angle wθw_\\theta. Increasing wsw_s makes the wind texture scroll faster. I used 0.0050.005 for cdc_d. I compute the blade\u0026rsquo;s initial rotation angle θ0\\theta_0 using its repeatable position-based hash xpx_p and the sampled clump value xcx_c, both in [0,1][0,1]. Here, τ=2π\\tau=2\\pi is a full turn in radians: θ0=(13xc+mix⁡(−0.15,0.15,xp))τ \\theta_0 = \\left(\\frac{1}{3}x_c+\\operatorname{mix}(-0.15,0.15,x_p)\\right)\\tau I also sample a wind strength wfw_f from the wind texture at s0s_0 and multiply it by a factor of 4/54/5. I blend these into the final rotation angle θ\\theta, using mix⁡(a,b,u)=(1−u)a+ub\\operatorname{mix}(a,b,u)=(1-u)a+ub: θ=mix⁡(θ0,wθ,wf) \\theta = \\operatorname{mix}(\\theta_0,w_\\theta,w_f) This angle is used to generate a rotation matrix about the y-axis MrM_r.\nI use the relative height of the vertex h=1−UVyh=1-\\mathrm{UV}_y, from 0 at the base to 1 at the tip, to compute an initial bend angle β0\\beta_0. The variation value x=(1−c)xp+cxcx=(1-c)x_p+cx_c combines the blade hash and clump value with clumping weight c=0.8c=0.8: β0=hπ mix⁡(0.10,0.2x,h) \\beta_0 = h\\pi\\,\\operatorname{mix}(0.10,0.2x,h) I then sample the wind texture at a second set of coordinates s1s_1. Here, th=t+0.25h2t_h=t+0.25h^2 offsets the time farther along the blade. The temporal sampling factor is 0.05, while the spatial factor remains cd=0.005c_d=0.005: s1=0.05(th,th)+cdposworldws s_1 = 0.05(t_h,t_h)+c_d\\frac{\\mathrm{pos}_{\\mathrm{world}}}{w_s} I call the value sampled here wbw_b. I remap it from [0,1][0,1] to [0.25,1][0.25,1] and square it, then multiply by wind speed and a bend variation factor qq. That factor blends a fixed value with the blade\u0026rsquo;s hash using the clump value: q=mix⁡(0.1,0.1+0.4xp,xc) q = \\operatorname{mix}(0.1,0.1+0.4x_p,x_c) The remapped strength produces the final bend angle β\\beta: β=β0+π ws q[mix⁡(0.25,1.0,wb)]2 \\beta = \\beta_0+\\pi\\,w_s\\,q\\left[\\operatorname{mix}(0.25,1.0,w_b)\\right]^2 This matches the shader\u0026rsquo;s mix(0, PI, wind_strength_bend), since blending from zero to π\\pi is just multiplication by π\\pi. The angle generates a rotation matrix about the x-axis, MbM_b. The height-dependent offset puts the blade tips ahead in the wind texture; the additional wind bend itself has no extra h2h^2 multiplier.\nFinally, I apply both rotation matrices to the vertex position v\\boldsymbol{v} and the normal: v←MrMbv \\boldsymbol{v} \\gets M_rM_b\\boldsymbol{v} to properly rotate and bend the blade mesh.\nUnless otherwise mentioned, I picked the constants by experimenting and seeing what looked good.\nScrolling this pattern makes neighboring blades move together. Settings are in wind_noise.tres.\nGiving Each Blade Some Variation Individual blades are different from each other, but remain similar to nearby blades. I combine a value unique to each blade with a value from the clump texture, then use the result for width and height variation. By using a mix of the unique and clump values, we get groups of blades with similar heights and orientations (clump factor) that are still slightly different from each other.\nI use cellular noise for the grass clump texture. Nearby values cluster together, with visible boundaries between groups of similar values. I generated this texture with FastNoiseLite too, using the default settings.\nClusters of similar gray values give neighboring blades similar sizes and shapes, creating clumps.\nIndependent random sizesClump-influenced sizes Cellular texture · each region shares a size value Clumping weight At zero, both tiles match. Increase the weight to give neighboring blades similar sizes; their positions stay fixed.\nThe left tile uses independent per-blade variation. The right blends those same values with a cellular texture, creating patches of similar blade sizes. Adjust the clumping weight to compare. This simulation was generated from the project source code using AI.\nTechnical details: blade appearance variation I use the blade\u0026rsquo;s world position to generate a hash value xpx_p in [0,1][0,1] and sample the cellular noise texture (or \u0026ldquo;clump texture\u0026rdquo;) to get a value xcx_c in [0,1][0,1]. I combine these into a variation parameter xx, using the clumping weight cc (set to 0.8). At c=0c=0, each blade uses only its own random value; at c=1c=1, it uses only the clump texture: x=(1−c)xp+cxc x = (1-c)x_p + cx_c that I use to scale the blade mesh\u0026rsquo;s vertex. I scale the vertex\u0026rsquo;s local horizontal coordinate vxv_x and vertical coordinate vyv_y with the following expressions. The function mix⁡(a,b,u)=(1−u)a+ub\\operatorname{mix}(a,b,u)=(1-u)a+ub blends between two values: vx=mix⁡(0.2,0.5,x) vxvy=mix⁡(0.5,2.0,x) vy \\begin{aligned}v_x \u0026amp;= \\operatorname{mix}(0.2,0.5,x)\\,v_x \\\\ v_y \u0026amp;= \\operatorname{mix}(0.5,2.0,x)\\,v_y\\end{aligned} for width and height variation.\nMaking Flat Blades Look Rounded The grass blades are simple, flat meshes. We can artificially round them without adding extra geometry by adjusting the angle of normals close to the edge of the blade. By pointing these further outwards we get the illusion of curvature on the flat mesh.\nFlat normalsRounded normals Normal curvature Both tiles use identical blades and lighting. Only the normals change.\nIdentical flat blade meshes, with flat normals on the left and rounded normals on the right. Adjust the curvature to see how lighting changes without adding geometry. This simulation was generated from the project source code using AI.\nWhat is a normal? The normal vector at a point on a mesh is the vector perpendicular to the surface. Lighting uses it to work out how the surface faces the light. By varying that direction across a flat blade, we can make the lighting behave as if the blade were curved.\nTechnical details: rounded normals In the vertex shader I compute leftward and rightward facing normals by rotating the normal about the y-axis by cnπc_n\\pi and −cnπ-c_n\\pi, respectively. The curvature parameter cnc_n corresponds to normal_curve_factor in the shader, with a default of 0.3 and a range of 0 to 0.5; multiplying by π\\pi gives an angle in radians. I pass these to the fragment shader and interpolate between them using the fragment\u0026rsquo;s horizontal coordinate UVx\\mathrm{UV}_x, which runs across the blade. This makes the flat blades look rounded.\nColoring the Grass I interpolate between a base color and a tip color to color every blade of grass. I also color distant fragments closer to the average color to simulate blending at far distances.\nEach blade transitions from its base color to its tip color.\nWithout distance blending\nWith distance blending\nLeft: individual colors stay distinct in the distance. Right: blending them makes the distant field look smoother. Technical details: fragment shader and color blending The fragment shader uses the specified base color b\\boldsymbol{b} and tip color b′\\boldsymbol{b}^{\\prime} to color every blade of grass. I also color distant fragments closer to the average color to simulate blending at far distances.\nI set the albedo using relative height h=1−UVyh=1-\\mathrm{UV}_y, from base (0) to tip (1). The function mix⁡(a,b,u)=(1−u)a+ub\\operatorname{mix}(a,b,u)=(1-u)a+ub blends between colors: mix⁡(b+b′4,mix⁡(b,b′,h2),e−0.04∥v∥) \\operatorname{mix}\\left(\\frac{\\boldsymbol{b}+\\boldsymbol{b}^{\\prime}}{4},\\operatorname{mix}(\\boldsymbol{b},\\boldsymbol{b}^{\\prime},h^2),e^{-0.04\\lVert\\boldsymbol{v}\\rVert}\\right) Here, v\\boldsymbol{v} is the fragment position in view space, so ∥v∥\\lVert\\boldsymbol{v}\\rVert is its distance from the camera. The constant 0.04 controls how quickly the distant color takes over, and ee is the base of the natural exponential. The divisor 4 is the shader\u0026rsquo;s chosen distant-color factor, rather than a literal average of the two colors.\nGiving the Field Depth To tie it all together, a custom lighting pass simulates ambient bounces and occlusion. I darken the base of the grass based on the field\u0026rsquo;s density, which gives the field more depth.\nWithout ambient occlusionWith ambient occlusion Grass density / occlusion Both tiles use identical blades and lighting. Only the base darkening changes.\nThe same tile without ambient occlusion on the left and with density-based base darkening on the right. Move the slider to see how occlusion adds depth. This simulation was generated from the project source code using AI.\nWhat is ambient occlusion? Ambient occlusion approximates how nearby surfaces block light from reaching each other. The bases of densely packed grass get less light than the exposed tips. Here, I approximate that with the grass density and the height along the blade in the lighting function (light()).\nTechnical details: lighting pass I emulate ambient occlusion using grass density dd and relative height h=1−UVyh=1-\\mathrm{UV}_y, from base to tip. The occlusion factor oo darkens the base more as density increases. Using mix⁡(a,b,u)=(1−u)a+ub\\operatorname{mix}(a,b,u)=(1-u)a+ub, I set it as o=mix⁡(1−34d,1,h2) o = \\operatorname{mix}\\left(1-\\frac{3}{4}d,1,h^2\\right) I then set the diffuse lighting color to α o lc 2n⋅l−2 \\boldsymbol{\\alpha}\\,o\\,\\boldsymbol{l}_c\\,2^{\\boldsymbol{n}\\cdot\\boldsymbol{l}-2} Here, α\\boldsymbol{\\alpha} is Godot\u0026rsquo;s light attenuation (ATTENUATION), lc\\boldsymbol{l}_c is the light color (LIGHT_COLOR), n\\boldsymbol{n} is the surface normal (NORMAL), and l\\boldsymbol{l} is the direction toward the light (LIGHT). The dot product n⋅l\\boldsymbol{n}\\cdot\\boldsymbol{l} measures how directly the surface faces the light. This gives a result I\u0026rsquo;m happy with.\nResults Wind · clumping · rounded normals · ambient occlusion Wind speed Pause animation A live miniature of the combined approach. Adjust wind speed or pause to inspect the blades.\nA simplified live WebGL scene combining scrolling wind, clumped blade sizes, rounded normals, and density-based ambient occlusion. The video at the top shows the full Godot implementation. This simulation was generated from the project source code using AI.\nThe full Godot implementation shown in the video runs at 60-230 FPS on my test computer, depending on the density. This measurement is for the Godot scene, rather than the simplified browser demo. Individual blades are different from each other but remain similar to nearby blades, and the scrolling Perlin noise texture does a pretty good job of emulating wind.\nThe finished field at a density of 0.4, using a terrain heightmap for the demonstration.\nThe ground mesh uses an albedo based on the base and tip color, specified density, and distance from the camera.\nConclusion I\u0026rsquo;m happy with how closely this captures the feel of Ghost of Tsushima\u0026rsquo;s grass. There\u0026rsquo;s still more I\u0026rsquo;d like to try, including the game\u0026rsquo;s specular lighting and shadows, and its bezier curve approach to bending individual blades.\nYou might notice that blades being viewed from their side look thin and not that great. The original Tsushima implementation rotates blades close to the camera to hide this problem. I\u0026rsquo;d like to work on this next if I do put it in a game.\nFor that, I\u0026rsquo;d also need to add performance optimizations like view frustum culling and LOD switching (this person implemented it).\nWhat are view frustum culling and LOD switching? View frustum culling skips objects outside the camera\u0026rsquo;s view. LOD (level of detail) switching uses simpler versions of objects farther away, where the extra detail is harder to see.\nSource Code and References Source code for the project.\nProcedural Grass in \u0026lsquo;Ghost of Tsushima\u0026rsquo; Texture Generation using Random Noise Similar implementation of this emulation How do Major Video Games Render Grass? Modern Foliage Rendering Acknowledgment A lot of the implementation details came from 2Retr0\u0026rsquo;s approach to the same idea. This is mostly a simplified version with reduced scope. Their version is also much better looking.\nI\u0026rsquo;d like to thank the maintainers and contributors of the Godot game engine for providing an excellent platform on which to experiment with computer graphics.\nI\u0026rsquo;d also like to thank the maintainers and contributors of the Blender 3D software for providing an excellent platform on which to create models and specify UV mappings.\n","permalink":"https://jairsantana.com/posts/grass/","summary":"\u003cp\u003eFor this project, I tried to recreate the feel of Ghost of Tsushima\u0026rsquo;s procedural grass in Godot, following their 2021 Advanced Graphics Summit talk, \u003cem\u003eProcedural Grass in \u0026lsquo;Ghost of Tsushima\u0026rsquo;\u003c/em\u003e. This was my final project for MIT\u0026rsquo;s 6.4400 Computer Graphics course.\u003c/p\u003e\n\u003cdiv style=\"position: relative; padding-bottom: 56.25%; height: 0; overflow: hidden;\"\u003e\n      \u003ciframe allow=\"accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share; fullscreen\" loading=\"eager\" referrerpolicy=\"strict-origin-when-cross-origin\" src=\"https://www.youtube.com/embed/MAErwFxp7_Y?autoplay=0\u0026amp;controls=1\u0026amp;end=0\u0026amp;loop=0\u0026amp;mute=0\u0026amp;start=0\" style=\"position: absolute; top: 0; left: 0; width: 100%; height: 100%; border:0;\" title=\"YouTube video\"\u003e\u003c/iframe\u003e\n    \u003c/div\u003e\n\n\u003cp\u003eHere\u0026rsquo;s a twenty-one second demo of an open field at a density of 0.4 (more on what that means later). For this scene, I used a terrain heightmap generated from simplex noise.\u003c/p\u003e","title":"Emulating Ghost of Tsushima's Grass in Godot"},{"content":"Hi, I\u0026rsquo;m Jair — a CS student working on full-stack and systems projects.\nThis site is a place to drop notes, project write-ups, and the occasional deep-dive. You can browse the blog, or find me on GitHub and LinkedIn.\n","permalink":"https://jairsantana.com/about/","summary":"About","title":"About"},{"content":"For MIT\u0026rsquo;s 6.4400 Computer Graphics course, I implemented a series of assignments covering spline curves and surfaces, skeletal animation, physically-based simulation, ray tracing, and shadow mapping. Most of the work used GLOO, the course\u0026rsquo;s lightweight C++ OpenGL framework, with GLM for vector and matrix operations.\nHere\u0026rsquo;s a small overview of what I learned.\nModeling Curves and Surfaces In Assignment 1, I implemented cubic Bézier and B-Spline curves, conversion between their control-point representations, and bicubic tensor product patches. This gave me a better understanding of non-polygonal representations for surfaces. Since the patches are continuous and differentiable, we can easily get the normal for lighting as well.\nAttaching a Mesh to a Skeleton Assignment 2 covered hierarchical transforms and skeletal subspace deformation, also called linear blend skinning. I built a node hierarchy and attached a mesh using per-vertex joint weights. Rotating a parent joint carries its descendants with it, while the weights let a vertex respond to several joints at once.\nThe approach is to first move a vertex from the character\u0026rsquo;s bind pose into a joint\u0026rsquo;s bind-pose coordinate frame, then move it back using that joint\u0026rsquo;s current transform. I also passed accumulated transforms down the scene tree when collecting objects for rendering, avoiding repeated walks up the hierarchy. I believe this is very similar (if not identical) to how videogames handle object rendering trees.\nCode: one joint\u0026#39;s contribution to a vertex glm::mat4x4 \u0026amp;bind_pose_joint_to_character_inverse_ = Bs_[j]; glm::mat4x4 \u0026amp;joint_to_character = joint_to_character_matrices[j]; glm::vec4 position_contribution = (attachment_weights_[i][j] * joint_to_character * bind_pose_joint_to_character_inverse_ * bind_pose_position); new_positions-\u0026gt;at(i) += glm::vec3(position_contribution); Summing these weighted contributions gives the deformed vertex position.\nSimulating Motion with Particles and Springs For Assignment 3, I implemented forward Euler, explicit trapezoidal, and fourth-order Runge–Kutta integration, then used them for circular motion, a spring-connected pendulum, and wireframe cloth. The particle system combines gravity, drag, and spring forces. The integrator then advances its positions and velocities over time.\nThe cloth is a grid with structural springs connecting neighbors, shear springs connecting diagonals, and flex springs spanning two grid edges. These resist stretching, diagonal collapse, and bending. This was more of a programming exercise, but it was still cool to see how to make a cloth simulation.\nRendering with Rays We were tasked with building a CPU ray tracer. I implemented sphere, plane, and triangle intersections, Phong lighting, shadow rays, and recursive reflections. Rays are transformed into object space for intersection, while normals return to world space using the inverse transpose of the object\u0026rsquo;s transform.\nCode: tracing a reflection if(bounces \u0026gt; 0) { // ray bounces const glm::vec3 ray_bounce_origin = hit_position + EPSILON * eye_reflection; const glm::vec3 ray_bounce_direction = eye_reflection; const Ray ray_bounce(ray_bounce_origin, ray_bounce_direction); HitRecord bounce_record = HitRecord(); color += material_specular_color * TraceRay(ray_bounce, bounces - 1, bounce_record); } eye_reflection is the reflected viewing-ray direction. The small offset keeps the new ray from immediately intersecting the surface it just left.\nCasting Shadows in OpenGL Assignment 5 involved texture mapping and shadows from a directional light. I rendered the scene from the light\u0026rsquo;s viewpoint into a depth texture, then sampled that texture while shading the camera\u0026rsquo;s view. If a fragment is farther from the light than the stored depth, another surface is blocking it.\nThis involved framebuffer objects, texture sampling, coordinate transforms, and GLSL. A small depth bias helps prevent a surface from incorrectly shadowing itself because of numerical error.\nCode: comparing against the shadow map vec3 proj_coords = (world_to_light_ndc_matrix * vec4(world_position, 1.0)).xyz; proj_coords = proj_coords * 0.5 + 0.5; float this_depth = proj_coords.z; float occluder_depth = texture(shadow_texture, proj_coords.xy).r; float bias = 0.005; if(occluder_depth \u0026lt; this_depth - bias) { return vec3(0.0, 0.0, 0.0); } These assignments preceded my final project, emulating Ghost of Tsushima\u0026rsquo;s procedural grass in Godot.\n","permalink":"https://jairsantana.com/portfolio/computer-graphics/","summary":"A collection of graphics assignments exploring how to model shapes, animate them, and turn a scene into an image.","title":"Computer Graphics: Curves, Animation, and Rendering"},{"content":"This project entailed the construction of an interactive visual program where the user can herd flocks of virtual birds known as \u0026ldquo;boids\u0026rdquo; — on-screen models which behave as birds in a flock, fish in a school, or mammals in a herd — which follow a simple behavioral model based on what each individual on-screen boid can perceive around them. The intended full model includes alignment with nearby flock mates as well as movement towards and around a target in the user\u0026rsquo;s hand.\nUpdating a singular boid\u0026rsquo;s position requires the normalization of several vectors, which presents the need to develop a resource-efficient and low-latency module for vector normalizations. To acquire the three-dimensional position of the target, we implemented stereo vision using a pair of FPGAs — each with their own camera — which communicate with each other through the SPI protocol. Using the target\u0026rsquo;s positional information, we computed the positions of the boids every frame in three dimensions and overlaid them on the camera footage via a double frame buffer.\nThe double frame buffer also presented a unique challenge since, due to overlaying the boids over the camera feed, there was a limited amount of additional pixels/DRAM requests that could be made every frame, amounting to roughly 25% of the screen\u0026rsquo;s area.\nFor this class project, we committed to having multiple boids follow the target, moving every frame and being rendered at their computed positions. To demonstrate proper depth calculation, we modified every boid\u0026rsquo;s size based on its depth. The implemented boid update module includes only the target-following force; alignment, cohesion, and separation remain future work.\nThe physical setup: two cameras attached to a pair of Urbana FPGAs.\nContributions This was a final project for MIT\u0026rsquo;s 6.2050 Digital Systems Laboratory, built with Sreeja Thipireddy. I was responsible for the implementation of the normalizer, boid manager, and boid update modules. Sreeja was responsible for the implementation of the inter-FPGA communication protocol, depth calculation module, double frame buffer, and modifications to the traffic generator. The renderer module was implemented in collaboration.\nVector Normalization To compute the necessary vectors in the boid computations, it is necessary to normalize vectors. To normalize a vector, it\u0026rsquo;s sufficient to multiply the vector\u0026rsquo;s components by the reciprocal of the vector\u0026rsquo;s magnitude.\nFor a vector v=(vx,vy,vz)\\mathbf{v} = (v_x, v_y, v_z), we first compute its squared magnitude:\nx=vx2+vy2+vz2,v^=v1x. x = v_x^2 + v_y^2 + v_z^2, \\qquad \\hat{\\mathbf{v}} = \\mathbf{v}\\frac{1}{\\sqrt{x}}. This lets us compute one reciprocal square root and reuse it for all three components.\nInitial Approximation We can compute an initial approximation to the reciprocal of the square root via the position of the number\u0026rsquo;s most significant bit.\nUsing x≈2bx \\approx 2^b, we obtain the initial approximation 1x≈2−b/2. \\frac{1}{\\sqrt{x}} \\approx 2^{-b/2}. For a nonzero integer xx, the position bb of its most significant bit is b=BIT_COUNT(x)−1−CLZ(x). b = \\mathrm{BIT\\_COUNT}(x) - 1 - \\mathrm{CLZ}(x). I implemented a custom combinational module with a tree-based approach which computes the count of leading zeroes. The result for any n-bit number can be derived from the result of its left and right halves. If the upper half of the number contains a 1 bit, the result comes from that half. Otherwise, the result is the width of the upper half plus the leading-zero count of the lower half. This gives us a combinational tree for finding the CLZ. The implementation starts with 32 two-bit pairs: 00 has two leading zeroes, 01 has one, and a pair starting with 1 has none. Five merge layers combine these counts into the result for the full 64-bit input. In clz_merge, the highest bit indicates that a whole block is zero; the remaining bits encode its leading-zero count. Code: tree-structured leading-zero counter From main/hdl/vec.sv. The source references this leading-zero count tree discussion.\nmodule clz_merge #( parameter WIDTH ) ( input wire [WIDTH-1:0] left, input wire [WIDTH-1:0] right, output logic [WIDTH:0] out ); logic both_full, left_not_full; assign both_full = left[WIDTH-1] \u0026amp; right[WIDTH-1]; assign left_not_full = left[WIDTH-1] == 1\u0026#39;b0; logic [WIDTH-2:0] right_chopped; assign right_chopped = right[WIDTH-2:0]; logic [WIDTH:0] result; always_comb begin if(both_full) begin result = {1\u0026#39;b1, {WIDTH{1\u0026#39;b0}}}; end else if(left_not_full) begin result = {1\u0026#39;b0, left}; end else begin result = {2\u0026#39;b01, right_chopped}; end end assign out = result; endmodule module clz_64 ( input wire [63:0] data, output logic [6:0] out ); logic [1:0] bit_pair [31:0]; logic [6:0] assembly_layer [5:0] [31:0]; genvar j; generate for (j = 0; j \u0026lt; 32; j++) begin assign bit_pair[j] = data[2*j+1:2*j]; assign assembly_layer[0][j] = (bit_pair[j] == 2\u0026#39;b00) ? 6\u0026#39;b000010 : (bit_pair[j] == 2\u0026#39;b01) ? 6\u0026#39;b000001 : 6\u0026#39;b000000 ; end endgenerate genvar layer, i; generate for (layer = 1; layer \u0026lt; 6; layer++) begin for (i = 0; i \u0026lt; (32 \u0026gt;\u0026gt; (layer - 1)); i += 2) begin clz_merge #( .WIDTH(layer + 1) ) m_clz_merge ( .left(assembly_layer[layer - 1][i + 1][layer:0]), .right(assembly_layer[layer - 1][i][layer:0]), .out(assembly_layer[layer][i \u0026gt;\u0026gt; 1][layer+1:0]) ); end end endgenerate assign out = assembly_layer[5][0]; endmodule The squared components of the vector are shifted right by FP_SHIFT before being added, so xx still has FP_SHIFT fractional bits. If F=FP_SHIFTF = \\mathrm{FP\\_SHIFT} and XX is the stored squared magnitude, its real value is X/2FX/2^F. The stored reciprocal-square-root estimate therefore needs the scale factor\n2F1X/2F≈2(3F−b)/2. 2^F\\frac{1}{\\sqrt{X/2^F}} \\approx 2^{(3F-b)/2}. The code uses e = (3*FP_SHIFT - b) \u0026gt;\u0026gt; 1 for this exponent. It zero-pads the squared magnitude to 64 bits before counting leading zeroes, then uses b = 63 - len_sqr_clz to find its most significant bit. The clocked assignment y_pipe[2] \u0026lt;= (1 \u0026lt;\u0026lt; e) \u0026lt;\u0026lt; FP_SHIFT gives the estimate an extra FP_SHIFT bits, which are removed by y_pipe_shifted before it is used in the next multiplication.\nCode: fixed-point initial estimate clz_64 m_clz_64 ( .data({{(64-DOT_PRODUCT_WIDTH){1\u0026#39;b0}},len_sqr_pipe[1]}), .out(len_sqr_clz) ); // 1 assign b = 63 - len_sqr_clz; // 1 assign e = (3*FP_SHIFT - b) \u0026gt;\u0026gt; 1; // 1 Newton-Raphson Refinement To improve this initial guess, I computed several iterations of a Newton-Raphson approximation scheme. Through a C simulation I found that 4 iterations was enough to achieve visually satisfactory results.\nFor an estimate yny_n of 1/x1/\\sqrt{x}, the iteration is\nyn+1=yn2(3−xyn2). y_{n+1} = \\frac{y_n}{2}\\left(3 - xy_n^2\\right). At the desired value, xyn2=1xy_n^2 = 1, so the correction factor is one. Each iteration employs only multiplication and substraction operations. In fixed-point arithmetic, the products also need to be shifted to account for the fractional bits. I split each iteration across three clock cycles: square the current estimate, multiply by the squared magnitude, and apply the correction. THREE_HALFS stores 1.5 in fixed-point format. The expression half_ay2_pipe[i] \u0026gt;\u0026gt; (2 * FP_SHIFT + 1) accounts for the product\u0026rsquo;s extra fractional bits and divides it by two. The generated stages implement the equivalent form yn+1=yn(1.5−xyn2/2)y_{n+1} = y_n(1.5 - xy_n^2/2). Code: three-cycle Newton-Raphson iteration genvar pipe_idx; generate for(pipe_idx = 3; pipe_idx \u0026lt; 3 + 3*ITERATIONS; pipe_idx += 3) begin always_ff @( posedge clk ) begin y2_pipe[pipe_idx] \u0026lt;= y_pipe_shifted[pipe_idx-1] * y_pipe_shifted[pipe_idx-1]; // 3*i // y2_pipe[pipe_idx+1] \u0026lt;= y2_pipe[pipe_idx]; // y2_pipe[pipe_idx+2] \u0026lt;= y2_pipe[pipe_idx+1]; // half_ay2_pipe[pipe_idx] \u0026lt;= half_ay2_pipe[pipe_idx-1]; half_ay2_pipe[pipe_idx+1] \u0026lt;= (len_sqr_pipe[pipe_idx] * y2_pipe[pipe_idx]); // 3*i + 1 // half_ay2_pipe[pipe_idx+2] \u0026lt;= half_ay2_pipe[pipe_idx+1]; y_pipe[pipe_idx] \u0026lt;= y_pipe[pipe_idx-1]; y_pipe[pipe_idx+1] \u0026lt;= y_pipe[pipe_idx]; y_pipe[pipe_idx+2] \u0026lt;= ($signed(y_pipe_shifted[pipe_idx+1]) * ($signed({1\u0026#39;b0, THREE_HALFS}) - $signed({1\u0026#39;b0, half_ay2_pipe_shifted[pipe_idx+1]}))); // 3*i + 2 end end endgenerate Precision and Pipeline Cost In the evaluation described in our final report, using an 11-bit integer range and an FP_SHIFT of 12, the computed vectors were within 10% of the desired vectors, which was enough accuracy for our application. This is an observed result for the evaluated inputs rather than a guarantee for every fixed-point input; sufficiently small vectors can have their squared components truncated to zero.\nWhen synthesized for an 11-bit integer range and 12 bits of precision, the normalizer consumes 48 DSP slices. The latency of the normalizer is 3+3I3 + 3I, where II is the number of iterations used for the inverse square root solver. While it is fully pipelined, all multiplications are assumed to take one cycle, which limits the range and precision of the input vectors due to the volume of operations required between frames. Too high a latency and we would not be able to finish computing updates for all boids.\nBoid Computations Every boid is associated with a position and velocity vector. Every frame, we add the velocity vector multiplied by a small Δt\\Delta t to its position. The intended full behavioral model calculates the acceleration of the boid based on:\nNormalized offset to the target\u0026rsquo;s position. Used in computing the target force. Average direction of other boids in the perception radius. Also known as the average flock heading. Used in computing the alignment force. Normalized offset to the average position of other boids in the perception radius. Also known as the flock center. Used in computing the cohesion force. Average of the negative offset to the position of other boids. Also known as the separation heading. In essence, this force makes sure that boids don\u0026rsquo;t fly over each other. Used in computing the separation force. In the full model, each of the above results is computed into a steering force by subtracting the boid\u0026rsquo;s velocity from the influencing vector. These results are then clamped to have a maximum magnitude equal to a constant maximum steering force. These contributions to the acceleration are then summed and added to the boid\u0026rsquo;s velocity.\nThe velocity\u0026rsquo;s magnitude is then clamped between a constant minimum and maximum speed. This new velocity is then used next frame to update the boid\u0026rsquo;s position.\nBoid Update The boid_update module computes the boid\u0026rsquo;s new velocity, which is passed back to the boid_manager module. In the submitted implementation, it only implemented the first of the four steering forces, which is responsible for leading the boid towards the target. To do this, it implements an FSM comprised of the following states:\nIDLE NORMALIZE_OFFSET GET_STEER NORMALIZE_STEER_TARGET CLAMP_STEER GET_ACCELERATION GET_VELOCITY NORMALIZE_VELOCITY CLAMP_VELOCITY Of relevance here is that, to clamp a vector between given minimum and maximum magnitudes, it is necessary to normalize it beforehand. Due to resource limitations, it was impossible to instantiate a separate normalizer module for each of these operations, so instead we opted for a blocking implementation during which the boid_update module changes the inputs to its normalizer submodule and waits for results.\nTo implement the rest of the steering forces, it would then be necessary to add stages for the computation of the necessary data as well as the necessary normalization stages before each steering force computation and subsequent clamping of the output value.\nOur final report recorded a 51-cycle latency for boid_update and estimated at least 19,000 boid updates per frame on the pixel clock, including boid_manager overhead. In the supplied source version, the FSM takes 56 cycles after accepting an update request, before manager overhead: three 16-cycle normalization states, three two-cycle steering or clamping states, and two single-cycle states. The report\u0026rsquo;s throughput estimate therefore needs to be recalculated for this version.\nBoid Manager The boid_manager module responsible for passing the necessary parameters to boid_update for every boid implements an FSM comprised of the following states:\nIDLE POS_UPDATE POS_UPDATE2 UPDATE_START UPDATE_WAIT During the IDLE state, the module awaits the new_frame signal before saving the currently inputted target coordinates and transitioning to the POS_UPDATE state. In this state, the module updates all boid positions by adding their respective velocities scaled by a small positive number. Then, during POS_UPDATE2, all positions are boundary-checked and wrapped around the screen if necessary.\nDuring UPDATE_START, the manager asserts the data_in_valid signal to the boid_update module and then transitions to the waiting state, where the current boid index is incremented after the boid_update module\u0026rsquo;s busy signal goes low. This continues until all boids\u0026rsquo; velocities have been updated, at which point the module returns to the IDLE state.\nDesign Evaluation Double Frame Buffer and Rendering Bottleneck The objective of the submitted implementation was to be able to render as many boids as possible and have them change sizes appropriately. Writing camera pixels into the frame buffer consumes most of the frame time, and the system cannot write an amount of boid pixels that surpasses the amount of cycles in the remaining frame time. This amount of pixels is roughly equal to the amount of cycles in the v-sync period.\nThis upper bounds the size and quantity of the boids by the amount of pixels for each boid when they\u0026rsquo;re closest to the screen. In the configuration evaluated in our final report, we found that we could comfortably render 16 boids as 96 × 96 squares — or 147,456 pixels — whilst remaining under the frame time constraint. This measurement differs from the supplied renderer\u0026rsquo;s current settings, which produce nominal square sizes of 4–49 pixels before eight-pixel width packing. This was the design\u0026rsquo;s bottleneck and would be the first area to be improved upon in the next iteration as both the depth calculation and boid updates can handle increased throughput.\nAdditionally, we could cut off the additional pixels sent via the eight-pixel packing by modifying the strobe signal to the DDR3 controller to only write pixels included in the square region. This would remove the current rendering artifact where box widths change only in increments of eight.\nCommunication Protocol and Depth Calculation Since the communication protocol only needs to communicate 22 bits for the center of mass information every frame regardless of any changes to the rest of the design, it would not need any changes or improvements. Similarly, the depth calculation module deals with known constants and is mostly a wrapper for a fixed-point divide. Even if a different camera were to be used, it would only require constant values and/or bit widths to be changed in this module.\nVector Normalization and Boid Updates The biggest source of resource allocation in the design comes from the normalizer instanced by the boid_update module. Forty-eight of the sixty DSP slices in the module come from the normalizer instance. Most of these come from the pipelined Newton-Raphson iterations, which instance multiplications between numbers with result bit widths of up to 4 FP_RANGE+3 FP_SHIFT+24\\,\\mathrm{FP\\_RANGE} + 3\\,\\mathrm{FP\\_SHIFT} + 2. This results in timing and resource utilization issues for higher integer ranges and precision widths.\nThis problem could be solved by splitting each multiplication into different steps — each of them fitting into a single DSP slice. This would drastically reduce resource utilization and improve throughput at the cost of increased latency. Increased latency, however, directly influences the amount of boids that can be updated every frame due to the boid manager\u0026rsquo;s blocking implementation of updates and the choice to instance only one normalizer for boid_update.\nProject source (MIT GitHub)\n","permalink":"https://jairsantana.com/portfolio/flock-herder/","summary":"Herding virtual birds with a tracked object, built on two FPGAs with stereo vision and a custom motion pipeline.","title":"Flock Herder: Interactive Boids on FPGAs"},{"content":"For MIT\u0026rsquo;s 6.1810 Operating System Engineering course, I implemented a series of assignments in xv6, a small Unix-like operating system for RISC-V. The work covered virtual memory, networking, synchronization, and file systems. Working inside a kernel gave me a better understanding of what happens behind operations I normally just call from a program.\nHere\u0026rsquo;s a small overview of my favorite parts of OS.\nCopy-On-Write I implemented copy-on-write for fork. Instead of immediately copying a process\u0026rsquo;s memory, the parent and child initially share physical pages. Pages that were writable become read-only, and a write triggers a page fault that creates a private copy.\nI added a page-table flag to distinguish these pages, reference counts to track their lifetime, and handling for writes made through the kernel\u0026rsquo;s copyout function.\nNIC Driver I implemented transmit and receive handling for the emulated E1000 Ethernet device, along with UDP port binding and packet queues. A process waiting for a packet sleeps until the receive path wakes it up.\nThe driver uses rings of descriptors shared with the OS via MMIO. I replace received buffers before handing packets to the network stack, and reclaim transmitted buffers once the device is done with them.\nCode: waiting for a packet char *packet; while((packet = (char *) udp_port-\u0026gt;packet_ring[udp_port-\u0026gt;head]) == 0) { sleep(udp_port, \u0026amp;netlock); } sleep releases the lock while the process waits, then reacquires it before returning. The loop checks the queue again after waking up.\nLetting CPUs Work in Parallel I split the physical memory allocator into per-CPU free lists, each with its own lock. When one list is empty, it can take a page from another CPU\u0026rsquo;s list. I also implemented a reader-writer spinlock that allows multiple readers, with pending writers preventing new readers from entering.\nExtending the File System I added doubly indirect blocks so files could grow beyond xv6\u0026rsquo;s original limit, including the cleanup needed when truncating them. I also implemented symbolic links, following chains when opening a file and supporting O_NOFOLLOW.\nMapping Files into Memory I added file-backed mmap and munmap. Each process records its mapped regions, and page faults load file contents as those pages are accessed. Shared mappings write data back during unmapping, while private mappings keep changes out of the file. I also added handling for these regions across fork, exit, and exec.\nAccessing a file through a pointer still involves allocating pages, checking permissions, and managing the underlying file\u0026rsquo;s lifetime.\n","permalink":"https://jairsantana.com/portfolio/operating-systems/","summary":"A collection of OS assignments exploring what happens behind familiar operations like fork, opening a file, and receiving a packet.","title":"Operating Systems: Memory, Networking, and Files"},{"content":"YGo is a collaborative editing app that uses near real-time peer-to-peer shared editing via CRDTs and a Go backend. We built it as a final project for MIT\u0026rsquo;s 6.5840 Distributed Systems.\nThe primary goal of YGo was to achieve availability: it is disruptive for a user to be unable to create and edit documents because all resources are currently exhausted. We designed the signaling layer around near-stateless servers, allowing for quick horizontal scaling in case of need. By “near-stateless” we mean that some state is stored in each signaling server\u0026rsquo;s memory, but losing the peer registry does not itself erase the clients\u0026rsquo; document state. Established peer connections can continue without signaling; connecting or reconnecting peers depends on recovering the room assignment.\nA second goal was to achieve fault tolerance: if one of the signaling servers dies, clients should still be able to create documents and establish P2P connections, provided the backend retains a Raft quorum. Since the servers relay signaling traffic between clients, we designed the system so that every client within the same document connects to the same signaling server. Without carefully addressing the failure of a hosting server, there could be split brain in the client network, causing two or more groups of clients to be unable to communicate with each other.\nContributions I built YGo with Carolina Moura, with whom I coauthored the final report. I worked on the client described in Section III: the YATA list and text CRDT, its interface, and the peer-to-peer synchronization protocol. The Raft and key-value server code also came from my 6.5840 labs. For this project, we adapted the storage system to use gRPC instead of the simulated RPC system provided for testing.\nSystem Overview A client can query any YGo node on its assigned HTTP port to create a document, referred to as a “room,” or look up its hosting signaling server. The client then opens a WebSocket to that host and receives the list of current collaborators. Via the WebSocket, clients exchange the information needed to establish WebRTC connections within the same document. Document edits travel over WebRTC data channels rather than through the Go server.\nOn startup, each backend node starts an HTTP server, a gRPC server, a key-value instance, and a Raft instance. Upon client requests, the HTTP server calls functions provided by the Clerk, which calls the KV service over the network. The replicated state machine under the KV server uses Raft as its consensus algorithm. Raft and KV share the gRPC listener and the client connections between nodes.\nFigure 1 from our final report: one backend node. HTTP and WebSocket traffic use port 8080; KV and Raft RPCs share port 7001. The browser-side CRDT is outside this backend diagram.\nThe distinction between document state and room metadata is central to the design. The replicated backend stores room assignments and signaling-server heartbeats. The document itself lives in the peers\u0026rsquo; CRDT instances. Replicating metadata therefore does not make document content durable.\nThe YATA List and Text CRDT We employed the YATA approach to a Conflict-free Replicated Data Type, whose basic primitive is the YATA list. Peers can operate concurrently on different versions of the list and share updates without requiring consensus amongst themselves. Convergence depends on integrating the same updates using deterministic ordering rules.\nA list item stores its identifier, creation-time origins, mutable linked-list neighbors, content, and a deletion flag:\ni=⟨ID,originLeft,originRight,left,right,content,deleted⟩. i = \\langle ID,\\mathrm{originLeft},\\mathrm{originRight},\\mathrm{left},\\mathrm{right},\\mathrm{content},\\mathrm{deleted}\\rangle. The origins refer to the IDs of the items immediately to the left and right of ii when it was first created. The left and right fields are mutable pointers used to maintain the local view of the linked list. A boolean flag, or tombstone, is used for deletions to preserve the structure: an item is marked deleted rather than removed, so concurrent insertions referencing its ID as an origin can still be resolved.\nEach item is assigned an ID consisting of a pair:\nID=(f,c), ID = (f,c), where ff identifies the client and cc is a monotonically increasing insertion counter maintained by that client. The report described ff as a UUID. In the supplied implementation, the browser hashes the signaling UUID with SHA-256 and uses the first signed 32-bit word as its numeric CRDT identifier. This is an identifier scheme rather than a guarantee of uniqueness; collisions remain possible.\nDeterministic Insertion When integrating item ii, the local list may already contain other insertions between originLeft and originRight, made after ii was created. To find its position deterministically, the algorithm traverses the local list from the left origin\u0026rsquo;s right neighbor toward the right origin.\nFor items sharing the same left origin, it breaks ties by client identifier. An item with a smaller client identifier is resolved to the left of the incoming item. If it has the same right origin and does not sort to the left, the scan stops. The algorithm also tracks items encountered during the scan and unresolved conflicting items, so insertions whose origins lie within that region retain their relationship to preceding items. An item with a left origin outside the scanned region ends the scan. The new item is then linked immediately after the rightmost item resolved as being to its left.\nThe report defined a counter-first total order over IDs, but the insertion code uses client identifiers for the same-origin tie-break. The excerpt below shows that implemented rule and the dependency tracking directly.\nCode: deterministic insertion scan From crdt/list.ts; excerpt from the supplied source.\nwhile (!ID.equals(scan.id, right.id)) { itemsBeforeOrigin.add(ID.key(scan.id)); conflictingItems.add(ID.key(scan.id)); if (ID.equals(scan.originLeft, item.originLeft)) { // tie-break by UUID if (scan.id.userID \u0026lt; item.id.userID) { left = scan; conflictingItems.clear(); } else if (ID.equals(scan.originRight, item.originRight)) { break; } } else if (itemsBeforeOrigin.has(ID.key(scan.originLeft))) { if (!conflictingItems.has(ID.key(scan.originLeft))) { left = scan; conflictingItems.clear(); } } else { break; } scan = this.getItem(scan.right); } When inserting multiple items at the same time, they may not be in dependency order. applyUpdate repeatedly integrates items whose origins are already known. If it drains the pending set, integration is complete; if a pass makes no progress and pending items remain, it throws an unresolved-origins error. For an item already present, it merges deletion state monotonically: an incoming tombstone can mark the existing item deleted, but cannot restore it.\nWe implemented a simple Text CRDT using the YATA list by making each insertion an individual character. Allowing arbitrary-length strings to be inserted as a single item, and splitting them only when required by later operations, remains a potential optimization.\nState Vectors and Deletions A state vector records how many insertions from each client the peer knows. In the source, the stored value is the next counter after the greatest integrated counter:\nSV[f]=max⁡integrated items from f(c+1), SV[f] = \\max_{\\text{integrated items from }f}(c+1), with zero used when none are known. makeUpdate sends items for which c≥SVrecipient[f]c \\geq SV_{\\mathrm{recipient}}[f]. This representation assumes that the counter summarizes a complete prefix; it does not encode arbitrary gaps in a client\u0026rsquo;s insertion history.\nCode: selecting items missing from a state vector From crdt/list.ts; excerpt from the supplied source.\nconst items: YataItem[] = []; for (const item of this.itemById.values()) { if (this.isSentinel(item)) continue; const known = target.get(item.id.userID) ?? 0; if (item.id.opCounter \u0026gt;= known) { items.push(item); } } Deletions do not allocate new insertion IDs. The protocol therefore sends tombstone keys separately from the insertion diff. getItems returns a left-to-right ordered list of items that have not been deleted, while the underlying tombstoned items remain available for resolving origins.\nPeer-to-Peer Synchronization A group of peers sharing the same text instance communicates edits through TextPeer, a protocol layer that wraps the CRDT. The wire format has two message kinds: stateVector, carrying the sender\u0026rsquo;s full state vector, and update, carrying CRDT items and tombstone keys.\nInitial Sync When a peer\u0026rsquo;s data channel to another peer first opens, it calls syncWithPeer, which sends a state-vector message. The receiver replies with an update containing every item the sender is missing, computed with makeUpdate, plus every tombstoned key. Because the data channel opens on both sides, the exchange runs symmetrically so each peer can acquire the other\u0026rsquo;s state at the time of synchronization.\nPeriodic Broadcast After initial sync, a timer in the application layer calls broadcastChanges. Each call builds an item diff against lastBroadcastSV and a deletion list from tombstones appended since the previous broadcast. If either list is nonempty, it sends the update to every connected peer and advances both broadcast cursors.\nCode: batched update broadcasts From web/src/lib/textPeer.ts; excerpt from the supplied source.\nbroadcastChanges(rtc: RTC): void { const items = this.text.makeUpdate(this.lastBroadcastSV); const deletes = this.tombstoneOrder.slice( this.lastBroadcastTombstoneCount, ); if (items.length === 0 \u0026amp;\u0026amp; deletes.length === 0) return; const msg: WireMessage = { kind: \u0026#34;update\u0026#34;, items, deletes }; rtc.broadcast(JSON.stringify(msg)); this.lastBroadcastSV = this.text.getStateVector(); this.lastBroadcastTombstoneCount = this.tombstoneOrder.length; console.log(\u0026#34;broadcastChanges\u0026#34;, JSON.stringify(msg)); } handleMessage applies incoming items and tombstones. In the supplied source, it leaves the broadcast cursors unchanged, allowing items and tombstones learned from one neighbor to ride out on the next broadcast to other neighbors. The cursor-advancing optimization in that method is commented out because it would suppress forwarding in a partially connected graph.\nA useful property of this design is that the graph of peers need not be fully connected for updates to propagate. The design uses forwarding to propagate updates along paths in a connected component, provided dependencies can be integrated and messages continue to be delivered. Disconnected components cannot exchange edits until a path is restored. The browser\u0026rsquo;s connection setup nevertheless attempts a mesh by connecting each new peer to the peers listed in its welcome message.\nThe supplied UI defaults to a two-second broadcast interval, adjustable from 200 ms to 10 seconds in 200 ms steps. These are batching settings, not measured end-to-end editing latencies. The local text changes immediately; remote visibility depends on the next broadcast, transport, and integration.\nWhy broadcasting? It\u0026rsquo;s possible to implement the peer-to-peer protocol as a pull-based protocol instead of a push-based protocol. This would entail deciding when to request updates instead of when to send them to peers. I decided to stick with a push-based protocol for several reasons. First, it is much simpler to think about and implement. Broadcasting peers need not wait for a reply and receiving peers don\u0026rsquo;t have to send one. Second, it removes needless probing that would occur with a pull-based approach. Peers can be optimized to only broadcast when they have something to share, reducing traffic. A notification-push and pull-based approach could circumvent these kinds of issues, but there is very little gain in complicating the approach in this way.\nSignaling and Room Assignment The signaling server fulfills three purposes: it manages the creation of rooms, answers requests about which server currently hosts a room, and acts as a middleman in establishing P2P connections if it is the hosting server.\nThe storage interface provides versioned reads and conditional writes:\nGET(k) → (value, version) PUT(k, v, version) → ok | ErrVersion | ErrMaybe A successful write increments the version. ErrVersion indicates a conflicting version; ErrMaybe represents an uncertain outcome after a retry. This distinction matters because a failed response does not necessarily mean the original write was not applied.\nThe report described separate server/\u0026lt;addr\u0026gt; lease keys. The supplied source instead stores a serialized address-to-timestamp map under one heartbeats key. Each server updates its entry with a versioned read-modify-write. Room ownership is stored separately as room/\u0026lt;id\u0026gt; → advertised address.\nHeartbeats and Room Creation Each server registers its advertised address and periodically renews its timestamp. With the supplied startup configuration, renewals are scheduled every 10 seconds and Signalings uses a 30-second freshness window. A server belongs to the live set when\ntlocal−theartbeat≤30,000 ms. t_{\\mathrm{local}} - t_{\\mathrm{heartbeat}} \\leq 30{,}000\\ \\mathrm{ms}. These values describe the implementation\u0026rsquo;s timers; they are not a bound on failover time. Divergent wall clocks, failed renewals, retries, and delayed requests all affect detection.\nOn POST /api/rooms, the server lists the live signaling set, picks a host uniformly at random, generates a fresh six-character room ID, and attempts a version-zero write. The report called these IDs alphanumeric; the current generator uses uppercase letters only. The receiving server need not be the chosen host: any signaling server can create a room on behalf of another. The current creation loop generates a new ID after any non-OK result, rather than distinguishing only version conflicts.\nLookup and Reassignment On GET /api/rooms/:room, the server reads the room assignment and checks whether the recorded host is still in the live set. If it is, its address is returned. If its heartbeat has expired, the server picks a new host and writes it with the version observed during lookup. If another lookup raced this update, ErrVersion causes the check to repeat.\nCode: conditional reassignment after a host expires From server.go; excerpt from the supplied source.\n// The server is dead, pick a new random server newServer := live[rand.Intn(len(live))] putErr := s.ck.Put(\u0026#34;room/\u0026#34;+room, newServer, version) if putErr == rpc.OK { fmt.Printf(\u0026#34;Room %q reassigned: %q (dead) -\u0026gt; %q\\n\u0026#34;, room, server, newServer) return newServer, rpc.OK } if putErr == rpc.ErrVersion { // someone raced us; prob will get out in the next iteration continue } return \u0026#34;\u0026#34;, putErr The host can die between the lookup and the client\u0026rsquo;s connection attempt. The client discovers this when connecting fails and repeats the lookup. In the supplied frontend, reconnection starts with a 500 ms backoff, doubles on repeated failures, and caps at 10 seconds. Lookup tries the page\u0026rsquo;s origin and a hard-coded list of three localhost servers, which would need to be replaced for deployment.\nEstablishing Peer Connections Before upgrading a WebSocket, the server validates the room and checks that it is still the assigned host. On a successful upgrade, it assigns a fresh peer UUID, inserts the connection into its in-memory room registry, and sends a welcome message carrying that ID and the IDs of current peers.\nPeers use signaling to exchange SDP offers, answers, and ICE candidates. The report described all peer JSON as broadcast; the source stamps the sender\u0026rsquo;s ID and unicasts messages with a to field to that peer. Messages without a target are broadcast to the other peers in the room. Once WebRTC data channels are established, edits use those channels.\nRecovering from Signaling Failure We addressed two kinds of failure. When the hosting signaling server dies, its heartbeat eventually expires. A subsequent lookup through a surviving node reassigns the room. Clients notice the WebSocket close and repeat the lookup, reconnecting to the new host. The frontend keeps its RTC instance and CRDT across the signaling outage, so the outage alone does not tear down established data channels or discard document state. Recovery behavior also depends on subsequent peer-membership messages and reconnection.\nA server can also be reassigned away from a room while still alive if it misses enough heartbeats. It may still hold open WebSockets for rooms it no longer owns. To address this stale registry, each server runs an eviction check every 10 seconds. If it discovers it is no longer the assigned host, it closes the room\u0026rsquo;s WebSockets, triggering client lookup and reconnection. This is a periodic recovery mechanism, not an instantaneous fencing guarantee.\nReplicated Key-Value Storage For the storage system, we used the code from Lab 4 and adapted it to gRPC. We also used the lab\u0026rsquo;s Raft code beneath the replicated state machine that sustains the KV server. Both Get and Put handlers submit operations through the RSM; the Clerk retries against other nodes when it receives ErrWrongLeader or cannot reach a server.\nThe KV state machine accepts a write only when its expected version matches the current value. For a new key, the expected version must be zero.\nCode: version checks in the KV state machine From kv/kv.go; excerpt from the supplied source.\nvv, ok := kv.store[args.Key] if !ok \u0026amp;\u0026amp; args.Version != 0 { reply.Err = rpc.ErrNoKey return } if ok \u0026amp;\u0026amp; vv.Version != args.Version { reply.Err = rpc.ErrVersion return } kv.store[args.Key] = VersionedValue{Value: args.Value, Version: args.Version + 1} reply.Err = rpc.OK The source uses a bbolt-backed persister for Raft state and snapshot bytes. This storage supports the replicated metadata layer; it does not store the browser\u0026rsquo;s text CRDT. KV snapshot and restore methods exist, while bootRaft passes -1 as the RSM\u0026rsquo;s snapshot threshold, disabling automatic size-triggered snapshots in this configuration.\nThe report assumed an always-available, strongly consistent storage service when explaining signaling. In practice, committing these operations requires a Raft majority. The supplied Compose configuration contains three nodes, so two are required for a quorum. Adding signaling capacity does not itself provide automatic Raft membership changes. Existing peer editing can continue independently of the backend, but room creation, lookup, and reassignment depend on the metadata service making progress.\nResults and Limitations We implemented the browser-side YATA list and character-based text wrapper, initial state exchange, periodic edit broadcasts, WebRTC signaling, versioned room assignment, heartbeat-based discovery, and periodic eviction of stale room registries. The final report described the architecture and recovery paths but did not include quantitative latency, throughput, scalability, or failover measurements. The timer values above are source configuration, and the recovery paths describe mechanisms rather than experimentally established guarantees.\nThe project has several limitations:\nDocument durability. Content is not saved to persistent storage. Once the last peer leaves a room, the document content is lost even if its room assignment remains in the database. Resource exhaustion. There is no authentication or rate limiting to prevent a user from creating large numbers of rooms. Room records accumulate in the metadata store; retained items and tombstones also grow with edit history. Wall-clock leases. Heartbeat freshness relies on each machine\u0026rsquo;s clock. Clock differences can cause a node to be declared dead prematurely, and storing all heartbeats in one map introduces contention on that key. Editor scope. The supplied UI handles single-character insertion, line breaks, and single-character deletion. It rejects range edits, paste, IME composition, and other multi-character inputs. Update handling. Unknown tombstone keys are ignored, and unresolved insertion origins cause an exception rather than entering a persistent retry queue. These paths limit the implementation\u0026rsquo;s tolerance of missing dependencies; connected topology alone is not an unconditional delivery or convergence guarantee. Planned work included a Sync operation to store CRDT state in the database, client snapshots for document persistence, authentication, rate limiting, a load balancer, and a more reliable lease implementation. Arbitrary-length text items remain another potential optimization. These features are not implemented in the supplied source.\nSource Code and References Project source.\nThe underlying Raft and KV code originated in my course labs. The report asks readers to obtain permission before accessing that lab repository.\nThe CRDT design followed P. Nicolaescu, K. Jahns, M. Derntl, and R. Klamma, Near real-time peer-to-peer shared editing on extensible data types, GROUP \u0026lsquo;16, pp. 39–49 (DOI).\n","permalink":"https://jairsantana.com/portfolio/ygo/","summary":"Collaborative text editing with a browser-side CRDT and a replicated backend for room assignment and signaling recovery.","title":"YGo: Peer-to-Peer Collaborative Editing"}]