Penumbra Development Journal

The Twist You Possibly Expected: Rust

2024-06-04T20:20:00Z

I’ve previously mentioned my intention to eventually switch from targeting JavaScript to targeting WebAssembly. Well, I’ve done it! I went through and reimplemented everything I have done so far to target WebAssembly using SIMD instructions. Here’s the previous sphere’s shadow demo alongside the new WebAssembly version:

Click and drag (or touch and drag) to change the position of the light source, and thus change the shape of the sphere’s shadow. Change the resolution via the dropdown to observe the effect on render times.

Eschewing AssemblyScript

I previously mentioned that my plan was to rewrite using AssemblyScript, which is a TypeScript based language that compiles directly to WebAssembly. This seemed really promising to me. My goal with this project was not to learn a new programming language but to learn about ray tracing and maybe do something fun with it.

Unfortunately, as I began to look more seriously into AssemblyScript I started to have my doubts about it. I briefly joined the AssemblyScript Discord server and it became apparent that established WebAssembly features like threads were not going to be implemented any time soon — seemingly because the authors have become disenchanted with one or more of the W3C working groups of which they were once a part of. In a long and somewhat inscrutable manifesto they list their objections, offenses, and demands. I can’t discount that they were mistreated but I nevertheless found it an off-putting read. They may well have some valid points — after all it is easy to be sympathetic about ensuring WebAssembly interoperates well with the web — but I can’t shake the feeling that maybe the WebAssembly standard will be better without their participation for a time.

Once I took AssemblyScript off the top of the list of possibilities I looked for alternatives, but ultimately Rust was the obvious choice. It’s a language I have wanted to learn anyway.

Adopting Rust

I have previously spent a little bit of time playing with Rust, but this was certainly a more thorough experience. It was helpful to understand that since I was building something that would solely target WebAssembly I could rely on wasm-pack to take care of a lot of the build details.

Tests

Getting tests working was also made pretty simple thanks to wasm-pack and wasm-bindgen. The standard way of writing tests in Rust is to put the tests in the same file as the code under test, inside a tests module. Here’s an example from ray.rs:

#[cfg(test)]
mod tests {
  use super::*;
  use wasm_bindgen_test::*;

  #[wasm_bindgen_test]
  pub fn creating_and_querying_a_ray() {
    let origin = Tuple::point(1.0, 2.0, 3.0);
    let direction = Tuple::vector(4.0, 5.0, 6.0);
    let r = Ray::new(origin, direction);

    assert_eq!(r.origin, origin);
    assert_eq!(r.direction, direction);
  }

One downside to having all these tests in Rust means there is no obvious way to get them running on this site. As a result, I’ve taken down the test page that allowed you to run the JavaScript tests in your browser. I might explore that at some point, but its not a priority for me right now.

Using SIMD Instructions

If you’re not familiar with “Single Instruction, Multiple Data,” it is a category of instruction sets CPUs implement to speed up calculations where you need to perform the same series of operations over different sets of variables. This comes in handy for things like matrix math. If you are old enough, you may even remember when SIMD instruction sets began to be added to processors: MMX on Intel and AltiVec for PowerPC. These days the common SIMD instruction sets are AVX on Intel or AMD processors and Neon on ARM processors.

Of course, WebAssembly is a virtual machine. You can’t just throw AVX and Neon instructions in the WASM file. Instead, WebAssembly has defined SIMD instructions that get compiled into the SIMD instructions for whatever architecture the browser is running on. In Rust these are exposed as compiler intrinsics in the std::arch::wasm32 module.

WebAssembly’s SIMD instructions are all fixed-width, operating on 128-bit wide operands. So for example there’s u8x16 instructions for addition, subtraction, multiplication, division, comparisons, etc. that operate on 16 8-bit unsigned integers packed into 128-bit wide registers. Likewise, there are instructions for i8x16 for signed 8-bit integers and f32x4 for 32-bit floating point values.^[1]

Here’s an excerpt from matrix.rs showing the multiplication operator implementation for Tuple and Matrix4.

impl Mul<Tuple> for &Matrix4 {
  type Output = Tuple;

  fn mul(self, other: Tuple) -> Tuple {
    let mut sum = f32x4_splat(0.0);
    for i in 0..4 {
      sum = f32x4_add(
        sum,
        f32x4_mul(
          f32x4_splat(other.get(i)),
          self.col_v128(i),
        )
      );
    }
    Tuple::from_v128(sum)
  }
}

Now that I have implemented everything using explicit WebAssembly SIMD intrinsics, I am wondering if I should have looked more closely at the still-experimental portable SIMD module. The main benefit is that I could also target non-SIMD WebAssembly and compare the performance. I could probably also more easily make use of WebAssembly’s relaxed SIMD which includes instructions like f32x4_relaxed_madd that would likely speed up matrix multiplication noticeably.

Speaking of `f32x4_relaxed_madd`…

…I’m curious how switching the above Tuple * Matrix4 implementation to it would perform. So I spent around two hours trying to get it to work, to no avail. I think the problem is that walrus, which is used by wasm-bindgen, does not yet seem to support the relaxed SIMD instructions. It throws this error when it hits the f32x4_relaxed_madd instruction:

Error: failed to deserialize wasm module

Caused by:
    0: failed to parse code section
    1: Unknown 0xfd subopcode: 0x105 (at offset 312376)

Maybe this should not be too much of a surprise, considering Relaxed SIMD doesn’t quite yet have wide support. It’s shipping in Chrome currently and behind a feature flag in Firefox. It looks like Safari may be getting it soon based on this WebKit commit from 2023, although it’s not a feature flag in Safari Technology Preview.

Wait, What’s That Weird Artifact in the Top Left of <sphere-shadow-wasm>?

I’m actually not sure! There’s 8 or pixels that are transparent, so if you are viewing the site in dark mode they will look black, and if in light mode they will look white. I haven’t spent too much time trying to figure it out, but I must be doing something wrong when copying the image data from WASM world to JS world, or thereabouts.

Detours

This entry took a while to get completed thanks to various detours I took. The bulk of the Rust work was actually done pretty quickly. It maybe took a month or less. But I took a number of detours to work on this site, including adding support for serving videos and breaking apart the repository into three repositories. There’s now the penumbra repo for the core library, penumbra-www for this website, and touch-pad which is now available as an independent npm package.

Update 2024-11-20: The penumbra-www repo has been promoted to the main repo for all of limulus.net! It’s been renamed to limulus-dot-net.

What’s Next?

With so much time spent on the above detours I’m looking forward to finally starting the next chapter of the book, which is “Light and Shading”. I’ll be screen recording as I work on it, so I might also produce some kind of video, likely focused on whatever demo I create.

In addition to u8x16, i8x16, f32x4, and f64x2 there are also instructions for u16x8, i16x8, u32x4, i32x4, u64x2, and i64x2. With all the ways you might want to slice 128-bit wide operands and all the different operations you want to do for each way of slicing, this makes for a lot of instructions! ↩︎

How I Am Using GitHub Copilot

2024-03-23T22:15:00Z

Above is a video I produced to demo GitHub Copilot to my coworkers. If you haven’t yet explored using a Large Language Model to you help you code, it is worth a watch. I screen-recorded myself developing an optimization for Penumbra (this project) and edited it down to about 6 minutes.

I use Copilot in other ways not covered in the video. It’s clearly been trained on other Ray Tracer Challenge implementations so it very quickly autocompletes tests with all the exact values. This has saved me a bunch of mindless typing. It also autocompletes production code that satisfies the tests, which is often less helpful for this project since I usually want to spend some time thinking about how to implement these things. But sometimes I turn it back on to get suggestions that prompt me to consider a different and potentially better solution.

If you’ve read my previous posts you’ll notice that I’ve switched to Rust targeting WebAssembly. There’s a story behind that! But it will have to wait for a future post.

Making this video was a lot of fun! A little less fun was navigating how to host video on this site without introducing a dependency on a third-party. There’s a good reason why just about everyone uploads to YouTube — doing this reasonably well is not easy. This may wind up needing to be a journal entry or even video of its own.

Chapter 5: Ray-Sphere Interactions

2023-12-29T21:00:00Z

In chapter 5 of The Ray Tracer Challenge you finally get to implement something that starts to resemble a ray tracer. You implement ray, sphere, and intersection related functions and the exercise at the end ties it all together to create an image.

Finding Intersections

The book does not go into the details of the math for how to determine the intersection points of a ray and sphere. I’m glad for that, but it bugged me that I do not have an intuitive understanding of why this intersect method works.

class Sphere {
  intersect(ray: Ray): IntersectionCollection {
    // Transform the ray into object space
    ray = ray.transform(this.transformInverse)

    // Vector from sphere origin to the ray origin
    const sphereToRayVec = ray.origin.sub(origin)

    // Supporting characters to determine the discriminant and intersection
    const a = ray.direction.dot(ray.direction)
    const b = 2 * ray.direction.dot(sphereToRayVec)
    const c = sphereToRayVec.dot(sphereToRayVec) - 1

    // Discriminant does not intersect sphere if it is negative
    const discriminant = b ** 2 - 4 * a * c
    if (discriminant < 0) return new IntersectionCollection()

    // Calculate the intersection points
    const sqrtDiscriminant = Math.sqrt(discriminant)
    const t1 = (-b - sqrtDiscriminant) / (2 * a)
    const t2 = (-b + sqrtDiscriminant) / (2 * a)
    return new IntersectionCollection(
      new Intersection(t1, this),
      new Intersection(t2, this)
    )
  }
}

The book does suggest some online resources for an explanation of the math at work. I took some time to read through this one. It includes two solutions: a geometric solution and an analytic solution. The geometric solution made sense to me but the analytic solution less so. Still — despite an error^[1] in that explanation — it did make some sense. One thing that helped was realizing that the discriminant being negative means there is no intersection because that would require taking the square root of a negative number.

The solution the book provides and I implemented above is the analytic solution. I would have a deeper understanding of it if it were the geometric solution, but at least I now have a better-than-tenuous idea of why this code works.

The Demo: `<sphere-shadow>`

The exercise at the end of the chapter is to render the shadow of a sphere by casting rays from a light source onto a “wall”. I’ve implemented that here, with the addition that you can change the position of the light source by dragging from the element.

I don’t normally test drive my demo code since it is more exploratory fun than writing code I intend to reuse. But I figured I would want to make use of the dragging interaction again, so I test-drove the creation of a TouchPad class to track all the mouse and touch events on (and off) of the element and emit only the needed move events. At some point I should add keyboard support to it as well.

Other than that there is not much new from the web technology side compared to the previous demo. Rendering of the canvas still happens in a single web worker.

What’s Next

In the demo I included an output to show the render time of the last frame. I get around 5.5ms in Chrome on my Mac Studio with an M2 Max. Firefox gets around 18.5ms and Safari around 9ms. I find this performance a little disappointing considering I feel like I have optimized things as much as I reasonably can. It makes me wonder if I should skip to targeting Web Assembly earlier than I was planning. I would like to keep the demos interactive in a real-time sort of way. Parallelization will help, but only so much on older devices with fewer CPU cores. Maybe now is the time…

At the time of writing this the issue with that page is that in the “Analytic Solution” section “equation 5” is a repeat of “equation 4”. It should actually be the quadratic formula:
$x = \frac{- b \pm \sqrt{b^{2} - 4 a c}}{2 a}$
Or, with the discriminant represented as $Δ$ :
$x = \frac{- b \pm \sqrt{Δ}}{2 a}$
If you decide to dig this deep hopefully the above can save you the intense head scratching that I went through. ↩︎

Chapters 2–4: Canvas and Matrices

2023-12-09T01:30:00Z

In the previous post I went a little further than the exercise at the end of the chapter asked for and created a web component that exercised my tuple implementation and included animation of the projectile. As it turns out, this wound up being very similar to the exercise at the end of chapter 2, which is about implementing a canvas. So I decided to skip that exercise and continue onto the next two chapters which walk you through implementing various matrix math operations.

Implementing the Canvas Class

Implementing a canvas class when targeting a web runtime is perhaps unnecessary since <canvas> provides a solid 2D canvas JavaScript API. However, the book has you implement color functions for tuples containing floats so a canvas that stores colors with floats instead of integer values seems like it might be a better bet going forward. So I decided to implement my own Canvas class backed by a Float32Array.

I went with 32-bit floats over the JavaScript-native 64-bit floats since I am still planning on porting this to AssemblyScript to take advantage of the v128 SIMD operations in WebAssembly. The “128” in “v128” implies you can either have an SIMD instruction operate either on 4 32-bit floats or 2 64-bit floats. Four-times-faster is better than two-times-faster. And based on a bit of research the extra precision is usually not needed or at least easy to avoid needing.

Using a TypedArray also opens the doors to backing a canvas with a SharedArrayBuffer or something similar. I can imagine this being useful by having the ray tracer running in many WebWorkers, all updating a shared canvas.

There’s a bit of a snag with SharedArrayBuffer however…

A Side Quest to Move Away From GitHub Pages

In response to the Spectre vulnerability, browser vendors updated the SharedArrayBuffer constructor to throw so that it could not be abused until they had a fix. The fix they ultimately adopted requires sending two HTTP headers with your HTML document. Well, you can’t set HTTP headers on GitHub Pages, where I was previously hosting this site.

I always planned on moving this site to my personal website, limulus.net. But my setup for limulus.net is very out-of-date. I have a GitHub repository for it but deployment is no longer automated. I just manually upload any changes to S3. None of the other infrastructure for it like the CloudFront distribution has been turned into CloudFormation templates so it’s all just sitting in AWS resources without any version control. I wanted to avoid adding to that mess for now by publishing to GitHub Pages.

Even though I know SharedArrayBuffer may not be how I ultimately choose to implement things I also didn’t want to be in the situation where I am forced into switching away from GitHub Pages in the middle of the project instead of early on. In the (ok, unlikely) event that anyone was subscribed to the RSS feed setting up redirects on GitHub Pages for that might be tricky. Better to just get it out of the way as soon as possible.

In a bid to get things done though I resisted the urge to write CloudFormation templates for everything, so unfortunately I have added to my AWS technical debt. However I did spend the time set things up in the new ways AWS recommends: I’m using GitHub’s OIDC provider to get temporary AWS credentials for the GitHub Action that publishes this site and I avoided setting up the S3 bucket to use public website mode. I learned that to get CloudFront to serve index.html files for directories served from a private S3 origin you need to write a CloudFront Function to rewrite the request. So unfortunately that means I now have a tiny bit of code for hosting this site that is not version controlled. But at least now I know how to set these things up in a more secure way.

Implementing the Matrix Class

There’s only a few things particularly interesting about the implementation of my Matrix class.

A Surprise Subclass

It likely should not have come as a surprise that the Tuple implementation would need to be treated as a matrix when doing matrix math operations. In fact, tuples need to be treated as matrixes of four rows, which is not my intuition about how to conceptualize an array of four items. It was seeming like I was going to need special handling in my Matrix class to account for whenever it was passed a Tuple and that felt messy. The solution I landed on was to create a TwoDimensionalArray class that would act as the base class for both the Tuple and Matrix classes. This way, the Tuple class can construct itself with 1 column and 4 rows and the Matrix class doesn’t have to treat Tuples as special cases.

This kind of refactor is definitely where having a robust test suite (in this case provided by the book) shines. I had confidence in a relatively substantial refactor without any added effort.

Chaining Matrix Transformations

While I won’t pretend to understand the reasons why (maybe I knew back when I took linear algebra?) if you want a transformation matrix to have multiple transformations you have to multiply them in reverse order. In other words, if you want a matrix that you can use to do a translation, then a rotation, then a scaling up, you need to first multiply the scaling matrix by the rotation matrix, and then the result by the translation matrix. The books suggests that you implement a “fluent” API of chainable methods that takes care of this for you. For example:

const twoOClock = Matrix.transformation()
  .translate(0, 1, 0)
  .rotateZ(-(2 / 12) * 2 * Math.PI)
  .scale(clockRadius, -clockRadius, 0)

What I’ve seen frequently in JS APIs is that they often require something like a final .done() method to perform the final calculations and produce the end result of the chain of operations. However, there is a way around this if you can structure your class to have the first attempt to read the values of the returned object do the finalization.

Here’s how that works with my Matrix class. The following is a selection of methods that demonstrate it. The .translate(), .rotateZ(), and scale() methods in the example above all call .#pushOperation() to push their operation onto the #operationStack array and return this.

export class Matrix extends TwoDimensionalArray {
  static transformation() {
    const chainable = Matrix.identity(4)
    chainable.#operationStack = []
    return chainable
  }

  #operationStack?: Matrix[]

  protected override get values() {
    if (this.#operationStack) {
      const operationStack = this.#operationStack
      this.#operationStack = undefined
      const result = operationStack.reduceRight(
        (result, operation) => result.mul(operation),
        this
      )
      super.values = result.values
    }
    return super.values
  }

  #pushOperation(operation: Matrix): this {
    if (!this.#operationStack) {
      throw new Error('Attempted to push operation to non-chainable matrix')
    }
    this.#operationStack.push(operation)
    return this
  }
}

The Demo: `<pixel-clock>`

The end-of-chapter exercise for chapter 5 is to use your matrix and canvas implementation to color in a pixel for every hour of a 12 hour analog clock. I did two things I really didn’t have to for this exercise: add animated “hands” and perform the rendering in a Web Worker.

Now that I have some hands-on experience with Web Workers I expect to be able to offload the work of the ray tracer off the main thread, and possibly even parallelize the work into multiple workers.

Up Next: Finally Casting Some Rays!

Now that these fundamentals are out of the way and I’ve got this site hosted where I want it, there should be less of a delay until the next post. With any luck the next post will also not be quite as long!

Chapter 1: Tuples

2023-11-19T02:51:00Z

The first chapter of the book focuses on setting up a foundational “tuple” library for operations on vectors and points. I didn’t really expect there to be much to show for this other than working tests. But the chapter ended with a suggestion for creating a small program to experiment with a mini physics simulation that fires projectiles at various angles and velocities and have their trajectories effected by gravity and wind.

I decided to take this a step further and create a web component that would animate the projectile on a canvas. Here it is in action:

I attempted to represent the $z$ axis by increasing/decreasing the size of the projectile. Something seems off with that though.

GitHub Copilot was helpful when writing the tuple tests. I could take the Gherkin test from the book, put it in a comment, and Copilot would generate the test code using the Gherkin-inspired test suite functions I wrote.

Looking Ahead

One thing that pained me as I wrote the implementation of the Tuple methods was knowing how inefficient they will be running in a JavaScript runtime. Vector math is the usual use case for [SIMD] instructions, but presumably JavaScript engines are not detecting that these operations could be compiled to SIMD instructions. I did a bit of forward research though and discovered that WebAssembly has SIMD support! At some point I plan to look at reimplementing Penumbra in AssemblyScript — but I first want a plain JavaScript baseline to compare against.

Onward to chapter 2!

One More Yak to Shave

2023-11-11T22:20:00Z

The site is getting published. The CSS needs work. Now it’s time to try and get an in-browser test runner working. For this I have two goals:

I should be able to take the Gherkin tests from the book and easily use them
The tests should be a part of this site

Initially I was thinking that I would use the Gherkin tests directly from the book — by using some existing tool to parse them and provide hooks for wiring up the steps. But I’m not really finding anything like that out there. I found this approach which defines functions for each of the Gherkin prefixes, maps them to Mocha functions, which seems like a better approach.

Mocha also seems like a good choice for the test runner. And after a decent amount of trial and error I got it working how I want. I even got Eleventy’s dev server to reload the page when the code changes, which will be nice for development.

Remaining yak shaving tasks:

Set up an RSS feed for this blog
Redo site styles

But at this point I would rather get started on the first test. This stuff has taken up way too much time already.

Getting Nerd Sniped

2023-10-31T22:52:00Z

The way The Ray Tracer Challenge starts is refreshing. The “Getting Started” section doesn’t spend time on any development environment setup. Instead it is a simple introduction to the Gherkin syntax then a notes on some typical pitfalls. Then the first chapter gives a brief introduction to points and vectors and throws its first test at you. It doesn’t even remind you that you need to choose a language.

I thought about what I wanted to use for my implementation. I could have chosen this as a way to introduce myself to a new language, but I decided to initially go with something I was familiar with: TypeScript compiled to JavaScript running in the browser. This way I could focus on the ray tracer itself and demos would be easy to share. After some research I also concluded that there is runway for performance enhancements like WebAssembly, WebGL and WebGPU.

Unfortunately this raises lots of questions that all have to get addressed before I even get started!

What would I name the project?
Where will I publish the site?
What will I use to build the site?
What test runner will I use? (My usual choice, Jest is great for a JSDOM environment, but that won’t work for advanced web features.)
How do I hook up the Gherkin tests from the book to my test runner?

For the name after some poking around on Wikipedia I settled on “Penumbra”. There’s probably some better names I could have chosen, but I couldn’t find any other ray tracers with that name already.

Deciding on where to publish the site was pretty straightforward. My personal site would make sense, but it’s all tied up with its own repository. I also don’t have any static site generator for it. My professional site does have a static site generator (Hugo) but I couldn’t convince myself it was an appropriate place for this project. So I decided to create a new repository for the project and use GitHub Pages to publish it.

My past experience with Hugo was alright, but Go template syntax kinda irks me. So I decided to give Jekyll a try, seeing as how it is the default for GitHub Pages. This was the first real nerd snipe.^[1] Jekyll is written in Ruby, and I figured I could just install the gem in my project. Unfortunately by default Ruby’s bundle tool does not by default install gems in the project directory like I expect a package manager to work in this decade. It’s possible to get it to work like this, but after frustration with current documentation on how to do it not matching the version of Ruby that macOS ships with I decided to try to do my development in a devcontainer. I got that working but wasn’t really happy with having to run Docker locally just for this project.

After going back to the drawing board I started researching other static site generators. That’s when I came across Eleventy, which is a fast Node.js based site generator. I actually had known about it for a number of months, but had completely forgotten about it. So now that is what I am setting up and is what this site is using.

Now of course I am getting nerd sniped trying to figure out how to use Eleventy and general web design things:

Why are WebC templates not working like I expect?
How do I get a list of blog posts to show up on the home page?
How do I get footnotes in a Markdown file to get rendered?

I’ve managed to work through these now, but I’ve run into issues with Eleventy’s dev server not updating when it should and mysterious issues with the webc:keep attribute in bundling mode. This has me wondering if my plan for using Eleventy’s dev server for development will work out. But that will have to be the next entry…

Because I have a an overly complicated project creation utility for personal TypeScript projects I always wind up starting a project by refreshing the dependencies for that utility. This time that led me to discovering that node-git has stagnated and is no longer providing up-to-date pre-compiled binaries for the latest versions of Node. This results in 3-6 minutes of compile time when installing node-git. Yikes! That’s a lot for a CLI utility that is supposed to be run via npm create. So I decided to spend the time to switch it over to simple-git which is pure JavaScript and doesn’t require any compilation. Thankfully I had written tests for the git functionality which did not mock out node-git and so swapping out the library was straightforward. ↩︎