limulus.net

Link: Why USA 1.0 failed

Eric McCarthy — Sat, 04 Jul 2026 16:40:00 GMT

It’s always struck me as a bit weird that we effectively treat Independence Day as the USA’s birthday. Sure, it’s an important date in the country’s history, but the signing of the Constitution is where our nation really got its start. What came before it is unrecognizable to us as our country.

Any way, today is a good day to watch this video and be reminded that change is possible and that revolutions don’t always require blood. Even the Constitution can be amended.

Link: Is it ethical to use AI?

Eric McCarthy — Sun, 28 Jun 2026 18:45:00 GMT

I previously suggested that I might write about my thoughts on the ethics of generative AI. Now I don’t need to, because Charity Majors covered just about everything I wanted to. Of course she did it way better than I could have.

I endorse the whole piece, but especially this:

I am troubled by the amount of motivated reasoning coming from the people I feel politically aligned with. It’s very easy to mock and write off people who vocally hate AI for a long list of things they never seemed to give a shit about before they realized they hated AI and went looking for reasons.

And oh man does this footnote resonate with me:

I have to say how much whiplash it gives me, as a child of open source and copyleft, to find that copyright law and internet advertising are now… the good guys? this timeline is WEIRD you guys

It really does continue to astound me how so many people seem to have decided that fair use shouldn’t be a thing anymore. We used to protest that kind of crap!

Using Lima for Coding Agent Isolation

Eric McCarthy — Fri, 19 Jun 2026 22:28:00 GMT

A number of months ago, before auto mode was made available, I had started using Claude Code’s --dangerously-skip-permissions flag (YOLO mode). Up until then I had been using claude’s sandbox, but it has (or maybe had) a number of annoying issues with things like git on macOS. So I decided I would try YOLO mode, but to mitigate the blast radius by only running it in a dev container.

I’d dabbled with dev containers previously, even using GitHub Codespaces until I discovered a way to effectively have private WIP branches. Local dev containers are nicer than having to pay for a slow cloud instance, but they are also annoying. I never established a good mental model of what VS Code plugins ran in the host or container and why some would not be automatically reinstalled after a container rebuild. One of the sillier annoyances is how there seems to be no simple way to get a container to adopt the host’s timezone. Then there’s the fact that the standard dev-container approach means having a container for each project, which seems like an annoying amount of duplicate configuration. And somehow containers running in Docker also just seemed slower than they ought to be.

So eventually I just stopped using them, and started running claude as my user directly on my Mac, relying on auto mode’s protections. Then a slightly unsettling thing happened with Claude Opus 4.8.

I had an item in the project’s BACKLOG.md to enable branch protection on the main branch to keep an agent in YOLO mode from pushing to main and triggering a release. I had claude in auto mode start working on this. Even though the backlog item spelled out why the protection should exist, claude nevertheless used the GitHub client gh command with my credentials to get around the branch protection on main and merge its own commits!

The irony of this was too much for me to bear, so I decided I really needed to make sure claude was properly isolated from commands that it can use with my credentials. Now, to be fair, my git credentials in my dev containers are just as readily available as they are on the host. And while I don’t think I have the gh command set up in any of my containers, it’s not an unreasonable thing to have set up either. What I really needed was an environment that — aside from Claude Code itself — does not have access to my credentials for any service.

This led me to do a bit of research, where I learned about Lima. It manages and runs Linux virtual machines that, on macOS, feel to me like a lightweight Docker. I’ve started using a single Lima VM to run claude in, mounting my entire ~/Developer directory^[1] and ~/.agents/skills directory. So long as I avoid putting credentials into this VM, it’s considerably safer to use Claude Code in auto mode from within it.

This setup addresses many of my gripes with using dev containers. It requires only one VM and configuration for all my projects, inherits the timezone of the host, and feels faster than Docker-based dev containers.

I’ve done a few things to the Lima configuration^[2] that help make this setup have as little friction as possible. First, I have the VM setup script install my dotfiles. I also have two custom zsh commands in the VM that help with node_modules isolation: nmlocal binds node_modules to VM-local storage and nmstatus lists those active mounts. This avoids issues with sharing node_modules between the host and guest platforms, though there’s some cognitive overhead with needing to maintain this. Finally, on the host, I added a zsh function for limactl shell that, when run inside a directory in ~/Developer, puts me in the matching directory in the VM.

Lima auto-forwards ports onto the host, similar to dev containers. This can be convenient for things like difit, but there’s a rough edge for things like dev servers. If your dev server watches files for changes, and you make edits using an editor running on the host, you’ll realize that file system events are not propagated from the host to the VM. However, this does work the other way around — a change to a file made in the VM does trigger file system events on the host. You can configure file watchers like Chokidar to poll the filesystem for changes, but just running dev servers from the host is probably a reasonable solution.

It won’t surprise me if Anthropic soon ships a feature to use a local VM built into Claude Code itself. (Claude Cowork actually already uses one for executing commands.) It’s ultimately still an awkward solution, but it’s what we have until macOS and other desktop operating systems get better APIs to ensure that processes spawned by agents do not automatically have the same access to the user’s credentials that user-spawned commands do.

If you didn’t know, ~/Developer has long been the canonical location for your software projects on macOS. It even gets its own special icon in the Finder. ↩︎
You might notice that the VM’s name is cochineal. My host naming scheme has been Sonoran Desert flora for quite some time, so it seems fitting to use the name of an insect that lives on prickly pears as a VM name. ↩︎

Link: Eight years of wanting, three months of building with AI

Eric McCarthy — Wed, 08 Apr 2026 04:10:01 GMT

Lalit Maganti wrote a remarkable account of building syntaqlite using Claude Code:

If you’re using AI to generate code at industrial scale, you have to refactor constantly and continuously. If you don’t, things immediately get out of hand. This was the central lesson of the vibe-coding month: I didn’t refactor enough, the codebase became something I couldn’t reason about, and I had to throw it all away. In the rewrite, refactoring became the core of my workflow. After every large batch of generated code, I’d step back and ask “is this ugly?” Sometimes AI could clean it up. Other times there was a large-scale abstraction that AI couldn’t see but I could; I’d give it the direction and let it execute.

The whole post is worth reading. With the project I have been working on I have been having similar experiences. Especially the need to always prioritize refactoring.

My current project recently reached the point where it was getting hard to reason about areas of responsibility, so I decided it needed to be refactored into a monorepo. In non-AI time, this is something I would have sat on for a while longer before begrudgingly taking it on. But in AI-time, delaying this refactor would mean compounded deterioration of my understanding of how the pieces of the codebase are interconnected.

Currently, I’m keeping a BACKLOG.md file in my project. It’s just a handful of sections with bullet lists of to-dos. Right at the top is a section for refactors. If I think of a refactor it becomes the very next thing to work on. I have yet to regret doing a refactor before doing more feature work.

Via simonwillison.net.

Choosing to Make Better Software With AI

Eric McCarthy — Sat, 04 Apr 2026 17:30:00 GMT

With some notable exceptions^[1], there’s something important missing from what I read about AI and how it’s affecting software engineering: the enormous opportunity that agentic engineering is giving us to significantly improve software quality.

If this sounds completely bonkers to you — keep reading. My goal with this piece is to help you understand why I think this is true.

If this makes you say “well of course” — keep reading. My goal is to help you communicate this idea to our rightfully skeptical peers. I think we may be intuiting something that is not obvious to everyone.

If you have other reservations about AI — I’m not addressing them here.^[2] But reading this may help you understand why so many software engineers are adopting agentic engineering despite qualms they may share with you.

The Contradiction

It seems impossible: how can generative AI — so well known for generating low quality output that everyone agrees to call it “slop” — actually produce quality software? Even when coding agents iterate on writing code, surely the end result would only get worse with each pass.

And it can be like this. In the hands of someone who is ignorant to what quality software looks like, the end result will be slop. Like a house where the owner adds room after room with no architect, no structural engineer, further changes become increasingly dangerous.

Through the Cone by Jaroslava Brychtová and Stanislav Libenský

But in the hands of someone who cares about quality and knows what it looks like, things can play out differently.

A Baseline Workflow For Quality

Let me set a baseline for what I think reflects the current state of how a software engineer that cares about quality typically works with agents to make a code change. At least, this is what I have found myself doing.

Plan the changes with the agent, using the agent’s planning mode.
Refine the plan with the agent by asking for clarifications, pushing back on assumptions, resolving ambiguities, discussing rationales, but also being open to approaches the model suggests.
Once the plan is acceptable, let the agent loose to edit files and iteratively run validation until it thinks the change is complete.
Review the changes, asking for either clarifications or refactors as potential issues are found. Repeat until it’s good to commit the changes.

A common lament is that reviewing changes to AI-generated code is tiresome. I think this manifests in two main ways: lots of changes that superficially look fine but may have subtle issues; and code that is very wrong and requires a frustrating number of changes. However, this can be mitigated:

Using the best (larger/slower) model. Today, that’s Claude Opus 4.6. Sure, some kinds of straightforward refactors a smaller/faster model will perform fine. But my experience is I typically regret being stingy with model choice. I find that Opus 4.6 avoids things like making meaningless test assertions that make review tricky.
Avoiding AI providers like GitHub Copilot that do per-prompt pricing, which punishes conversational approaches towards refinement and understanding.
Iterating on the plan before letting the agent make changes helps prevent too many things being wrong.
Demanding 100% test coverage as part of the validation in step 3 ensures every branch gets a test, which further reduces the chance for the agent to introduce non-functional code.

What I’m finding with this process is that the refactors I ask for as part of step 4 are largely about simplification, organization, addressing code smells, or discovering there’s some unwanted consequence of the implementation I didn’t foresee during planning. These kinds of problems are far more interesting to find and tackle, making code review less tiresome and more engaging.

Going Beyond the Baseline

If we stopped here I’d say we are getting about the same level of quality software as we’d get with doing proper Test Driven Development, but with the speed advantage of not having to type everything out manually. I would argue this alone justifies adopting agentic engineering, so long as we are keeping our changes small and incremental enough to avoid accumulating cognitive debt.

Lynx After Dürer by Marta Klonowska

But we don’t have to stop here! There’s a whole lot more we can now do to improve the quality of our software with a coding agent’s help.

Testing Infrastructure

Have you ever felt like you had no choice but to put a timeout or sleep in a test? You know it’s bad because this makes tests slow and flaky, but you do it anyway because the alternative is creating some kind of event system.

Or have you ever avoided writing integration tests and smoke tests, because staging an environment they can run in is high effort?

Have you avoided things like performance tests, automated mutation testing, fuzz testing, fault injection, static analyzers, and accessibility scanners?

Well, now you’re probably only a prompt or two away from integrating these things in your codebase. All of these things got dramatically more accessible with coding agents and they can have a marked effect on both quality and security.

Of course, this testing infrastructure is code too, so we do need to consider its quality as well. It needs to be understandable, fast, and deterministic.

Documentation

Something I have started doing is creating Architecture Decision Records. These are documents that detail various decisions about how the software works or is written, and the rationale for the decisions. These documents come out of either planning sessions or refactors. Writing these things by hand would be a bore, so I have the agent write them, and give them a quick proofread.

This is something I would never bother with if it weren’t for coding agents. Not just because writing them is a chore, but because I know that no one is going to bother reading them. But the trick here is that during planning an agent will read the ones relevant to their prompt! This is the sort of thing that is incredibly helpful to keep a codebase on the rails.

Commit messages are another place where I typically let the agent with context for the changes do the writing for me. It does a good enough job for the value that commit messages provide. (I find that few people read my commit messages, despite the effort I put into them.) If your commit messages read like updated reticulator.ts, leveraging the agent’s context is an easy win for better communication around your changes.

Refactors

If you’ve got an older codebase, chances are there are lots of large refactors you want to do but don‘t have the time for. If you have good test coverage, agents can reduce the effort required and let you refactor sooner.

Forest Glass by Katherine Gray

Alas, if it’s an active codebase with more than a few contributors, this won’t help avoid a large refactor being disruptive. Still, agents can reduce the effort of splitting out inactive areas of the code into microservices or micro-frontends to allow a refactor to a portion of the code.

Mitigating the Maintenance Burden

A high quality codebase feels heavy. It’s got lots of tests — probably an order of magnitude more test code than production code. It’s also got lots of tools to format, lint, type check, and otherwise validate code. It makes judicious use of libraries to avoid repeating the mistakes of others. Keeping these tools and libraries up-to-date has significant maintenance cost. Attempts to automate this work have not been that successful, but I suspect that this is going to start to change as we find ways to safely integrate agents into automation, with things like agentic workflows.

Where This Is Heading

Having been experimenting with agentic engineering for a while now^[3], I’ve come to the conclusion that it gives us the ability to do more to ensure quality — whether that’s creating tests we would have otherwise skipped or planning changes with a model that can suggest approaches we may not have otherwise considered. As more people get familiar with this workflow, I think the common wisdom of using AI in the process of creating software is likely to flip from “it will result in lower quality software” to “not using it results in lower quality software.”

I think that will happen even if LLM capability has already plateaued.

As the common wisdom changes, more and more folks who have been resistant to adopting this practice are going to get pushed to adopt it, in spite of their reservations. Being forced through this kind of change stinks, which is why I’m glad that I’ve been exposing myself to it.

For better or for worse, it seems likely that AI is going to happen to all of us. We’ll need to find a way to be resilient in the face of that change. Using agentic engineering as an opportunity to allocate more of my energy to improving the reliability, user experience, and security of my software is how I am trying to do that.

Except of course Simon Willison wrote something along these lines as I was sitting on a first draft. Go read that too!

It looks like Gergely Orosz, writing for Substack, may have also posted about this. I have not read the full post, since I cancelled my paid Substack subscriptions back in 2023, due to their insistence on hosting white-supremacist content. ↩︎
This is not to say that I have not given a great deal of thought to environmental and ethical concerns around generative AI. I have. But a lot of the anti-AI discourse feels to me like moral panic where many claims do not hold up to scrutiny and a fair assessment does not result in a slam dunk case against it. Maybe I will write about this some day, but my bandwidth to write long-form is not as wide as I wish it was.

Update: I fully endorse Charity Majors’ piece on AI ethics. ↩︎
Last year I built tcx2webvtt using Claude Code and what was an early form of a plan/code/review/refactor loop, using Sonnet 3.7 and 4.0. ↩︎

Link: Dual-loop BDD is the new Red-green TDD

Eric McCarthy — Sun, 15 Mar 2026 17:47:46 GMT

Justin Searls:

My initial prompt simply told the agent to practice “red-green TDD” (a phrase I had never heard of until it was discovered that LLMs apparently interpret it as “real TDD”). This approach turned out to be woefully insufficient. Why? Because agents follow the path of least resistance and will invariably write a shitload of unit tests chasing the local maximum of code coverage without any regard for the global maximum of making sure shit actually works.

While I haven’t had bad results from the red-green TDD prompt trick, Claude Code has not been doing what I expected with this instruction. What I see is that it:

Writes most, if not all, of the tests first.
Runs the tests to see them fail.
Writes the implementation.
Runs tests again, and if they don’t pass, fixes the implementation and/or the tests in a single step. (Opus 4.6 will often say something along the lines of “I’ll fix the tests and the code simultaneously.”)

This is not TDD! At least not as I know it and use it. What I expect the agent to do when told to TDD is something more like this:

Write a single test.
Run the tests to make sure it fails in a nontrivial and expected way.
Write a minimal amount of code to make the test pass (and run the test again to make sure it passes).
Refactor the code, and run the tests to make sure they still pass.
Repeat until the planned change is complete.

This is why it has been puzzling to me to see folks say that the red-green TDD prompt is actually getting agents to do TDD. You’re not doing TDD if you are writing all the tests at once and seeing them all fail for trivial reasons like nonexistent symbols/methods/properties. You actually need to see the expectations of the tests fail for you to be sure that they are justifying their existence.

I suspect part of the confusion here is that TDD is a buzzword that has lost its original meaning, so when people say their agents are doing TDD they don’t know that they’re not.

Apparently though, Justin has gotten Opus 4.6 to do something a lot closer to what I have been expecting:

So yesterday I updated the prompt with the more sophisticated dual-loop approach developed by folks like Dan North and other adherents of behavior-driven development in the late aughts. It is best illustrated by two concentric circles: you begin each feature with a failing integration test, then dive into an inner loop for numerous red-green-refactor iterations of unit tests, then pop back out again once the outer loop’s integration test passes.

I was expecting that we’d need a mode or custom agent workflow to pull this off. I’m still doubtful that strictly adhering to a prompted workflow like this is a strength of agents, but I’ll have to give this a try.

On a related note, I’ve started wondering if agents really need such a fine-grained TDD loop. Maybe we just need to ask them to present the tests they plan to write during the planning phase, in BDD style.

Surviving the AI Revolution in Software Development

Eric McCarthy — Mon, 02 Mar 2026 04:40:08 GMT

A wonderful essay by Ed Lyons:

Up front, here are the lessons:

Stop listening to people who are afraid

Seek first-hand testimony, not opinions

Go with someone much more enthusiastic than you

Do not look down

You must get different equipment

Put the summit out of your mind

Yet I hope you stay for the hike up.

Via laughingmeme.org.

Good Taste

Eric McCarthy — Sat, 07 Feb 2026 19:55:26 GMT

embedding-shapes writes about how having good taste is the human differentiator:

I get why many of us fear AI and carry this sense of doom with us, but at the same time, as a creative I’m not super worried about it, and not because I think AI cannot generate things, or even generate good things, because I think it can. But because the hard thing has never been to just produce something. What’s always been hard, is producing something good, something made by someone with Good Taste. A real human being that stand in front of hundreds of choices and knows (or feels) what gets to stay, what gets cut, what to push and what should be refused, and finally sharing the choices they made with you, through the medium.

They’re right about this, but I have to wonder if enough employers care about good taste.

Tahoe’s Windows

Eric McCarthy — Sat, 07 Feb 2026 17:45:09 GMT

John Gruber:

The windows on MacOS 26 Tahoe don’t really have comically large, childish corner radiuses. They just look like they do because some jackasses at Apple — all of whom, I pray, are now at Meta — thought they looked better that way. It’s a straight-up inversion of Steve Jobs’s maxim that design is about how things work, not how they look. I can think of no better example to prove that the new UI in MacOS 26 Tahoe was designed by people who do not understand or care about the most basic fundamental principles of good design.

In general I think the larger corner radiuses of windows in macOS Tahoe look fine. But if Apple really wanted to make this new look functional they really needed to rethink how windows work at a more fundamental level. The window’s corners should not be clipping the scroll bar. The resize target needs to be increased. Toolbar items should feel like they naturally fit in a window’s corner.

What Apple shipped is how I expect an overly-ambitious Linux theme to work within the constraints of X11 — not macOS.

Where Are the Test Driven Agentic Workflows, Anyway?

Eric McCarthy — Tue, 03 Feb 2026 23:56:10 GMT

Jesse Vincent:

By my very uncharitable math, agentic software engineering methodologies have gotten to somewhere around the 1970s. Like everything in AI, this is, of course, a speed run. We’re very quickly relearning why software engineering management and software engineering project management matter.

So by next Tuesday, maybe folks will start thinking about what they can steal from XP.

I’ve been thinking that we’re probably not that far away from seeing a decent boost in quality from coding agents that implement a Test Driven Development workflow.

Based on my experience, it’s not really possible to get a coding agent to follow TDD on its own by instructing it to. Even with the rules of TDD spelled out in an AGENTS.md or similar, I’ve found that they will write tests simultaneously with the production code. They don’t bother ensuring they see a test fail for the expected reason. I don’t know if that is because the models themselves have trouble following the instructions, or if the agent’s system prompts are interfering.

I honestly wouldn’t be surprised if next Tuesday Claude Code does ship a TDD mode that explicitly takes the LLM through the TDD process.

Update: Justin Searls has had some success getting Opus 4.6 to follow a dual-loop BDD workflow.

I’m Not Sweating My AI Energy Usage

Eric McCarthy — Sat, 31 Jan 2026 06:58:00 GMT

Simon P. Couch used energy-per-token estimates to calculate his AI coding energy use:

On a median day, I estimate I consume 1,300 Wh through Claude Code — 4,400 “typical queries” worth. (For those wondering, this is $15-20 of token spend on a typical day.)

This number matches my mental model for my own AI energy use. When Claude Code is churning away, I picture an ultra-high-end GPU at full utilization dedicated to my request for a minute or two.

Recently I built a gaming PC and hooked it up to my UPS with a wattage output display. When playing graphics-intensive games, I see that I’m consuming about 500 W more than I would be if I was just using my Mac Studio.

My median day of AI coding undoubtedly uses less than $15–$20 of token spend. But even if it did use that much, it would still be less energy use than 3 hours of flight simming.^[1]

Granted, both of these newish behaviors (AI coding and PC gaming) are net increases in my energy use. But I’m finding it hard to feel bad about either of them, especially when no one is telling heavy gamers they should feel ashamed of their energy usage.

Via simonwillison.net.

Flight simming for 3 hours ( $500 W \times 3 hours = 1,500 Wh$ ) is more than a typical day of AI coding at the 1,300 Wh estimate. ↩︎

Node.js Now Has TypeScript Type Stripping On By Default

Eric McCarthy — Sat, 01 Nov 2025 20:45:00 GMT

Despite keeping up better with my RSS feed reader lately, I still missed that Node.js v22.18.0 enables TypeScript type stripping by default! Apparently it has also been enabled in v24 since its initial release.

This is great news, because it means that with some important caveats, it is now possible to run TypeScript natively in Node without needing to specify the --experimental-strip-types flag. The caveats boil down to the fact that not every TypeScript language feature can be cleanly stripped to create valid JavaScript — a small handful of features require the TypeScript compiler to generate JavaScript code in order to fully function.

Luckily, in TypeScript v5.8 they introduced the --erasableSyntaxOnly option. This raises errors when you use one of these features that would cause Node.js to throw a JS parse error even after type stripping.

The full list of these non-erasable language features is thankfully small, and, in my usage of TypeScript, quite manageable:

namespaces and modules with runtime code
import = aliases
Parameter properties in class constructors (the shorthand where class { constructor(readonly foo: Bar) {} } auto-creates a foo prop)
enum declarations

I don’t think I’ve ever used executable code in a namespace or a module — the only time I use those is if I need to extend global or package types. I don’t think I’ve ever used import = aliases. And I can easily live without parameter properties.

Lack of enums is probably the toughest pill to swallow. But the TypeScript docs for enums have a good suggestion — instead of creating an enum create an object with as const:

export const Direction = {
  Up: 1,
  Right: 2,
  Down: 3,
  Left: 4,
} as const

export type DirectionEnum = (typeof Direction)[keyof typeof Direction]

Hopefully this proposal for an as enum assertion gets some traction, since it would be nice to not need an extra [Name]Enum type.

There are two other TS options that are helpful for running type-stripped TS in Node: rewriteRelativeImportExtensions and verbatimModuleSyntax. The former helps ensure you can use .ts extensions in your import statements, and the latter avoids surprises from top-level side-effects when importing a TS module.

Private Branches in Public Repos

Eric McCarthy — Sat, 11 Oct 2025 21:47:00 GMT

Something that’s semi-frequently bugged me about having this site be statically generated from flat files in a public GitHub repository is how to deal with drafts of posts. I want to be able to save my progress “in the cloud” so that I can pick it up on other machines without making anything half-baked publicly accessible.

The way I have been dealing with this is to use a persistent GitHub Codespace where changes can just sit untracked. But this means waiting for the Codespace to initialize when I start working on something, dealing with it pausing after inactivity, or even paying a bit for it if I go over GitHub’s free allotment. Without increasing the number of vCPUs on the Codespace, it’s also significantly slower than any machine I would use.

So a few times I’ve gone looking for a better solution to this problem and I’ve come up empty. But last night I decided to try again, and I gave this prompt to Claude:

I wish I could have encrypted WIP branches in public GitHub repos. Does such a thing exist?

The response told me a bit about encryption options, but the first two alternatives it gave hints at something better:

Private branches - Just use a private repo for WIP, then move to public when ready

Fork workflow - Keep your fork private while the upstream is public

This made something click for me. If I created a private fork that only exists to store branches of work-in-progress, then I could add it as a remote to clones of the public repo. This would effectively let me have “private branches”! Finally!

I’ve implemented this by creating an empty private GitHub repo, added it as a remote named private, pushed up everything from the original, and renamed the origin remote to public. This rename makes sure I will need to explicitly specify a remote when pushing new branches, preventing accidental pushes to the public repo.

git remote add private https://github.com/limulus/limulus-dot-net-private.git
git push private --all
git remote rename origin public
git checkout -b my-wip
git push -u private my-wip

When it comes time to merge my work into the public main branch, I can do this easily with plain old git merge. If there’s intermediate commits in the branch that I don’t want to be public I can always rebase or squash.

GitHub unfortunately does not track this separate private repo as a fork. That (plus the public/private dichotomy) means I can’t directly make PRs from the private repo. If I were using a PR based workflow, I expect that would have to reset the upstream of the branch to public and then make a PR. But that’s not something I’m planning on doing with this project.

This is one of those things where I feel a little silly for not realizing this was possible. It’s obvious in retrospect that the ability to have multiple remotes in a repository would allow for this. But it’s a git feature I’ve used at most once before.

The Social Contract

Eric McCarthy — Sat, 26 Apr 2025 18:12:00 GMT

Maybe you’ve heard of this thing called the social contract. It’s a concept in philosophy popularized during the Enlightenment that helps explain why anyone should submit themselves to the authority of a government. The idea is simple: we agree that the government gets to do things like imprison some of us, so long as the government meets a standard of evidence and the law is fair.

In the United States we call a good chunk of things covered under our social contract due process. If the government wishes to do something adverse to someone, they are due a process that ensures their rights are respected and that the actions of the government are carried out lawfully. Without due process, there is nothing to constrain the government from harming someone for any arbitrary reason.

There is a sense among many people in this country that they are not beneficiaries of this arrangement — that only criminals benefit from due process. This could not be further from the truth. Due process is what prevents a local politician from seizing your property simply because they want to. It’s what keeps you out of jail for accidentally insulting the mayor’s buddy. It’s what saves you from undue prosecution if you harm someone while acting in self-defense.

In this country we have been gifted a high standard of due process throughout all levels of government. From the town police, to the county sheriff, to the state police, to the federal Department of Justice. Many seem to believe we have too-high a standard; that it’s high time that we abandon this principle — at least for some people.

Due process does not mean everyone is treated exactly the same without regard for efficiency. For example, our immigration courts are operated by the executive branch rather than the judiciary. Noncitizens in an immigration court are not entitled to a jury trial. Nevertheless, this process is established by laws passed by Congress and its decisions are ultimately reviewable by the judicial branch. It does not mean the executive gets to do whatever it wants without regards to what the law says or a court orders.

What you probably already figured this post was about

This post is mostly about Abrego Garcia. While I do find the evidence he is an MS-13 gang member thin and speculative, that is ultimately immaterial. He could be incapable of hurting a fly or he could be a bona fide terrorist. The court order preventing his deportation to El Salvador should not have been defied. The court order the Supreme Court upheld with no dissents should not have been defied. These acts of defiance violate our long established social contract.

I want to convince everyone of this, but sadly I doubt it is possible to convince those who believe the government is justified in their defiance. I find it unlikely they can be made to understand that someday this will be used against them. The prose of Judge Wilkinson’s order in this case — even though it is as compelling an appeal to American values as any decision of a court can be — seems unlikely to persuade many Americans who have never held these values in high regard.

When discussing rights, we often talk about how things like due process protects your rights when you are accused of a crime. Heck, it is how I wrote the first draft of this post. Unfortunately, I don’t believe such people are capable of thinking of themselves as the you in this scenario. It is unimaginable to them that the leopards will eat their face. I’m not sure why so many are like this but I suspect it’s fundamental to how their brains work and the only thing that can sometimes change that wiring is when the leopard finally eats their face.

When talking to folks about this, it may be more effective to make the argument about how they been benefiting from due process all along. It’s what stands between us and things like corrupt local party officials. We must continue to exercise this bedrock principle of our social contract lest we lose it completely and begin a descent to into a culture tolerant of corruption.

The consequences

When it became clear that the Trump administration had decided to defy court orders, I posted this:

An executive that defies court orders is undeserving of the safety that the social contract would otherwise afford them.

“Safety” here is a spectrum, but it includes feeling secure in one’s office. Repeated defiance of the courts must be met with serious threats of removal through lawful means, such as impeachment. While that is unlikely to succeed immediately given the current make-up of Congress and President Trump’s continued popularity among the Republican base — it is not impossible if these factors shift. To that end, there is a path to bring an early end to Trump’s un-American pursuit of authoritarianism: make the 2026 midterm elections a referendum on impeaching Trump and Vice President Vance, likewise making the newly-minted Speaker Hakeem Jeffries president for the remainder of the term.

A contract with someone acting in bad faith only works if there are consequences for violating it.

Workout Exporter

Eric McCarthy — Sun, 16 Mar 2025 06:10:00 GMT

Last year, I got a GoPro HERO12 Black, which I have been using to record my bike rides. Naturally, I also track these as workouts using my Apple Watch. I’ve been thinking it might be neat to synchronize a map, heart rate, and other data from HealthKit with the video. It probably won’t make for a particularly compelling watch, but it’s something I’d like to have.

The rough plan to make this work is to do the following:

Make a little iOS app for exporting workouts into a machine readable format, preferably a standardized format so it can be used for other purposes.
Write a program that reads both the workout data and an XML export for a Final Cut Pro project and generates a WebVTT file with the workout data in JSON. This WebVTT file would then be included as a track alongside the video.
Create web components that overlay or go alongside the video to display the data.

One of these components is a map, something I’ve already worked on for my photos. For example, scroll down on One Long Cut to see a map of where it was taken.

Anyway, I’ve been working on the first step of this plan and now have something to show for it! I’m calling it simply Workout Exporter. It’s a SwiftUI app that lets you choose a recent workout, writes a TCX file with the workout data, and lets you export it using a share sheet where you can AirDrop it or save it to Files.

There are still some rough edges, particularly with export naming and the fact that it shares is a .txt instead of a .xml or .tcx file. I’ll refine this over time, as I’ll undoubtedly need to tweak the export at some point.

This was also my first time doing assisted coding with Claude as an LLM and also with Claude Code — or any coding agent for that matter. Since my experience with SwiftUI is limited, this is a case where LLM assistance was helpful in learning idioms that Apple’s documentation alone rarely conveys. I’ll be trying it out more with the next step of this project to get a better feel what working with an agent is like when it comes to a language I am considerably more familiar with.

I Wrote To My Representative

Eric McCarthy — Sun, 09 Feb 2025 21:35:00 GMT

One thing I suspect partisans like myself underappreciate is the extent to which members of Congress are driven by the concerns their constituents choose to share. Back in my days on Twitter, I followed someone who used to work as a congressional staffer and this was their common refrain.

I used to write my congressperson occasionally, but it has been well over a decade since I have. Part of the reason for this has been a creeping cynicism that this doesn’t do much good. But I think I’ve been getting this wrong. So yesterday I wrote the following to my Republican congressman, Juan Ciscomani:

Dear Representative Ciscomani,

I am writing to express my deep concern over the recent events unfolding in the Executive Branch, particularly those in which involve the Department of Government Efficiency (DOGE) and its leader, Elon Musk.

As a professional software engineer who was on Twitter during Musk’s takeover, I watched in disbelief as Musk publicly ridiculed and derided the engineering work of his new employees. While many of the layoffs were in no doubt necessary for Twitter to remain profitable after the buyout, Musk could have chosen to treat those who had to be let go with the respect they deserved, given their contributions to making and keeping Twitter a company worth $44 billion. Instead Musk chose a chaotic, mean spirited, and reckless path. By all accounts, Twitter (now X) is greatly devalued with advertisers increasingly avoiding engaging in business with it.

This very public display of disrespect was a turning point of opinion for many experienced software engineers of Musk. It seems he now surrounds himself with young and inexperienced acolytes — perhaps these are the only kind of people who do not challenge him. Sadly, it seems he has brought these people into DOGE and installed them in places where their inexperience and past associations with online gangs such as “The Com” could put government computer systems in danger of compromise.

Musk’s vendetta against the United States Agency for International Development (USAID) is another area where his capriciousness threatens the security of the United States. By disrupting and potentially completely stopping aid programs the standing of the U.S. will worsen and China will step in and fill the void.

I do not believe most of those who voted for Trump in Arizona’s 6th Congressional District had this sort of mayhem and disregard for security in mind when they cast their votes. My hope is that you will sponsor congressional action to rein in Musk and DOGE.

Sincerely,
Eric McCarthy

I’ll be resending this to my Democratic senators as well.

My approach to writing these letters is to attempt to persuade without being overly emotive and establish my credibility. I like to make it clear what my area of expertise is and how it is relevant to my concern. Keeping it to the facts and tying those facts to my concern make it more difficult for my concern to be dismissed.

I also don’t like to bring up my party affiliation or whether or not I voted for them. I’m their constituent regardless. It’s tempting to say that my vote for them is contingent on them taking action to address my concern, but I think leaving the door open to the possibility that I am a swing voter without ever explicitly mentioning it is more than enough.

One thing I did not do a good job of in this letter is being specific as to what action I’d like to see. When it comes to votes on legislation this is pretty straightforward. But in this instance I really don’t know what a good course of action for my Republican congressman would be. It may not be something specifically legislative in nature. Even just talking to a journalist and saying that he has reservations about Musk and DOGE would be welcome as it would open the doors for more Republicans to do the same.

If you want to write to your members of Congress, enter your address on the Find Your Members form of congress.gov. The “Contact” link should go directly to a web form to submit your note. My understanding is that this is generally the preferred way to express written concerns — snail mail requires a longer processing time. I’m not sure if by phone is any better or worse, but I’m a much better writer than I am a phone-talker so I just go with that.

New York 2024

Eric McCarthy — Sat, 28 Dec 2024 06:05:00 GMT

In October Rebecca and I visited my parents on Long Island and took a week to drive around to various parks in upstate New York. It was a great trip! We took over 3 hours of video which I edited down to 25 minutes to create the above travelogue.

At some point in the future I’ll get around to making linkable chapters in videos, but here’s what you can expect if you watch the entire thing:

We milled around Bailie Beach. There’s an interesting structure seemingly made from driftwood which was presumably constructed by the beachfront property owners.
Some still shots from the Aurora Borealis which was visible to the naked eye, but shows up impressively on cell phone cameras. The last two photos are also from Bailie Beach.
A bit of footage from our voyage on the P.T. Barnum of the Port Jefferson Ferry from Long Island to Connecticut on our way to stay in Ithaca, New York.
Our walk through Watkins Glen. It was very crowded that day!
Our walk to the beautiful Taughannock Falls.
We spent a day at the Corning Museum of Glass and I recorded Manolo Aguilera creating a glass nopal (prickly pear pad) for a greater glass sculpture. This one — I think.
We hiked over 6 miles at Letchworth State Park and saw the three major waterfalls.
Then we visited Niagara Falls and went on the Nikola Tesla, a Maid of the Mist boat.
Finally we went Behind the Falls after crossing the Rainbow Bridge into Ontario.

Most of this was shot on my iPhone 16 Pro in 4K or my GoPro HERO12 Black in 4K, with some HD footage from my phone or Rebecca’s iPhone 14 Pro. I edited it in Final Cut Pro and got it ready for hosting here with the help of my media tools.

Not included in this video is our exploration of Ithaca and Rochester. We spent a good amount of time wandering around my alma mater, University of Rochester, and the Eastman Museum.

I still need to go through the photos and post more here, though you can see some from Letchworth that I used for The Rebound.

The Rebound

Eric McCarthy — Sun, 24 Nov 2024 18:35:00 GMT

This used to be a proper website.

It had a blog where I would write on politics. It had a page for a small bit of open source software I maintained. At one point it even had a photo gallery.

Over the years it moved from Movable Type, to Wordpress, to some custom PHP. And then bit by bit it became a chore to maintain. “Platforms” like Twitter and Facebook grabbed my attention and made posting easy.

So limulus.net languished. I deleted it piece by piece. Until all that remained was a single little homepage with some hand-written HTML. I even 410ed my blog.

Middle Falls, Letchworth State Park

The Last Glacial Period

The Laurentide Ice Sheet once covered much of North America. Over tens of thousands of years it accumulated frozen water, spreading further south. It didn’t simply cover land as its glaciers rolled forward — it scraped, crushed, and ground the earth under enormous pressure. It and its sibling ice sheets of the period sequestered so much of the planet’s water that sea levels fell 120 meters.

Elon Musk’s Personal Homepage on the World Wide Web

The reality of the online world that we built didn’t really make itself fully apparent to me until Elon Musk’s purchase of Twitter. The company was flawed before his ownership but his gross public treatment of Twitter’s engineers — now his own employees! — prompted me to swear to myself that I would never work for a company of his nor would I ever purchase a product from one of his companies given a choice.

Prior to this — like many others — I had scaled back my usage of Facebook after the 2016 U.S. Presidential election. Seeing the under-informed and often outright racist opinions of distant family members and high school friends put me off using it. At some point Facebook started down-ranking political posts in their feed algorithm both to avoid a perception of political bias but also because it was clearly turning off users. Paradoxically this decreased Facebook’s relevancy to me — what good is it if my posts won’t be seen by people?

I stayed on Twitter for a number of months after the purchase, but when it became apparent that my values could not abide staying on Twitter I began my exit to decentralized social media. I’ll be honest, I was surprised such a thing existed or even could exist. My assumption had always been that social media at a large scale would require centralization. But I was wrong. It turned out this entire time folks had been working on a web standard to make decentralization work and open-source social media server software like Mastodon was proving it out.

Mist, Middle Falls, Letchworth State Park

Getting on Mastodon awakened in me a renewed enthusiasm for the web and having a place of my own. After a too-brief period on an instance named mastodon.lol, I migrated my account to my own Mastodon instance at mastodon.limulus.net. (Go follow me at eric@limulus.net!)

The Glaciers of New York

As Earth started to warm about 20,000 years ago the Laurentide Ice Sheet began its recession. But it was not a linear process. Some 14,000 years ago a glacier flowed south from current-day Rochester, New York. It covered an area almost as far south as the Pennsylvania state line. The glaciers were not yet done transforming the land.

Billionaires Only Want One Thing and It’s Disgusting

As I write this it is a little over a week after Donald Trump’s second presidential election win. It seems I am not alone in looking at how things have turned out and being dismayed at the outsized influence billionaires have on us. Trump himself is a billionaire. Musk became one of Trump’s largest contributors and has been warping Twitter further and further right-wing and away from free-speech principles. Peter Thiel, a billionaire early investor of Meta, believes that freedom and democracy are incompatible, and is also an early contributor to Vice President-elect J.D. Vance. Rupert Murdoch is a billionaire that owns a right-wing entertainment machine that has done more to create disturbing information silos than any other entity.

I doubt these people get together to conspire. Surely their egos can only stand each other’s company for a short time. But they all individually want the same thing: to further cement their status as oligarchs. The only thing that stands in their way is a well informed populace with an agreement on basic facts — so they seek to destroy even that.

Base of the Genesee Arch Bridge, Letchworth State Park

A Fair and Famous Stream

Finally the glaciers receded. The land, no longer being crushed by heavy ice sheets, rebounded. New valleys were revealed. The Genesee River — which flows north towards Rochester and drains into Lake Ontario — found a new route. Over thousands of years this new route carved Letchworth Gorge, home to a number of impressive waterfalls. This is still new territory for our own dear Genesee.

Rebuilding limulus.net

What can I do in the face of this threat to my values? Well, there is only so much any one of us can do to solve the problems of the world. But one small thing I think I can do is rebuild this space and start regularly creating things to put on it. My hope here is to help find ways to make “platforms” less necessary, while also promoting ideas like subscribing to web feeds instead of following on LinkedIn, subscribing to a YouTube channel, or subscribing to a Substack. Part of this will — somewhat ironically — involve posting limulus.net links to some of these platforms.

This is not an approach that I came up with, nor is it even a new idea. It’s known as POSSE: “Publish (on your) Own Site, Syndicate Elsewhere.” Shamefully, I have only recently come to know about the IndieWeb movement that has been championing this approach.

A more technical article about how I am building this site with Eleventy and other tools will inevitably come later. But working on this article and working through some of the remaining technical challenges has helped alleviate some of my election-outcome despair. I am back to feeling like there is some hope. A lot of that hope is coming from the potential the web has to make a rebound after a capitalism-induced ice-age.

The Twist You Possibly Expected: Rust

Eric McCarthy — Tue, 04 Jun 2024 20:20:00 GMT

I’ve previously mentioned my intention to eventually switch from targeting JavaScript to targeting WebAssembly. Well, I’ve done it! I went through and reimplemented everything I have done so far to target WebAssembly using SIMD instructions. Here’s the previous sphere’s shadow demo alongside the new WebAssembly version:

Click and drag (or touch and drag) to change the position of the light source, and thus change the shape of the sphere’s shadow. Change the resolution via the dropdown to observe the effect on render times.

Eschewing AssemblyScript

I previously mentioned that my plan was to rewrite using AssemblyScript, which is a TypeScript based language that compiles directly to WebAssembly. This seemed really promising to me. My goal with this project was not to learn a new programming language but to learn about ray tracing and maybe do something fun with it.

Unfortunately, as I began to look more seriously into AssemblyScript I started to have my doubts about it. I briefly joined the AssemblyScript Discord server and it became apparent that established WebAssembly features like threads were not going to be implemented any time soon — seemingly because the authors have become disenchanted with one or more of the W3C working groups of which they were once a part of. In a long and somewhat inscrutable manifesto they list their objections, offenses, and demands. I can’t discount that they were mistreated but I nevertheless found it an off-putting read. They may well have some valid points — after all it is easy to be sympathetic about ensuring WebAssembly interoperates well with the web — but I can’t shake the feeling that maybe the WebAssembly standard will be better without their participation for a time.

Once I took AssemblyScript off the top of the list of possibilities I looked for alternatives, but ultimately Rust was the obvious choice. It’s a language I have wanted to learn anyway.

Adopting Rust

I have previously spent a little bit of time playing with Rust, but this was certainly a more thorough experience. It was helpful to understand that since I was building something that would solely target WebAssembly I could rely on wasm-pack to take care of a lot of the build details.

Tests

Getting tests working was also made pretty simple thanks to wasm-pack and wasm-bindgen. The standard way of writing tests in Rust is to put the tests in the same file as the code under test, inside a tests module. Here’s an example from ray.rs:

#[cfg(test)]
mod tests {
  use super::*;
  use wasm_bindgen_test::*;

  #[wasm_bindgen_test]
  pub fn creating_and_querying_a_ray() {
    let origin = Tuple::point(1.0, 2.0, 3.0);
    let direction = Tuple::vector(4.0, 5.0, 6.0);
    let r = Ray::new(origin, direction);

    assert_eq!(r.origin, origin);
    assert_eq!(r.direction, direction);
  }

One downside to having all these tests in Rust means there is no obvious way to get them running on this site. As a result, I’ve taken down the test page that allowed you to run the JavaScript tests in your browser. I might explore that at some point, but its not a priority for me right now.

Using SIMD Instructions

If you’re not familiar with “Single Instruction, Multiple Data,” it is a category of instruction sets CPUs implement to speed up calculations where you need to perform the same series of operations over different sets of variables. This comes in handy for things like matrix math. If you are old enough, you may even remember when SIMD instruction sets began to be added to processors: MMX on Intel and AltiVec for PowerPC. These days the common SIMD instruction sets are AVX on Intel or AMD processors and Neon on ARM processors.

Of course, WebAssembly is a virtual machine. You can’t just throw AVX and Neon instructions in the WASM file. Instead, WebAssembly has defined SIMD instructions that get compiled into the SIMD instructions for whatever architecture the browser is running on. In Rust these are exposed as compiler intrinsics in the std::arch::wasm32 module.

WebAssembly’s SIMD instructions are all fixed-width, operating on 128-bit wide operands. So for example there’s u8x16 instructions for addition, subtraction, multiplication, division, comparisons, etc. that operate on 16 8-bit unsigned integers packed into 128-bit wide registers. Likewise, there are instructions for i8x16 for signed 8-bit integers and f32x4 for 32-bit floating point values.^[1]

Here’s an excerpt from matrix.rs showing the multiplication operator implementation for Tuple and Matrix4.

impl Mul<Tuple> for &Matrix4 {
  type Output = Tuple;

  fn mul(self, other: Tuple) -> Tuple {
    let mut sum = f32x4_splat(0.0);
    for i in 0..4 {
      sum = f32x4_add(
        sum,
        f32x4_mul(
          f32x4_splat(other.get(i)),
          self.col_v128(i),
        )
      );
    }
    Tuple::from_v128(sum)
  }
}

Now that I have implemented everything using explicit WebAssembly SIMD intrinsics, I am wondering if I should have looked more closely at the still-experimental portable SIMD module. The main benefit is that I could also target non-SIMD WebAssembly and compare the performance. I could probably also more easily make use of WebAssembly’s relaxed SIMD which includes instructions like f32x4_relaxed_madd that would likely speed up matrix multiplication noticeably.

Speaking of `f32x4_relaxed_madd`…

…I’m curious how switching the above Tuple * Matrix4 implementation to it would perform. So I spent around two hours trying to get it to work, to no avail. I think the problem is that walrus, which is used by wasm-bindgen, does not yet seem to support the relaxed SIMD instructions. It throws this error when it hits the f32x4_relaxed_madd instruction:

Error: failed to deserialize wasm module

Caused by:
    0: failed to parse code section
    1: Unknown 0xfd subopcode: 0x105 (at offset 312376)

Maybe this should not be too much of a surprise, considering Relaxed SIMD doesn’t quite yet have wide support. It’s shipping in Chrome currently and behind a feature flag in Firefox. It looks like Safari may be getting it soon based on this WebKit commit from 2023, although it’s not a feature flag in Safari Technology Preview.

Wait, What’s That Weird Artifact in the Top Left of <sphere-shadow-wasm>?

I’m actually not sure! There’s 8 or pixels that are transparent, so if you are viewing the site in dark mode they will look black, and if in light mode they will look white. I haven’t spent too much time trying to figure it out, but I must be doing something wrong when copying the image data from WASM world to JS world, or thereabouts.

Detours

This entry took a while to get completed thanks to various detours I took. The bulk of the Rust work was actually done pretty quickly. It maybe took a month or less. But I took a number of detours to work on this site, including adding support for serving videos and breaking apart the repository into three repositories. There’s now the penumbra repo for the core library, penumbra-www for this website, and touch-pad which is now available as an independent npm package.

Update 2024-11-20: The penumbra-www repo has been promoted to the main repo for all of limulus.net! It’s been renamed to limulus-dot-net.

What’s Next?

With so much time spent on the above detours I’m looking forward to finally starting the next chapter of the book, which is “Light and Shading”. I’ll be screen recording as I work on it, so I might also produce some kind of video, likely focused on whatever demo I create.

In addition to u8x16, i8x16, f32x4, and f64x2 there are also instructions for u16x8, i16x8, u32x4, i32x4, u64x2, and i64x2. With all the ways you might want to slice 128-bit wide operands and all the different operations you want to do for each way of slicing, this makes for a lot of instructions! ↩︎

How I Am Using GitHub Copilot

Eric McCarthy — Sat, 23 Mar 2024 22:15:00 GMT

Above is a video I produced to demo GitHub Copilot to my coworkers. If you haven’t yet explored using a Large Language Model to you help you code, it is worth a watch. I screen-recorded myself developing an optimization for Penumbra (this project) and edited it down to about 6 minutes.

I use Copilot in other ways not covered in the video. It’s clearly been trained on other Ray Tracer Challenge implementations so it very quickly autocompletes tests with all the exact values. This has saved me a bunch of mindless typing. It also autocompletes production code that satisfies the tests, which is often less helpful for this project since I usually want to spend some time thinking about how to implement these things. But sometimes I turn it back on to get suggestions that prompt me to consider a different and potentially better solution.

If you’ve read my previous posts you’ll notice that I’ve switched to Rust targeting WebAssembly. There’s a story behind that! But it will have to wait for a future post.

Making this video was a lot of fun! A little less fun was navigating how to host video on this site without introducing a dependency on a third-party. There’s a good reason why just about everyone uploads to YouTube — doing this reasonably well is not easy. This may wind up needing to be a journal entry or even video of its own.

Chapter 5: Ray-Sphere Interactions

Eric McCarthy — Fri, 29 Dec 2023 21:00:00 GMT

In chapter 5 of The Ray Tracer Challenge you finally get to implement something that starts to resemble a ray tracer. You implement ray, sphere, and intersection related functions and the exercise at the end ties it all together to create an image.

Finding Intersections

The book does not go into the details of the math for how to determine the intersection points of a ray and sphere. I’m glad for that, but it bugged me that I do not have an intuitive understanding of why this intersect method works.

class Sphere {
  intersect(ray: Ray): IntersectionCollection {
    // Transform the ray into object space
    ray = ray.transform(this.transformInverse)

    // Vector from sphere origin to the ray origin
    const sphereToRayVec = ray.origin.sub(origin)

    // Supporting characters to determine the discriminant and intersection
    const a = ray.direction.dot(ray.direction)
    const b = 2 * ray.direction.dot(sphereToRayVec)
    const c = sphereToRayVec.dot(sphereToRayVec) - 1

    // Discriminant does not intersect sphere if it is negative
    const discriminant = b ** 2 - 4 * a * c
    if (discriminant < 0) return new IntersectionCollection()

    // Calculate the intersection points
    const sqrtDiscriminant = Math.sqrt(discriminant)
    const t1 = (-b - sqrtDiscriminant) / (2 * a)
    const t2 = (-b + sqrtDiscriminant) / (2 * a)
    return new IntersectionCollection(
      new Intersection(t1, this),
      new Intersection(t2, this)
    )
  }
}

The book does suggest some online resources for an explanation of the math at work. I took some time to read through this one. It includes two solutions: a geometric solution and an analytic solution. The geometric solution made sense to me but the analytic solution less so. Still — despite an error^[1] in that explanation — it did make some sense. One thing that helped was realizing that the discriminant being negative means there is no intersection because that would require taking the square root of a negative number.

The solution the book provides and I implemented above is the analytic solution. I would have a deeper understanding of it if it were the geometric solution, but at least I now have a better-than-tenuous idea of why this code works.

The Demo: `<sphere-shadow>`

The exercise at the end of the chapter is to render the shadow of a sphere by casting rays from a light source onto a “wall”. I’ve implemented that here, with the addition that you can change the position of the light source by dragging from the element.

I don’t normally test drive my demo code since it is more exploratory fun than writing code I intend to reuse. But I figured I would want to make use of the dragging interaction again, so I test-drove the creation of a TouchPad class to track all the mouse and touch events on (and off) of the element and emit only the needed move events. At some point I should add keyboard support to it as well.

Other than that there is not much new from the web technology side compared to the previous demo. Rendering of the canvas still happens in a single web worker.

What’s Next

In the demo I included an output to show the render time of the last frame. I get around 5.5ms in Chrome on my Mac Studio with an M2 Max. Firefox gets around 18.5ms and Safari around 9ms. I find this performance a little disappointing considering I feel like I have optimized things as much as I reasonably can. It makes me wonder if I should skip to targeting Web Assembly earlier than I was planning. I would like to keep the demos interactive in a real-time sort of way. Parallelization will help, but only so much on older devices with fewer CPU cores. Maybe now is the time…

At the time of writing this the issue with that page is that in the “Analytic Solution” section “equation 5” is a repeat of “equation 4”. It should actually be the quadratic formula:
$x = \frac{- b \pm \sqrt{b^{2} - 4 a c}}{2 a}$
Or, with the discriminant represented as $Δ$ :
$x = \frac{- b \pm \sqrt{Δ}}{2 a}$
If you decide to dig this deep hopefully the above can save you the intense head scratching that I went through. ↩︎

Chapters 2–4: Canvas and Matrices

Eric McCarthy — Sat, 09 Dec 2023 01:30:00 GMT

In the previous post I went a little further than the exercise at the end of the chapter asked for and created a web component that exercised my tuple implementation and included animation of the projectile. As it turns out, this wound up being very similar to the exercise at the end of chapter 2, which is about implementing a canvas. So I decided to skip that exercise and continue onto the next two chapters which walk you through implementing various matrix math operations.

Implementing the Canvas Class

Implementing a canvas class when targeting a web runtime is perhaps unnecessary since <canvas> provides a solid 2D canvas JavaScript API. However, the book has you implement color functions for tuples containing floats so a canvas that stores colors with floats instead of integer values seems like it might be a better bet going forward. So I decided to implement my own Canvas class backed by a Float32Array.

I went with 32-bit floats over the JavaScript-native 64-bit floats since I am still planning on porting this to AssemblyScript to take advantage of the v128 SIMD operations in WebAssembly. The “128” in “v128” implies you can either have an SIMD instruction operate either on 4 32-bit floats or 2 64-bit floats. Four-times-faster is better than two-times-faster. And based on a bit of research the extra precision is usually not needed or at least easy to avoid needing.

Using a TypedArray also opens the doors to backing a canvas with a SharedArrayBuffer or something similar. I can imagine this being useful by having the ray tracer running in many WebWorkers, all updating a shared canvas.

There’s a bit of a snag with SharedArrayBuffer however…

A Side Quest to Move Away From GitHub Pages

In response to the Spectre vulnerability, browser vendors updated the SharedArrayBuffer constructor to throw so that it could not be abused until they had a fix. The fix they ultimately adopted requires sending two HTTP headers with your HTML document. Well, you can’t set HTTP headers on GitHub Pages, where I was previously hosting this site.

I always planned on moving this site to my personal website, limulus.net. But my setup for limulus.net is very out-of-date. I have a GitHub repository for it but deployment is no longer automated. I just manually upload any changes to S3. None of the other infrastructure for it like the CloudFront distribution has been turned into CloudFormation templates so it’s all just sitting in AWS resources without any version control. I wanted to avoid adding to that mess for now by publishing to GitHub Pages.

Even though I know SharedArrayBuffer may not be how I ultimately choose to implement things I also didn’t want to be in the situation where I am forced into switching away from GitHub Pages in the middle of the project instead of early on. In the (ok, unlikely) event that anyone was subscribed to the RSS feed setting up redirects on GitHub Pages for that might be tricky. Better to just get it out of the way as soon as possible.

In a bid to get things done though I resisted the urge to write CloudFormation templates for everything, so unfortunately I have added to my AWS technical debt. However I did spend the time set things up in the new ways AWS recommends: I’m using GitHub’s OIDC provider to get temporary AWS credentials for the GitHub Action that publishes this site and I avoided setting up the S3 bucket to use public website mode. I learned that to get CloudFront to serve index.html files for directories served from a private S3 origin you need to write a CloudFront Function to rewrite the request. So unfortunately that means I now have a tiny bit of code for hosting this site that is not version controlled. But at least now I know how to set these things up in a more secure way.

Implementing the Matrix Class

There’s only a few things particularly interesting about the implementation of my Matrix class.

A Surprise Subclass

It likely should not have come as a surprise that the Tuple implementation would need to be treated as a matrix when doing matrix math operations. In fact, tuples need to be treated as matrixes of four rows, which is not my intuition about how to conceptualize an array of four items. It was seeming like I was going to need special handling in my Matrix class to account for whenever it was passed a Tuple and that felt messy. The solution I landed on was to create a TwoDimensionalArray class that would act as the base class for both the Tuple and Matrix classes. This way, the Tuple class can construct itself with 1 column and 4 rows and the Matrix class doesn’t have to treat Tuples as special cases.

This kind of refactor is definitely where having a robust test suite (in this case provided by the book) shines. I had confidence in a relatively substantial refactor without any added effort.

Chaining Matrix Transformations

While I won’t pretend to understand the reasons why (maybe I knew back when I took linear algebra?) if you want a transformation matrix to have multiple transformations you have to multiply them in reverse order. In other words, if you want a matrix that you can use to do a translation, then a rotation, then a scaling up, you need to first multiply the scaling matrix by the rotation matrix, and then the result by the translation matrix. The books suggests that you implement a “fluent” API of chainable methods that takes care of this for you. For example:

const twoOClock = Matrix.transformation()
  .translate(0, 1, 0)
  .rotateZ(-(2 / 12) * 2 * Math.PI)
  .scale(clockRadius, -clockRadius, 0)

What I’ve seen frequently in JS APIs is that they often require something like a final .done() method to perform the final calculations and produce the end result of the chain of operations. However, there is a way around this if you can structure your class to have the first attempt to read the values of the returned object do the finalization.

Here’s how that works with my Matrix class. The following is a selection of methods that demonstrate it. The .translate(), .rotateZ(), and scale() methods in the example above all call .#pushOperation() to push their operation onto the #operationStack array and return this.

export class Matrix extends TwoDimensionalArray {
  static transformation() {
    const chainable = Matrix.identity(4)
    chainable.#operationStack = []
    return chainable
  }

  #operationStack?: Matrix[]

  protected override get values() {
    if (this.#operationStack) {
      const operationStack = this.#operationStack
      this.#operationStack = undefined
      const result = operationStack.reduceRight(
        (result, operation) => result.mul(operation),
        this
      )
      super.values = result.values
    }
    return super.values
  }

  #pushOperation(operation: Matrix): this {
    if (!this.#operationStack) {
      throw new Error('Attempted to push operation to non-chainable matrix')
    }
    this.#operationStack.push(operation)
    return this
  }
}

The Demo: `<pixel-clock>`

The end-of-chapter exercise for chapter 5 is to use your matrix and canvas implementation to color in a pixel for every hour of a 12 hour analog clock. I did two things I really didn’t have to for this exercise: add animated “hands” and perform the rendering in a Web Worker.

Now that I have some hands-on experience with Web Workers I expect to be able to offload the work of the ray tracer off the main thread, and possibly even parallelize the work into multiple workers.

Up Next: Finally Casting Some Rays!

Now that these fundamentals are out of the way and I’ve got this site hosted where I want it, there should be less of a delay until the next post. With any luck the next post will also not be quite as long!

Chapter 1: Tuples

Eric McCarthy — Sun, 19 Nov 2023 02:51:00 GMT

The first chapter of the book focuses on setting up a foundational “tuple” library for operations on vectors and points. I didn’t really expect there to be much to show for this other than working tests. But the chapter ended with a suggestion for creating a small program to experiment with a mini physics simulation that fires projectiles at various angles and velocities and have their trajectories effected by gravity and wind.

I decided to take this a step further and create a web component that would animate the projectile on a canvas. Here it is in action:

I attempted to represent the $z$ axis by increasing/decreasing the size of the projectile. Something seems off with that though.

GitHub Copilot was helpful when writing the tuple tests. I could take the Gherkin test from the book, put it in a comment, and Copilot would generate the test code using the Gherkin-inspired test suite functions I wrote.

Looking Ahead

One thing that pained me as I wrote the implementation of the Tuple methods was knowing how inefficient they will be running in a JavaScript runtime. Vector math is the usual use case for [SIMD] instructions, but presumably JavaScript engines are not detecting that these operations could be compiled to SIMD instructions. I did a bit of forward research though and discovered that WebAssembly has SIMD support! At some point I plan to look at reimplementing Penumbra in AssemblyScript — but I first want a plain JavaScript baseline to compare against.

Onward to chapter 2!

One More Yak to Shave

Eric McCarthy — Sat, 11 Nov 2023 22:20:00 GMT

The site is getting published. The CSS needs work. Now it’s time to try and get an in-browser test runner working. For this I have two goals:

I should be able to take the Gherkin tests from the book and easily use them
The tests should be a part of this site

Initially I was thinking that I would use the Gherkin tests directly from the book — by using some existing tool to parse them and provide hooks for wiring up the steps. But I’m not really finding anything like that out there. I found this approach which defines functions for each of the Gherkin prefixes, maps them to Mocha functions, which seems like a better approach.

Mocha also seems like a good choice for the test runner. And after a decent amount of trial and error I got it working how I want. I even got Eleventy’s dev server to reload the page when the code changes, which will be nice for development.

Remaining yak shaving tasks:

Set up an RSS feed for this blog
Redo site styles

But at this point I would rather get started on the first test. This stuff has taken up way too much time already.

Getting Nerd Sniped

Eric McCarthy — Tue, 31 Oct 2023 22:52:00 GMT

The way The Ray Tracer Challenge starts is refreshing. The “Getting Started” section doesn’t spend time on any development environment setup. Instead it is a simple introduction to the Gherkin syntax then a notes on some typical pitfalls. Then the first chapter gives a brief introduction to points and vectors and throws its first test at you. It doesn’t even remind you that you need to choose a language.

I thought about what I wanted to use for my implementation. I could have chosen this as a way to introduce myself to a new language, but I decided to initially go with something I was familiar with: TypeScript compiled to JavaScript running in the browser. This way I could focus on the ray tracer itself and demos would be easy to share. After some research I also concluded that there is runway for performance enhancements like WebAssembly, WebGL and WebGPU.

Unfortunately this raises lots of questions that all have to get addressed before I even get started!

What would I name the project?
Where will I publish the site?
What will I use to build the site?
What test runner will I use? (My usual choice, Jest is great for a JSDOM environment, but that won’t work for advanced web features.)
How do I hook up the Gherkin tests from the book to my test runner?

For the name after some poking around on Wikipedia I settled on “Penumbra”. There’s probably some better names I could have chosen, but I couldn’t find any other ray tracers with that name already.

Deciding on where to publish the site was pretty straightforward. My personal site would make sense, but it’s all tied up with its own repository. I also don’t have any static site generator for it. My professional site does have a static site generator (Hugo) but I couldn’t convince myself it was an appropriate place for this project. So I decided to create a new repository for the project and use GitHub Pages to publish it.

My past experience with Hugo was alright, but Go template syntax kinda irks me. So I decided to give Jekyll a try, seeing as how it is the default for GitHub Pages. This was the first real nerd snipe.^[1] Jekyll is written in Ruby, and I figured I could just install the gem in my project. Unfortunately by default Ruby’s bundle tool does not by default install gems in the project directory like I expect a package manager to work in this decade. It’s possible to get it to work like this, but after frustration with current documentation on how to do it not matching the version of Ruby that macOS ships with I decided to try to do my development in a devcontainer. I got that working but wasn’t really happy with having to run Docker locally just for this project.

After going back to the drawing board I started researching other static site generators. That’s when I came across Eleventy, which is a fast Node.js based site generator. I actually had known about it for a number of months, but had completely forgotten about it. So now that is what I am setting up and is what this site is using.

Now of course I am getting nerd sniped trying to figure out how to use Eleventy and general web design things:

Why are WebC templates not working like I expect?
How do I get a list of blog posts to show up on the home page?
How do I get footnotes in a Markdown file to get rendered?

I’ve managed to work through these now, but I’ve run into issues with Eleventy’s dev server not updating when it should and mysterious issues with the webc:keep attribute in bundling mode. This has me wondering if my plan for using Eleventy’s dev server for development will work out. But that will have to be the next entry…

Because I have a an overly complicated project creation utility for personal TypeScript projects I always wind up starting a project by refreshing the dependencies for that utility. This time that led me to discovering that node-git has stagnated and is no longer providing up-to-date pre-compiled binaries for the latest versions of Node. This results in 3-6 minutes of compile time when installing node-git. Yikes! That’s a lot for a CLI utility that is supposed to be run via npm create. So I decided to spend the time to switch it over to simple-git which is pure JavaScript and doesn’t require any compilation. Thankfully I had written tests for the git functionality which did not mock out node-git and so swapping out the library was straightforward. ↩︎