The New Engineer

Programming is changing. I'm not telling you to discard your grief over the loss of coding, if you have it. But it is changing and I'm here with you. I'm just here to pull back the curtain and remind you: It is changing. And a lot of the changes are unimaginably good.

Most of us had to learn the command line in the beginning. That's not going away. At least not yet. The command line is a straightforward, opinionated way to talk to the computer or to talk to many computers.

Now there's a new abstraction. Coding agents. We love abstractions, right? It's far more powerful than the command line. And far less exact. If the command line is a hammer, then coding agents are clay.

This week at Fractal Tech, I made several small tools to satisfy my curiosity as we were learning about training LLMs. The new abstraction is writing down what software you want, putting it into one side of a box, and pulling the software out the other side.

I know, it's not that easy. Sometimes you have to more prescriptively ask for what you want. Like making a wish with an adversarial genie. Regardless, it works.

I've been aware of this abstraction for months, yet it's taken me time to integrate that knowledge. Small tools are the new bash one-liners. They take a little longer to make, but not that much longer. Oftentimes, only five minutes. These small pieces of software help us satisfy and grow our curiosity.

On Monday, we learned about using local, paid, and frontier (Anthropic, OpenAI) AI models. The challenge for the day was to rebuild a dictation app like Wispr Flow.

I had several recordings of meetings with friends and decided, instead, to make a transcription app for multiple speakers. And the bulk of it was done within one to two hours:

Making this tool exposed me to contrastive training: the model trains on audio clips of one person versus multiple people, learning to differentiate and cluster by voice profile.

Before coding agents, I would have either not made the tool or used an existing tool. Both ways lead to no more understanding. This small tool solved a real problem I had too: transcribing meetings so I could extract action items. And I can always add to it…

Tuesday was the same. We learned about fine-tuning AI models using a technique called LoRA, which involves compressing the changes to the model into something you can load up when you want to use the model.

The coding agent helped me set up a fine-tuning process for the image generation model Stable Diffusion XL. I used photos of myself to generate fake family photos of me and myself and myselves:

Did I need this tool? No. But managing the model checkpoints (snapshots of the model during its training) on Google Colab taught me about practical training aspects. And figuring out how to run the model locally and later on "serverless" GPUs exposed me to a large surface area.

I was so quickly able to accelerate from conceptually understanding a piece of the training space to building something on the frontier. The loop from noticing your curiosity to executing on it and satisfying it is so tight now.

I believe that deep curiosity leads to expertise. I also believe that satisfying curiosity makes people happy. Therefore these coding agents are promoting a generation of happy experts.

The new abstraction for development is moving up the stack from libraries and frameworks to entire tools. The New Engineer is a curious individual that advances the plot quickly like a storyteller… an inventor with enough personal prototypes to fill a Hall of Demos at their own World's Fair… a just-in-time specialized tool developer.

Another example of that is Thermostat, a simple web app I made to visualize how the temperature parameter affects LLM output (blue meaning cool, predictable outputs and red meaning hot, uncommon outputs):

As a technical artist, I've spent a lot of time writing graphics code that transforms values using functions that repeat themselves.

Discovering that changing the temperature of LLM output was as simple as dividing all the predicted outputs by a constant made me curious: "What if this number wasn't constant?" This is the kind of question that often informs my art direction.

Again, within an hour, I had a functional tool to test out my ideas. It's a project that I learned from and can put in my portfolio.

The New Engineer has questions and finds answers quickly. The New Engineer plays with their work. On a whim, I wanted to see how straightforward it would be to create an ASCII post-processing effect on my computer:

I had a guess as to how it worked. I talked through what I wanted with the coding agent and within the hour it had created a small tool to my specification:

Don't suppress your grief or fear about how fast everything is changing. But don't let it keep you from indulging in the curiosity and excitement of the new frontier as a new kind of engineer.

Curious about hiring me or working together? Check out my hiring page. You can also hear more from me on YouTube or Twitter.