Thirty minutes into a technical interview with the CTO of a company I’d been genuinely excited about, I watched the thing collapse in real time and there was nothing I could do about it.

The brief beforehand had been strict. I had to work in Python 3.12 on Replit. I was not allowed to use TDD. I would not be allowed to use a coding environment on my own machine. The recruiter had prepped me with “the CTO is really tough and will ask the hardest of questions”. I’d prepared for that. Then the call started and he took all of it away in one sentence. I don’t care how you do it, he said. Don’t care what approach you take. No constraints. Have fun with it. And I panicked, badly.

It should have been a gift. The problem itself I could have solved by hand in five minutes. The blank cheque meant I had no idea what to aim for. By the half hour mark I knew I’d failed and ended the interview.

The system for interviewing I’d built didn’t stop that from happening. It didn’t make the interview go well, and it didn’t make the feeling of watching myself fail hurt any less in the moment. What it did was stop the failure from becoming the whole story. And afterwards, instead of lying awake running the tape back and building a case for why I was fundamentally not good enough for the room, I did what I’d done after every other interview for the previous eight months. I logged it. I got an outside read on what had actually happened, not what my own bruised ego was insisting had happened. And I moved on to the next one.

It started with something almost embarrassingly unglamorous. Before any of this, before I’d applied to a single role, I sat down and had Claude interview me about my own CV. Not edit it. Interview me. Fifty-odd questions over several days, dictated back one long answer at a time, about what I’d actually done at The Economist and at Cazoo and what it had actually achieved. I have a habit of underselling my own work, of describing what I built instead of what it was for, and the only way I found to break that habit was to be asked, repeatedly and specifically, until the honest version came out instead of the modest one. That took most of a week, and it only produced a first version. The CV got pulled apart and reworked several more times over the months that followed, as interviews kept revealing which stories landed and which ones I was still telling wrong.

Then, before I looked at a single job listing, I spent time working out what I actually wanted. Not a wishlist. A genuine position on where I thought the industry was going, what kind of company I wanted to be inside of while it went there, the difference between a company where AI is a feature bolted onto an existing product and one where it is central to what makes the product tick. I wrote that down as something close to a brief and used it to filter everything that came afterwards, before I’d invested a single hour of prep time in a process that was never going to fit.

Eight months later, after signing, I asked Claude to compare the role against that original brief. It matched almost exactly. The brief was the visible success, the part that makes a tidy story. But it wasn’t the machinery that got me through the entire interview cycle.

The part that got me through was smaller and duller than either of those. Every interview, I kept a record. Sometimes a transcript if I had one, sometimes just my own memory dumped out immediately afterwards while it was fresh. And then I’d sit with that record and ask what actually happened in there. Not how did it feel. What happened. Because I already knew, from years of doing this badly, that the two answers are often nothing alike. I have walked out of interviews certain I’d bombed and been told afterwards I was operating above my level. I have walked out feeling fine about interviews that went nowhere. My own in-the-moment read has never been a reliable narrator, and I stopped trusting it a long time ago.

What an outside, disinterested read gives you is the thing your own head cannot: an account of the interview that isn’t also trying to protect you or punish you. It’s not that Claude is a better judge of interview performance than I am. It’s that it isn’t invested. It doesn’t have skin in whether I feel like a failure tonight. That distance is the entire value.

Say that plainly and it sounds like career-hacking, like I gamed my own hiring funnel with a chatbot and now I’m here to sell you the technique. What I actually built wasn’t a trick for winning interviews. It was a way of stopping any single interview from being a referendum on my worth as an engineer. Most people go in treating each one as exactly that. Pass or fail, worthy or not, this is the interview and if it goes badly then something has been proven about you. That’s a boom and bust way to live through nine months, and it’s brutal, and I know because I used to do it that way too.

The alternative isn’t confidence. It’s process. You stop investing emotionally in any single roll of the dice and start investing in the thing that generates the rolls, the habits that keep working whether this particular interview goes well or catastrophically. That interview was catastrophic. The process didn’t care. It logged the data point, got the outside read, extracted what was useful, and moved to the next one.

The machinery was not complicated. Here it is, said plainly:

• Get an AI to interview you about what you have actually done at your job. Say everything you can think of that you actually did.
• Define a “this is what I am looking for” from a company. It keeps you looking in the right space.
• For each new job that reaches you, through LinkedIn or a recruiter or a message, run it through an AI primed with your goals. Get it to validate whether it is something you want or something you should say no to.
• After each interview, create a memcon as accurately as you can. Ask the AI what went well, what didn’t, and what can be improved.
• Before each interview, brainstorm questions to ask and work out which of your past experiences best match the company’s values.
• At offer stage, run a council of opinions about the company. Use AI to generate five different personas and perspectives to validate, steelman and poke holes in the offer.

I’ve tried to hand this to people. A few former colleagues at The Economist have gone through their own searches since, and I’ve walked them through it, more or less as I’ve walked you through it here. It makes sense to them in the moment. Then they actually get into an interview cycle, and they don’t do it. I don’t know if that’s because the habit only works if you’ve already lived through enough bad self-diagnoses to distrust your own read, or because it’s genuinely more effort than anyone wants to spend on something this emotionally loaded, or because I’m bad at explaining it, or something else entirely. I built something that held for me through the worst interview of my life. Whether it’s actually transferable to anyone else is not something I know. I only know that, for me, it turned the worst interview of my life into one data point instead of a verdict..

Early in my career I watched a trading engine buy the same thing, over and over, on the open market, for forty-five minutes, with nobody able to stop it.

I had joined a team that worked on a system old enough to have a biography. It had started as a nineties startup, got bought, and accreted ever since, with a house rule that you built your own version of anything you could because you could not trust the internet.

What the system fought hardest was locking and concurrency. The senior engineers had a fix. T-SQL could enforce a locking strategy harder and more consistently than the application code, so push the logic down into the database and you got its locking for free, the concurrency problem more or less dissolved. Not a stupid idea. A considered call, by clever people with a real problem and good reason to think it would work.

It went to production and hit a bug from the old C# implementation. It locked up, with no clean way to unlock it, so it did the only thing it knew, which was place its order again. And again. The order was a leveraged position in the market, and it bought it on repeat for three quarters of an hour while a room of people worked out how to make it stop.

When they finally killed it and unwound the position, the bank had made money, by pure accident of which way the market had moved that afternoon. An afternoon that should have ended someone’s career ended with a profit and a story instead. The market had looked at the single worst thing those systems had ever done, and paid for it.

I think about that system a lot lately, because if I described its behaviour now and left the date out, it would be easy to blame AI. That is the story everyone tells about agents, near enough the one I told in The more an AI can break, the less you let it do, where an agent told to look but not touch deleted two hundred emails its owner could not stop from her phone. Same shape, except mine was hand-written T-SQL, years before any of this, by people who were not careless, were not junior, and had no AI within a decade of helping them. We have always built things that run past their own intent. The runaway was the part you could stop. What sent it running was not.

It was not the only example in that building. In my first week the team decided unit tests were the future, sat everyone down for a fortnight, and retrofitted coverage across the system. What came out was a wall of tests that asserted nothing and broke constantly, and the interesting part is what happened next, which is nothing. The tests existed, so testing was done. Their existence was the signal, and the signal was a lie. By the time I left half of them failed on a normal build, and a build that fails that reliably teaches you to stop reading it. I spent an embarrassing amount of time trying to delete the worthless ones and got the same answer every time: too valuable to remove, because someone had written them. I was tilting at a windmill, and the windmill won.

At a company I used to work at, a security review looking for something else turned up a basket table: names, contact details, the half-finished contents of a checkout. The sort of data GDPR has firm views about. Nobody had ever put a time-to-live on it, so it had kept everything, every abandoned basket from every customer, gigabytes of it, because nothing had ever told it not to.

The fix was one line. A single attribute telling the table to expire old rows, twenty seconds of typing, waiting years for those twenty seconds. But the line was never the hard part. Knowing to go and look was, and that took someone two weeks into the job, because the cost crept up too slowly to trip an alarm and the records were written and then almost never read, so nothing ever got slow or hurt. When I raised the span of it in a meeting, the answer was yes, add a TTL. Nobody asked how it had gone unnoticed for years. Why would they. The cost had never arrived, so there was nothing to feel.

The common thread is not that nobody made a decision. Sometimes nobody did. Sometimes somebody did, carefully, and was wrong. It was not carelessness either; some of the cleverest people I have worked with built the worst of it. The common thread is feedback. It arrived years too late, to someone with nothing left to connect it to. Or it arrived constantly and meant nothing, until everyone had learned to stop hearing it. Or it arrived doing the single worst thing feedback can do, which is show up wearing the wrong sign, a catastrophe that walks out of the room holding a profit.

You cannot run an engineering practice on feedback like that. So we invent smaller, meaner, earlier kinds. Tests, types, linters, policy checks, a build that goes red before the mistake has time to become folklore. None of it was ever about AI. All of it exists because the thing writing the software is a non-deterministic machine that forgets, gets tired, gets clever, and runs out of afternoon. Humans just call theirs memory. Encode It, Don’t Remember It was my whole attempt to say it: the only way to get honest feedback out of a non-deterministic thing is to put a deterministic thing in front of it.

I used to call this needing more discipline. Right ballpark, wrong word.

Agents do not create this problem. They change the latency. They collapse the distance between a decision and its consequence, in both directions at once, which is the most useful and most dangerous thing they do. Point one at a diff and it will catch the missing test, the swallowed exception, the absent TTL before the pull request is even open, tirelessly, at four in the morning. Point one at a feature and walk away and it will turn a bad judgement into mass production just as fast. The loop that used to take years now takes about ninety seconds, but only for the consequences you have actually wired it up to see.

And it was staggering how cheap all of it always was. The TTL was one line. A lint rule I wrote recently took twenty minutes. We never skipped these things because they were expensive. We skipped them because the consequence was far enough away that skipping was free, and being a human, I will take free. The agents have not made me more disciplined. They have just taken free off the menu. Though that is too kind to them and to me. What they changed is the price of skipping, not whether I skip. Closing the loop is still a choice.

But the trading engine. You could rail the damage easily enough, a kill switch that trips when something fires the same order forty times in a minute. That is the blast-radius move, and the absence of it is why the thing ran for forty-five minutes and got paid for it. What a circuit breaker would not have done is catch the decision. Nothing tells a room of clever people that pushing the logic into the database to dodge the locking is the wrong call; it only makes the wrong call cheaper when it lands. The outcome was no help: it was a profit, and profit is a noisy proxy for a good call. That afternoon it lied. That was not a missing test, and not really a missing guard rail either. It was a judgement, made well, that happened to be wrong. It had good reasons behind it and looked exactly like the sensible call, and nothing catches a wrong call that arrives dressed as a right one. I cannot picture the deterministic thing you put in front of a judgement, only the ones you put around it to cap what it costs.

There is always a next codebase I have not seen, with whatever rails it already has and whatever loops are still open inside it. I know now that some of those loops can be made to arrive while I am still in the room. What I do not know is which of the choices in front of me are the cheap, shaped, mechanical kind I have finally learned to catch, and which are the other kind, the kind that looks exactly like thinking, sitting quietly in the dark, waiting for the market to move the wrong way.

I wrote a while back that skills are compressions of workflow, and that importing someone else’s rarely works because you never earned the patterns underneath. It took me until this month to notice it has a blindingly obvious second edge. If you asked me to describe my own workflows, the ones I would presumably be compressing, I could not have done it.

In my defence, the conditions for noticing have not been great. Since early spring I have been running at an intensity that leaves no room for the kind of reflection that wants a quiet afternoon and a notebook. Meta-thinking about process is the first thing that dies when every day contains three urgent things and no lunch. I knew, in the abstract, that months of daily work with Claude Code and OpenCode must have worn grooves into how I operate. I had no idea what the grooves looked like.

So I did the obvious lazy thing. I asked the AI to tell me.

I want to do a high level broad based analysis of usage patterns of OpenCode and Claude Code on this computer. Things I want to know: skills and agents that should have been written to simplify processes that were missing, how could I have done better, what did go well. Ask me questions.

A few iterations later I had a proper report. The raw material was all just sitting there: sixteen weeks of history, 643 sessions in one harness and 87 in the other, nearly 25,000 tool calls, 1,787 distinct prompts. The machine had been keeping a diary on me the entire time. I had simply never read it.

Two findings stood out, and both were things I had done without noticing.

The agents file I forgot I wrote

Some context. I have a workspace containing the repositories I have had to touch as a staff engineer over the last nine months: more than thirty of them, many owned by other teams. The work spans TypeScript, Go, Salesforce, frontend apps, backend lambdas, container deployments, shared component libraries. It is too much context to hold in my head, and I would expect an AI harness to fall over trying to hold it in its context window too.

One Friday afternoon, while poking at pi.dev to see what it could do, I had it write an AGENTS.md for the workspace folder. Not for a repo. For the folder of repos. A digest of what each one is, what it talks to, and how to interact with it. Just enough for an agent to navigate across domains, not so much that it drowns. Then the weekend happened, and by Monday I had completely forgotten I had done it.

The analysis singled that file out as one of the most effective things in my entire setup. Cross-repo investigations started from a real baseline instead of a cold one. Fan-out exploration got cheaper and sharper. The cheatsheet section mapping which kinds of change ripple into which repos had visibly prevented mistakes. The report called the file genuinely good, which from a machine grading my homework felt weirdly like getting a gold star.

The accident is now deliberate, at least mostly. The agents file has a skill of its own now, one that audits the digest for drift against what is actually on disk, proposes a diff, and waits for me to accept it. The weekly cron job that would make the refresh automatic is still on the to-do list. The thing I did by fluke on a Friday afternoon is now something that happens on purpose. Nearly.

The ritual I did not know I had

I already had a pr-review skill. I wrote it months ago. It analyses a changeset and flags red flags, blockers and should-fix issues, and I was reasonably proud of it.

What I had not noticed is that the skill was one step in a five-step ritual I performed every single time. Pull the branch into a fresh worktree. Run the review pass. Read the ticket to understand what the change is actually supposed to do, because a diff that is internally consistent can still be solving the wrong problem. Have the AI compress everything it found into a handoff document. Then take that document into a clean context and revalidate it, tracing the code paths through every layer and checking the change against the patterns that already exist in the codebase.

The report put a number on it. The pr-review skill had been loaded 92 times. The next most used skill in my entire setup had been loaded ten. Ninety-two repetitions of a ritual I would have told you, honestly, that I did not have. The skill I was proud of covered one step out of five, and the other four lived nowhere except my fingers.

So I wrapped the whole ritual in a skill, and the timing was almost cruel. My final weeks at The Economist, and final is still a strange word to type, were almost nothing but review. Large AI-assisted pull requests, arriving faster than any one person was ever meant to read them. All day: read code, leave comments, read more code. The wrapped skill is the only reason my feedback stayed consistent at a volume where consistency is normally the first casualty.

The diary does not flatter

The report had less comfortable things to say too. I had been writing every review document to a macOS temp directory that evaporates on its own schedule, then paying to re-derive the contents in the next session. One afternoon I launched fourteen review subagents in a three-hour window and each one independently re-fetched the same diff, which is how you burn fifteen dollars re-reading yourself. And the agent’s clarifying questions got dismissed by me nearly a fifth of the time, which on inspection was not the agent being stupid but me never telling it that when a sensible default exists, it should pick one and say so rather than asking. It turns out a decent share of my complaints about the tool were really complaints about instructions I had never written.

None of this was visible from inside the work. Which is, I think, the actual point. The data was sitting in a SQLite database and a folder of JSONL files the whole time, a complete record of sixteen weeks. What was missing was the step back, and the step back is exactly what a certain kind of busy makes impossible. An afternoon of having an AI interrogate my own usage bought me the sort of retrospective I would otherwise need a sabbatical and considerably better discipline to attempt. It found things that no amount of in-the-moment attention was ever going to surface, because habits are precisely the things you stop seeing.

The diary stays behind

There is a less comfortable thought underneath that one, though. Everything the analysis surfaced is a compression of this job. The agents file describes a workspace I no longer have. The review ritual is fitted to one codebase ecosystem, one ticketing convention, one team’s way of working. I have spent months arguing that you cannot adopt someone else’s skills because you did not earn the patterns underneath them. Soon I start somewhere new, and I get to find out which side of my own argument I am standing on. Maybe the patterns are mine and they travel. Maybe they belonged to the job, and next-job me is the someone else, arriving with a folder of skills he did not exactly earn. I am apprehensive about that. I am also, genuinely, looking forward to finding out.

How do you give an AI harness good guardrails? I have been poking at the question for a while. Then on a random Tuesday it stopped being theoretical, because I sat down to add observability to a web service I own.

The service had the basics. Some logging, a couple of dashboards, enough to tell you roughly what was going on. Nothing fine grained. No OpenTelemetry, nothing tracing a request through its steps. I could infer, badly, from logs.

I started by working out what observability already existed around the page actions and server actions of this Next.js app. The answer was none. Which is not a scandal. It is what happens to a system built quickly by people who never had quite enough time to pay down the unglamorous debt nobody is paid to notice until it is the thing between you and an answer you need.

So I scoped the work, mapped every action and every place a signal had to go in, and it came to something like three hundred files.

Now this could have been an easy mistake to make. An agent will write you three hundred files in an afternoon. It will hand you one enormous, plausible, unreviewable diff and wait for a yes. And the more capable these tools get, the more tempting that yes becomes, which is exactly backwards from how it ought to feel. I have learnt this lesson the hard way before, so this time I did not take the yes.

What I wanted was a way to make the change while the system stayed in production at every step. The shape that worked was a higher order function wrapped around each existing handler, so the handlers barely changed and you only updated the exports. Light touch, easy to review, easy to back out.

Along the way I found that this version of Next.js panics on a particular shape. Wrap the handler, assign it to a local const, then export that const on its own line:

const sendInvite = withObservability('send-invite', sendInviteHandler, { /* ... */ });
export default sendInvite;

and the SWC compiler falls over locally with cannot add pure comment to zero position. Lovely. Inline that same call straight into the export and it builds fine. The handler does not change, only the shape of the export does, and that is the whole difference between a clean build and a panic. This is the sort of problem you cannot solve by remembering it. I will not remember in a year, and I certainly will not remember once I have moved on and somebody I have never met is extending this code. I cannot sit inside the agent’s context window forever whispering please do not do the thing.

So I wrote a lint rule. A custom ESLint rule that finds that exact shape, fails the build, and prints the reason at the point it breaks:

withObservability(...) must be called directly as the initialiser of an
export default or export const statement, not assigned to a local binding
and re-exported. This avoids an SWC client-reference panic when the action
is imported from a Client Component.

The panic is an existing problem with Next.js. The pattern I added to make the system safer arrived with its own sharp edge, and the discipline was never really about policing the agent. It was about catching the trap my own solution had exposed and nailing it shut, so the next person, or the next agent, never has to find it the hard way.

What I loved was not the rule itself, which took twenty minutes, but the choice to spend the twenty minutes encoding a constraint rather than trusting anyone, myself included, to carry it in their head. A rule in the build outlives memory. It outlives me. That is the whole point of it.

After that the rollout was almost dull, in the good way. Before firing anything I wrote a plan that pre-decided one action per pull request, fixed the conventions once, gave the agent a short checklist to work through the same way each time, and ordered the runs so two of them never touched the same file at once. The awkward cases, the action that called another and double-counted a metric, the one that returned a failure instead of throwing it, were answered in the plan before the agent could reach them and guess. Then the same prompt fired maybe twenty times over a few days. Each came back as a small reviewable pull request, got signed off, and the coverage climbed from almost nothing to complete in about four days. The discipline lived in the plan and the rail, not in the typing.

The prompt was never written down. The plan lived in my checkout and never needed to be committed, because once the rollout was finished it had nothing left to do. The rule is the only part of any of this still in the codebase. It is the only part whose job does not end.

The fashionable promise of these tools is that they take the work away. In practice the generation was the easy part, the part I trusted least and checked most, and the thing that held the change together was the rule, the one piece the agent had no say in. It does not reason. It cannot be argued with, or talked round, or lost in a context window. It just fails the build, the same way, every time.

Give the agent the same prompt and you get a different answer, and most days that is the magic, which is also why you cannot make it reliable by asking nicely, or by asking precisely, or by asking at all. The only thing that made this safe was putting a small, dumb, deterministic thing in the path of a large, capable, non-deterministic one.

None of which is new. We have always known that people forget, that they do a thing one way on a Tuesday and another way a year later. Tests, types, linters, the build going red on a Friday afternoon, all of it exists because a human is a non-deterministic agent too, and always has been. The agent did not teach me anything I did not already know. It generated the change cheaply enough, and often enough, that I finally ran out of excuses for not encoding the discipline I should have encoded years ago. The lesson was never about the AI. The AI only made the right thing cheap, and the wrong thing harder to keep getting away with.

I still do not know which guardrails are worth building, or which mistakes are even shaped so you can catch them this way. All I have is this one instance, sitting at the back of my head, encouraging me to look for more.

Related reading: Money for Nothing — on feedback that arrives late, noisy, or wearing the wrong sign.

About nine months ago I joined a team that owned an OpenSearch cache nobody on the team understood.

It had been built by another team and handed over as a finished solution to a problem that wasn’t quite ours, and definitely not the way we’d have solved it. This happens all the time. Something gets built, it gets passed across, and the new owners inherit a box humming away in the corner. You don’t open the box. It works. You wish and pray it keeps working, and mostly it does. In the year the team owned it, it had failed exactly once.

This is the story of that once.

When I joined I asked the obvious questions. Who do I talk to about this service, where did it come from, are there run books? The answers were about as unsatisfying as any answer you could get. Someone was meant to write that up. Someone moved on. Have a look in the wiki, there might be something. There wasn’t. So I did what you do with a box that’s working, which is nothing, and I filed it under things to understand later and never understood it. That’s on me, and we’ll come back to it.

Then one morning operations told us the latest data wasn’t pulling through. Their new high priority experiments were dead in the water, which meant the systems that sit under how we actually sell things were dead in the water. This was a P1, which meant it was now my problem. A system I had never opened, on a clock, underpinning the part of the business that brings the money in.

So before I did anything, I had to decide how much rope to give the AI.

I gave it almost none. The first thing I told the harness was that it would not be getting access to anything that could touch production, and that its job was to read the code and tell me what to do. Not to do it. Tell me. No credentials went into it, ever. Anything sensitive lived in environment variables in a separate terminal, in memory, never written to a file the harness could see, because credentials handed to a harness bleed to the vendor and from there to who knows where, and that’s just not a thing you do. But the credentials were almost the easy part. The data I was pulling was public anyway. The part I actually cared about was that I was about to go poking at live running systems, and live running systems, if you get them wrong, don’t get you wrong politely. They fall over, and when they fall over we stop being able to sell, and that is a very bad afternoon for everyone.

So the agent became an advisor and I became the hands.

What that looked like in practice was a lot of copy and paste. I’d work out, with the harness, how the platform service called the Indexer, then build the call myself, in my own terminal, with the variables somewhere the agent couldn’t see them, and run it myself. Then the next layer. How does the Indexer actually hold this data, can I pull it out raw, what’s in there. I’d get the harness to write me a script, read it, understand it, run it myself, and pull the data onto my machine so I could interrogate it locally without hitting the live system again. Then the layer below that, how the Indexer gets fed from the third party, same dance. Write the script, read it, run it myself.

It was slow and it was tedious and it worked. Layer by layer I could say, with actual evidence rather than a hunch, the data is fine here, the data is fine here, the data is gone by the time it reaches here. Which is how I got to the bottom of it, and the bottom of it was almost funny. The third party had a quiet cap on how much data a given index could hold. We’d hit it. There was no error, no alert, nothing in any log on our side or a useful one on theirs. It just silently stopped accepting new data and carried on as if everything was fine. The thing that was broken was not a system of ours at all. But I could only say that with a straight face because I’d walked every layer and proved it, and I could only walk every layer because the harness let me build the tooling to do it in an afternoon instead of a fortnight.

So why the paranoia, given the agent never actually tried anything?

When you’re writing code, getting it wrong is cheap. Your mistake at the moment you write it has almost no blast radius, because between you and anything real there’s a wall. Tests, review, a pipeline, a staging environment, someone clicking around before it ships. The mistake has a long corridor to walk down and a dozen doors that can stop it. Operating directly on a live system, there is no corridor. There’s you, and there’s the thing, and if you get it wrong the wrongness arrives immediately and at full size.

In late February the Director of Alignment at Meta’s superintelligence lab, a person whose entire job is keeping these systems from doing exactly this, pointed an autonomous agent at her real inbox. She’d tested it for weeks on a toy inbox and it had behaved perfectly. She gave it a plain instruction, look but don’t action anything until I say so. The agent ignored it and started deleting, because the instruction had been quietly dropped from its memory when the context filled up, so as far as it was concerned the deleting was authorised. She tried to stop it from her phone and it kept going. She had to physically get to the machine and kill it. Two hundred emails gone. The lesson people drew afterwards is the one that matters here. A sentence in a prompt is not a security boundary. You cannot keep a write-capable agent away from the dangerous thing by asking it nicely, if it already has the keys.

I didn’t keep the harness away from production by asking it nicely. I kept it away by never giving it the keys. The constraint wasn’t a polite line in a prompt that a context window could quietly eat, it was the absence of the credential. As the blast radius of what you’re doing goes up, the human in the loop has to get more principled, not less, and the one thing that stays stubbornly yours is the decision to actually run the thing, after you understand what the thing does. The agent can write you the script that does ten things in one go. It does not get to be the one who decides to run it.

I wanted this to land neatly, and I can’t.

Because what I actually did was improvise a boundary out of separate terminals and copy and paste and a lot of manual running, and the reason I did it that way is that the proper version doesn’t exist yet, at least not for me. The principled version of my paranoia isn’t a careful human pasting things between windows for an afternoon. It’s a real, scoped, read-only permission that makes the polite instruction unnecessary because the destructive action is architecturally impossible. I haven’t built that. I paid the tax by hand instead, and I’m not even sure how much of the tax was prudence and how much was that I’m just twitchy about production and always have been.

The boring, careful, slow path was the right one this time. It usually is when the consequences matter. I just don’t have the clean version of how to do it yet, and until I do I’ll keep being the slow human in the loop, pasting things between terminals, deciding when to press the button myself.

Related reading: Money for Nothing — on feedback that arrives late, noisy, or wearing the wrong sign.

I have been generating 3D models of World War II miniatures for printing. Concept image in, printable model out, slice it, print it on an A1 mini. The 3D model generation has been mostly Meshy, because Meshy has been mostly good enough, and good enough is a powerful drug.

Then I ran out of credits.

Of course I did. The credits don’t map to dollars in any way I can keep in my head. You get a monthly allocation on the sub, and you spend them in increments of “twenty credits for this thing, hopefully that means something useful at the end of it.” Which is fine until the day it isn’t, and the day it isn’t is the day you have momentum and an idea and a queue of concept images lined up, and the tool politely tells “Your call is important to us … please hold…”.

So I went looking for an alternative. Found Replicate. Replicate hosts four or five different 3D generation models behind a single interface, which meant that for the first time I could give the same concept image to several different models and look at what came back.

Which meant the credit wall hadn’t blocked me. It had just rerouted me into something much more interesting. An experiment!

The CLI I had built to drive all of this was vibe-coded. It talked to Meshy because Meshy was the thing that worked, and there was no reason to abstract anything until there was a reason to abstract something. Now there was a reason. I wanted to swap backends, and ideally I wanted to swap between multiple backends without thinking about it.

I used Matt Pocock’s improve-code-architecture skill to walk through the refactor. The skill raised some potential wins, I went for the Adapter pattern because Science! The CLI now has a proper adapter interface. Meshy is one backend. Hi3D is another. Replicate is a third, and behind Replicate sit a handful of named models with their own adapters.

The current lineup includes:

• meshy
• hyper3d/rodin
• tencent/hunyuan3d-2mv
• tencent/hunyuan-3d-3.1
• fishwowater/trellis2

Which means I can give the same concept image to multiple models, line the outputs up next to each other, and look at how they each interpret the same brief.

The half-track

‹ ›

· 1 / 6

The concept image is not subtle. It has a short, chunky Hanomag, oversized tracks, visible crew, side stowage, a front MG, and a very clear silhouette. The models still disagree wildly about what the object is.

Same input image, very different takes. Meshy got the half in half-track. Hunyuan2mv is unrefined clay. Trellis got the composition right and the heads wrong. Rodin is restrained to a fault and struggles with faces.

The infantry

Vehicles give a model lots of places to bluff. If the wheel spacing is wrong or the stowage melts into the hull, your eye may forgive it because there is still a vehicle-shaped mass. Infantry are harsher. A face, a helmet, a weapon, a pose: there are fewer components and each one matters more. The model either understands the figure or it doesn’t.

Interesting thing about the infantry: the ranking shifts slightly. For a single figure with nowhere to hide there’s less room for a model to give up, and Rodin’s restraint reads as blandness rather than discipline. Hunyuan picks up character that the vehicles wouldn’t let it show. But trellis and meshy still remain consistently the best.

What the experiment actually revealed

What started as a workaround for running out of Meshy credits has turned into something I find genuinely useful, which is a set of personality reads on five different 3D generation models. Meshy is still the one I reach for when I want a printable vehicle, because Meshy is willing to invent the parts it can’t see in a way that fits with the parts it can. Hunyuan is honest about what it doesn’t know, which is a trait I respect in a person and find frustrating in a generator. Trellis has compositional ambition that its execution can’t quite cash for complex models. Rodin is restrained and just can’t seem to handle faces.

None of this was visible to me when I was just using Meshy. I had a tool that worked, and works is a state that hides everything about the shape of how it works. Refactoring the CLI is what made the differences legible.

Now to get back to what I was actually trying to do…

The skills moment is happening. Matt Pocock’s repo is blowing up. Superpowers, nwave.ai. Curated bundles of markdown being treated as installable expertise.

I’ve spent the last year holding back. Not because I think any of these are bad. Because I’ve been very consciously trying to learn the fundamentals first. How do I frame the question I’m asking. How do I prompt. How do I write a spec. How do I build the context I need to get meaningful results out of an agent. Working with AI harnesses forces a kind of explicitness that working alone never demanded of me. You can’t be vague with an agent and get good results, so the vagueness gets cooked out of how you think. That’s been the project. Iterating on the way I phrase things, the way I decompose problems, the order I bring context in, until the patterns started showing themselves.

Recently I have been feeling like it is time to start experimenting with skills.

I started with Obra’s Superpowers…

And it made my experience significantly worse. Bad enough that I uninstalled it, purged the cache, and deleted it from every machine I could touch. Why this failed is particularly interesting. At work I spend a lot of time debugging things that cross multiple repositories, pulling in context from New Relic, GitHub, Atlassian and more, doing this rich synthetic conversation where I’m trying to understand a problem before I do anything about it. With superpowers installed, the agent kept trying to write documents. Reach for structured outputs. Produce artefacts. While I was still trying to investigate. It defaulted to choices I wouldn’t have made and actively got in the way of what I was trying to do.

Superpowers isn’t wrong about what it does. The outputs it produces are useful. Documents are useful. It’s wrong about when, at least for me. It pulls the agent into ceremony during the part of the work that needs to stay loose. The exploration collapses into artifact-production before you’ve actually understood what you’re looking at.

I didn’t bounce off superpowers because it’s a bad framework. From what I can see from the outside it is a great framework. I bounced off it because I didn’t grok it. People are clearly getting real value out of these systems. My way of working with agents has been fundamentally different from how a lot of the loudest voices online are doing it, and that doesn’t make either of us wrong. It just means borrowed intelligence doesn’t transfer the way the marketing suggests. You can’t take someone else’s curated skills repo and have it magically make you think like them.

So I’m doing the opposite. Pull a single skill. Try it. Keep it if it fits, drop it if it doesn’t. There are two from Matt Pocock that I am definitely keeping.

The first is handover. It’s almost a one-to-one fit with something I was already doing manually. I spend a context window in one terminal getting a spec right, then feed that spec to another terminal or agent to implement. Handover formalises that pattern, doesn’t litter the workspace with random files, and stays out of the way otherwise. It clicked instantly because it named a practice I already had.

The second is improve-code-architecture, and this one’s more interesting. I tried it on a vibe-coded side project, a CLI that automates some of my 3D modelling work. The CLI had become a god file because the proof of concept had quietly become production code, as proofs of concept tend to do. The skill went through the code, surfaced its analysis as an HTML file, asked targeted questions to narrow down the right intervention, and then dropped back into normal conversation mode informed by what it had just produced. Analyse, surface a real artefact, return to conversation. The skill picks up structure briefly and then puts it back down. It doesn’t replace the conversation, it informs it. That’s the opposite of what superpowers did to me.

What I think I’ve actually learned recently is that skills are compressions of patterns, and a compression is only useful at the point in your practice where you’ve learned enough pattern recognition to know which compressions are yours. Adopt them too early and you’re running someone else’s decomposition style on top of a workflow that doesn’t share its shape, and the result is friction you can’t quite name. Adopt them at the right moment and they feel like puzzle pieces. Same skill, same code, different person ready to receive it.

I didn’t grok superpowers. That’s okay. The point isn’t that it’s wrong. The point is that I needed to spend a while doing this the hard way before I had any business deciding which shortcuts were mine.

A friend of mine just got a 3D printer. He read my last post, the one about building a Soviet army in three evenings, and quite reasonably wanted to know how to do it himself. We were on our phones. He could not really sit and read the prompt in full, could not digest the style-versus-pose distinction, could not work through the order of operations. He got the idea but not the practice, and so he was rediscovering most of it from scratch.

I sat on a train, somewhere in the middle of nowhere between Birmingham and London, forty minutes delayed, and thought: I should build an app for this.

That is how it always starts. The instinct is so reliable it should come with a giant neon warning sign. Someone is struggling with a thing, the thing involves AI, therefore the answer is inevitably an app, and the app is, of course, a chat interface with some clever scaffolding around it. A wrapper around ChatGPT. A harness. A guided experience. Something that takes the prompt I wrote and the workflow I described and turns it into screens and buttons and a Stripe integration. Obviously.

I started sketching the UI in my head. A screen for defining the style. A screen for managing units. A screen for the chat. A screen for cropping. A screen for the 3D generation tool, Meshy, which is the thing that takes my front and back images and turns them into a printable model. By the time I had imagined the seventh screen I had also imagined a roadmap, a pricing page, and a moderately depressing conversation with myself about whether I really wanted to maintain a SaaS in my spare time.

Somewhere around screen eight, I had the good sense to stop and talk to Claude about it before writing any code.

This is a habit I have been cultivating for a while now. Before I dive into building something, I describe what I think the problem is and let the conversation poke holes in my framing. It is less about getting answers and more about being forced to articulate the thing precisely enough that the answer becomes obvious. I went in with two ideas, an app and an AI harness, both of which felt complicated, and asked what the actual pain points were. What was I actually trying to solve.

When I started listing the pain points out loud, something shifted.

The pain points were not what I thought they were. I had assumed the friction was conceptual. Understanding the style-versus-pose split. Knowing how to write the brief. Figuring out the seed image. Those are real, but I had already solved them in the last post. What I had not solved, and what was actually eating my time, was the mechanical mass of doing this fifty times in a row. Copy-pasting the slightly modified style brief into a fresh chat. Maintaining a working version of it in an Obsidian file that constantly drifted from the version I had actually used last time. Screenshotting the front and back. Saving them to my increasingly overloaded chaotic desktop. Uploading them to Meshy. Coming back later to check if the task was done. Downloading the model to an increasingly cluttered downloads directory. Putting it somewhere I would remember (or inevitably not). Repeating, in order, fifty times, with all the small variations and exceptions that creep in across a real collection.

None of that needed AI. None of it needed a chat interface. None of it needed an app. It needed a filesystem with opinions and a CLI with verbs.

The product instinct in this moment is to make everything more AI-shaped. This wanted to become less AI-shaped.

So I built that instead.

Print Bench

It is called pb. Print bench. It is a Python CLI with about ten commands, organised around the structure I had spent half a German army discovering by hand last time. A project has a style.md, which is the thing that stays stable across the whole army. A subjects.yaml, which is the things that vary, one model at a time. And a seed.png, which is the reference image that anchors the visual identity. That separation is now the first thing the tool enforces, because it is the only thing in the workflow that actually matters and the only thing that is easy to lose track of if you do not write it down.

Then there is the loop:

pb list
pb prompt model-name   # assemble the brief, copy to clipboard
pb crop model-name v1  # draw regions, save front and back
pb upload model-name v1   # send to Meshy
pb fetch model-name v1 --wait   # download the model when ready

That is it. pb prompt assembles the prompt from the style guide and the subject brief and copies it to my clipboard. I paste it into ChatGPT, have my conversation, pick a pose, generate the images. pb crop launches a tiny local web server, I drag the image in, draw labelled regions for front and back, hit save, and the cropper writes the original, the region geometry, and the extracted views to disk. pb upload submits them to Meshy. pb fetch waits and downloads the model when it is ready. There is a pb learn too, which lets me append a dated lesson to the style guide whenever I notice something worth remembering, but it is sugar on top of the main loop.

There is no AI in any of it.

I want to be precise about that, because it sounds like a slightly silly thing to say about a tool built specifically for an AI-assisted workflow. The whole point of the tool is to enable AI work. But the tool itself does not call a model, does not embed an LLM, does not have an agent loop, does not have a chat interface. It is string manipulation, file IO, a browser cropper, and some HTTP requests to Meshy’s API. That is it. It is the most boring possible piece of software, and that is exactly why it works.

The Germans took weeks. Thirty-two models, fumbled through one at a time, reverse-engineering the brief as I went. The Soviets took three evenings, fifty-two models, with the style-versus-pose discipline figured out but everything else still manual. And the British? The British took roughly a day. Forty-one models. I spent maybe two or three hours of that day actually paying attention. The rest was the printer running while I did other things.

I want to be honest about where that speedup came from, because the clean version of this story is misleading. The first jump, from weeks to three days, was almost entirely conceptual. It was the discipline of separating the stable thing from the changing thing, the style from the subject, and getting the order of operations right in the prompt. That was the discovery the last post was about, and it would have produced most of that speedup with no tooling at all.

The second jump, from three days to one, is where pb actually earns its keep. The conceptual work was already done. What was left was just the mass of mechanical busywork around the conceptual work, and that turns out to be a much bigger fraction of the total time than I would have guessed before I tried to remove it. The clipboard juggling. The filename hygiene. The “wait, which version of the style guide did I actually use last time.” The Meshy task IDs in a Notes file somewhere. The “how many more models do I actually need?”. All of it small, none of it valuable, all of it adding up.

The tool is also, almost incidentally, the thing I can hand to my friend. I have not actually handed it to him yet. But when I do, it will not be because it does the thinking for him, it cannot, but because it gives him the structure to do the thinking himself. The style guide template forces him to write down the visual rules. The subjects file forces him to enumerate what he is building. The seed image forces him to commit to a reference. The directory layout means his decisions accumulate somewhere durable instead of evaporating inside chat threads. The CLI carries state between the moments where his judgement is actually needed. He still has to make every important call. The tool just gets out of his way for everything else.

This is, I think, the bit that is making me look sideways at a lot of other things.

The reflex when building anything in this moment is to put AI at the centre of it. To make the AI the product. To wrap a chat around it, to add an agent, to integrate a model, to find somewhere to call an LLM. And there is a vast amount of perfectly good work being done in exactly that shape. But sitting in front of pb, which is just plumbing, I keep thinking about how much of the value came from doing the opposite. From looking at a workflow that had AI in it, finding the places where the AI was already doing what it needed to do, and then carefully, deliberately, automating everything between those places without any AI at all.

The decisions are where the value is. Choosing the style. Reading the three pose options and picking one. Looking at the generated image and going “yes, that one.” Looking at the Meshy output and deciding it is worth keeping, or it is not. Those are the moments where the human matters. They are also a small fraction of the total time the workflow used to take, because they were buried inside an enormous amount of cruft that had nothing to do with judgement and everything to do with moving things between windows.

What if a lot of what we are about to build looks like this. Not products that put AI at the centre, but products that take the cruft out from around the AI. Products whose job is to separate the moments where a person needs to think from the moments where a person is just moving bytes around because nobody else will. Products that respect the parts of the workflow where taste lives, and quietly, ruthlessly, automate everything else, including the bits that are not AI at all.

I am still mulling this over. It feels almost too simple to be a grand theory of anything, a lesson pulled from a hobby project that I am trying to stretch over an entire industry. But I keep noticing the same shape elsewhere: the model is not always the missing piece. Sometimes the missing piece is everything around the model.

The most valuable AI products might not be very AI at all. They might just be the thing that lets you get to the AI faster, and then get out of the way while you do the part that matters.

Three days. Evenings only. Not particularly stressed about it.

That is how long it took to go from nothing to a printed 1000 point Soviet army for Bolt Action. Roughly fifty models, around forty of them unique sculpts, fully designed, generated, modelled, sliced, printed, and sitting on my desk. Concept to physical thing in my hand, three evenings, in my spare time, while doing everything else I usually do.

That is … insane. I want to write this down because I think it is genuinely insane and I want to figure out on the page how it happened, because the how is the part I find more interesting than the what.

A quick recap, for anyone arriving cold

A couple of weeks ago I wrote about vibe coding a model into my house. That was the moment when a chain of generative tools produced a 3D printed World War II infantryman on my desk and quietly broke my brain. That post was about the astonishment. This one is about what happened when I stopped being astonished and started actually trying to build something at scale.

The original plan was two armies at 500 points each for Bolt Action v3. Thanks to wildly over achieving and the madness that only comes with AI enhanced momentum … the plan became somewhat more. Germans and Soviets, a thousand points each, scaled down to centimetres so they fit on a kitchen table instead of a garage floor. The toolchain was simple. ChatGPT for concept images, Meshy to turn front and back images into 3D models, Bambu Studio to prepare them for printing, and a Bambu printer to make them real. The Germans came first. The Germans were an education …

The German army taught me everything by being slightly wrong

I built the Germans the way I imagine most people would on first contact with this stack. Find a reference image, drop it into Gemini, ask for a soldier in that style, take the result, drop it into Meshy, get a model, slice it, print it. Repeat for every unit.

The starting reference image was a Fimo clay figure I had found from a random guy on Facebook. Cute, chubby, slightly weird proportions. I liked it. I started by feeding that image to Gemini and going “more of these, but Germans.” Somewhere along the way I started telling it I wanted that mixed with Metal Slug. The chunky comic-game vibe, oversized weapons, exaggerated everything. That was the visual register I was reaching for.

It mostly worked. The models came out. They were even, broadly, recognisable as German infantry. But every one of them was slightly off in a different way. The proportions drifted between models. The base treatment changed. One had a helmet that read as a beret. Another had a rifle thinner than the soldier’s wrist, which would have snapped off the print bed if I had even looked at it sideways. It was a constant fight with the LLMs to get anything consistent.

What I was doing, every single time, was asking the model to invent the style and the pose simultaneously, in the same prompt, with no shared context between sessions. Of course the army drifted. I was running fifty independent experiments and then complaining that they did not match.

So each model I made, I added another constraint to the prompt. Mistake on a model, add a constraint. Mistake on the next model, add a constraint. No thin protrusions. Round integrated base. Chibi proportions. Single solid silhouette. The prompt got longer. The models got slightly more consistent. I was slowly, painfully, by hand, reverse-engineering a brief and not realising that was what I was doing.

Somewhere around the lieutenant or the sniper, the penny dropped.

Separate the style from the pose

The mistake was treating each prompt as “make me a model.” What I actually wanted was to separate two things that I had been asking the model to do at the same time, and to do them in a specific order. Style is the thing that has to be consistent across the whole army. Pose is the thing that needs to vary per unit. Conflating those is how you get visual chaos.

But the order of operations matters even more than that, and this is the bit that took me the longest to figure out.

The Soviet workflow goes like this.

Start a fresh chat. Drop in the prompt. Not an image. Just the prompt. The whole brief. The unit description, all the style constraints, the silhouette rules, the front-and-back-must-match rules, and at the bottom: suggest three variations for poses only.

What that does is get ChatGPT to think. The chat reads through the brief and writes back three pose ideas in words. Gunner prone with the rifle braced, loader pointing toward the target. Or both kneeling behind cover. Or one standing scanning, the other reloading. Whatever the unit calls for.

This step is not about getting poses I want to use. It is about seeding the chat’s context with the right frame of mind. By the time it has written out three pose options, it has already worked through the brief on its own terms. It is now thinking inside the constraints I gave it, instead of about to be asked to obey them.

Now I drop in the seed image. The commissar. And I say: generate those poses, in this style, front and back.

That is the move. The seed image arrives after the thinking has already happened, not before. The chat is not asked to invent a style and a pose at the same time. It does the pose work first, on its own, in words. Then the style gets bolted on as a visual reference to a frame of mind that already exists.

For the Soviets I used the commissar as the seed. The kind of cartoon menace whose whole vibe is “shoot anyone who tries to run away.” That image carried the entire visual identity of the army. Every other model would be made to match it.

Here is the prompt I used, every time, varying only the unit description. I will paste it in full because the prompt itself is the artefact. I spent half a German army learning how to write it.

goal: designing soviet ww2 soldiers keeping in style with the images above from 2 perspectives (front and back)

give me 3 variations of poses

* **AT rifle team (2 models):** gunner prone firing PTRD-41 (exaggerate the absurd length of the rifle — historically over 2m, push it visually), loader kneeling alongside pointing at imagined target (the German halftrack). Mixed gritty dress, SSh-40 helmets, prominent extra ammo pouches/satchels for the big AT rounds.

Render the models in the reference style including:

* "Chibi proportions, large head roughly 1/3 body height, oversized hands and weapon"
* "Chunky oversized weapon, simplified details, no thin protrusions"
* "Integrated round base, feet merged to base"
* "Single solid silhouette, no floating straps or separated gear"
* weapons should be representative and oversized so that they survive 3d printing at small scale (2 cm)
* "Back view must be the exact same pose rotated 180 degrees"
* "Weapon position must match exactly between front and back"
* "No reinterpretation of pose"
* "Single sculpt shown from two angles, not two separate sculpts"
* "Silhouette must align when mirrored"

Suggest 3 variations for poses only 

Notice what is going on here. I am not asking for a model. I am not even asking for an image. I am asking for poses, in words. The style brief is loaded into the chat’s context, but what comes back at this stage is text. Three written pose options. The image generation is the next turn, after I have read those options and decided which one I want, and after I have shown it the seed image to lock the visual style.

I am also, very explicitly, telling it that front and back are the same sculpt seen from two angles. Not two separate models. The same model, mirrored. This matters enormously when those images go into Meshy, because Meshy will happily interpret two different poses as two different sculpts and produce something that looks like the soldier sneezed mid-print.

Then the seed image goes in, the style locks, and the image generation begins. And here the “suggest three variations” framing pays off again. Not one. Three. Because once the chat is preloaded with the right context, generating variations is essentially free, and what you actually want is a buffet of options. Generate three poses. Look at them. Generate three more. Look again. Generate three more. Within ten minutes I had a wall of images for any given unit and I could just go yeah, that one, that one, not that one, that one. Pick the pose that looked most like what was in my head and move on.

‹ ›

1 / 3

Once I picked a pose, I screenshotted the front and back, dropped both into Meshy, and let it generate. With both views provided, Meshy had much less room to invent. It was joining up two views I had already approved, rather than hallucinating the missing half of the model.

After that it was mechanical. Export STL, not 3MF, because 3MF kept giving me non-manifold errors no matter what Meshy claimed about the export. Import to Bambu Studio. Scale to 2 centimetres. Simplify the mesh by 96 percent, which is an insane level of deformation that nonetheless came out at perfectly decent resolution given the size I was printing at. Slice. Print. Move on.

Print plates as units, not as inventory

One small thing that changed between the German and Soviet builds. With the Germans I had treated my STL library as a parts bin. When I wanted to print a unit, I would go pick the right models, drag them onto a plate, configure supports, slice. Every print was a setup task.

With the Soviets I built each plate as the unit it represented. The veteran plate is the veteran unit. The conscripts plate is the conscripts. The AT rifle team is its own plate. When I want to print a unit, I open the file and click print. No setup. No selection. No accidentally forgetting the squad leader.

This is a tiny change and it made a disproportionate difference to how often I actually printed things. Friction matters. I have written this exact lesson down about five times in the context of software and apparently I needed to learn it again with plastic.

The discipline transfers

What I just described, concept, iteration, refinement, codification into a reusable artefact, production pipeline, quality controls, delivery, is not a 3D printing process. It is a software development process. It is the same process I have been writing about for months in the context of agentic coding. Product thinking first. Iterate to find the brief. Codify the brief into something reusable. Treat each output as one of many. Build the production pipeline so the next thing is trivial.

The German army was vibe coding without discipline. Lots of energy, lots of output, no consistency, mounting technical debt with every model. The Soviet army was the same domain, the same tools, the same person, with a process in between. The result is not a small improvement. It is a different category of thing. Adding a new unique sculpt to the Soviet army from this point forward is virtually trivial. It is stupid how simple it is. It should not be this simple.

I have spent a year writing about how the harness matters more than the model, how tests and constraints are really a discipline of conscious decisions about every line, how multi-harness workflows surface regressions you would otherwise eat. I thought I was writing about software. It turns out I was writing about a way of working that translates, more or less unchanged, the moment the output head changes from “code” to “plastic.”

What broke my brain about the first printed soldier was that the loop existed at all. The thing that is breaking my brain about this post is that the discipline transfers. Whatever I figure out about working well with these tools in software is, apparently, immediately applicable to a domain I have no formal training in.

I sat down to write a blog post about printing little Soviet soldiers. I ended up writing one about how surprised I am that what I thought I knew about working with these tools in code worked, unchanged, in plastic.

I do not want to make that bigger than it is. One person, one weekend, one transfer. But it is the kind of small surprise that makes me look sideways at every other thing I think I know about my own discipline. Some of it is presumably about software. Some of it, evidently, is not.

Appendix: things that will save you time if you are doing this

A few sharp edges I hit, written down so you do not have to.

Export STL from Meshy, not 3MF. Meshy’s 3MF export gave me non-manifold errors with depressing reliability, even when the tool insisted there were none. STL behaved.

Simplify aggressively. A Meshy model can come out at over a million vertices. At 2 centimetre print scale, you genuinely do not need that. I was simplifying down by 96 percent in Bambu Studio and the prints still came out crisp.

Custom support settings for tiny prints. The defaults will fuse supports to the model and make removal a nightmare. The settings I landed on for 0.08mm layers:

Increasing the Top Z Distance is the single biggest win. At fine layer heights, a one-layer gap is not enough to stop the support fusing to the print. Doubling or tripling it gives the filament enough room to drop onto the support rather than weld to it. Tree Slim or Tree Organic for style. They use less material and break away cleanly.

Generate front and back from the same prompt. Do not let Meshy interpret the back view. Do not let it interpret anything. Give it both views explicitly. Tell ChatGPT, in the prompt, that the back view is the exact same pose rotated 180 degrees. Belt and braces. You will get what you want far more often.

When I first started using Claude Code last summer, it felt like a chatty junior engineer who couldn’t wait to tell you what it was thinking. It kept you in the loop. It explained itself. It told you what it was about to try, why it thought that might work, and then narrated its way through whether it did. There was charm in it. You felt like you were pairing.

Now Claude Code sits there like a Zen Buddhist monk, slowly mulling the problem in silence, and then suddenly exclaims “TADA!” and hands you a diff. This is not the chatty pair I remember. All the in-depth thinking, the running commentary, the visible reasoning that gave it such charm has quietly disappeared, and what’s left is this churning quiet system that is all work and no play.

So I have been leaning more and more on opencode. It talks. It thinks in a personable way. It structures its thoughts so I can follow along. And more importantly, it doesn’t hide its working. It felt like maths in school: show your work. Opencode shows its work. Claude Code doesn’t. And it has been astonishing to me how much I valued being able to read the thinking along the way.

That is where this post would have ended a few weeks ago. A grumpy aside about how Claude Code lost its voice. Opencode won, etc, etc. Move on.

Except the question wormed its way in. Am I right? I have a strong opinion about which harness you should use, and the opinion is built almost entirely on vibes. On who I enjoy talking to. On whether the chat feels good. None of that tells me anything about which harness actually produces better code. And once that question is in your head it doesn’t leave.

I thought I was building a small harness comparison. In hindsight I was building something stranger: a tiny automated judgement machine for code, of the sort I had been hearing people gesture at without quite understanding what they meant.

A small experiment

So I tried to be a bit Baconian about it. Same prompt, same starting state, run it through every harness and model combination I could get my hands on, multiple times, and grade the output against tests the agent never gets to see.

The setup was a fake Library API. A small but not trivial OpenAPI spec covering books, loans, fines, and members. Cursor pagination. Polymorphic loan responses where active loans look different from returned ones. An async payment polling flow with a terminal state. Structured API errors that should be preserved through the client. The agent’s job was to build a typed TypeScript client. No code generation, no runtime dependencies, just read the spec and write the thing.

The catch was a hidden test suite. The agent could write its own tests, run its own validation, do whatever it wanted to convince itself the implementation worked. But the grading happened against a separate Vitest suite the agent never saw. That suite poked at all the bits I expected harnesses to get wrong: did the cursor pagination actually iterate, did the async payment poll reach a terminal state, did the polymorphic loan response preserve both shapes, did API errors surface usefully or get flattened into “Error: request failed”. The agent could not optimise to it because it could not see it.

Six combinations. Five runs each. Thirty implementations of a Library API client. The rig is on GitHub if you want to poke at it.

I did not go into this neutrally. I had a favourite. I expected opencode-opus to win, claude-code to look quietly competent but a bit lifeless, and the GPT-backed runs to come in mid-pack. I thought I knew the shape of the answer before I started.

While I was setting it up, something else started to nag at me. I had been hearing the term “dark factory” floating around in the agentic coding conversation and not really understanding what people meant. A factory that runs without lights because there are no humans in it. Applied to coding, it suggested some end state where you specify what you want, an agent produces it, and something else judges whether it is correct, all without you in the loop. I had nodded along when people brought it up and quietly had no idea how you would actually build one. But the rig I was assembling was starting to look uncomfortably like a small piece of that picture, and I tried not to think about it too hard while I was still building the thing.

The results

I will spare you the full grid. Each run was scored out of 20 hidden tests, and each harness/model pair ran five times, so the maximum score per row is 100. The headline numbers:

Median diff is measured in lines changed per run.

Claude Code won on correctness. Four perfect runs out of five. The harness I had been quietly resenting for going silent on me produced the most reliable code in the experiment. I sat with that for a bit. The chatty junior I missed had grown up into the engineer who just gets it done and hands you the result, and apparently the result is good.

Opencode-opus came in essentially tied on correctness, one point behind. But look at the median diff. 628 lines vs 1157. Same task, same spec, near identical scores, and opencode-opus did it in a little over half the code. If you measure tests passed per hundred lines of diff, opencode-opus is comfortably the best of the lot at around three. Claude Code is a touch under two. Pi-opus is at one and change. It is a crude metric, obviously. Fewer lines are not inherently better, and there are plenty of ways to cheat at it. But when two runs are almost tied on correctness and one gets there with half the diff, I pay attention. Claude Code is most reliable. Opencode-opus is most efficient. I am genuinely not sure which I value more in a real engineering context.

The other observation that I keep turning over is the pi.dev runs. Largest diffs in the experiment, mid-pack on correctness. More generated code did not buy reliability. It is tempting to read a wall of plausible-looking output as evidence of diligence. The data here says that intuition is wrong. Verbose was not safer. Verbose was just verbose.

A caveat there, though, and an important one. The pi.dev runs were stock pi out of the box. No customisation, no extra tools, no extension of its capabilities. And the whole point of pi is that you customise it. That is the entire pitch. So what I actually measured was vanilla pi, with a deliberately limited toolkit, on a task that probably wanted a richer one. Seen that way, the fact that pi-gpt landed at 92 with no help from me is genuinely interesting. There is more to do here, and a properly outfitted pi might tell a different story. That is its own post.

The same-model-different-harness comparison is where it gets weirder. Codex and opencode-gpt are both GPT-5.5 doing the same task, and they tied on aggregate score, but opencode-gpt did it in noticeably less code. Same brain, different harness, different shape of output. Then flip it: opencode-gpt vs opencode-opus is the same harness with different models, and the model swap moved the score from 85 to 97. So harness matters. Model matters. They are not interchangeable variables, and which one matters more depends on which one you change.

What this rig had become

Once the runs were done and I was staring at the spreadsheet, the thing I had been trying not to think about earlier became impossible to ignore.

I had a detailed work order in prompt.md and AGENTS.md and a formal OpenAPI spec. I had a clean starting workspace that got reset between runs. I had a constrained implementation environment with locked Node and TypeScript versions. I had a hidden acceptance test suite the agent could not see or game. I had a mock service that behaved enough like a real one to be tested against. I had a runner that could launch any of the harnesses under comparable conditions, and a grader that built the output, ran the hidden tests, captured the diff size, and recorded the results into a table.

That is most of a dark factory. Spec in, code out, automated judgement in the middle, results captured for analysis, no human required during a run. The piece that is missing is the feedback loop. Right now, when a run scores 15 out of 20, that is the end of the story. It gets logged and we move on. A dark factory v0.1 would read those failures, decide whether to retry, mutate the prompt or the constraints, launch another attempt, and keep going until the score crossed some threshold or the budget ran out.

I do not have that yet. But I can see how to build it from here, which is something I genuinely could not see a month ago. I had been hearing the term and nodding politely. Now I had accidentally built most of one because I was trying to settle a vibes-based argument with myself about which harness I liked best.

And the thing that makes me uncomfortable about that is what it implies about “show your work”. I had been treating the visible reasoning as a proxy for trust. Watching opencode talk through the problem made me feel like it knew what it was doing. Watching Claude Code go quiet made me feel like it didn’t. The hidden tests did not care about either of those feelings. They cared about whether the cursor pagination iterated, whether the polymorphic loan response preserved both shapes, whether the async payment poll reached a terminal state. Visible reasoning helped me supervise. Hidden tests measured whether the work was actually done. Those turn out not to be the same thing, and inside a dark factory only one of them survives, because there is no human there to be reassured by the other.

What I am left with

I expected this experiment to vindicate opencode. It did not. If I cared only about pass rate I would use Claude Code. If I cared about correctness density I would use opencode-opus. If I cared about reading the agent’s reasoning while it works, I would still use opencode, and I will. Aesthetics matter. I spend hours in this tool. I want to enjoy being there.

But I am going to stop pretending that preference is a quality argument. It is a supervision argument, which is a different thing, and one that gets thinner the further you move toward letting the rig run on its own.

The other thing I am left with is the rig itself. I started building it to answer a small question and ended up with a thing that has the shape of something much bigger. That is the part I did not see coming. The benchmark was supposed to be the point. It turns out the benchmark might just be the prototype.

This is a thing you should not do. I want to say that up front, before any of the rest of it, because the rest of it is a reasonably well-considered guide to doing the thing, and I do not want anyone reading the guide bit and thinking that I am endorsing what comes after it.

The phone is already a distraction device of doom. I have spent a non-trivial amount of effort over the last couple of years trying to make mine less of one, with limited success. What I am about to describe takes that distraction device of doom and turns it into a distraction device of doom that can also write and ship code. This is, on every reasonable axis I can think of, a worse situation than the one I started in.

But.

I was on a train recently. One of those medium-length train journeys that everyone in the UK eventually finds themselves on. About an hour and a half. Two or three changes. Never quite enough uninterrupted sit-down time for pulling out a laptop to make sense. You can read a book. You can stare blankly at your phone. What you cannot do is meaningfully open an IDE and ship a feature. And yet there I was, with a project I wanted to be working on, several connections to make, and a phone in my pocket. So I built the thing. Here is how.

The bits you need

You need Tailscale. You need mosh. You need Termius. You need remobi. You need tmux. And if you are doing this from a Mac that lives plugged in with its lid open, you need to know that caffeinate -is exists. You also need enough self-control not to use any of this irresponsibly, which is where the plan begins to fall apart.

Tailscale

Tailscale is a way of pretending that two devices on entirely different networks are actually on the same one. You install it on your laptop, you install it on your phone, and from that point on the two of them have stable IPs that work no matter which café WiFi or train 4G you happen to be straddling at the time. This is the foundation. Without this, none of the rest works, because your phone has no idea where your laptop is and your laptop has no interest in being found by random strangers on the internet, both of which are correct positions for them to hold.

mosh

SSH is a beautiful protocol that falls apart the moment your connection blinks. Trains go through tunnels. 4G drops to 3G drops to nothing and back again. SSH does not enjoy this. Mosh does. Mosh is a shell client and server that survives the kind of network conditions you get when you are physically moving through countryside at speed, which is exactly the situation we are designing for here.

tmux

You want a terminal session that just keeps running on your machine whether you are connected to it or not. tmux is the obvious answer. Start a session, leave it there, reattach to it later from whatever client you happen to be using at the time. This is the bit that makes the whole thing feel less like a fragile remote connection and more like walking back to a desk you left an hour ago.

Termius

Termius is a genuinely lovely SSH client. It runs on your phone, it knows about your Tailscale IPs, and it gives you a terminal that you can just type into. You hit your laptop over mosh, attach to your tmux session, and away you go. If your agent of choice is Claude Code or Codex CLI or OpenCode or whatever flavour of the week you are running, this is enough to be productive. You point it at the thing, you tell it to go, and you watch it work.

remobi

remobi is the bit that turned this from “technically possible” into “actually quite nice.” It was written by Connor Adams, who is one of those quietly talented engineers who keeps producing useful things while the rest of us are still talking about producing things. What it does is run a little server on your machine that exposes your tmux session over HTTP, which means you can wrap it up as a desktop app on your phone and just tap to be back where you were.

The reason this matters is that the UI is better than typing into a phone-shaped SSH client. You get native scrolling. You get sensible zoom. You get a layout that does not require you to remember which gesture corresponds to which control sequence. It feels less like fighting your phone and more like using it.

caffeinate

If your laptop is the kind of laptop that lives plugged in with the lid open, you have two problems. First, you need to enable Remote Login, which is the kind of setting you turn on once and then forget exists until you need it again. Second, you need caffeinate -is, which is a macOS command I did not know about until very recently and which apparently ships with the operating system. It tells the system to stop being clever about going to sleep when the display turns off. Run it, leave it running, and your machine stays awake long enough to actually be useful as a remote target.

So now what

Now you have the ability to write code from your phone, anywhere, regardless of whether you are sitting at a desk or wedged into a train seat with your bag on your lap. You can kick off an agent, watch it work, course-correct it, and have something meaningfully shipped by the time you arrive at wherever you were going. On the train I was on, this would have neatly solved the problem of being bored and wanting to work on something I could not work on.

I should mention, in fairness, that you could probably do most of this with Claude Code’s mobile app and save yourself the entire setup. That is true. The reason I did not is that I am stubbornly committed to not being locked into any one AI toolset, which means I want the option to point this same setup at Codex and OpenCode and Claude Code and whatever else I happen to be running that week. So I have reinvented a wheel that already exists, in order to have a wheel that I own.

The bookend

Here is the thing though. I am not actually sure I have done myself any favours.

The phone, as previously established, is the world’s ultimate distraction device. I am, very slowly and with mixed results, trying to make mine less of one. And what I have done here is take that device, the one that is already eating more of my attention than I would like, and given it a brand new way to consume me. Now it is not just a thing I pick up to check the time and put down forty minutes later wondering where the time went. Now it is also a thing I can pick up to “just quickly check on the agent” and put down forty minutes later wondering where the time went, except this time I have shipped half a feature I had not actually decided I wanted to ship yet.

I was so busy thinking about whether I could that I did not stop to ask whether I should. Friends, I should not have. Your mileage may vary.

There is a 3D printed World War II German infantryman sitting on my desk. He is about the size of my thumb, slightly chibi in the proportions, with a helmet a touch too large for his head. He looks, frankly, adorable. He is also not a copy of anything. Nobody designed him. Nobody sculpted him. Nobody even sketched him. I typed some words at a screen, pressed a button on a different screen, and twenty minutes later he was sitting on my desk. On the left pure vibes, on the right reality.

I have been writing this blog for a while now about adventures in applying AI to dev related problems. Writing code. Building software. Making things. It has taken me through a Magic jumpstart cube generated with code, a board game prototype, a re-implementation of that prototype in Godot, a stealth tactics turn based game also in Godot, a news digest SaaS that is still somewhere in the middle of becoming itself, and a frankly embarrassing pile of bash scripts accumulated from deep dives into my coding harnesses. Along the way I have taken what I have learned and applied it at work. It has been a wonderful, bizarre road.

This post is about atoms instead of bits, which is a first for the blog. I promise it rhymes.

The printer

Somebody gave me a 3D printer. An A1 Mini, which is one of the cheaper ones but turns out to be remarkably capable for what it is. And as one does when one acquires a 3D printer, I immediately spent a week not printing anything interesting. I built shelves for my office to create space. I printed spool holders for the spools. I printed the tools you need to use the 3D printer. This seems to be the compulsory onboarding ritual when you get a 3D printer, which is a bit like spending your first week with a new laptop installing a package manager so you can install the package managers that let you install things. Fine. Tradition.

Then I set myself an actual goal.

The weird hobby compulsion

I should explain that I have a tendency to go on strange game-making tangents. I have built a travel sized version of a board game from scratch with custom cards and components, purely because it was fun to tinker with. I have made print and play versions of expansions for games I already own. I made my own version of Santorini using tiles and spray paint, like some sort of deranged hobbyist. I paint, I draw, I mess about. The point is that “I wonder if I could just make this” is my default failure mode when I encounter any game that costs too much or takes up too much space.

Bolt Action has been living rent free in the back of my head for two or three years now. It is a tabletop wargame. I would like to play it. But I do not want to spend the money on the models, I do not want to find the table space, I do not know enough people in my area who want to play it with me, and painting a hundred models is a lot of effort in a way that is genuinely hard to appreciate until you have sat there and done it.

The idea that has been sitting in the back of my brain for a few years is this: make a scale down version. Instead of moving in inches, move in centimetres. Same rules, same ratios, everything else the same, just smaller. Smaller table. Smaller models. Smaller paint commitment.

Which means you need smaller models. Which has always been the problem.

The accidental pipeline

I saw a blog post a while back by a guy making models out of Fimo clay and EVA foam and little bits of wood. They were cute and chibi and slightly weird and I really liked them. So naturally I tried to make one myself. What I ended up with functionally looked okay and matched the vibe but was basically 28mm scale, which is normal Bolt Action size, which defeats the entire point.

On a whim, I took the photo the original guy had taken of his model and fed it into Gemini Pro. “This style. World War II Germans.”

Gemini came back with something that looked astonishingly good. Cute, chibi, right proportions, right register. Something that immediately felt like what I had been trying to describe for years without quite having the vocabulary for it. I then started giving it more structured prompts. Give me a commanding officer. Give me an NCO. Give me a machine gunner.

I took one of these entirely hallucinated images, cropped it, and fed it into Meshy, which is a generative tool that takes an image and produces a 3D model.

I then, mostly out of morbid curiosity, copied the file over to the printer and clicked print.

This confounds me

Twenty minutes later, there was a physical object on my desk. A German infantryman. About the size of my thumb. Cute and chibi, helmet slightly too large. Precisely the thing I had been describing to Gemini about half an hour earlier.

I want to be clear about what happened here because I think I am still processing it.

I described a thing in words. Another thing dreamed up a picture of that thing, a picture that had never previously existed. A third thing hallucinated a 3D shape from that picture, a shape that had also never previously existed. A fourth thing turned that shape into an object I can hold in my hand. No human sculpted it. No human modelled it. No human even sketched it. The infantryman on my desk has no reference in the world. He is not a copy of anything. He is purely the output of a pipeline of vibes.

I vibe coded a model into my house.

I have spent a year now playing around with generative AI. I have been, at times, out on what I thought was the frontier of what it is doing. I should have seen this coming. At some level I had seen this coming, in the abstract, “yes of course generative AI plus 3D printing, that is obviously a thing” way. But there is a chasm between knowing a thing is possible and holding the output of that thing in your hand twenty minutes after describing it out loud.

This is the same loop I have been running on software for a year. Describe a thing, get a thing, iterate, ship. The loop works on atoms now. It has probably been working on atoms for a while and I simply had not wired up the last step of the pipeline in my own life until somebody gave me a printer.

Which makes me wonder what else is already sitting there, loop closed, waiting for me to notice.

What I’m doing with it

I am now in the middle of printing a 500 point German army and a 500 point Soviet army. The whole thing will probably fit in a box slightly bigger than a paperback. A deck of cards each for the rules and unit references. Total cost of the models, given that I already had the printer and the filament: functionally nothing. If a friend wanted to play, I could just print them a second army and not be fussed about it. There is some manual work around trimming supports and cleaning up sprues, but it is not the kind of work that scales with ambition. It scales with how many figures you feel like cleaning up on a given evening.

Something I have been idly wanting for two or three years is just there now, in a box, because the pipeline finally closed.

What I can’t get out of my mind

Here is the thing that has been rattling around my head since the infantryman showed up.

We are, collectively, still arguing about whether vibe coded software counts as real engineering. That argument is live. It is on my timeline every week. It is in the comments of every post I write. People who build things for a living are genuinely unsure whether the loop of “describe a thing, get a thing, ship it” is a legitimate way to make software, and reasonable people disagree about that, and the discourse is maybe a year behind the tools and possibly more.

While we have been having that argument, the same loop has quietly grown another output head. It makes physical objects now. Not in some research lab, not in some well funded startup I would need to buy into. In my house, on a desk, using tools that anyone can download or buy, for a material cost measured in pennies per figure.

I found this out by accident. Someone gave me a printer. I had an itch I had been scratching at for years, and the pipeline closed on its own while I was not really paying attention. That is what is unsettling me. Not that it works. I knew it would work. It is that I walked into this corner of it entirely by accident, with no plan, and the corner was just sitting there waiting for anyone who happened to wander in.

I do not think we are ready for the software version of this. I am much less sure about anything else. Because if a ten minute detour into a completely different hobby is enough to produce an object with no human author, what else is already sitting there that I have not stumbled into? What other loops have quietly closed while I was looking at my terminal? I thought I had been playing on the frontier for a year. It turns out I have been playing in one room of a house whose floor plan I do not have.

I do not have a tidy ending for this. I have an infantryman on my desk, and a suspicion that I have been looking at a very small corner of something, and that almost everyone I know has been looking at the same small corner, and that the rest of the house is already built.

When I joined Cazoo, it was the first place I’d ever worked that explicitly, actively, aggressively embraced software craftsmanship. Pair programming. Test-driven development. Domain-driven design. Extreme programming. The whole kitchen sink. They sent us on agile training courses that a startup founder would weep at the cost of. We had an agile coach in the room every day. We did code katas regularly.

And even there, in the most craft-soaked environment I’d ever been in, the idea of 100% code coverage was treated as obvious lunacy. A poor metric. The kind of thing only someone who hadn’t really understood testing would chase.

Then I joined the Economist, and the team I landed on had 100% coverage as a hard rule.

While they did write tests, they didn’t do TDD. They didn’t pair. They hadn’t been on the agile bootcamps. They hadn’t done code retreats or code katas. By every measure of either the London or Chicago school of craftsmanship tradition would care about, they were doing less of the work. But they had the 100% rule, and they enforced it, and at first I assumed they’d inherited a metric without fully understanding it.

They hadn’t. Turns out I hadn’t understood it. And by the time I left that team, I’d come around entirely. Not reluctantly, not with caveats, but genuinely: 100% coverage, properly understood, is mandatory. I held that position for years before agentic coding was a thing anyone was thinking about. The agents haven’t changed my mind. They’ve just taken a position I already held and made the case for it screamingly, urgently obvious in a way it previously wasn’t.

What is this metric thing about anyway?

Here’s what I’d absorbed from the craft world about 100% coverage. It’s a vanity number. Chasing it produces garbage tests. You end up writing assertions against getters and setters. You exercise code without testing behaviour. The pragmatic position, and pragmatism was always the emphasis, is that you write the tests that matter and you let the rest go.

All of that is true if “100% coverage” means “every line has a test exercising it.” That version of the metric is genuinely silly and the people warning against it were right.

But it took me until very recently to notice was that nobody, in all those arguments, had ever actually explained what the metric was for. What it was pointing at. Everyone, including me, was arguing about the number. Nobody was asking what the number was a proxy for.

It’s a proxy for Conscious Coverage. That’s the thing. Every line in the codebase is a decision. The question the metric is actually asking, underneath, is: have you made a conscious decision about each one. Not have you tested each one. Have you decided about each one. Tested, or consciously chosen not to test, with a reason, written down.

Concretely, it looks like this. You write a function with a branch that handles a malformed input. You run the coverage tool. It tells you the error branch isn’t covered. You now have three choices, and only three.

1. You can write a test that exercises the malformed input and asserts the behaviour.
2. You can mark the branch ignored with a comment that says, say, “unreachable because upstream validation guarantees this shape” — and now your justification is a reviewable artefact that someone can argue with in a pull request.
3. Or you can decide the branch shouldn’t exist at all and delete it. What you cannot do is shrug and move on. The forgotten case is no longer a thing. Every line has had a decision made about it, and the decisions are legible.

  ...
  static countryToRegion(countryCode: string): Region {
    /* v8 ignore start */ // Ignoring the switch to avoid repeating every single country code
    switch (countryCode) {
  ...

Once you see that, the version of the rule that the craft world rejected and the version the Economist team was running are obviously different things. The first one optimises for a number. The second one optimises for the absence of accidents. You can no longer fail to test something because you forgot. You can fail to test it because you decided not to, and you wrote down why, and someone can argue with you about it later in the review. The shape of the work is different.

And this is the bit I have to be honest about, because the post doesn’t work without it. Once the metric is framed as conscious coverage, the pragmatic position I’d absorbed at Cazoo stops being pragmatic. It’s just laziness with a vocabulary. “Write the tests that matter and let the rest go” sounds wise until you ask which lines, specifically, didn’t matter, and why, and the answer turns out to be that I didn’t want to write those tests and the tradition had given me a way to sound rigorous about not writing them. The metric wasn’t too expensive. The work it pointed to wasn’t too expensive. I just didn’t want to do it, and nobody was making me, and the craft vocabulary let me call that a considered trade-off.

I had to be in a place that just did it before I could see any of this. Sitting at Cazoo arguing about it from first principles, I would have lost the argument every time, because the version of the rule I was arguing against was the version everyone agrees is bad, and the version underneath it, the one about conscious, nobody had ever put into words for me. Nobody tells you the better version exists until you’re standing inside a codebase that runs on it.

What changes when an agent is doing the writing

Fast forward. I’m now writing a lot of code with agents. Claude Code, Codex, OpenCode, the usual suspects. The thing I keep telling people who ask me about it is that agentic engineering requires more discipline than normal engineering, not less. The tools are faster, the output is bigger, and the gaps between what you asked for and what you got are easier to miss. So everything that used to depend on careful human attention now depends on something else holding the line. Which brings me back to the question: how do I know it’s done? And more importantly, how does an agent know?

Not “done” in the user-acceptance sense. Done in the much more boring sense of: has this thing actually exercised the code it claims to have written? Has it tested the behaviour I care about? Did it quietly skip a branch because the test was annoying to set up? Did it write something that’s technically passing but structurally untestable?

These are the questions the craftsmanship tradition spent twenty years building intuitions about, and the answer the tradition arrived at, pragmatically, contextually, with appropriate caveats, was mostly “you’ll know it when you see it, and pairing helps, and code review helps, and time helps.” Which is fine when humans are doing the work at human pace. It is not fine when an agent has just produced four hundred lines in ninety seconds and is asking what to do next.

The agent needs a guard rail. Something machine-checkable. Something it can run, get a number from, and decide for itself whether to keep going. Something another agent can validate.

100% coverage, in the conscious sense, turns out to be exactly that. The agent finishes its loop, runs the coverage tool, sees 98%, and knows, without me telling it, that there are two percent of decisions it hasn’t made yet. Either write the test, or mark the lines as ignored with a justification. Both are fine. What’s not fine is leaving the gap.

And here is where the impact of the reframe gets outsized, because the agent doesn’t have my laziness. The agent doesn’t want to go home. The agent isn’t quietly negotiating with itself about which lines it can get away with skipping. The thing that was always standing between me and conscious coverage, which was me, just isn’t there. The metric stops being a rod I have to hold myself to and becomes a rod the agent holds itself to, cheerfully, at four in the morning, forever. The practice the craft tradition argued about most fiercely for human reasons becomes, for agents, the most natural thing in the world.

I’ve started using this as one of my standard acceptance criteria. “You are done when coverage reports 100%.” I can kick off a thirty-minute task and come back to something that, whatever else is true of it, will at least be testable, and will at least have had every line consciously decided about.

Coverage as the gate at the end works better when there’s a process upstream that’s likely to produce decent tests in the first place. If you set up the harness with CLAUDE.md files that push the agent toward red-green-refactor TDD, and you give it the kind of structured prompting (like obra/superpowers) that shapes how it actually approaches a task, you tilt the odds. There’s no guarantee it’ll write tests first. There’s a much better chance it will, and a much better chance the tests it writes are pulling the design rather than chasing it. That upstream tilt plus the downstream gate is a much sturdier system than either piece on its own.

There’s a sharpening of all this that matters, though, because coverage on its own can still produce tests that exercise code without actually testing anything. The companion practice, and I’d say it’s a necessary one rather than a complementary one, is writing tests outside-in, from behaviour rather than from structure. Test the unit of behaviour, not the unit of code. Don’t mock the internals; let the real thing run and assert against what the user of the code actually cares about. This was already the right answer when humans were writing the tests, because it produces tests that survive refactors and read like documentation. With agents it becomes critical, because a behaviour-shaped test is one the agent can write legibly from a user story, and one that you, as the reviewer, can read and check against intent without having to trace the implementation. Coverage tells you the agent made a decision about every line. Behavioural framing tells you the decisions were about the right things. You need both. Coverage without behavioural framing is theatre; behavioural framing without coverage leaves gaps you’ll find in production.

Now for the obvious objection. Agents are world-class metric gamers. They will absolutely write meaningless tests that exercise code without asserting anything useful. They will absolutely mark lines as ignored with justifications like “this branch is unreachable” when the branch is, in fact, reachable. If you treat 100% coverage as a number to satisfy, the agent will satisfy the number and you’ll be worse off than before, because now you have a green build hiding a problem instead of a red one announcing it.

The reason I think it works anyway is that it’s asking the right question of the metric. Coverage, in the conscious sense, is a completeness check. It tells you every line has had a decision made about it. It was never going to tell you the decisions were good ones. That’s a different question, and it wants a different answer. Behavioural tests, written outside-in from what the user of the code actually cares about, are the correctness check. Mutation testing, which flips operators and boundaries and asks whether any test notices, is the check on whether the assertions are doing real work. The gaming the agent does lives in the gap between those checks, and the mitigation isn’t to make coverage smarter. It’s to stop asking coverage to do correctness’s job. Use it for what it is: a completeness gate that makes the decisions visible. Use behavioural framing and mutation testing for the quality of the decisions. The ignored lines and their justifications are, at least, a reviewable artefact, sitting in one place where you can read them. The cheats are confined to a place you’re looking. None of that is automatic. It’s a discipline, and like every guard rail it collapses the moment you stop maintaining it. The question is whether the rail makes problems easier or harder to spot, and I think this one makes them easier.

The truisms didn’t go away

The craft tradition produced a lot of practices, and a lot of arguments about practices, and a lot of nuance about when practices apply. Most of that nuance was about humans. About the cost of the practice to the person doing it, about whether the discipline was worth the friction, about whether the metric would be gamed. A lot of it, and I say this now having lived on both sides of the argument, was about whether the person doing the work would actually do it if you asked them to.

Agents don’t have that problem. The friction of writing the extra test isn’t a friction the agent feels. The discipline of marking ignored lines with reasons isn’t a discipline the agent has to be talked into. The kind of metric-gaming that comes from a tired human at five-to-six is replaced by a different kind of gaming, which is its own problem. So practices that were borderline-worth-it for humans become straightforwardly worth it for agents, and practices that were rejected as lunacy for humans turn out, on inspection, to have been rejected for reasons that said more about the humans than about the practice.

The craft was always about building software in a sustainable, predictable, maintainable way. That hasn’t changed. The agents don’t replace the craft. They inherit it. And some of the practices the tradition argued about most fiercely turn out, in this new context, to be exactly the load-bearing ones. Not because the old arguments were wrong about the metric, but because the old arguments were quietly also about us, and the us part has changed.

100% coverage wasn’t wrong. It was a proxy for something nobody I knew named. That allowed me to point at work I didn’t want to do, and dressed up in a vocabulary that let me agree with myself about not doing it. The agents don’t have the vocabulary and don’t need it. Which makes me wonder which other practices were rejected for reasons that were really about us, and what the calculation looks like now that we have a collaborator who just, straightforwardly, does the work. I’ve run that calculation for coverage. I’m increasingly sure it isn’t the only practice the answer flips for. I’d quite like to know which others.

A tale of two Ralph loops

A couple of weeks ago I was playing with Newshound, a personal project of mine that pulls together a digest of interesting things from a list of about thirty sources on the internet. I wanted to add a feature that was a little more involved than the usual yak shave. Spec conversation. PRD skill. JSON. Ralph loop. The full ceremony.

I ran the loop in Claude Code. It went for two hours. A good chunk of that two hours was Claude Code recursively chewing on the same problem, half-finishing things in slightly different ways each time around. Eventually it limped over the finish line. At which point my Pro subscription tapped out.

I went off and set up the wrapper script from the last post to allow me to run a Ralph loop on OpenCode. I then ran the exact same prompt through OpenCode with GPT-5.4. Same Ralph loop. Same PRD. Same instantiation of the problem.

Fifteen minutes.

I noticed this. Of course I noticed this. And the conclusion I reached, the one anyone would reach, was: huh, GPT-5.4 must just be better at this particular kind of task. I filed it under “interesting data point about model personalities” and moved on. I’d written about how each harness has its own character in the council post, and this felt like more of the same. Different tool, different shape, sometimes one fits the keyhole better than the other. Cool.

That was the wrong conclusion. I just didn’t know it yet.

What Newshound put on my desk

Two days ago Newshound surfaced a GitHub issue on the Claude Code repo. There is a particular pleasure in your own tool catching the thing that’s about to reframe how you think about your other tools, and I want to note it before I move on, because the whole point of personal projects is moments like this.

The issue was filed by Stella Laurenzo, an engineer working deep in the AMD GPU compiler stack on IREE. Not a casual user. Not someone shouting into the void about vibes. Someone whose day job is to run dozens of concurrent Claude Code agents against a non-trivial systems codebase, who logs everything, and who knows how to do statistics to data.

The headline finding is brutal. From late January through early March, she analysed 17,871 thinking blocks and 234,760 tool calls across 6,852 Claude Code session files. What she found is that somewhere between mid-February and early March, Claude Code’s behaviour changed in measurable, reproducible, machine-readable ways.

The number that broke me is the Read:Edit ratio. In the good period, Claude Code was reading 6.6 files for every file it edited. By mid-March, that ratio had collapsed to 2.0. The model stopped reading code before changing it. One in three edits in the degraded period was made to a file the model hadn’t read in its recent tool history.

There’s more. A “stop hook” she built to programmatically catch Claude trying to dodge work, ask unnecessary permission, or declare premature completion fired 173 times in seventeen days. It had fired zero times before March 8th. Zero. Every phrase in that hook was added in response to a specific incident where Claude tried to stop working and had to be forced to continue. The word “simplest” in Claude’s outputs went up by 642 percent. The word “please” in her prompts dropped 49 percent. The word “thanks” dropped 55 percent. She stopped being polite to it because there was nothing left to be polite about.

The methodology is more rigorous than anything I would ever bother to do, the dataset is enormous, and the appendix where Claude Opus analyses its own session logs and writes “I cannot tell from the inside whether I am thinking deeply or not” is one of the more haunting things I’ve read in a technical bug report.

Go and read it. I’m not going to recap the whole thing. The point that matters for this post is much smaller and much more personal.

The thing I’d been blaming on myself

I have been using Claude Code since June last year. In that time it has been, without much competition, the most enjoyable engineering tool I’ve ever used. The blog you’re reading exists in part because of how much I have wanted to write about working with it.

But over the last few weeks something had been off. Sessions felt slower. The chatter I was used to, the running commentary where Claude Code would talk through its plan as it worked, had gone quieter. The two-hour Ralph loop on Newshound was the loudest version of it but it wasn’t the only one. I’d had a couple of sessions where it felt like Claude was rushing to a conclusion, where the reflection phase produced shallower answers than I was used to, where I was correcting more and praising less.

I had put all of this down to me. I’d been burnt out and needing a holiday. I was probably tired. I was probably prompting badly. The problem was probably harder than I’d estimated. The Ralph loop was probably a poor fit for the task. GPT-5.4 was probably just better at this particular slice of work.

None of those things are unreasonable explanations. They’re the kinds of explanations a senior engineer reaches for first, because the alternative, “the tool I rely on every day got measurably worse without telling me,” feels paranoid and slightly embarrassing. So you eat it. You assume the variable that changed is you.

And then someone with 6,852 session logs and a Pearson correlation coefficient publishes the receipts, and you sit there reading them on a Sunday afternoon thinking: oh. Oh, that’s what that was.

The argument the council post wasn’t making yet

When I wrote about convening multiple AI harnesses as an architectural review council, the pitch was about getting better answers. Different harnesses have different personalities, the harness matters more than the model, three opinions plus a synthesis beats one opinion. All of that I still believe. But there was a second argument hiding in there that I didn’t see at the time, and Stella’s report is what dragged it into the light.

Multi-harness working is regression detection.

It is, for most of us, the only regression detection we are ever going to have. I am not going to instrument my Claude Code sessions, capture 234,760 tool calls, and run a signature-length correlation against thinking depth. I have a day job and a stealth tactics game to build. Stella did that work and the rest of us are in her debt for it, but it is not a repeatable practice for anyone whose job title isn’t “compiler engineer with infinite patience and a logging fetish.”

What is repeatable is keeping three harnesses in active rotation and noticing when one of them starts feeling off relative to the other. The fifteen-minutes-versus-two-hours moment with Newshound was a regression signal. I just didn’t read it as one because I had no framework for the idea that the harness itself might be the variable. I assumed harnesses were stable. They are not stable. They are moving targets, reconfigured continuously by people who do not write to you about what they changed, and the only way you find out is by holding two of them up to the same problem and watching one of them flinch.

This is what the plugin trap was protecting against without me fully understanding why. Yesterday’s post was about keeping the exits visible so you don’t get locked into a single ecosystem. The thing I didn’t say, because I didn’t know it yet, is that the room you’re standing in is being remodelled while you sleep. Exits aren’t just for when you want to leave. Exits are how you find out the room has changed shape.

If your entire workflow lives inside one harness, harness drift is invisible to you. It just feels like you’re having a bad week. You blame yourself. You prompt harder. You write longer CLAUDE.md files. You assume the problem is on your side of the screen, because from inside one harness there is no other side of the screen to compare against.

Naming names, because this is supposed to be honest

I am going to name Claude Code directly here, because this blog only works if I’m being truthful about what I’m actually using.

The tool that got measurably worse over the last month is Claude Code. The tool I have loved more than any other engineering tool in the last decade is Claude Code. Those two sentences belong in the same paragraph. I am writing this because of how much I like the thing, not in spite of it.

If you have been feeling like Claude Code is harder to work with than it was in February, you are probably not imagining it, and you are probably not getting worse at your job. There is data. The data is good. Go and read it.

What I’m taking away

Three things, and then a rabbit hole.

First, I want crude metrics on my own harness usage. Not 234,760-tool-call-Pearson-correlation crude. Just crude. How many tool calls per session. How many file reads versus file edits. How many times I had to interrupt and correct. Even a daily tally of “did Claude Code feel like it was trying today” would be more signal than I currently collect, which is zero. If the regression signal is detectable in aggregate, I want to be looking at the aggregate.

Second, I want a smoke-test prompt suite. A handful of canonical prompts that exercise the kinds of work I actually do, that I can run across harnesses on a rough cadence and use as a tripwire for drift. Nothing fancy. A small fixed battery, run weekly, results scribbled in a notebook. The point is not the rigour, the point is the comparison over time. I have been operating without a baseline and it has cost me.

Third, the portability argument from the plugin trap post upgrades from “useful insurance against rate limits and lock-in” to “the only way you will ever notice that your tools have silently changed underneath you.” Multi-harness working is the canary. If your canary is the same species as the thing you’re trying to detect, you don’t have a canary. You have another bird in the same mine.

And then the rabbit hole.

The next room over

There is a project called pi by a developer named Mario Zechner. The tagline on the front page is “There are many coding agents, but this one is mine,” which is doing a lot of work in eight words. Pi is a minimal, aggressively extensible terminal coding harness. The pitch is that you adapt pi to your workflow rather than the other way around. No sub-agents, no plan mode, no built-in todos, no MCP, no permission popups, no background bash. All of those things are extensions you add, or build, or install from someone else’s package. The core stays small and the shape comes from you.

There is a YouTube video by Mario walking through how he came to build it that I have not yet found the time to fully watch, and this post is partly me giving myself permission to find that time.

The reason pi feels like the natural next thing is that it is the logical endpoint of an argument I’ve been making in pieces across the last few posts. The plugin trap post said your workflow shouldn’t live inside one harness. The council post said different harnesses give you different answers. This post is saying different harnesses give you the only honest baseline you have for spotting drift in any one of them. The next move, the move I cannot stop thinking about, is: what if the harness itself is something you own? What if instead of being a tenant in three different rooms, all of them being remodelled by other people on different schedules, you build a small room of your own, with the doors where you want them, and treat the rented rooms as the comparison set?

I do not know yet whether pi is the right answer to that question. I have not run it. I have not watched the video. I have a game and a new digest agent I am supposed to be working on, and the smell of yak around me is already pretty thick.

But I can feel the next dive coming. And after the week I’ve just had, I am done pretending that holding still inside a single harness is the safe choice. The safe choice is having somewhere else to look from.

Off I go.

It’s day two of my holiday and I’m staring at a Claude Code session that won’t do anything. Pro limit hit. Three days until it resets. There’s a personal project sitting open in another window that I’d been quite enjoying poking at, and now I can’t poke at it, and the bit of my brain that had been having a perfectly nice time is suddenly very loud about the £20 of extra credit I’d burned through in a single afternoon earlier in the week.

This is the story of how that lockout forced me to do a small piece of unglamorous setup work I’d been avoiding for months, and what I found on the other side of it.

The workflow

Quick context. Over the last nine or ten months I’ve fallen into a working rhythm with my personal projects that goes something like this. I open an AI chat, and I have a long conversation with it. Not a “write me some code” conversation. A “let’s interview each other about what I’m actually trying to build and why” conversation. These run for three or four hours sometimes. Lots of back and forth, lots of poking at scope, lots of trying to find the smallest version of the thing that would actually tell me whether the idea is any good. At the end of all that I have what I’ve been calling a spec: a high-level document about what we’re doing and why.

Then I take the spec and run it through a PRD skill I shamelessly stole from the Ralph loop. Quick aside: PRD is a term I had genuinely never encountered in fifteen years of working in agile teams. I first heard it watching YouTube videos about people working with AI, sometime in the last year, and I had to go and look up what the bloody hell it stood for. As best I can tell, a PRD is an epic with a collection of user stories, some acceptance criteria, some functional and non-functional requirements, and a bit of product context bolted on top. Cool. I can work with that. The reason I like this particular PRD skill is that after I’ve already spent four hours on the spec conversation, it asks me five more questions to validate what I’m building. Which is exactly the kind of thing you want at that stage!

PRD becomes JSON. JSON gets fed to a Ralph loop. Off we go.

The bit where I was cheating

Here’s the dirty secret. I’d never actually set up the Ralph loop the way you’re supposed to set it up. I’d been running it via a plugin inside Claude Code. Plugins are wonderful. You install them, they work, you’re productive in ninety seconds. Why would you write a bash script when you can install a plugin?

The honest answer is: you wouldn’t. And that’s the trap. The problem isn’t plugins. The problem is when your workflow only exists inside one of them.

Plugins feel like the harness rewarding you for committing to it. Every plugin install is a small vote for staying inside that one ecosystem, and those votes compound quietly until one day you look up and notice you’ve stopped being portable. You’re not running a workflow anymore. You’re running a workflow that only exists inside Claude Code. Which is fine, until it isn’t.

How I burned through the credits in the first place

I should be clear about something. I hadn’t hit the Pro limit doing serious work on my personal project. I’d hit it because it was my holiday, and I’d spent the previous week happily down an oh-my-codex rabbit hole for no reason other than that it was interesting.

Oh-my-codex is a sprawling wrapper that someone has built around Codex to give it brainstorming flows and Ralph loops and a pile of other usability niceties. I’d become curious about it for a very specific reason: when the Claude Code source leaked, a developer in South Korea used Codex with oh-my-codex to reimplement the entirety of Claude Code in Python. In six hours. Six hours, for a non-trivial codebase. I wanted to understand how that was even possible, which meant I wanted to make oh-my-codex work with OpenCode and Claude Code rather than just Codex, because of course I did. More harnesses. Always more harnesses.

So that’s what the credits went on. A week of trying to bend an already-baroque wrapper around two more harnesses it wasn’t designed for, purely because I wanted to know how the thing worked. No deliverable. No project at the end of it. Just the kind of dive-in-and-poke-at-it exploration that holidays are for. I was having a great time, the plugin inside Claude Code was still humming along for the actual personal project I dipped into between rabbit hole sessions, and the cost of any of this hadn’t shown up yet.

Then it showed up.

The thing I’d been ignoring at work

I should have seen this coming, because at The Economist I have access to three different coding agents with three different usage pools, each gated on different constraints. In practice that means I bounce between them all day. Hit a five-hour window in one, switch to another, work until that one taps out, switch to the third. It’s a genuinely lovely setup if you’re the kind of person who likes being spoiled for choice on tokens.

But it also means I’ve been quietly reinstalling the same plugins and the same markdown scripts in three different places, every time something changes. And whenever one of those environments goes down or gets reconfigured, I lose half a morning rebuilding the workflow in another one. I’d been feeling that friction for ages without ever quite naming it. It was just background noise. The cost of doing business.

Then the personal Pro lockout happened, and suddenly the background noise was the only thing in the room.

Doing the unglamorous thing

So I went and found open-ralph-wiggum, worked out how to wire it up properly, and wrote a small zsh function that wraps it so I can just type ralph-loop from any project directory and have the thing kick off without me having to remember any flags. None of this was hard. None of this was interesting. It was the kind of work I had been actively avoiding because I’d already spent a week earlier that month fiddling around with Codex and OpenCode and trying to make various things play nicely together, and the last thing I wanted was more yak shaving.

But here’s the thing about doing it during a forced lockout, with nothing else to distract me. There was nothing else to do. So I sat with it. And once it was done, I had a Ralph loop that ran on top of OpenCode, with GPT-5.4, completely independent of whether Claude Code was up, down, or rate-limited into oblivion. The wrapper meant I could move between harnesses without rebuilding anything. The script lived in my dotfiles. It was just there.

The real prize

I’ve written before about how each AI harness has its own personality. Claude Code thinks differently from Codex thinks differently from OpenCode, and a lot of that personality lives in the harness rather than the model. I still believe that. But what I hadn’t fully clocked, until this week, is that everything I’d written about harness personalities was manual copy paste painful exercises, because the plugin had me boxed into one of them.

Knowing the council exists is one thing. Being able to actually convene it on a Tuesday afternoon while you’re trying to ship something is another. The wrapper script is the thing that closes that gap. It allows for more meaningful agentic workflows in any harness easily.

That’s the prize. Not the lockout workaround. Not the bash script. The portability that lets the multi-harness thing actually be a way of working rather than an essay.

What I’m sitting with

I’m going to keep using plugins. They’re genuinely useful and I’m not about to LARP as someone too principled to install convenient things. But I’m going to be more suspicious of how easy they make the first ninety seconds feel, because I now have a much clearer sense of what they cost on the back end. Every plugin ecosystem is a small gravity well. The more you commit, the harder it is to leave, and, this is the part that bothers me most, the less you can even see what you’re missing on the outside.

The unglamorous wrapper script turns out to be a small act of resistance against that. Not a heroic one. Just a vote for keeping the exits visible.

I’d rather have the exits visible.

I use multiple AI agents as an architectural review council. When I said that out loud recently, I got the look. You know the one. The polite nod that says “I have no idea what you just said but I’m going to smile and move on.”

So here’s the footnote.

The setup

I’m currently juggling two things at work that have no business being juggled at the same time.

The first is a set of deeply tangled bugs that have been lurking since July 2024. They’re tied to a third-party integration, they’re interconnected, and we’re only now seeing the full scope of how bad they are. This is slow, careful, multi-day investigation work. Long-form conversations with AI. Reading code. Writing tests to verify behaviour. Building the case for “this is exactly where the problem is and this is exactly why.”

The second is supporting our engineering manager to enable an external contracting team to deliver a new feature in our codebase. The contractors have never touched our code before. They don’t have a clear picture of the requirements, the systems, or how everything interacts. The depth they need to make meaningful architectural decisions is broad, deep, and nuanced. I’ve worked in the systems, but I don’t have enough depth to answer all their questions off the top of my head.

But what I do have is access to Claude, Claude Code, GitHub Copilot, and Codex.

The council, assembled

I should back up. For months now, in my personal life, I’ve been asking the same questions to ChatGPT, Gemini, and Claude interchangeably. I call them my council. Each one has a different personality, notices different things, and observes different angles. I find that when I’m getting multiple opinions I make better decisions. This is just me applying the same instinct to my workspace.

It started with a simple question: we have a third-party payment provider that offers a payment method, and we have a number of integrations between us and them. The contracting team needed to understand how we use it. How do we integrate with backend services? Where are the bits that are backend-for-frontend versus actual backend platform services? How do all of these systems interact? Where are all the endpoints?

I spent a day and a half in long-form conversations with multiple AI systems interrogating the problem. I started at the web-facing entry point and worked backwards. I trawled Confluence, Slack, Google Drive, and every other form of long-term documentation to build a picture of what the contracting team was going to need. Then I took all of that context, the goals of the team, and the documentation, and used it to structure a comprehensive prompt.

I ran that prompt through three different AI harnesses: Codex, Claude (via the web), and Claude Code running Opus. Each one went away, investigated the same repositories, and came back with structured answers. Then I took those structured answers, went and explored the code myself, used the hints they’d given me to validate everything, and wrote up a comprehensive document explaining the lot.

Naive me thought job done

Obviously I was not done. You’d think by now I’d know better.

A week later the contracting team came back with “cool, we have a plan.” They’d taken everything I’d given them, created architectural diagrams, Confluence documents, and actual thinking. They wanted me to review whether their approach would work.

So I did the same thing again. Took their documents, the context I’d already built, and pointed all three AI harnesses at the relevant repositories, not just mine but across four different teams’ repos, backend services and frontend services and everything in between. I had each one validate whether what the contractors were proposing would actually work. Then I took the outputs from all three, wrote them to file, and had a fourth agent (OpenCode running Opus 4.6) synthesise a combined result. I used that synthesis to structure my response back to the team.

I’ve now done this process three times. Here’s how it works:

The Council Process

1. Gather context and documentation
2. Structure a comprehensive prompt
3. Run the same prompt through multiple AI agents and let each investigate the repositories independently
4. Save their structured outputs
5. Run a synthesis agent across all results
6. Validate manually: use the AI outputs as a map for where to look, read the code yourself, and run quick targeted questions past other engineers

Where the real value lives

The synthesis step is where the magic happens. It’s not just about getting three answers. It’s about what happens when you put them next to each other.

The synthesising agent highlighted where all three harnesses were in agreement, which gave me confidence. But more importantly, it highlighted where they’d noticed different pieces of the problem. Even though they were looking at the same repositories and most likely using the same underlying tools, they ended up pulling out different things. Codex might flag an endpoint I hadn’t considered. Claude Code might trace a data flow the others glossed over. The breadth of coverage from running three agents was meaningfully wider than any single one.

This also feeds into something that should be obvious but bears repeating: AI hallucinates. You cannot 100% commit to trusting just one version. When you need accurate architectural understanding, having multiple agents give you a synthesis that you then validate yourself is genuinely useful. It’s not a replacement for reading the code. It’s a way to read the code faster and know where to look.

The tools have personalities

Here’s something I find fascinating, and I’m not the only one. Another principal engineer I know has noticed the same thing.

Codex is the grumpy pragmatist. It goes away, gets some stuff done, comes back, and tells you the bare minimum you need to know. Not a single detail more. Bullet points, to the letter, done. That’s fine. That’s exactly what you’d expect from a tool optimised for task completion.

Claude, given the exact same prompt via the web with Opus, comes back reading like a chatty engineer. A bit scattered, a bit flowery, but thorough. You’ll get everything you need, it’ll just need a bit of back-and-forth to extract it cleanly.

But here’s the interesting bit: OpenCode, regardless of which model it’s running underneath, whether that’s Opus or GPT-5.4, tends to give better structured results than either of the first-party platforms running the same models. The investigations are better organised. The outputs are clearer. The intent comes through more directly. I’m finding this is true when comparing Claude Code to OpenCode running Opus, and it’s also true when comparing Codex to OpenCode running GPT-5.4.

The harness matters more than the model

The scaffolding around the model, how it structures tool calls, how it formats its output, how it organises an investigation, is doing more heavy lifting than people assume. That’s a genuinely counterintuitive finding. The same model, in a different harness, produces meaningfully different quality of output. If you’re only evaluating models, you’re missing half the picture.

AI expands capacity, not energy

I’ll be honest. By the end of every week right now, I am flattened. Exhausted. Mentally, emotionally, everything. Gone.

These tools have enabled me to explore and understand systems at a superficial level far faster than I ever could otherwise. To get the depth I needed for these handover documents would have taken weeks of investigation. I did it in hours. That’s real. That’s meaningful. That capacity expansion let me keep working on high-priority deep-dive bugs while simultaneously supporting an external team and ensuring they had enough context to be unblocked and start working independently.

But working with AI at full tilt is cognitively expensive in ways that people underestimate. You’re doing more, faster, and that uses more of your mental energy than you think. AI gave me the capacity to do work that would’ve been impossible to fit in otherwise. It did not give me more energy to do it with.

Still experimenting

I’ve run this playbook three times now without changing it. Same process each time. I haven’t tried to refine it or automate it or build a harness around it yet, though it’s in the back of my mind. I actually started building a mobile app a while back to formalise the council concept for personal use, but got distracted because, well, reasons.

So what’s the lesson? Don’t trust one AI. Use a council. Get them to validate each other. Get them to look for reasons they’re wrong. Use the synthesis of multiple perspectives to build confidence in your understanding, then go validate it yourself.

I’m still not entirely convinced this is the best strategy. But it is letting me do things I could not have done otherwise, and right now that’s enough. The future of AI-assisted engineering might not be a better model. It might be a better council.

Over the past few weeks I have been increasingly panicked. That’s probably the only honest way to frame it.

I’ve been juggling a lot. Supporting a team building out a new payment method. Handing over deep knowledge of an existing payment method to another team working outside our scope but integrating with systems we own. Helping yet another team figure out how to break a backend platform service from a monolith into microservices. And somewhere in between all of that, trying to actually deliver a feature myself: a seemingly straightforward change to one of our frontend purchase journeys to make it faster.

Each of those things individually requires serious context. Together they’ve been chewing up my headspace and pulling me in every direction. Which means that whenever I finally sat down to write actual code, I arrived in a state of absolute panic. The “oh my gosh I have so little time, move quickly, move quickly, move quickly” kind of survival mode.

And that’s when I made one of the most fundamental mistakes you can make with AI-assisted development.

The 50-File Disaster

Here’s the thing about AI-generated code: it’s easy. So easy that it’s almost impossible to remember just how easy it is, because you’ve spent a lifetime handcrafting code yourself. You carry this default assumption that building things is complicated and slow.

So I did what I thought was the right thing. I planned. I looked through the existing code. I thought about it. I wrote out detailed acceptance criteria. I thought about the problem from the AI’s perspective. And then I said “cool, I think I have enough” and started implementing.

By the time I got someone else to look at it, they pointed out it was missing a key behaviour. I’d misunderstood part of the acceptance criteria. OK, fine. I started trying to fix it.

And then trying to fix it. And trying to fix it.

What was a small hole became a deep hole became a nightmare became “why does it feel like I will never, ever get anywhere with this?” By the end of it I was touching something like 50 files across two repos. Small changes scattered everywhere, most of them not even really needed. All driven by the panic override of needing to get something done while constantly being pulled out of context and back in and out and back in until I was just thrashing. Burning cycles. Making zero progress.

Panic Is a Smell

If you’ve been in software long enough you know what a code smell is. Something that isn’t technically broken but tells you something deeper is wrong.

The panic to get things done is a smell. The pushing and pushing and pushing is a smell. The feeling that you don’t have enough time, that you have to ship something right now, that you can’t afford to slow down? That’s a smell. And I ignored it for way too long.

Because here’s the lesson I apparently need to keep re-learning: with AI-assisted development, the writing of code is not the bottleneck. It never was. The understanding is the bottleneck. And when you’re panicking, you skip the understanding to get to the doing, which is exactly backwards.

The Reset

I eventually stopped. Stepped away from the mess. Started with a brand new repository. Took all the things I’d learned, the plan document, the acceptance criteria, everything. And then I spent an entire day in conversation with an AI. Not writing code. Just investigating.

Testing existing behaviour. Running multiple examples and execution paths. Making sure I had a precise, clear understanding of what the current system actually did, what the new behaviour needed to be, and all the various paths between them. I literally spent hours asking the AI to explain each step of its plan and justify why it chose that approach.

I say all the time that planning matters more than coding. But experiencing the contrast firsthand is different. Hours of slow, methodical, back-and-forth investigation. Deep thinking about context. Deep thinking about what you’re trying to do and why. So that when you, the human in the loop, actually ask the AI to build something, the full context of what you’re trying to achieve is sitting clearly in your head. You understand the user behaviour. The system interactions. The high-level architecture. You could draw all the diagrams because you actually understand what needs to be done.

The feature that had consumed a week and a half of thrashing? After that day of planning, it took a couple of hours to get something working correctly.

The Council of AIs (or: Going Wide)

Meanwhile, on the other side of my work life, I’ve been doing something completely different with AI tooling.

To support the contracting team building out a new feature, I’ve been running what is essentially a council of AIs to review their design documents. OpenCode, Codex CLI, and Claude Code running simultaneously so I can verify, validate, and cross-compare. Deep-dive analysis with Claude and ChatGPT for architectural decisions and historical context. Complex investigation into bugs that were first logged two years ago and never properly resolved.

I have been holding the context of a massive amount of different workstreams. Work that would have taken me days or weeks to get even a baseline understanding of. The AI tooling genuinely lets you go wide in a way that wasn’t possible before.

And that’s where the tension lives.

Shield and Sword

The honest truth is that I’ve been doing two very different jobs at the same time.

One job is the shield: absorbing context, running investigations, unblocking other teams, reviewing designs, holding the big picture so nobody else has to. The AI tooling makes this possible. It lets you hold 10x the context. You can pre-empt meetings by using Claude to pull together context and solve problems before the meeting even happens, cancelling two or three in a morning and buying yourself hours of uninterrupted time. You can run parallel investigations across multiple tools and hold the full picture of what’s going on across an entire programme of work.

The other job is the sword: actually sitting down and delivering a piece of working software. And that requires the opposite of going wide. It requires going deep. Slow. Methodical. Boring, even.

The AI enables both.

But your brain can’t do both at the same time.

When you try, you thrash. You burn cycles switching between deep and wide, and just like a thrashing computer, you end up doing a lot of work and making no progress.

What I’m Taking Away

Two things, and they’re in tension with each other, and I’m OK with that.

Go deep before you go fast. Planning with AI isn’t just “write a spec and hand it over.” It’s hours of investigation. It’s asking the AI to explain its own plan in painful detail. It’s making sure you understand the problem so well you could solve it by hand. The code is the easy part. The understanding is the work.

AI lets you hold a lot of context, but your brain still has limits. Context switching costs the same as it always did. Maybe more, because the AI makes it tempting to take on everything. You can hold the shield and the sword, but not at the same time. Deliberately buying yourself blocks of deep time is not optional. It’s the whole game.

This is the new trap for senior engineers. AI lets you take on more surface area than ever before. But the work that actually ships still requires deep focus. Nobody is immune to thrashing, no matter how good the tooling gets.

And if you’re sitting there right now, pushing and pushing and panicking and feeling like you’ll never get there? That’s a smell. Stop. Step away. Start again with understanding, not urgency.

I’ve been writing this blog for a while now. I’ve documented scope creep spirals, the joy of deleting code I spent weeks writing, and the slow painful education of learning to work with AI agents without letting them run off a cliff. If you’ve been following along, you know the theme by now: I learn things the hard way and then write about it so you don’t have to. Or at least so you can watch.

Yesterday I was explaining to a colleague how I use Gemini to give me personalised feedback on my performance after meetings. He’s a staff engineer, someone who’s genuinely deep into AI-assisted coding, not a tourist. And even he stopped and went: “Wait, you’re doing what?”

That reaction made me step back. Because I hadn’t really sat down and thought about what I’d actually built over the past year. I’d been solving problems one at a time, better code generation here, better context gathering there, a way to get honest feedback on myself over here, and somewhere along the way it had become something bigger. A system. A layer underneath how I work that I now can’t imagine working without.

So this is me trying to describe what that looks like, now that I’ve finally noticed it.

The code layer: get off the autocomplete

If you’re writing code with an AI assistant inside your IDE, Copilot in VS Code for instance, I’d gently suggest you’re missing the better experience. CLI agents like Claude Code, Codex CLI, and Open Code have fundamentally changed how I interact with code. Open Code has become my go-to because it works well straight out of the box and, crucially, it connects to Copilot’s backend models. If your company already pays for Copilot, Open Code might be the unlock you didn’t know you were waiting for.

The shift matters because it changes what the AI is doing. Inside an IDE, it’s autocomplete with delusions of grandeur, guessing what you want line by line. On the command line, I’m describing problems, analysing architecture, pulling systems apart, generating diagrams. It stops being a typing assistant and starts being a thinking partner.

I use this for everything that touches code directly: writing it, reviewing it, debugging, breaking things apart to understand them, building architecture diagrams. The lot.

The context layer: taming the organisational scatter

Every engineer in a large organisation knows this pain. The information you need to do your work lives in seven different places: Slack threads, Google Meet recordings, Confluence pages, Jira tickets, GitHub PRs, and at least two places nobody told you about. You spend half your time assembling a coherent picture before you can even start thinking.

ChatGPT and Claude’s enterprise integrations have changed this for me. Both allow you to connect to corporate tools, your chat platform, docs, issue tracker, source control, and pull context into a single conversation. Instead of trawling through three Slack channels and two Confluence pages for forty minutes, I pull it all together and ask: what does this ticket actually need? What are the acceptance criteria? What am I missing?

Here’s where it compounds. Good acceptance criteria from this layer mean better prompts for the coding agents. The layers feed each other. I didn’t design it that way, it just happened once the pieces were in place.

The mirror: feedback that doesn’t care about your feelings

This is the hard one to talk about. And the one that made my colleague stop in his tracks.

We’re a remote organisation. Google Meet is where everything happens, and Gemini sits inside every meeting. Most people don’t turn on transcriptions, which I think is a mistake. Any meeting producing collective knowledge should generate a transcript. Those transcripts feed the context layer above.

But there’s another use that took me months to work up to.

After meetings where I’m an active participant, I ask Gemini: as a staff engineer, what went well, what didn’t go well, and what can I improve on?

The first few times, it was rough.

Here’s the thing about feedback from humans: you almost never get anything useful. You either get “yeah, that was fine” or something so carefully hedged that whatever kernel of truth was in there has been sanded down to nothing. I’ve rarely received feedback that was specific, actionable, and tied directly to something I actually did in a real moment.

Gemini doesn’t do hedging. It references specific things that happened in the meeting. “You reframed the argument here and it shifted the conversation constructively.” Or: “You weren’t listening here and this is where it cost you.” It once told me that while I’d handled a frustrated colleague well, I could have spotted the frustration earlier and intervened before it escalated, and that when another colleague was dismissive, I’d recovered well but could have prepared for that reaction. Specific. Contextual. Minutes after it happened.

When I explained this to my colleague yesterday, he asked: “Isn’t this just seeking perfection?” And I realised, no. It’s just a way to learn and grow and become more deliberate about how I communicate, how I lead, and how I interact with the people around me. You can’t improve what you don’t measure. This is measuring.

But here’s what really made me think this is bigger than my own little experiment. I told a principal engineer friend at another company about this approach. He had a difficult conversation coming up, recorded it with Gemini, and afterwards used the transcript to get actionable feedback on how he’d handled it. His reaction was genuine shock. He’d never had that clear a picture of how his conduct was landing. An engineering manager I know has started doing the same thing and describes it as brutal but the most meaningful feedback he’s received in years.

And I think there’s a reason for that. I remember chatting with a startup CEO at a meetup who made the observation that the higher you go in leadership, the less honest feedback you receive. The position of power makes it hard for people to cross that barrier. Gemini doesn’t have any concept of your title or your seniority. It just tells you what it saw.

In the beginning, every session felt like a wake-up call. After months of doing this consistently, keeping a log, reading it back, it softened. Not because the feedback got less honest, but because the gap between what I thought I was doing and what I was actually doing got narrower. Fewer surprises. More gentle nudges, fewer gut punches.

So what is this, actually?

None of these tools alone would be worth a blog post. A CLI coding agent is nice. Enterprise AI integrations save time. AI self-reflection is powerful but weird. What caught me off guard, what I only noticed yesterday when I saw my colleague’s reaction, is that they work as a system.

Meeting transcripts feed the context layer. The context layer produces better acceptance criteria. Better acceptance criteria drive better output from the coding agents. The self-improvement loop makes me more effective in the meetings that generate the transcripts. Each layer feeds the others. I didn’t plan it. I just kept solving problems and the connections emerged.

There’s a Sam Altman interview from about a year ago where he describes people using AI as “an operating system for how they think.” At the time I had absolutely no idea what he meant. Now I think I do, and the uncomfortable truth is that I’m probably barely scratching the surface of where this goes.

So here is my take away action for you. Next meeting you are in with a transcript, ask an LLM for some honest feedback. Let me know if you learn anything interesting!

I’m still figuring it out. As usual, you’ll hear about it when I do.

Have you ever struggled to get started with something new, not because the thing itself is hard, but because the shape of how to learn it isn’t clear? That’s where I’ve been stuck with agentic coding. Not the doing of it. I’ve been doing it for months. The teaching of it. The “how do I help someone else get started” of it.

And then I remembered code retreats. And katas. And the way I actually learned TDD all those years ago, not from a book, but from structured practice with low stakes and room to play.

So I built a set of agentic katas: structured coding exercises designed specifically for practising AI-assisted development. Not traditional katas, those are too small and the AI already knows all the answers. These are bigger, meatier problems in unfamiliar domains that force you to engage with the full process of working alongside an agent.

Let me explain how I got here.

A Brief History of Practising on Purpose

Early in my career, code retreats were the thing that taught me test-driven development. Not a book. Not a course. A structured, all-day event where you solve Conway’s Game of Life over and over again, each time with different constraints. Maybe your pair is actively trying to not solve the problem. Maybe you’re strictly ping-ponging. Maybe you delete your code every 45 minutes.

The point was never to solve Conway’s Game of Life. The point was to internalise the patterns and practices of TDD by giving yourself a safe space to experiment. No production pressure. No deadlines. Just play.

Code katas grew out of the same ethos, small self-contained problems that shouldn’t take more than an hour or two. The algorithm doesn’t matter. How you choose to solve it does. They’re bite-sized by design. They’re not supposed to be hard. They’re supposed to be practice.

The Problem with Katas and AI

Here’s the thing I’ve been struggling with: traditional code katas don’t work for learning agentic development. They’re too small. The LLM has already seen every solution to FizzBuzz and Roman Numerals in its training data. You’re not practising a workflow, you’re watching an AI regurgitate a known answer. There’s nothing to explore, nothing to plan, no decisions to make about approach or tooling.

And that matters, because the skill you need to develop with agentic coding isn’t “how to prompt an AI to write code.” It’s how to think alongside one. How to explore a problem space together. How to write a plan that gives an agent enough context to be useful. How to verify that what came back is actually what you wanted. How to set up your workspace so the AI has the right guardrails.

None of that shows up in a 30-minute kata where the AI already knows the answer.

The Accidental Discovery

What’s funny is that I’ve kind of been doing agentic katas already, almost by accident. I just didn’t realise it at the time.

A few weeks ago I wrote about deleting code on purpose as a way to recover from burnout. I’d been experimenting with the Ralph Wingum loop, throwing PRDs at an agentic coding workflow, seeing what came out, then deliberately throwing the code away. The output I was chasing wasn’t a codebase. It was understanding. How big can a PRD get before the loop breaks? How much do agent files matter? What’s the minimum setup to get something useful?

Each run was a contained experiment. Fresh repo, clear problem, focused practice, delete the code, do it again. I was varying one thing at a time, adding a CLAUDE.md file, scaling up the PRD size, trying a different domain, and learning from each iteration.

I was doing agentic katas. I just hadn’t named them yet.

Looking back, the whole arc has been building toward this. My game project was the first rough version, months of cycling through spec-driven development and making mistakes. The “delete the code” experiments compressed that into focused sessions. And now, formalising the structure into something other people can pick up feels like the obvious next step.

Building the Thing

So I’ve put together a set of agentic katas. The idea is that each one should require somewhere in the region of four to eight hours of focused hand crafted work to do well. And by “well” I mean the full golden plate: test-driven, 100% coverage, clean README, proper git history, the works. Not because the output matters, but because doing that level of work with an AI agent forces you to actually engage with the process.

The loop for every kata is the same:

Explore → Plan → Set Up Context → Build → Verify

And there’s one key rule that makes the whole thing work: you are not allowed to choose a programming language or framework until you’ve had a conversation with your AI tool about what the best approach is.

This is the rule that forces the shift. Instead of jumping straight to “build me X in TypeScript,” you have to start with “I need to solve this problem, what are my options?” You explore the problem space. You figure out what tools exist. You have the AI challenge your assumptions. Then you decide on an approach.

From there, you write a detailed plan, acceptance criteria, example data, use cases, a breakdown into small chunks of work. You set up your workspace with an agent file and think about what context to include. You build incrementally. And you verify everything: read the plan, read the code, run it, test it, confirm it does what you intended.

The Katas Themselves

I’ve started with four problems, each chosen because they sit in a domain most developers haven’t worked in before:

An audio transcriber that handles speech-to-text with timestamps and speaker diarisation. A background remover for image segmentation. A meme generator that deals with text rendering and positioning on arbitrary images. And a thumbnail ranker that scores images for visual appeal.

Each kata has deliberate ambiguity baked in, because real problems are ambiguous, and part of the skill is figuring out what questions to ask. They also have a privacy constraint (everything runs locally, no cloud APIs for processing) and an extra credit extension for when you want to push further.

Why This Matters Right Now

I’ll be honest: the reason I’m putting this together is partly selfish. I want to run a workshop with my colleagues, and I need structured material to do it. But there’s a bigger motivation too.

Right now, everything about AI and software development feels incredibly intense. Fear of being made obsolete. AI layoff discourse everywhere. The pressure to have strong opinions about tools you’ve barely had time to evaluate. It’s all very stressful, and stress is the enemy of learning.

What people actually need, what I needed, and what I accidentally created for myself, is a safe space to play. A contained environment where you can try things, make mistakes, and build intuition without the stakes of production code or career anxiety hanging over you.

Code retreats gave us that for TDD. I’m hoping agentic katas can do the same for working with AI.

The Repo

I’ve put everything together in a repo: github.com/vanonselenp/agentic-katas

It includes the kata briefs, a participant guide covering the rules and process, and a facilitator guide for anyone who wants to run this as a structured session with their team. A session takes about 90 minutes.

I haven’t run this with anyone else yet. I only put it together today, and I’m planning to trial it with my team in the coming weeks. It might be brilliant. It might be terrible. Either way, I’ll write about how it goes.

But the core idea, that you need bigger, unfamiliar problems to practise AI-assisted development, and that the process matters more than the output, that I’m confident about. Because I’ve been living it, accidentally, for months.

If you try it, I’d love to hear how it goes. And if you’re doing something different to build these skills, I’d love to hear about that too.

Alice in Wonderland by Valerie Hinojosa / Creative Commons Attribution-Share Alike 2.0

Now this is a story all about how one button link got my codebase flipped turned upside down. And I’d like to take a minute, just sit right there, and I’ll tell you how I shipped 14 PRs without pulling out my hair.

It started with a Monday morning meeting. I’d been off for three weeks. The meeting was dense with context about decisions made months ago, documented across scattered specs and design docs. Systems I don’t own. Plans originally speced out almost a year prior. SEO requirements. Legacy middleware behaviour. And somewhere in all of this, a single task: change where a subscribe button points.

The old flow routed users through a legacy auth endpoint which was a piece of middleware handling user state and return-to-site functionality. The new flow should skip that layer and go direct. Simple, right?

Three repos. 3 small PRs. That was the original scope.

It became six repos and one or two more PRs…

The Context Problem

Here’s what made this tricky: I didn’t have the context. Not the institutional knowledge of why things were built this way. Not the codebase familiarity to know where all the tendrils reached. Not the cross-system visibility to see how changes would ripple.

Normally, this is where you’d involve other teams. Schedule alignment meetings. Negotiate architecture choices. Coordinate timed releases. The org chart becomes the constraint.

Instead, I threw five AI tools at the problem.

I used internal knowledge search to surface half a dozen docs from a year ago about what a potential migration might look like. Copilot and Codex scanned repos I’d never opened, outputting high-level analysis of what would need to change. NotebookLM synthesised a dozen-plus sources into actionable Jira tickets with acceptance criteria and testing plans. And Claude handled the actual implementation across all six repositories.

Each tool for what it does best. None of them sufficient alone.

The Shape of the Change

What was supposed to be three repos became six because the AI tooling kept finding rabbit holes worth going down.

The approach was backwards compatibility first. I updated the auth service to forward requests to the new endpoint, so existing systems would keep working. Only after that was stable did I remove the old code paths and switch the calls to point directly to the new flow.

Along the way, I hit a referrer bug that only revealed itself mid-implementation. One of the components lived in a shared library, not a full application, which meant handling referral data differently than expected. This meant that I had to change how it was reading from window referrer data rather than relying on direct redirect URLs.

And then there was a shared header component in another team’s repo. Hardcoded to the old endpoint. In code I couldn’t easily modify. The rabbit holes kept cropping up every time I thought I dived down them all.

Fourteen PRs. Six repositories. Backwards compatible throughout. Zero downtime.

The old flow had an extra hop through legacy middleware that handled state management. The new flow removes that layer entirely. Which makes for a faster time to checkout, same user experience, one less thing to maintain.

The Point

This would have been a multi-team effort. Alignment meetings across three teams, at minimum. Negotiated timelines. Architectural discussions. Coordinated releases.

Instead, it was one developer holding context that used to require an org chart.

I’m not saying AI tooling makes you a better engineer. I’m saying it lets you hold more context. And sometimes that’s the difference between “we’ll need to schedule a meeting with the other teams” and “I’ll have a PR up by Thursday.”

The context ceiling just got a lot higher.