For the past five months, I’ve been co-leading development of an enterprise platform where nearly 100% of the code is written by AI coding agents. This isn’t POC or experiment mode. We’re adopting agent generation as a production engineering practice for a first-class product.

We were lucky in our timing. The models keep getting meaningfully better: September 2025, November, again this year. It feels like we started at the point right when the models became viable for this sort of operation. We aren’t vibe coding and we’re not hand-writing with autocomplete. It’s a different relationship between the engineer and the code.

LLM codegen is Lethal Weapon#

I’ve been trying to find the best words to describe the boundary of responsibility and the needed approach to this technology. When I see engineers slow down or get blocked in adoption, it’s usually because of trust issues, not a lack of understanding. I get it. This technology will make you uncomfortable. That’s rational. It makes me uncomfortable, sometimes. But I want to help engineers get past that initial set of trust issues, because I have learned a lot by pushing past discomfort.

I have hit on this: You are Danny Glover. You’ve been doing this for years, by the book. You’re a veteran who has seen it all and knows the score. Then you get a new partner with unconventional, disruptive methods that bring stunning results.

Your new partner is also an erratic psychopath who is going to get you killed. If you don’t have a story of the robot doing something completely insane, you will have one soon.

The dynamic is real: you have to rein your partner in while letting them push you out of your comfort zone. But, for me, that’s led to mutual growth: I get faster, the agent gets better-directed, and the output improves because we’re both bringing something the other can’t.

It doesn’t hurt that the models have kept getting better. (Maybe it’s like Lethal Weapon 2? or 3? Doesn’t he get less insane over time?) But I’ve seen Opus 4.6 try to brute-force its way into my application because it forgot where the test credentials lived. The new models are amazing, but I’m not in the fully-autonomous, dangerously-skip-permissions camp. Don’t give them the car keys.

(I have never seen Lethal Weapon.)

The prompt is the product#

Prompt engineering is a real thing, although I think for the most part good prompting is just good communication. I have colleagues who can zero-shot in VSCode and produce great components. I am awestruck by that, but I can’t do it. I’m very spec -> plan driven.

I create a small to moderate description of what I need - the draft spec. I then work interactively with the robot to turn that into a full spec of the code to be built. (I always ask it to ask one question at a time, because it wants to be overwhelming.) Migrations, structs, business logic, web components are all detailed, with base or extended components labeled when they exist. There are code examples. I’m building bigger components right now, so these are usually hundreds of lines, and I’ll sometimes run a simplification pass on the spec before I execute. Our spec process has a documented prompt in our repo, so anyone on the team can initiate it as a slash command, and we can also improve it over time.

I then hand this off to an implementation plan. The robot turns the spec into the steps it will take to implement the spec. This feels like overhead right now, and I’m thinking about how it can be streamlined. But there are some nice things about this process:

  • It’s easier to turn the spec into things that aren’t just the implementation steps - like user stories, or a functional spec for stakeholders to review.
  • In the plan, I can add to the spec details all the practice points I want to emphasize. The spec tells the robot what to build, but the plan reminds them how to build it well.

The plan also forces an agent-friendly sequential structure that matches the onboard TODO list it commonly runs in the IDE. The robot checks the boxes as it goes through steps. It can organize the plan into an orchestrator with multiple phase files, so that it can take on easier execute-review rounds, or pick up in new sessions when context dictates, or delegate to subagents when appropriate.

Planning is also encapsulated in a prompt in our repo. None of this approach is original - you’ve probably heard about this all before. I got most of it from Harper Reed and the robot helped me build the details. But it works.

And I cannot emphasize this enough: tests#

Test-driven development has always been a good idea that few of us actually practiced. With agentic development, it is not only easy but a non-negotiable technique to give the robot accountability and its own means of verification. Unaccountable robots make mistakes, and will then tell you the results are perfect, because they are insane. And this can become a loop:

  • ROBOT: Done! It’s perfect.
  • YOU: It’s not perfect. These things are broken: broken things
  • ROBOT: You’re absolutely right. roboting… Done! It’s perfect.
  • YOU: These things are still broken: broken things, additional broken things

The above example has 1-2 too many cycles.

Build verifiability into the plan. The robot knows how to do this. It will implements in red → green cycles to declare a testing contract and then fulfill it. This obviously inverts the usual TDD complaint (“it slows us down”) because the agent handles the implementation cost.

If you are building a full-stack application, then apply this across the whole stack.

  • Unit tests, of course, server- and client-side
  • BDD on the middleware layer
  • API testing
  • We are also telling it to write end-to-end tests from the start. We have Playwright in the repo, not as a separate QA automation base, and the robot proves its own frontend implementation, which I find very important, because it’s the worst at UI.

Make the robot write the tests before the code. Make sure they’re strong and representative. Know where the agent is stubbing or mocking. Our tests are built to use a local or CI/CD hosted database, using the same schema migrations as production and test seed data. It’s maybe heavier or less clean than it could be, but it’s also very real.

You’ll have to pay attention to test performance, because the agent is writing a lot and executing them often.

But… the agent is writing the tests#

Even if you are reviewing the tests, there’s a lot of robot here. It’s writing a lot of code, and it’s writing the code that it’s using to verify the code that it wrote. It’s robots all the way down.

Early on in this development, I talked to a different robot about my coding robot (for a unique perspective, ha) The other robot told me that under no circumstances should I trust the first robot. It insisted that all codegen should be reviewed, line by line, by a human. At this point, I think we were generating to the tune of 30,000 lines of code a day across two engineers.

So… I can’t agree with the robot-hating robot. If I’m reviewing every line of code, I’m now back to human velocity. I don’t want to go back there. But there has to be some oversight. That layer can’t be trivial. It has to be as considered as the codegen itself. See above re: erratic psychopath.

Steve Harvey realizing how much code the agent is writing. His demeanor shifts from joy to concern.

There are answers to this.

  1. Quality defense in depth. Do more with your CI pipeline. Much more:

    • Mutation testing — inject small faults into your code and verify your tests catch them.
    • Flaky test analysis — detect tests that pass or fail randomly.
    • Static code analysis and linting — catches bugs and security issues that tests won’t surface, and reduces the by-line code review that is the biggest time suck.
    • Race detection — find unsafe concurrent access to shared memory.
    • Architecture analysis — enforce dependency direction, layer boundaries, and coupling rules. Keeps the agent from quietly violating your intended structure.
    • (This is just the implementation side. There’s all kinds of additional, important QA engineering work. I think that’s another blog post or two. I think QA engineer / SDET becomes a very, very important role in the future.)

  2. Approach code review the way you are approaching codegen. Engineers still need to review the code, but they can have the robot do a first-pass analysis as a code reviewer to give a stronger summary and ideas of key areas to focus. An onboard PR reviewer (like Copilot) can help with this, but it doesn’t always catch the bigger-picture concerns. We have been assembling our own “architect review” prompt that focuses on aspects like DRY, implementation simplicity, application security.

  3. I’m also thinking about robot-assisted repo reviews as part of our regular practice. In this codebase - just like human-generated codebases - we’re going to get debt over time, but the robot can facilitate the pay-down, too. We can have it review the codebase every few iterations, identify the common smells: redundant components when reuse would have worked, giant files, duplicated logic, missed application hardening, Rube Goldberg-machines of connected capabilities instead of clean implementations.

I want to pit robots against robots. Robot solutions for robot problems. I’m still figuring this part out - we all are, I guess - but I think this is the direction.

What I’m still figuring out#

Architecture decisions that span multiple components are still hard to delegate. The robot doesn’t like to touch things that aren’t part of the immediate mission. So I have to nudge it to do “good neighbor” things, like fix breaking tests that aren’t part of their current implementation. This makes me concerned about how much it’s thinking about component-to-component interactions. See point 3 above.

The boundary between “let it cook” and “this needs a human” is something I’m calibrating daily. And the practices that work for a small, senior team may not transfer directly to a larger organization.

I said above that isn’t the stereotype of “vibe coding.” It’s not not vibe coding, though. There is a little bit of letting go and expanding your trust boundaries, here. But you can tend to this code and make sure it’s not slop. That takes scaffolding, oversight, and a willingness to watch the agent work instead of just approving the output.

But five months in, I’m convinced this is the direction.

  • I don’t think it replaces engineering as a practice - we will need engineers, because someone needs to direct this robot.
  • But I also don’t think it’s true to say that this is the illusion of velocity, or a slop machine that you’ll regret in a few months. That’s not an honest assessment of what the tools are doing now.
  • Humans do important work, but the idea that most human-generated code is poetry, or that human-generated codebases are curated sites of flawless care, are fantasies to anyone that’s spent some time working with a codebase created and maintained by multiple humans.
  • In fact, I’m excited that I get to think about all the dimensions of software development - architectural review, QA engineering, iterative refactoring - that now are allowed more time, because it’s faster to ship features.

There are valid arguments all over the place about whether agentic coding is good or bad. All I know is we’re onboarding customers now to a production app that we built with a robot.

This post was developed using Claude Code as a writing partner.