Writing coding tools in the age of AI

For most of its life, PhpSpec talked to developers. Test results, opinionated restrictions on design choices, offers to generate tests or code. All of it composed for a human interacting with the tool on the terminal. Increasingly, the thing reading the terminal output is not a human. And the output of testing tools is not optimised for code agents. I decided to rewrite PhpSpec to, in addition to the beloved human-friendly features, interface better with AI.

So PhpSpec 9.0 is able to speak directly to code agents. I added an Agent Formatter (--format=agent) and made it available in all commands:

$ phpspec run --format=agent

The response sent to the agent was co-designed with the agent. And as we build PhpSpec with PhpSpec itself, I asked my code agent to use the agent formatter whilst building the agent formatter and collect feedback, which we reviewed in the iterations for developing this. So this is the output that has been optimised for agents:

{"v":2,"event":"run_started","suite":"default","seed":null,"php":"8.3.16","coverage":false,"guard":"off"}
{"v":2,"event":"example","id":"6fd046add251","example":"App\\Basket > totals the prices of its products","state":"failing","expectation":{"matcher":"toBe","expected":4000,"actual":3500,"negated":false},"message":"Expected 3500 to be 4000","spec":"spec/App/Basket.spec.php:6","rerun":"run spec/App/Basket.spec.php:6","rerun_argv":["run","spec/App/Basket.spec.php:6","--format=agent"]}
{"v":2,"event":"summary","examples":1,"scenarios":0,"steps":0,"passing":0,"failing":1,"errors":0,"pending":0,"skipped":0,"risky":0,"warnings":0,"deprecations":0,"notices":0,"actionable":1,"duration_ms":4,"rerun":"run spec/App/Basket.spec.php:6","rerun_argv":["run","spec/App/Basket.spec.php:6","--format=agent"],"offers":[{"id":"o_7f3a1c2d","action":"fake_method","target":"App\\Basket::total","value":"4000"}]}

It is JSON Lines, streamed: a line when the run starts, a line per example as it happens, a line if the run dies, and a summary line last, so an agent can act on the first failure before the suite has finished. Each entry prints the expectation, the actual result, the matcher, the exact rerun command, and carries a stable id. The summary carries an offer: what the framework would generate next, if invited to. An offer is applied with phpspec accept o_7f3a1c2d, or with --accept-offers on the run, and never on the run's own initiative. The failure message has become the prompt. If you have any experience with PhpSpec at all, you know that PhpSpec will offer to create a spec or a class or a method for you. Imagine that... but with steroids.

PhpSpec prevented us from bad design, now it will restrict the code agent

Code generation has made code cheap, which moved the entire weight of engineering onto the one artefact that states what "correct" means: the executable specification. And the industry has half-discovered this under the name Spec-Driven Development, and that SpecBDD is that idea with the missing half attached, because a Markdown spec cannot fail and a spec suite can. A BDD tool, rebuilt deliberately, is the best harness an agent can work inside: its DSL guides what gets written, its suite is the fastest honest sensor of whether the behaviour holds, and its cycle, red, green, refactor, is an agent loop we happened to sharpen for twenty years before agents existed. PhpSpec 9.0 is that rebuild.

The bottleneck moved

Every generation of tooling optimises whatever is scarce. When typing code was expensive, we built IDEs, snippets, scaffolding, frameworks. When integration was expensive, we built CI. When deployment was expensive, we built pipelines and containers.

Code is no longer scarce. A model produces a plausible class in seconds, an entire module in a minute. Anyone who has worked seriously with coding agents knows the strange new shape of the work: generation is instant, and the hours go into deciding what to ask for and verifying what came back.

Which means the scarce resources now are exactly two: a precise statement of intended behaviour, and a trustworthy, fast way to check that behaviour holds. We have had a name for the combination of those two things for over twenty years. It is called an executable specification.

Here is the inversion nobody in the BDD community should be shy about saying out loud: for two decades we told developers "the spec is the design document, the code follows from it," and the industry politely heard "a testing tool with unusual vocabulary." Now that the primary author of implementation code is becoming a machine, the sentence is no longer philosophy. The spec is the part the human means. The code is a build artefact: regenerable, disposable, cheap. You would not hand-review compiler output; increasingly, you will not hand-write implementation either. What you will do is own the specification, because the specification is where the engineering lives.

SpecBDD is SDD with teeth

If you have watched the agentic tooling space, you have seen Spec-Driven Development arrive: GitHub's Spec Kit and its cousins, with their structured pipeline of specify, plan, tasks, implement, each phase producing a Markdown artefact that feeds the next, the spec anchored as the source of truth that any of thirty different coding agents can implement against.

I think SDD is the most important correction the AI-coding world has made to itself. It is the industry discovering, at scale and under pressure, what the BDD community learned the long way: examples and intent first, implementation second. If you squint, a Spec Kit specification with acceptance scenarios is a .feature file wearing a Markdown coat.

But SDD as currently practised has a gap, and it is a load-bearing gap: a Markdown spec cannot fail.

A prose specification cannot go red. It cannot catch the model drifting on step 14 of a 20-step plan. It verifies nothing; it merely intends. The current answer is to have another model read the code and judge whether it matches the spec: inference checking inference, a slow and probabilistic sensor stacked on a probabilistic generator. It is better than nothing. It is much worse than a deterministic answer in forty milliseconds.

SpecBDD is SDD with teeth. It closes the loop the Markdown leaves open:

Story BDD, the Gherkin scenarios, captures the what: stakeholder-legible intent, acceptance criteria in the ubiquitous language of the domain. This is the layer that maps onto the SDD "specify" phase, except it executes.

Spec BDD, the object specs, captures the how: the design contract of every class, message by message, example by example. This is the layer SDD does not even attempt, and it is where design quality is won or lost.

PhpSpec 9.0 runs both in one tool. That was true before it was an AI feature: no more juggling phpspec and Behat, one runner, one feedback loop from scenario down to method. In the age of SDD it becomes something bigger: a compilation target for intent. The plan does not just get described; it gets encoded in artefacts that can turn red.

Harnesses, guides, and sensors

There is a second conversation happening in parallel, under the banner of harness engineering: the emerging craft of building everything around the model (the environment, constraints, and feedback loops within which an agent works), written about by Birgitta Böckeler on martinfowler.com, by OpenAI's Codex team, by Addy Osmani.

The framing I find most clarifying splits a harness into guides and sensors. Guides are feedforward: they shape behaviour before generation through conventions, structure, and context. Sensors are feedback: they detect problems after the fact. The best sensors are computational, deterministic and fast, rather than inferential, which is another model slowly forming an opinion. An agent with only guides repeats its mistakes forever; an agent with only sensors flails before it converges. And in every honest account of the state of the art, the same admission appears: verifying behaviour (not style, not types, but functional correctness) remains the unsolved part. The elephant in the room.

Read that list again as a BDD practitioner and try not to smile.

A spec DSL is a guide: it constrains what the agent writes first into a small, behavioural, example-shaped language. A spec suite is a sensor: the highest-signal computational feedback that exists, a machine-checkable answer to "is this what we meant?" in milliseconds. And the BDD cycle of red, green, refactor, taken in baby steps, is an agent loop that happened to be designed twenty years before agents: red gives the model a goal, green gives it a stop condition, refactor gives it a safe improvement phase with a tripwire.

We spent two decades teaching humans to work in small steps against fast feedback. Humans found it hard to sustain: we get bored, we batch, we skip the red. Models are the perfect students of this discipline: they never tire of the loop. They just need the loop to exist, and to be legible to them.

Here is the part I have come to believe, watching agents work: PhpSpec was an agent harness before there were agents. Since 2011 the workflow has been describe a class, run, the tool notices the class does not exist, offers to generate it, run again, the tool notices the method does not exist, offers to generate it. A resolution loop, driven by failures, with code generation on the other side of each prompt. It was a tool-use protocol waiting for something intelligent to sit on the other end. Now there is something.

What 9.0 is

9.0 is a ground-up rewrite. The ObjectBehavior class is gone; specs are closures in the Jasmine/RSpec style, in .spec.php files:

describe(Basket::class, function () {
    this('basket', fn () => new Basket(new NullTaxRegime()));
 
    it('totals the prices of its products', function () {
        $this->basket->add(new Product('Spec-Fu', 1500));
        $this->basket->add(new Product('BDD in Action', 2500));
 
        expect($this->basket->total())->toBe(4000);
    });
 
    it('notifies the ledger when checked out', function (Ledger $ledger) {
        $basket = new Basket(new NullTaxRegime(), $ledger);
        $basket->add(new Product('Spec-Fu', 1500));
 
        $basket->checkout();
 
        expect($ledger->record(any()))->toHaveBeenCalled()->once();
    });
});

Prophecy is gone; mocking is built in, with type-hinted doubles injected straight into the example. Story BDD is built in: Gherkin .feature files with given()/when()/then() step definitions, scenario outlines, data tables, one runner for the whole cycle, and step titles that register once across the suite, so two files cannot quietly shadow each other. Parallel execution runs on worker processes that report over a wire format, so an entry from a parallel run carries the same detail as one from a single process. There is native per-example line coverage in a JSON format designed for Infection, because mutation testing matters more than ever (hold that thought). And --fake generates method bodies with hardcoded return values derived from your spec's expectations: write the example, watch it pass against a stub, then make it real. Sliming, as a flag.

When something goes wrong, the report is written as a briefing rather than a log, whoever the reader is. End-of-run detail is grouped into Failures, Errors, Warnings, Deprecations, and Skipped; a failed expectation reads as an aligned pair under the matcher's sentence ("expected: ... / got: ..."); a step whose code threw is an error with the offending line shown in context, and a step's PHP warnings are captured into the report instead of leaking raw onto your terminal. The JSON you saw at the top is the same briefing, serialised: every command speaks it, so next --format=agent emits its suggestion together with the exact command that acts on it, and generate --format=agent offers the change under an id for phpspec accept <id> to apply, because generate writes nothing without someone to answer.

Then there is the conversational layer. vendor/bin/phpspec pair opens a REPL with real pairing roles: the AI navigates while you drive, or you swap and it drives. vendor/bin/phpspec next reads the actual suite state and proposes the single most impactful next step, feature-first, outside-in. vendor/bin/phpspec generate turns one natural-language instruction into one artefact. vendor/bin/phpspec refactor works through a plan the model names, with the technique, the reason and the baby steps: you approve the plan, then each step as its diff is shown; the whole suite runs before the plan and after every step, a step that breaks anything is undone, and however the run stops, it stops green. It journals what it did so it never re-polishes the same class out of boredom, and under --format=agent the plan is one receipt that accept carries out. The spec decides what "correct" means; the model operates inside that boundary.

All of that is AI for the developer. It is the smaller half of the story.

PhpSpec as the harness

The half that matters more is the reverse direction: not what AI does for PhpSpec users, but what PhpSpec does for AI.

One pipeline, and the harness is data. Underneath, every AI command is the same verb: an agent resolves the current TDD step deterministically from your words and the suite state, grounds the model in the real files, and asks for exactly one artefact. Each command is described by a manifest: a text file with YAML frontmatter declaring its tools, its answer channel, its grounding, its model parameters, followed by its prose. The pairing roles are manifests too. Which means the whole harness is editable without touching code, and a project can override any prompt by dropping a file of the same name under .phpspec/prompts/: committable, team-shared, surviving composer updates. Tuning how your agent behaves is a text edit and a diff in review, not a fork.

Deterministic first. When the step fully determines the artefact, no model is consulted at all. Ask for the step definitions of a feature and the scaffolding is derived from the Gherkin, offline, in milliseconds, with titles another steps file already owns skipped rather than duplicated. The model is reserved for the part that genuinely needs imagination, and everything it proposes flows through the same generators and guards the tool has always had. Structure only ever arrives as tool calls; a model that answers in prose where an artefact was required gets exactly one corrective re-ask and then a clean failure, never a half-parsed blob.

The discipline is enforced, not requested. In pair mode the roles are mechanical, not rhetorical. A navigating AI is not asked nicely to avoid writing files; the write tools are simply not on its surface, and it channels concrete advice through a single offer per turn that you see as a diff and accept or decline. A driving AI gets one artefact per turn: the change lands through a confirm gate that shows the driver's stated plan, the suite auto-verifies immediately (the whole suite when the change touched a feature or steps file), the result is fed back to the model, and the turn winds down and hands you the keyboard. Specs cannot be rewritten destructively: content that would drop existing examples is rejected with the reason, as is anything in last decade's syntax. You do not fix the confident, sweeping change nobody asked for with a sternly worded system prompt. You fix it with a harness that will not accept it.

The session is a recording, and recordings are specs. Every exchange with a model is captured: the exact composed prompt, the tool calls, the outcome, with credentials stripped, and a whole pair conversation accumulates into a session file. Here is why that matters more than debugging: a captured session replays. PhpSpec's own AI features are specified by replaying recorded model behaviour through the real pipeline and asserting on what lands, deterministically, in CI, with no API key. When a session misbehaves in the wild, promoting it to a permanent regression fixture is one copy plus a grader. The tool eats its own philosophy: the AI layer is itself developed against executable specifications of model behaviour.

The adversarial loop. If a model writes the implementation and the same model writes the examples, you have an echo chamber wearing a green badge. Mutation testing breaks the mirror: Infection, fed by 9.0's native per-example coverage format, mutates the implementation and asks whether the suite notices. Every surviving mutant is a precise, machine-generated statement of behaviour your specs do not pin down, which makes it a work item. One process generates, another attacks, the framework referees, and the human arbitrates intent.

And because the models themselves are configuration, not architecture, the provider layer stays out of your way: Gemini, Claude, GPT, Grok, DeepSeek, or a local Ollama, with reasoning effort a one-line setting that maps to each provider's native thinking controls. Misconfigure any of it and the error names the exact key and the fix, because a harness that misdiagnoses its own configuration has no business supervising anyone else's code.

Notice what is absent from this list: nothing here asks you to adopt an exotic workflow, a new methodology with a manifesto, a YAML file of vibes. It is the BDD cycle you already know (describe, run, generate, verify), with the human's exclusive grip on the keyboard relaxed. The trend line of this industry is agents working in loops against fast feedback. I am simply arguing the loop should be the one we spent twenty years sharpening.

Better engineers, not fewer

There is a fear under all of this, and I would rather name it than talk around it: if the model writes the code, what am I for?

In classic pair programming we had names for the two chairs: the driver, hands on the keyboard, and the navigator, watching the trajectory. The driver's seat is taken now, permanently, by something that types at four hundred words a minute and never gets tired. What remains is the chair that was always harder to sit in. Navigation. Deciding what to build, naming behaviour so precisely that a machine cannot misunderstand it, decomposing a domain into objects whose responsibilities you can state in a sentence, hearing the design feedback in a spec that has become painful to write.

Those are exactly the muscles BDD builds. That was always the point; "BDD is not about testing" was the community's oldest, most-ignored slogan. The vocabulary shift from test to spec, from assert to expect, from method to behaviour, was never cosmetic. It was training your attention on intent. For twenty years that training was optional, a craft interest for people who cared. AI makes it the job description. The engineer who can specify (turn a fuzzy business conversation into a Gherkin scenario, and a scenario into a crisp object contract) now wields tools that amplify that clarity a thousandfold. The engineer who can only type is competing with the price of tokens.

AI does not lower the bar for engineering. It moves the bar to the one place the machines cannot reach on their own: knowing, and stating, what ought to be true. The developers who thrive in the next decade will be better engineers than we were, not because they hand-craft superior for-loops, but because they will have practised specification the way we practised syntax. A junior who learns to spec will out-engineer a senior who only learned to code.

What we just told you

So, the three moves, once more, now that you have seen the machinery. Code became cheap, and the engineering moved into the executable specification: the one artefact that states intent and can prove it holds. The industry's SDD instinct points the right way, and SpecBDD gives it what Markdown cannot: a spec that goes red. And PhpSpec 9.0 turns the BDD cycle into the agent's working environment: the JSON briefing you saw first, so the sensor speaks the agent's language; manifests and prompts as data, so the guides are yours to edit; roles and gates enforced mechanically, so the discipline is a property of the environment; recorded sessions replayed as specs, so even the AI layer answers to the loop; and mutation testing to keep the generator honest. The developer develops through the agent. The agent develops through the harness. The engineering survives the chain.

The cycle starts the same way it always has: describe the behaviour you want, watch it fail, make it pass in the smallest steps you can manage, and listen to the feedback as you go. The only thing that has changed is who sits at the keyboard.

composer require phpspec/phpspec:^9.0@RC --dev
vendor/bin/phpspec pair

Fifteen years ago, Konstantin and myself were throwing ideas for PhpSpec T-shirts, I am resurrecting this for the prompt: "Once you spec, you never go back." It was written for humans, but my Claude Code is hooked on it already.