Experiments

You’re already dangerous with these tools. Here’s what actually works.

You use Claude Code, Cursor, Codex, and Lovable. You change the settings and push forward. I run the experiments, show you the receipts, and hand you the loop I use, for free.

Episode 1 result boards. Clipped names: 0 of 5 working features on run 1 and 0 of 5 on run 2. Full names: 2 of 5 on run 1 and 4 of 5 on run 2.
Episode 1: the same pet-tracker task on clipped names and full names, two runs each.

Episode 1

The cheap run

The short version

Clipped names cost about 30% less per AI run and delivered 0 of 5 working features, twice. Full names delivered 2 of 5, then 4 of 5.

The numbers

  • $0.44per run, clipped names
  • $0.62per run, full names
  • 0 and 0working features of 5, clipped names
  • 2 and 4working features of 5, full names

The clipped version also read about 40% less code, 61% as much as the full-names version. Average costs were $0.4378 and $0.6246 a run.

In both clipped runs the pet list never showed up: run 1 rendered empty and run 2 returned an HTTP 500.

A cheap run is not a cheap feature

You have heard “keep it short, tokens cost money.” The cheaper run saved thirty percent and delivered nothing you could use. The full-names run cost a little more and got working features. Measure what ships.

How we ran it

One task, a pet-tracker app. Two runs per version, the same model. The full-names version also had its naming rules written into its instructions.

How the code was checked

A bigger model reviewed each run’s built code task by task against one rubric. It read the code and did not run the app.

Where the evidence lives

The reviewer’s notes and both branches are public in the aed-conventions repository under evidence/buildoff_2026-08-11. The naming rules behind the full-names version are in the AED book.

Free

The plan, execute, reflect kit

The loop I use on every experiment: plan the work, run it, then reflect on what the run proved before the next one. It is free and MIT licensed, works with Claude Code and Codex, installs in about 10 minutes, and comes with four sample business tasks to try it on.

Free kit, version 0.1.0

The release page has the download and its SHA-256 checksums.

Get the free kit

Want the receipts on your own project?

Bring the tools you already use. We run the experiment on your codebase, keep the numbers, and set up the loop so it keeps working after we leave.

Building on a Mac? I made Scribe Pro for talking to your agents.