01 · The question
Is an AGENTS.md file instructions, or decoration?
I ran a personal experiment on my own time. Every run got the same prompt: a todo app with CRUD, a markdown editor with preview, search with match highlighting, and coloured tags with a tag filter. Every run started from the same empty template: React 19, Vite 7, Tailwind 4, shadcn/ui, TypeScript, localStorage, no backend. The only thing that changed between runs was the instructions file.
02 · The setup
Three rounds between 26 September and 7 October 2025, 14 runs in total.
The variables:
- AGENTS.md present or absent (3 baselines had none)
- the same rules written as markdown or as plain text, with every heading and bullet stripped
- the editor and agent running it: Zed, VS Code and Claude
I recorded the folder structure each run produced and timed three tasks by hand in a RUN.md.
03 · Structure moved first
With no instructions, every run produced a flat src/components/.
With an AGENTS.md, runs moved to a feature-based layout like src/features/todos, src/features/tags and src/features/search, with documentation next to the code. The file did steer the agent, but mostly on where things go, not on how well they work.
04 · Plain text beat markdown
The most surprising result: the same content without formatting was followed most closely.
The plain-text version gave every feature its own KNOWLEDGE.md and pulled search logic into a useSearchFilter hook. The markdown version, with identical rules, wrote one KNOWLEDGE.md for todos and stopped. My best guess is that headings and bullets read as a document to skim, while sentences read as instructions to follow.
05 · Tests only when it is a rule
Only one variant added tests without being asked in the prompt.
The monolithic AGENTS.md, which spelled out testing as a requirement, produced a Vitest suite. Its README claims 14 of 14 passing; I didn't re-run that count. One Claude run went the other way and built only a priority feature nobody asked for. Scope drift is real even with good instructions.
06 · What it doesn't prove
This is a direction, not a benchmark.
One prompt, one run per variant, timing by hand, results judged by reading the output instead of a scoring rubric, and the model versions weren't recorded. A proper version would repeat each variant several times and score the output against the spec.
07 · What I kept
The per-feature KNOWLEDGE.md survived the experiment.
It became the .docs/knowledge convention I now use in real projects, and I write agent instructions as plain sentences with hard rules stated as rules.
