# The AI Agent Complexity Ratchet: Why 90% Test Coverage Is Required
**作者**: Garry Tan
**日期**: 2026-05-12T05:03:37.000Z
**来源**: [https://x.com/garrytan/status/2054064931515855118](https://x.com/garrytan/status/2054064931515855118)
---
I've been coding with AI for the past year. Not just prompting -- building real software. Two open-source projects: GStack, which makes AI coding agents better, and GBrain, which turns everything you read and write into a searchable knowledge base your AI can use. Between them, about 970,000 lines of code and 665 test files. Pretty much all written by Claude Code and Codex at my direction (15 simultaneous Conductor sessions most of the time).
Last week I merged fourteen pull requests in 72 hours. Nearly 29,000 lines of new code. Each release was better tested than the one before it.
That's supposed to be impossible. Speed and quality are supposed to trade off. Ship fast, break things. Move slow, ship right. Pick one.
You don't have to pick anymore. The unlock is 90% test coverage -- and AI agents made it free to get there. For fifty years, that level of verification cost too much human willpower to sustain. Now the agent writes the tests alongside the code. The result is what I call the complexity ratchet: a system that can only get better, never worse.
(This is the seventh in a series about building with AI: 1 2 3 4 5 6)
## Software used to be brittle
For fifty years, the whole discipline of software engineering was organized around one idea: prevent errors, because errors are catastrophic.
You had to get the code right the first time. Miss one edge case and you crash in production. Ship a bad database migration and you lose customer data. Write a function that does something subtle, and when the one person who understands it quits, nobody knows why it works. The whole system depended on humans being careful, and humans are not careful. So we built elaborate processes -- code reviews, staging environments, QA teams, release trains -- all designed to catch mistakes before they reached users.
It kind of worked. But it was slow. And it meant that the complexity of any software system had a hard ceiling: the number of things one team could hold in their heads simultaneously.
## Now software is squishy
I don't mean sloppy. I mean resilient in a way that wasn't possible before.
When I say "the models are here," I mean that AI coding agents -- Claude, GPT, Codex, and the ecosystem growing around them -- can now read code, understand context, diagnose errors, and write fixes. Not perfectly. But well enough that the error model for software has changed.
The migration breaks? The agent reads the error message, understands the database schema history across 45 versions, writes the fix, writes the test. The file sync hangs on a million symlinks? Agent diagnoses the parser timeout, bounds it at 30 seconds, ships the fix with tests. An extraction pipeline has an attribution bug? A cross-model evaluation catches it, the prompt gets iterated, enforcement gets added at the database layer.
For most code-level errors -- logic bugs, parsing failures, broken edge cases -- agents can now diagnose and fix them in the next turn. That's genuinely new. The errors that remain catastrophic are the ones that destroy state: bad migrations on production data, security holes exploited before detection, privacy leaks that can't be un-leaked. The ratchet helps here too (good tests catch most of these before production) but the real shift is that the vast majority of errors in a codebase are the fixable kind.
This is a phase change for how software gets built. But it only works if you have the ratchet.
## The Agent Complexity Ratchet
A ratchet is a mechanism that allows motion in one direction only. A socket wrench turns a bolt forward and prevents it from turning back. That's the metaphor.
In agent-coded software, every coding session with an AI agent adds three things to the codebase:
1. Tests that encode what "correct" means -- automated checks that run every time someone changes the code, and fail loudly if the change breaks something
2. Documentation that records why decisions were made -- not just what the code does, but the reasoning and tradeoffs behind it
3. Evaluation results that establish quality thresholds -- structured assessments of output quality with scores, so you know if the next version is better or worse
The next time an agent works on the codebase, it loads all three into its context window (the text the AI can see and reason about). It can't regress below the test suite -- the tests would fail. It can't ignore the documentation -- it's right there in context. It can't ship quality below the evaluation baseline -- the scores are recorded.
The quality floor goes up with every turn. Forward-only motion. That's the ratchet.
## What this looks like in practice
I'll make this concrete. GBrain is a knowledge system I'm building -- it gives AI agents long-term memory by storing, indexing, and searching through a person's notes, meetings, conversations, and research. Think of it as a second brain that your AI assistant can actually read.
One of its features is epistemological extraction: it reads through thousands of pages and extracts who believes what, with what confidence, over time. "Garry thinks Bitcoin will hit $300K (confidence: 0.45)." "Jared thinks this startup has strong retention (confidence: 0.80)." Like that, but across 28,000 pages.
The first extraction run pulled 100,720 claims. I used a cross-model evaluation to grade the quality -- I had GPT-5.5 and Claude both independently score the output. Overall: 6.8 out of 10.
The biggest problem? Something I call holder confusion. Take the claim "AI will replace 80% of software engineers by 2027." Who holds that belief? Is it the person who wrote it? Is it someone they're quoting? Or is it the system's analysis engine, which inferred it from a podcast transcript? Version 1 got this distinction wrong 35% of the time. That matters -- if you're building a system that tracks what people believe, you need to know WHO believes it.
So the evaluation results got documented. Six specific failure modes got identified. The version 2 prompt addressed all six. Weight rounding (the confidence scores) got enforced at the database layer -- no more false precision like 0.74 when 0.75 is the honest answer. Seventeen tests locked in the contract.
Now no future version of the extraction can ship without those 17 tests passing. Nobody has to remember why weight rounding matters or what holder confusion is. The tests remember.
The quality floor went up permanently. That's one turn of the ratchet.
## Why most vibecoded projects die
"Vibecoding" is Andrej Karpathy's term for coding with AI by describing what you want in natural language and letting the model generate the code. It's powerful and it's how I build. But from what I've seen across YC applications and open-source repos, most vibecoded projects that skip tests start falling apart once they reach moderate complexity -- a few thousand lines, a handful of interacting features.
They skip the ratchet. No tests, no docs, no evals. The agent adds complexity but nothing prevents regression. Every new feature has a chance of breaking an old one, and without tests, you don't find out until a user reports it. By version 0.5 the codebase is a haunted house where every change breaks something unexpected. Then the developer writes a blog post about how AI coding doesn't work.
AI coding works fine. They just didn't build the ratchet.
You could argue that the kind of person who writes tests is also the kind who writes good architecture in the first place. Fair. But the ratchet mechanism isn't about the person -- it's about what happens on the next turn. When a new contributor opens a PR, or when a model version changes, or when you're coding at 2am and your judgment is impaired, the tests catch regressions regardless of who wrote them. The ratchet works even when the human isn't at their best. That's the point.
Without tests, improvement is a noisy process -- agents try to make things better, but without regression signals, good changes and bad changes are equally invisible. With a dense test suite, you get a ratchet on the tested surface: quality can only go up for the behaviors you've encoded. That's most of the system, not all of it. But it's enough to sustain forward motion at speed.
## Tests as institutional memory
In traditional software companies, institutional memory lives in humans. The senior engineer who knows why that caching layer exists. The architect who remembers the migration that almost destroyed the database. The tech lead who can explain the weird edge case in the billing system.
Humans leave. They retire, they get poached, they burn out. When they leave, the knowledge goes with them. Every software company has had the experience of opening a critical file and finding a comment that says // DO NOT CHANGE THIS -- ask Dave and Dave left three years ago.
The agent's context window doesn't quit. It doesn't get poached. It doesn't forget. When the test suite encodes "weight rounding must use 0.05 increments" and the documentation explains "because cross-modal eval showed false precision degrades trust in the confidence scores," that knowledge is durable. Any agent, any model, any time can load that context and understand the constraint.
Tests are institutional memory that survives employee turnover. For a one-person project, they're even more critical -- they're the only institutional memory you have.
## Everything harnessable is testable
The ratchet doesn't just work for traditional code. It works for anything a computer can observe.
Think about the layers of a modern system. The OS gives you process trees, file system state, network sockets, cron schedules. The terminal gives you every keystroke, every line of output, every interactive prompt. The browser gives you rendered pages, button states, navigation events. APIs give you structured responses you can parse and validate. And AI agents give you observable behavior -- what they say, what tools they call, what order they do things in, whether they ask before acting.
All of these are harnessable. And if you can harness it, you can observe it. If you can observe it, you can assert on it. If you can assert on it, you can ratchet it.
This is a much bigger surface area than traditional unit tests. Let me show you.
GStack is my open-source coding agent framework -- 93,000 GitHub stars, 701,000 lines of code, 46 skills. One of its core features is interactive plan review: you ask it to review your architecture, and it walks through the plan section by section, asking questions, probing edge cases, challenging your assumptions. Like having an engineering manager who actually reads the code.
The problem: Claude Code would sometimes skip the whole interactive part. It would read the plan file, dump every finding in one shot, and exit -- without asking the user a single question. The entire point of the review is the back-and-forth dialogue. Skipping it defeats the purpose.
How do you even test that? You can't unit test "did the AI have a conversation." No traditional testing framework covers this.
So I used Bun's TTY functionality to build a test harness (PR #1354) that literally spawns Claude Code inside a pseudo-terminal, feeds it a specific repo scenario, triggers the review skill, and watches the terminal output in real time. The test observes whether the agent fires an interactive question before finishing. If it dumps findings and exits without asking anything, the test fails.
That's not testing code. That's testing whether an AI agent follows a behavioral contract. At the TTY level. By literally watching it work.
The ratchet response was three layers:
1. STOP gates in the skill instructions -- explicit rules that say "you MUST ask the user before proceeding to the next section," with anti-rationalization clauses that name the specific failure mode so the model can't talk itself into skipping
2. Anti-shortcut clause -- "the plan file is the OUTPUT of the interactive review, not a substitute for it." One sentence that closes the exact loophole the model kept exploiting.
3. Gate-tier floor tests -- the TTY harness tests that spawn Claude Code in controlled scenarios and fail if the agent doesn't ask at least one interactive question
Now when Anthropic ships a new model version, or when I change a skill prompt, the test suite catches any regression in the interactive contract. The agent can't silently stop asking questions. The test watches the terminal and checks.
Or take PR #880, which shipped a new OpenClaw plugin. The test doesn't just check that the code compiles. It builds the plugin from source, spawns a real OpenClaw instance in an isolated profile, installs the plugin via the CLI, runs plugins inspect to verify the runtime loaded it, sets the config slot, validates the config, and runs plugins doctor to confirm zero diagnostics. A full end-to-end round trip across two separate programs. 359 lines of test code. The kind of test a human would almost never write by hand because the setup is too tedious. Claude wrote it in about five minutes. That's the effort wall disappearing in real time.
The principle generalizes. You can test at the OS level: did the migration create the right tables, did the cron job fire, is the process still alive? At the browser level: did the page render, did the agent fill in the form correctly. At the API level: did the model return valid JSON with the right schema. At the behavioral level: did the agent follow the protocol, did it ask before deleting, did it stop when told to stop.
The whole stack is testable. The ratchet applies to all of it. Most people haven't realized this yet because they're still thinking about test coverage as "did my function return the right number." The real test surface is everything the computer can see.
## The 90% number
So what does 90% test coverage actually buy you?
Capers Jones studied over 10,000 software projects and measured defect removal efficiency (DRE) -- the percentage of bugs caught before they reach users. His data from Applied Software Measurement shows a nonlinear curve: below 70% coverage, DRE sits around 65-75%. At 85-95% coverage, DRE jumps to 92-97%. The relationship isn't linear. There's a knee in the curve around 85% where defect escapes drop sharply.
The avionics industry figured this out decades ago. DO-178C, the FAA standard for flight-critical software, requires modified condition/decision coverage (MC/DC) for Level A systems -- the ones where a bug means a plane crash. Branch coverage alone misses 10-20% of faults. MC/DC, which is stricter than line coverage, achieves >99% DRE. They don't mandate this because bureaucrats like paperwork. They mandate it because the data showed that below certain coverage thresholds, critical defects escape at rates incompatible with not killing people.
The reliability engineering parallel is clean. Factories use a system called Six Sigma to measure quality. The idea: count how many defects you get per million units produced, then express that as a "sigma level" -- higher sigma means fewer defects. A 3-sigma process produces about 67,000 defects per million (pretty bad). A 4-sigma process produces about 6,200 (ten times better). A 5-sigma process produces 233 (another 27x better). The jump from 4 to 5 sigma is not incremental improvement. It's a phase change.
Test coverage follows the same curve. Going from 70% to 90% coverage isn't 30% better. It's an order of magnitude fewer escapes. The defects that slip through at 70% are hiding in the 30% of untested code. At 90%, the hiding places shrink to 10% and most of the dangerous paths are locked down.
Now, I should be honest about what the research also shows. Mockus, Nagappan, and Dinh-Trong studied Windows Vista and found that while coverage correlates with fewer post-release defects, the effort to reach 90%+ rises sharply. The last 20% of coverage takes disproportionately more work than the first 70%. This has been true for decades. It's why most teams stop at 70-80% and call it good enough.
But something changed: AI coding agents don't experience effort.
They don't get bored writing the fourteenth edge-case test. They don't cut corners at 5pm on a Friday. They don't look at a gnarly integration test and think "I'll come back to this later." The effort curve that stopped human teams at 70% doesn't apply to agents. You can ask Claude to write tests for every edge case in a module and it will do it cheerfully, thoroughly, at 2am, without complaining. The brutal last 20% that made 90% coverage impractical for human teams is exactly the kind of work AI agents are best at.
This is the real unlock. It's not that AI lets you write code faster. Plenty of people have noticed that. It's that AI lets you verify at a level that was previously too expensive to sustain. The 90% threshold that the data says is magical? It used to cost too much human willpower to reach. Now it's free.
That's the key distinction. The ratchet isn't about line coverage as a vanity metric. It's about tests that encode behavioral contracts -- the holder confusion test, the weight rounding test, the interactive review gate. Each test locks in a specific lesson learned. Coverage is the proxy that tells you how much of the system's behavior is under contract. At 90%, nearly every behavior change triggers a test signal. The agent either passes (safe to ship) or breaks a test (caught immediately).
The remaining 10% is integration points, infrastructure plumbing, and edge cases that are genuinely hard to test. That's fine. The 90% is what turns chaos into a ratchet.
Getting to 90% used to be a heroic effort. Now it's a Tuesday. That's the game change.
## Proof of concept
I started both projects alone. They're not solo anymore.
GStack now has 37 contributors. v1.30 incorporated 21 community PRs in a single release. GBrain has 25 contributors. v0.31.1.1 landed 22 community fixes in one PR -- auth flow, schema bootstrapping, sync, privacy.
The ratchet is what makes this safe. Every external PR has to pass the existing test suite. A new contributor doesn't need to understand the whole system. They need to make the tests pass.
Last week's GBrain releases tell the story:
- v0.31.0: a new facts table for real-time memory, plus a dream consolidation phase that promotes short-term memories into long-term knowledge
- v0.31.1: fixed 25 CLI commands that were silently routing to an empty local database instead of the user's actual brain
- v0.31.1.1: twenty-two community-reported fixes in one PR
- v0.31.2: fixed a code sync that hung forever on large repos with symlinks by adding a 30-second timeout
Each release shipped with more tests than the last. The agent writes the tests alongside the code. The coverage doesn't slip because the effort to maintain it is no longer a human burden.
## The new complexity ceiling
The complexity ceiling for software just got a lot higher.
It used to be bounded by one team's ability to hold the system in their heads. Now it's bounded by one person plus agents who can load the full codebase, schema history, test suite, and documentation into context.
That's a much bigger number. And it keeps growing as context windows get larger and models get better at reasoning about code.
Every software company that doesn't adopt this model -- agents plus taste plus a test suite that only goes up -- is already shipping slower and with less quality than one person who has.
The tools are here. The code is open. Tests are the ratchet. 90% coverage, every PR, no exceptions.
For fifty years, 90% coverage was a luxury reserved for avionics and medical devices -- teams with the budget to throw human hours at the effort wall. AI agents demolished that wall. The coverage threshold that makes software reliable is no longer expensive. It's just a setting. The question isn't whether you can afford 90%. It's whether you can afford not to.
The ratchet, the skills, and the whole knowledge system are open source and free on GitHub. Go build.
My MIT-licensed open source projects:
- GStack -- makes Claude Code dramatically better. 93K stars. Free.
- GBrain -- your second brain for AI agents. 14K stars. Free.
The AI Explainer series:
1. Fat Skills, Fat Code, Thin Harness -- the architecture
2. Resolvers -- the routing table for intelligence
3. The LOC Controversy -- what 600K lines actually produced
4. Naked Models Are Stupider -- the model is the engine, not the car
5. The Skillify Manifesto -- every workflow becomes a testable skill
6. Meta-Meta-Prompting -- compounding skills produce emergent capabilities
7. The Agent Complexity Ratchet -- you are here
## 相关链接
- [Garry Tan](https://x.com/garrytan)
- [@garrytan](https://x.com/garrytan)
- [72K](https://x.com/garrytan/status/2054064931515855118/analytics)
- [GStack](https://github.com/garrytan/gstack)
- [GBrain](https://github.com/garrytan/gbrain)
- [merged fourteen pull requests in 72 hours](https://x.com/garrytan/status/2054055071017538028)
- [GBrain](https://github.com/garrytan/gbrain)
- [Vibecoding](https://x.com/karpathy/status/1886192184808149383)
- [GStack](https://github.com/garrytan/gstack)
- [build a test harness](https://github.com/garrytan/gstack/pull/1354)
- [PR #880](https://github.com/garrytan/gbrain/pull/880)
- [Applied Software Measurement](https://www.amazon.com/Applied-Software-Measurement-Analysis-Productivity/dp/0071502440)
- [DO-178C](https://en.wikipedia.org/wiki/DO-178C)
- [Mockus, Nagappan, and Dinh-Trong](https://ieeexplore.ieee.org/document/4343756)
- [GStack](https://github.com/garrytan/gstack)
- [GBrain](https://github.com/garrytan/gbrain)
- [GStack](https://github.com/garrytan/gstack)
- [GBrain](https://github.com/garrytan/gbrain)
- [Fat Skills, Fat Code, Thin Harness](https://x.com/garrytan/status/2042925773300908103)
- [Resolvers](https://x.com/garrytan/status/2044479509874020852)
- [The LOC Controversy](https://x.com/garrytan/status/2045404377226285538)
- [Naked Models Are Stupider](https://x.com/garrytan/status/2045798603059548364)
- [The Skillify Manifesto](https://x.com/garrytan/status/2046876981711769720)
- [Meta-Meta-Prompting](https://x.com/garrytan/status/2053127519872614419)
- [Upgrade to Premium](https://x.com/i/premium_sign_up)
- [1:03 PM · May 12, 2026](https://x.com/garrytan/status/2054064931515855118)
- [72.9K Views](https://x.com/garrytan/status/2054064931515855118/analytics)
- [View quotes](https://x.com/garrytan/status/2054064931515855118/quotes)
---
*导出时间: 2026/5/12 21:13:50*
---
## 中文翻译
# AI 代理的棘轮效应：为什么 90% 的测试覆盖率是必须的
**作者**：Garry Tan
**日期**：2026-05-12T05:03:37.000Z
**来源**：[https://x.com/garrytan/status/2054064931515855118](https://x.com/garrytan/status/2054064931515855118)
---
过去一年里，我一直在用 AI 编写代码。不仅仅是生成提示词（prompting）——而是构建真正的软件。两个开源项目：GStack（让 AI 编程代理变得更强大）和 GBrain（将你阅读和编写的一切转化为 AI 可用的可搜索知识库）。这两个项目加起来大约有 97 万行代码和 665 个测试文件。这些代码几乎全是由 Claude Code 和 Codex 在我的指导下编写的（大部分时间同时运行着 15 个 Conductor 会话）。
上周，我在 72 小时内合并了十四个拉取请求（Pull Request）。新增了近 29,000 行代码。每次发布的版本都比前一个经过了更好的测试。
这本该是不可能的。速度和质量通常被视为一种权衡。要么快速发布，制造问题；要么稳步前行，保证质量。你只能二选一。
你不必再为此纠结了。解锁这一困境的关键在于 90% 的测试覆盖率——而 AI 代理让实现这一目标的成本几乎为零。在过去的五十年里，这种级别的验证需要消耗太多的人力毅力，难以维持。现在，代理会与编写代码的同时编写测试。其结果就是我所说的“复杂度棘轮”：一个只会变得更好、绝不会变得更差的系统。
（这是关于使用 AI 进行构建的系列文章中的第七篇：1 2 3 4 5 6）
## 过去的软件是脆弱的
五十年以来，整个软件工程学科都是围绕一个理念组织的：防止错误，因为错误是灾难性的。
你必须一次性把代码写对。漏掉一个边缘情况，生产环境就会崩溃。发布一次糟糕的数据库迁移，你就会丢失客户数据。写一个带有微妙逻辑的函数，当唯一理解它的人离职后，就没人知道它为什么能工作了。整个系统依赖于人类的谨慎，但人类并不谨慎。所以我们建立了繁杂的流程——代码审查、预发布环境、QA 团队、发布列车——所有这些都是为了在错误到达用户之前将其捕获。
这种方法某种程度上奏效了。但这很慢。这也意味着任何软件系统的复杂度都有一个硬性上限：即一个团队能同时掌握在脑海中的事物数量。
## 现在的软件是柔性的
我不是指“马虎”。我是指以前无法实现的那种韧性。
当我说“模型已经到来”时，我的意思是 AI 编程代理——Claude、GPT、Codex 以及围绕它们不断发展的生态系统——现在可以阅读代码、理解上下文、诊断错误并编写修复补丁。虽然不完美，但已经足够好，以至于软件的错误模型已经发生了改变。
迁移出错了？代理会读取错误信息，理解跨越 45 个版本的数据库模式历史，编写修复代码，并编写测试。文件同步在百万个符号链接上卡住了？代理会诊断解析器超时问题，将其限制在 30 秒，并附带测试发布修复。提取管道存在归属错误？跨模型评估会捕获它，提示词得到迭代，在数据库层增加了强制执行机制。
对于大多数代码级别的错误——逻辑 Bug、解析失败、边缘情况损坏——代理现在可以在下一轮中诊断并修复它们。这确实是前所未有的。剩下的真正灾难性错误是那些会破坏状态的错误：生产数据上的糟糕迁移、在被发现前被利用的安全漏洞、无法撤回的隐私泄露。棘轮在这里也有帮助（好的测试能在生产前捕获大部分此类问题），但真正的转变在于，代码库中绝大多数错误都是可修复的类型。
这是软件构建方式的一次相变。但这只有在拥有“棘轮”的情况下才有效。
## 代理复杂度棘轮
棘轮是一种只允许单向运动的机制。套筒扳手向前拧动螺栓，并防止其倒转。这就是其隐喻。
在由代理编写代码的软件中，每一次与 AI 代理的编码会话都会为代码库增加三样东西：
1. **编码了“正确”定义的测试**——自动化检查，每当有人更改代码时运行，如果更改破坏了某些东西，就会大声报错。
2. **记录了决策原因的文档**——不仅是代码做什么，还有背后的推理和权衡。
3. **确立质量阈值的评估结果**——对输出质量的结构化评估并带有评分，这样你就知道下一个版本是更好还是更差。
下一次代理处理该代码库时，它会将这三者全部加载到其上下文窗口（AI 可以看到并推理的文本）中。它的质量不能低于测试套件的标准——否则测试会失败。它无法忽略文档——因为文档就在上下文中。它不能发布低于评估基线的质量——因为分数已被记录。
质量底线随着每一轮提升。只能向前运动。这就是棘轮。
## 实际应用中的样子
我来具体说明一下。GBrain 是我正在构建的一个知识系统——它通过存储、索引和搜索一个人的笔记、会议、对话和研究，为 AI 代理提供长期记忆。你可以把它看作是你的 AI 助手真正能阅读的“第二大脑”。
它的一个功能是认识论提取：它会通读数千页内容，提取出是谁相信什么，随着时间的推移置信度如何。“Garry 认为比特币会达到 30 万美元（置信度：0.45）。”“Jared 认为这家初创公司有很强的留存率（置信度：0.80）。”就像这样，但是跨越 28,000 页。
第一次提取运行提取了 100,720 条主张。我使用跨模型评估来对质量进行打分——我让 GPT-5.5 和 Claude 独立对输出进行评分。总体得分：6.8/10。
最大的问题？是我称之为“持有者混淆”的问题。以“到 2027 年，AI 将取代 80% 的软件工程师”这一主张为例。谁持有这种信念？是写这句话的人吗？是他们引用的某个人吗？还是系统的分析引擎从播客逐字稿中推断出来的？版本 1 在 35% 的时间里都搞错了这种区别。这很重要——如果你正在构建一个追踪人们信仰的系统，你需要知道是**谁**相信它。
因此，评估结果被记录下来。确定了六种具体的失败模式。版本 2 的提示词解决了所有这六种模式。权重取整（置信度分数）在数据库层得到了强制执行——不再有像 0.74 这样的虚假精度，而 0.75 才是诚实的答案。17 个测试锁定了契约。
现在，未来的任何提取版本如果不通过这 17 个测试就无法发布。没有人需要记住为什么权重取整很重要，或者什么是持有者混淆。测试会记住。
质量底线永久地提高了。这就是棘轮转动了一圈。
## 为什么大多数“氛围编程”项目会死掉
“氛围编程”是 Andrej Karpathy 提出的术语，指通过自然语言描述你想要什么，并让模型生成代码的 AI 编程方式。它很强大，也是我的构建方式。但根据我从 YC 申请和开源仓库中看到的情况，大多数跳过测试的氛围编程项目一旦达到中等复杂度（几千行代码、几个相互交互的功能）就会开始分崩离析。
他们跳过了棘轮。没有测试，没有文档，没有评估。代理增加了复杂度，但没有什么能阻止回退。每个新功能都有破坏旧功能的机会，而没有测试，直到用户报告你才会发现。到了 0.5 版本，代码库就像一个鬼屋，每次改动都会破坏意想不到的东西。然后开发者写一篇博文，说 AI 编程不管用。
AI 编程完全管用。他们只是没有构建棘轮。
你可能会争辩说，会写测试的那种人，原本也就是会首先写出好架构的那种人。公平。但棘轮机制不在于人——而在于下一轮会发生什么。当新的贡献者打开 PR，或者模型版本发生变化，或者你在凌晨 2 点编程且判断力下降时，无论测试是谁写的，它们都能捕获回退。即使人类不在最佳状态，棘轮依然有效。这才是重点。
没有测试，改进是一个充满噪音的过程——代理试图让事情变得更好，但由于没有回退信号，好的改动和坏的改动同样不可见。有了密集的测试套件，你就在被测试的表面上获得了一个棘轮：对于你已经编码的行为，质量只能上升。这涵盖了系统的大部分，但不是全部。但这足以在高速的情况下维持向前的运动。
## 测试即制度记忆
在传统的软件公司中，制度记忆存在于人脑中。知道为什么存在那个缓存层的高级工程师。记得那次几乎毁掉数据库的迁移的架构师。能解释计费系统中奇怪的边缘情况的技术负责人。
人会离开。他们退休、被挖走、职业倦怠。当他们离开时，知识也随之带走。每家软件公司都有过这样的经历：打开一个关键文件，发现一条注释写着 // 不要修改这个——去问 Dave，而 Dave 三年前就走了。
代理的上下文窗口不会辞职。它不会被挖走。它不会忘记。当测试套件编码了“权重取整必须使用 0.05 的增量”并且文档解释了“因为跨模态评估表明虚假精度会降低对置信度分数的信任”时，这种知识是持久的。任何代理、任何模型、任何时候都可以加载该上下文并理解该约束。
测试是能经受住员工流动的制度记忆。对于个人项目来说，它们甚至更关键——它们是你拥有的唯一的制度记忆。
## 一切可被观察的皆是可测试的
棘轮不仅适用于传统代码。它适用于计算机能观察到的任何事物。
考虑一下现代系统的各个层级。操作系统为你提供进程树、文件系统状态、网络套接字、cron 计划。终端为你提供每一次击键、每一行输出、每一个交互式提示。浏览器为你提供渲染的页面、按钮状态、导航事件。API 提供你可以解析和验证的结构化响应。而 AI 代理为你提供可观察的行为——它们说什么、调用什么工具、按什么顺序做事、是否在行动前询问。
所有这些都是可被利用的。如果你可以利用它，你就能观察它。如果你能观察它，你就能对它进行断言。如果你能对它进行断言，你就能给它装上棘轮。
这比传统的单元测试的表面积要大得多。让我展示给你看。
GStack 是我的开源编程代理框架——9.3 万个 GitHub 星标，70.1 万行代码，46 个技能。其核心功能之一是交互式计划审查：你让它审查你的架构，它会逐节浏览计划，提出问题、探索边缘情况、挑战你的假设。就像有一个真正阅读代码的工程经理一样。
问题是：Claude Code 有时会跳过整个交互部分。它会读取计划文件，一次性抛出所有发现，然后退出——而没有问用户任何问题。审查的全部意义在于来回的对话。跳过它就违背了初衷。
你甚至该如何测试那个？你无法单元测试“AI 是否进行了对话”。没有传统的测试框架能涵盖这一点。
所以我使用了 Bun 的 TTY 功能构建了一个测试工具（PR #1354），它实际上是在伪终端中生成 Claude Code，向其提供特定的仓库场景，触发审查技能，并实时观察终端输出。测试会观察代理是否在完成之前发出了一个交互式问题。如果它抛出发现并在没有询问任何东西的情况下退出，测试就会失败。
这不是在测试代码。这是在测试 AI 代理是否遵循了行为契约。在 TTY 层面上。通过字面意思观察它工作。
棘轮的响应分为三层：
1. 技能指令中的“停止门”——明确的规则，写着“在进入下一节之前你必须询问用户”，并带有反合理化条款，命名了特定的失败模式，以便模型无法自圆其说地跳过。
2. 反捷径条款——“计划文件是交互式审查的输出，而不是它的替代品。”一句话，恰好关闭了模型一直在利用的确切漏洞。
3. 门级底层测试——那些 TTY 工具测试，在受控场景下生成 Claude Code，如果代理没有至少询问一个交互式问题，测试就会失败。
现在，当 Anthropic 发布一个新的模型版本，或者当我更改一个技能提示词时，测试套件会捕获交互契约中的任何回退。代理无法悄无声息地停止提问。测试会监视终端并检查。
或者以 PR #880 为例，它发布了一个新的 OpenClaw 插件。测试不仅检查代码是否编译。它从源代码构建插件，在隔离的配置文件中生成一个真正的 OpenClaw 实例，通过 CLI 安装插件，运行插件检查以验证运行时加载了它，设置配置槽，验证配置，并运行插件诊断以确认零诊断错误。一个跨越两个独立程序的完整端到端往返。359 行测试代码。这是人类几乎永远不会手动编写的测试，因为设置太繁琐了。Claude 在大约五分钟内写好了。这就是努力之墙在实时消失。
这个原则是通用的。你可以在操作系统级别测试：迁移是否创建了正确的表，cron 作业是否触发，进程是否仍然存活？在浏览器级别：页面是否渲染，代理是否正确填写了表单。在 API 级别：模型是否返回了具有正确模式的有效 JSON。在行为级别：代理是否遵循了协议，删除前是否询问，被告知停止时是否停止。
整个堆栈都是可测试的。棘轮适用于所有这些。大多数人还没有意识到这一点，因为他们仍然认为测试覆盖率是“我的函数是否返回了正确的数字”。真正的测试表面是计算机能看到的一切。
## 90% 这个数字
那么，90% 的测试覆盖率到底能给你带来什么？
Capers Jones 研究了超过 10,000 个软件项目，并测量了缺陷移除效率（DRE）——即在到达用户之前捕获的 Bug 百分比。他来自《应用软件测量》的数据显示了一条非线性曲线：低于 70% 的覆盖率时，DRE 大约在 65-75% 之间。在 85-95% 的覆盖率时，DRE 跃升至 92-97%。这种关系不是线性的。在 85% 左右曲线有一个拐点，缺陷逃逸率急剧下降。
航空业几十年前就搞清楚了这一点。DO-178C 是 FAA 对飞行关键软件的标准，它要求 A 级系统（那些 Bug 意味着坠机的系统）必须满足修改条件/决策覆盖率（MC/DC）。仅分支覆盖率会漏掉 10-20% 的故障。MC/DC 比行覆盖率更严格，能达到 >99% 的 DRE。他们强制要求这一点不是因为官僚喜欢文书工作。他们强制要求是因为数据显示，低于某些覆盖率阈值，关键缺陷的逃逸率与不伤人这一要求是不相容的。
可靠性工程的类比很清晰。工厂使用一种称为六西格码的系统来衡量质量。其理念是：计算每生产一百万个单位有多少缺陷，然后将其表示为“西格玛水平”——西格玛越高意味着缺陷越少。3 西格玛的过程每百万大约产生 67,000 个缺陷（相当糟糕）。4 西格玛的过程产生大约 6,200 个（好十倍）。5 西格玛的过程产生 233 个（又好了 27 倍）。从 4 西格码到 5 西格码的飞跃不是渐进的改进。它是相变。
测试覆盖率遵循同样的曲线。将覆盖率从 70% 提高到 90% 并不是好了 30%。逃逸的缺陷数量少了一个数量级。在 70% 时漏掉的缺陷隐藏在 30% 的未测试代码中。在 90% 时，藏身之处缩小到 10%，且大多数危险的路径都被锁定了。
现在，我应该诚实地说明