# How to Build AI Loops That Are Replacing Prompt Engineering
**作者**: Lummox
**日期**: 2026-10-16T14:23:47.000Z
**来源**: [https://x.com/Lummox_eth/status/2111100800625787154](https://x.com/Lummox_eth/status/2111100800625787154)
---
For two years, getting better results from AI meant writing better prompts.
Longer instructions, smarter system messages, increasingly complicated frameworks. Engineers spent hours figuring out which words made an agent perform better.
That approach is starting to lose its advantage.
Andrej Karpathy's autonomous research system ran roughly 700 experiments in two days and discovered around 20 improvements. Shopify's CEO reportedly used a similar approach to improve a smaller model by 19%. Boris Cherny, the creator of Claude Code, said he had gone eight months without writing code by hand. [philschmid.de](https://www.philschmid.de/autoresearch?utm_source=chatgpt.com)
The common thread is not a better prompt.
It is a loop.
A system that gives AI a goal, lets it work, checks the result, and keeps going until the objective is reached.
Here is how these loops work, why they are becoming so powerful, and how to build your first one.
## 1. The engineer who ran 700 experiments
In March 2026, Andrej Karpathy released [autoresearch](https://github.com/karpathy/autoresearch), a small open-source system designed to automate machine learning experiments.
The architecture was surprisingly simple.
Three important files controlled the process.
[train.py](//train.py) contained the training code the agent could modify.
[prepare.py](//prepare.py) handled data preparation and evaluation.
program.md described the objective, constraints, and experimental rules.
The agent had one job: improve model performance without changing the evaluation system.
It would inspect the code, propose a modification, train for five minutes, compare the result against the previous experiment, and decide whether to keep the change.
Then it started again.
Over two days, the system reportedly completed around 700 experiments and identified roughly 20 improvements. The combined changes reduced a time-to-GPT-2 benchmark from 2.02 to 1.80 hours.
One discovery involved a missing scaling factor in the attention mechanism, something that ordinary syntax checks would never flag.
The significance was not that AI suddenly became a better researcher than Karpathy.
It was that an agent could investigate hundreds of measurable hypotheses without requiring someone to approve each experiment.
Humans get tired after dozens of unsuccessful attempts.
A properly controlled loop can keep exploring as long as its budget allows.
## 2. A loop is not a prompt running every hour
Most people confuse automation with autonomous iteration.
They schedule a prompt, receive an answer, and call it an agent.
But running the same instruction repeatedly does not create a useful feedback system.
A real loop has five stages:
Discover → Plan → Execute → Verify → Iterate
The agent examines the problem, chooses an approach, performs the work, evaluates the outcome, and uses that evidence to determine what happens next.
Three components make the difference.
Verification: The result must pass an objective test. A compiler, benchmark, test suite, or deterministic validation script.
Memory: The system records what was attempted, what failed, and which changes produced improvements.
Stopping conditions: The loop ends when the objective is achieved or a hard limit is reached.
Without verification, the agent can confidently repeat mistakes.
Without memory, it keeps rediscovering the same failures.
Without stopping conditions, it can consume resources indefinitely.
A loop is not useful because it runs without supervision.
It is useful because its progress can be measured without constant supervision.
## 3. The self-healing GitHub repository
Now imagine applying the same architecture to ordinary software development.
A developer pushes code to GitHub.
The CI pipeline detects a failing test.
Instead of waiting for someone to inspect the failure manually, an automated workflow sends the logs and relevant repository context to Claude Code or Codex.
The agent investigates the error, creates a candidate fix, and runs the tests again.
If verification fails, it examines the new output and tries another approach.
If the tests pass, it opens a pull request containing the proposed correction and supporting evidence.
A human reviews the change before merging.
This is a practical example of a self-healing repository.
The workflow does not require an enormous multi-agent system.
It needs an event trigger, a coding agent, a reproducible environment, persistent state, and an independent verifier.
The same pattern can handle dependency updates, formatting violations, reproducible bugs, and routine maintenance.
The important distinction is that the agent does not decide whether its work is successful.
The repository's verification system does.
## 4. Build your first AI loop with three files
The easiest mistake is starting with ten agents, multiple orchestration frameworks, and an expensive monitoring dashboard.
Start smaller.
You can organize a basic coding loop around three files.
AGENTS.md — Project instructions
Define the objective, repository conventions, permitted operations, and restrictions.
For Claude Code, the equivalent project-specific instructions can live in CLAUDE.md.
[loop.py](//loop.py) — The controller
This script invokes the coding agent, manages retries, records progress, and enforces execution limits.
[verify.sh](//verify.sh) — The verification gate
This file contains the commands that determine whether the required work has succeeded.
A simplified controller looks like this:
MAX_ATTEMPTS = 10for attempt in range(MAX_ATTEMPTS): [agent.run](//agent.run)(goal="Fix failing tests") result = verify() if result.passed: print("Objective completed") break save_feedback(result)
This is conceptual code. A working implementation needs an agent integration, error handling, rollback support, timeouts, and persistent state.
But the architecture is already there.
The agent produces a candidate.
The verifier evaluates it.
The feedback informs another attempt.
The controller determines when to stop.
Once one supervised execution reliably produces acceptable results, the process can be connected to a schedule or GitHub event.
Get one loop working before trying to build an entire autonomous engineering team.
## 5. Why the verifier matters more than another agent
There is a common misconception in AI engineering.
When one model makes mistakes, add another model to review its output.
Sometimes independent reviewers help.
But two AI agents agreeing does not mean their conclusion is correct.
Imagine an agent optimizing a Python function.
It rewrites the implementation and claims a 30% performance improvement.
A reviewer examines the code and agrees that the optimization looks reasonable.
Neither opinion proves anything.
The system needs to run correctness tests and reproducible benchmarks.
If the new function produces incorrect results, reject it.
If performance decreases, reject it.
If correctness is preserved and the measured improvement is real, keep the change.
This is the principle behind Karpathy's experiments.
The agent generates possibilities.
An external evaluation determines which possibilities survive.
Better models can improve the quality of proposed solutions.
Better verification improves the reliability of the entire system.
## 6. The two problems that can destroy an autonomous loop
The first problem is uncontrolled execution.
An agent might attempt dozens of variations without making meaningful progress.
Every iteration consumes tokens, compute, and sometimes paid infrastructure.
A loop needs hard budgets, maximum attempt counts, and a way to detect repeated failures.
The second problem is more subtle.
Comprehension debt.
An autonomous coding system can generate changes faster than its maintainers can understand them.
Tests pass. Pull requests merge. Development appears faster.
But the gap between what the repository contains and what the engineering team understands keeps growing.
Eventually, an unexpected production failure exposes that gap.
Passing tests cannot guarantee maintainability, architectural quality, or complete security.
That is why autonomous loops should initially operate on narrow, verifiable tasks.
Authentication, payments, critical infrastructure, and irreversible operations still require stronger controls and human judgment.
The objective is not to eliminate engineers from the process.
It is to eliminate repetitive work while preserving engineering responsibility.
## 7. The metric that actually matters
The number of agents running is not a productivity metric.
Neither is the number of prompts executed, tokens consumed, or pull requests opened.
The useful measure is cost per accepted result.
How much time, compute, and human review did the system require to produce one verified, useful change?
A loop that generates fifty patches and gets two accepted may be less efficient than one producing five carefully verified improvements.
This is why the best starting point is rarely an ambitious autonomous development platform.
It is one annoying recurring task with an objective definition of success.
Make that task reliable.
Add memory.
Enforce a budget.
Automate execution.
Then expand.
Karpathy's research experiments illustrate what happens when AI can repeatedly test its own proposals against meaningful evidence.
The same principle can transform ordinary software development.
Prompt engineering still matters, but the competitive advantage is moving toward something larger.
Designing the environment.
Defining success.
Building verification.
Creating systems that can continue making progress without requiring another human instruction every few minutes.
A prompt tells an agent what to do.
A loop gives it a way to discover whether it actually worked.
The engineers gaining the biggest advantage are not necessarily using smarter models.
They are building better systems around the models they already have.
And while everyone else is still rewriting prompts, those systems can keep experimenting, correcting mistakes, and producing verified work long after the conversation ends.
## 相关链接
- [Lummox](https://x.com/Lummox_eth)
- [@Lummox_eth](https://x.com/Lummox_eth)
- [101K](https://x.com/Lummox_eth/status/2111100800625787154/analytics)
- [autoresearch](https://github.com/karpathy/autoresearch)
- [train.py](https://train.py/)
- [prepare.py](https://prepare.py/)
- [loop.py](https://loop.py/)
- [verify.sh](https://verify.sh/)
- [agent.run](https://agent.run/)
- [Upgrade to Plus](https://x.com/i/premium_sign_up)
- [10:23 PM · Oct 16, 2026](https://x.com/Lummox_eth/status/2111100800625787154)
- [101.1K Views](https://x.com/Lummox_eth/status/2111100800625787154/analytics)
- [View quotes](https://x.com/Lummox_eth/status/2111100800625787154/quotes)
---
*导出时间: 2026/10/21 09:20:34*
---
## 中文翻译
# 如何构建正在取代提示词工程的 AI 循环
**作者**： Lummox
**日期**： 2026-10-16T14:23:47.000Z
**来源**： [https://x.com/Lummox_eth/status/2111100800625787154](https://x.com/Lummox_eth/status/2111100800625787154)
**阅读**: 101K
**点赞**： 92
**转发**： 22
**回复**： 4
---
两年来，从 AI 获得更好结果的方法一直是编写更好的提示词。
更长的指令、更聪明的系统消息、日益复杂的框架。工程师们花费数小时琢磨哪些措辞能让智能体表现更好。
这种方法的正开始失去优势。
Andrej Karpathy 的自主研究系统在两天内运行了大约 700 次实验，发现了约 20 项改进。据报道，Shopify 的 CEO 用类似方法将一个较小模型的性能提升了 19%。Claude Code 的创造者 Boris Cherny 表示，他已经八个月没有手写代码了。[philschmid.de](https://www.philschmid.de/autoresearch?utm_source=chatgpt.com)
共同点不是更好的提示词。
而是循环。
一个给 AI 设定目标、让它工作、检查结果并持续运行直到达成目标的系统。
下面介绍这些循环是如何运作的，为什么它们正变得如此强大，以及如何构建你的第一个循环。
## 1. 运行了 700 次实验的工程师
2026 年 3 月，Andrej Karpathy 发布了 [autoresearch](https://github.com/karpathy/autoresearch)，一个旨在自动化机器学习实验的小型开源系统。
其架构出奇地简单。
三个关键文件控制着整个流程。
[train.py](//train.py) 包含智能体可以修改的训练代码。
[prepare.py](//prepare.py) 负责数据准备和评估。
program.md 描述了目标、约束和实验规则。
智能体只有一个任务：在不改变评估系统的前提下提升模型性能。
它会检查代码、提出修改、训练五分钟、将结果与上一次实验对比，然后决定是否保留这次修改。
然后再次开始。
据报道，在两天内，该系统完成了约 700 次实验，并识别出约 20 项改进。这些改动综合起来，将一个“达到 GPT-2 水平”的基准时间从 2.02 小时缩短到 1.80 小时。
其中一项发现是注意力机制中一个缺失的缩放因子，这是普通的语法检查永远不会标记的问题。
其意义不在于 AI 突然变成了比 Karpathy 更优秀的研究者。
而在于智能体能够自主调查数百个可度量的假设，无需有人逐一批准每个实验。
人类在几十次失败的尝试后就会疲惫。
而一个控制得当的循环，只要预算允许，就能一直探索下去。
## 2. 循环不是每小时运行一次的提示词
大多数人把自动化与自主迭代混为一谈。
他们定时运行一个提示词，收到一个答案，就称之为智能体。
但反复运行相同的指令并不能构成有用的反馈系统。
真正的循环有五个阶段：
发现 → 规划 → 执行 → 验证 → 迭代
智能体审视问题、选择方法、执行工作、评估结果，并利用这些证据决定下一步做什么。
三个组件决定了成败。
验证：结果必须通过客观测试。可以是编译器、基准测试、测试套件或确定性的验证脚本。
记忆：系统记录尝试过什么、什么失败了、哪些改动带来了改进。
停止条件：当目标达成或达到硬性限制时，循环结束。
没有验证，智能体会自信地重复错误。
没有记忆，它会不断重新发现同样的失败。
没有停止条件，它会无限消耗资源。
循环之所以有用，不是因为它能在无人监督下运行。
而是因为它的进展可以在无需持续监督的情况下被衡量。
## 3. 自愈式 GitHub 仓库
现在想象把同样的架构应用到普通的软件开发中。
开发者将代码推送到 GitHub。
CI 流水线检测到测试失败。
与其等待有人手动排查失败原因，自动化工作流会将日志和相关仓库上下文发送给 Claude Code 或 Codex。
智能体调查错误、创建候选修复方案，并再次运行测试。
如果验证失败，它会检查新的输出并尝试另一种方法。
如果测试通过，它会发起一个包含所提议修复及支持证据的拉取请求。
在合并之前，由人类审查这次改动。
这就是一个自愈式仓库的实际示例。
这样的工作流不需要庞大的多智能体系统。
它需要事件触发器、编码智能体、可复现的环境、持久化状态，以及一个独立的验证器。
同样的模式可以处理依赖更新、格式违规、可复现的 bug 和日常维护。
关键区别在于：智能体不决定自己的工作是否成功。
仓库的验证系统来决定。
## 4. 用三个文件构建你的第一个 AI 循环
最容易犯的错误是一上来就用十个智能体、多个编排框架和一个昂贵的监控仪表盘。
从更小开始。
你可以围绕三个文件组织一个基础的编码循环。
AGENTS.md — 项目指令
定义目标、仓库规范、允许的操作和限制。
对于 Claude Code，等效的项目专属指令可以放在 CLAUDE.md 中。
[loop.py](//loop.py) — 控制器
这个脚本调用编码智能体、管理重试、记录进度，并强制执行运行限制。
[verify.sh](//verify.sh) — 验证门
这个文件包含判断所需工作是否成功的命令。
一个简化的控制器如下：
MAX_ATTEMPTS = 10for attempt in range(MAX_ATTEMPTS): [agent.run](//agent.run)(goal="Fix failing tests") result = verify() if result.passed: print("Objective completed") break save_feedback(result)
这是概念性代码。可运行的实现还需要智能体集成、错误处理、回滚支持、超时机制和持久化状态。
但架构已经在这里了。
智能体产出候选方案。
验证器评估它。
反馈用于下一次尝试。
控制器决定何时停止。
一旦一次受监督的执行能可靠地产生可接受的结果，就可以把这个流程接入定时任务或 GitHub 事件。
先让一个循环跑通，再尝试构建整个自主工程团队。
## 5. 为什么验证器比再加一个智能体更重要
AI 工程领域有一个常见的误解。
当一个模型犯错时，就加另一个模型来审查它的输出。
有时独立审查者确实有帮助。
但两个 AI 智能体达成一致，并不意味着它们的结论就是正确的。
想象一个智能体在优化一个 Python 函数。
它重写了实现，并声称带来了 30% 的性能提升。
一个审查者检查代码后，同意这个优化看起来合理。
两种意见都不能证明任何东西。
系统需要运行正确性测试和可复现的基准测试。
如果新函数产生了错误的结果，拒绝它。
如果性能下降了，拒绝它。
如果正确性得到保留，且测得的提升是真实的，就保留这个改动。
这就是 Karpathy 实验背后的原则。
智能体生成可能性。
外部评估决定哪些可能性得以存活。
更好的模型可以提升候选解决方案的质量。
更好的验证可以提升整个系统的可靠性。
## 6. 两个可能摧毁自主循环的问题
第一个问题是不受控制的执行。
智能体可能尝试几十种变体，却没有取得有意义的进展。
每次迭代都消耗令牌、算力，有时还有付费的基础设施。
循环需要硬性预算、最大尝试次数，以及检测重复失败的方法。
第二个问题更为微妙。
理解债务。
自主编码系统生成改动的速度，可能超过维护者理解它们的速度。
测试通过。拉取请求合并。开发看起来更快了。
但仓库所包含的内容与工程团队所理解的内容之间的差距在不断扩大。
最终，一次意外的生产事故会暴露这个差距。
通过测试并不能保证可维护性、架构质量或完整的安全性。
这就是为什么自主循环最初应该在狭窄、可验证的任务上运行。
身份验证、支付、关键基础设施和不可逆操作仍然需要更强的控制与人类判断。
目标不是把工程师从流程中 elimination 掉。
而是在保留工程责任的同时，eliminate 重复性工作。
## 7. 真正重要的指标
运行中的智能体数量不是生产力指标。
执行的提示词数量、消耗的令牌数量、发起的拉取请求数量，也都不是。
有用的衡量标准是每个被接受结果的成本。
系统需要多少时间、算力和人工审查，才能产出一个经过验证的有用改动？
一个生成五十个补丁却只有两个被接受的循环，可能比一个产出五项经过仔细验证的改进的循环效率更低。
这就是为什么最佳起点很少是一个宏大的自主开发平台。
而是一个令人烦扰的、成功标准可以客观定义的周期性任务。
让这个任务变得可靠。
加入记忆。
强制执行预算。
自动化执行。
然后再扩展。
Karpathy 的研究实验展示了当 AI 能够反复用自己的提议对照有意义的证据进行检验时，会发生什么。
同样的原则可以改变普通的软件开发。
提示词工程仍然重要，但竞争优势正转向更大的东西。
设计环境。
定义成功。
构建验证。
创建无需每隔几分钟就下一条人类指令也能持续取得进展的系统。
提示词告诉智能体该做什么。
循环则给了它一种方法，去发现它是否真的做成了。
获得最大优势的工程师，未必是在使用更聪明的模型。
他们是在围绕已有的模型构建更好的系统。
当其他人还在重写提示词时，这些系统能在对话结束之后，长久地继续实验、纠正错误、产出经过验证的工作。
## 相关链接
- [Lummox](https://x.com/Lummox_eth)
- [@Lummox_eth](https://x.com/Lummox_eth)
- [101K](https://x.com/Lummox_eth/status/2111100800625787154/analytics)
- [autoresearch](https://github.com/karpathy/autoresearch)
- [train.py](https://train.py/)
- [prepare.py](https://prepare.py/)
- [loop.py](https://loop.py/)
- [verify.sh](https://verify.sh/)
- [agent.run](https://agent.run/)
- [Upgrade to Plus](https://x.com/i/premium_sign_up)
- [10:23 PM · Oct 16, 2026](https://x.com/Lummox_eth/status/2111100800625787154)
- [101.1K Views](https://x.com/Lummox_eth/status/2111100800625787154/analytics)
- [View quotes](https://x.com/Lummox_eth/status/2111100800625787154/quotes)
---
*导出时间: 2026/10/21 09:20:34* 
