# SPEEDRUNNING THE PROGRESS
**作者**: Mihura
**日期**: 2026-07-22T09:33:05.000Z
**来源**: [https://x.com/XMihura/status/2079862285221113938](https://x.com/XMihura/status/2079862285221113938)
---
I believe technology is at an inflection point
artificial intelligence is opening a historic window of opportunity: it is going to displace entire sectors, create new vacuums of power, and redistribute the economy. but more importantly, it is going to revolutionize the way we discover new knowledge and accelerate the pace of progress.
I really think that the present moment is critical, carrying an immense opportunity cost for inaction. REFUGIO was my small attempt to act within the reach of my capacity and ability on my hometown community
last month I gathered in Madrid 48 of the smartest people I could find with the reach of this account and locked them in a room for four hours
they were mostly computer scientist, physicists, mathematicians, engineers...
the goal was simple: i wanted to see if a room full of highly capable humans augmented by AI coding agents could outperform the best fully autonomous frontier models at a complex optimization task
I just published the full technical report (link in the comments of this article to avoid algo shadowbanning), but since I know most of you have your dopaminergic systems too fried by the algorithmic feed to read a 20-page pdf, here is the short version
Speedrunning as a model of progress
if you want to understand how to accelerate scientific and technological progress, don't look at academia. traditional peer review is slow, bureaucratic, and increasingly flooded with AI-generated slop. instead, look at video game speedrunners
> A speedrun is a play-through of a video game with the intention of finishing the game it as fast as possible. Speedruns are usually performed in games that have a clear end goal, like defeating the final boss or completing all the levels. It’s generally a well defined task with a clear metric to measure progress: time.
if you've never stayed up until 2 am watching a Summoning Salt documentary you are probably confused and don't understand what I'm talking about, let me explain.
Summoning Salt is a great Youtube Channel that makes feature-length documentaries about the history of retro video game world records. He meticulously documents how, over decades, thousands of isolated gamers form a collective hive-mind to discover glitches, exploit memory limits, and shave literal fractions of seconds to complete games like super mario 64
watching his videos is watching the absolute peak of human optimization. I really think speedrunning communities are the ultimate search engines exploring impossibly large, highly non-convex search spaces and we can learn a lot about them
how do they actually do it? basically it boils down to a few key elements:
first, priors.
speedrunners aren't blank-slate reinforcement learning agents starting from randomnly initalized weights. they are general intelligences equipped with highly calibrated models of the world. For example, think about the classic 80s game Montezuma's Revenge. if you put a 7-year-old kid in front of it, they instantly know that the pixelated ladders are for climbing up and down. why? because they've seen ladders in real life and they successfully map real-world ladder properties to the game. This is because they share with the game's developers a common understanding of the concept of a ladder and its representation. howeer, a vanilla RL agent just sees an array of pixels and has to blunder around randomly for millions of episodes just to learn what a ladder is. It's in a sense, inductive inference
A room from Montezuma's Revenge. Humans immediately read the ladders, rope, key, and skull as functional objects; a blank-slate RL agent sees only pixels until it explores.
why does this inductive inference work? here is a brief dose of algorithmic information theory: solomonoff induction, basically the mathematical formalization of occam's razor. the universe (and by extension, the physics engine of a video game) is generated by sets of highly compact rules. it has low kolmogorov complexity. therefore, if you observe a structural pattern in one small corner of a search space, it is mathematically highly probable that it's a fundamental rule of the system instead of a random coincidence
humans intuitively understand this. we use these priors to prune the search space. If you are interested about this read the full report
second, competition
nothing pushes limits like knowing some autist on the other side of the planet is trying to take your #1 spot.
third, and most importantly: forced communication and collaboration
when a speedrunner finds a game-breaking glitch, like removing the wii remote batteries mid-swing to hit a golf ball twice, they are forced to publish the video proof of the gameplay to claim the record. instantly, the entire global community that competes with them learns that discovery and they just start their search from this new frontier like a hive mind
basically, these are the same principles of open science: you need to publish your results to get them recognized
## progress is search
zoom out for a second and you'll see the bigger picture: progress IS search. advancing the state of the art of any task is literally just finding a point in a vast search space that beats every candidate found before. and if progress is search, then accelerating progress stops being a philosophical question and becomes an engineering problem: how do you configure a population of searchers (their incentives, their communication channels, etc.) so they explore the search space as efficiently as possible?
we are entering an era where AI agents can propose, implement, run and evaluate experiments on their own at just the coste of the compute. the bottleneck of science is shifting from execution to searching strategies. We need to seriously think about how to engineer these hybrid populations of humans and AI agents, which I believe will drive the next period of human research
the hackathon was a toy example of thinking about this: a tiny, controlled sandbox to watch these dynamics play out in four hours
to be fair, REFUGIO was not a serious scientific experiment, it was mainly a networking event to connect the technical talent of Madrid with its ecosystem. I wrote my reflections on that, and on the state of the madrid scene, in a separate post that you can find in my X articles
## the challenge
Representation of the warehouse: the red dots are robots, the yellow squares are the packages, black squares are shelves, and blue squares are the delivery stations.
I gave the teams a blank slate: a 52x52 warehouse grid. they had to program the layout for 960 shelves and write the python policy to route 96 robots to make as many deliveries as possible in 300 ticks.
the incentives were inspired by the speedrunning dynamics explained above
- you only earn hackathon points if you break the global high score
- the moment your code validates and takes the lead, your entire source code is instantly open-sourced and published to every other team in the room so they can iterate over it
I managed to get 3 Mac Minis for the winner team of the event too
Hackathon morning: participants at work while the public leaderboard updates live on the screen
The room during the hackathon: 16 teams of three, each working with AI coding agents on the same problem
## the infrastructure: safety without humans in the loop
now, letting 16 teams spam untrusted, AI-generated python code to a server every few minutes is a devops nightmare. if you run that on a shared machine, someone is going to try to os.system("rm -rf /")
since i was running the whole event alone with my wife, i needed a zero-ops platform
With help of some friends at Cursor, I built a pipeline on Vercel: Supabase Postgres for the queue and leaderboard, a Vercel Blob to store the code and replays, and a Vercel Workflows for orchestration. We used a Vercel Sandbox. every single submission was booted into its own isolated Firecracker microVM, ran the simulation, to then be destroyed.
since I was going to have 48 people blindly executing AI vibe coded Python on their computers, sometimes generated by other participants, I needed to be extra-careful with safety. Before the code even touched a CPU, it had to survive a two-stage security filter.
- first, a static filter that blocked obvious script-kiddie imports (like socket or os).
- second, an LLM reviewer. i hooked up GPT-5.5 to the pipeline throuhg the OpenAI API with one strict instruction: "reject anything that looks like malware, data exfiltration, or a sandbox escape, but approve ANY game strategy, no matter how weird."
It's simple but it actually worked perfectly. the platform handled 93 submissions with zero crashes. 7 were rejected. the static filter caught the basic stuff and the LLM caught the big brain moves: teams ended literally trying to monkey-patch the internal python referee module to force a favorable seed or just print a fake score, things that the standard regex didn't caught
The life of a submission, as one clockwise loop: submitted, screened at the gates, queued, claimed by the orchestrator, run inside the sandbox trust boundary, recorded in the storage layer, published on the frontend.
## how the humans broke the game
what happens when you force open-source discovery? progress goes very fast
Video of one of the first submissions of the hackathon. You can see how robots get trapped in jams and unresolved collisions.
at first, teams used simple pathfinding. it worked, but caused massive traffic jams. then, one team (or their agent) found out one of the exploits I left purposedly: because the evaluator ran the robots' functions sequentially in the same python module, they could use global variables to let the robots communicate. they built a centralized space-time planner. boom. new record. the code was published, and within minutes, every team had assimilated this architectural jump.
then they started grinding the warehouse layout, optimizing the aisles for traffic flow.
but the absolute kino moment came two hours in. one team realized the platform was leaking the exact evaluation seeds in the frontend json payload. This was a mistake of mine, I trusted Codex too much with the vibe coding of the platform. however, in retrospective it was fun bc it gave another dimension to the search space. So at least four teams, instead of writing a general algorithm, they wrote a script that fingerprinted the seed and executed a hardcoded, offline-optimized path specifically tailored for that exact scenario
Video of the best policy found during the hackathon. Robots don't collide and the layout shape is overfitted to the leaked hidden seeds.
they absolutely destroyed the problem with a final score: 1008 deliveries.
The human–AI collective frontier over the four-hour event. Numbered markers are the milestones of the run, from the first frontier push to the winning instance-fitted layout.
Evolution of the warehouse layout along the human frontier. From the starter long racks to the winning instance-fitted layout.]
## the autonomous AI solo runs
after the event, i wanted to see how the machines would do on their own. i took the 3 best models available that day (Claude Opus 4.8, Gemini 3.1 Pro, and GPT-5.5) and gave them the exact same problem, the same evaluator, and similar conditions (I couldn't give the exact compute budget bc it's difficult, they were 48 people in the room). One rule was clear tho: zero human intervention.
Gemini 3.1 Pro and GPT-5.5 missed all the exploits. they got stuck in local optima, endlessly tweaking turning penalties and variable weights without ever making a structural leap.
Opus 4.8 was best and got to 805 deliveries, although it took around 40min to just make the fist submission. This because it found the leaked seeds right away and used them to search locally before submission. It actually found the global state exploit but was too autistic to evolve it into a full central planner. Even though it had the leaked seeds in its context window, it never realized it could exploit them.
The performance of the three autonomous agents available on the day of the event compared to the human collective
## the new frontier models
but the technocapital machine never sleeps. a few weeks later, the new generation of models dropped. so i reran the experiment with Fable 5 (well, Fable 5 was available before the hackathon, but not during the hackathon thanks to the USG), Grok 4.5, and OpenAI's GPT-5.6 Sol.
the results were interesting
GPT-5.6 Sol (Extra High) was genuinely terrible at this. it scored 488, missing every exploit and getting stuck in useless parameter sweeps. OpenAI seriously needs to get its shit together with this v2 subagent architecture.
Grok 4.5 did... great (876). it found the global communication exploit and built a central planner, but missed the seed leak.
but Fable 5... bro. Fable 5 is scary
Fable 5 was the only autonomous model that independently discovered BOTH the global communication channel AND the seed leak. it went down the exact same dark path as the winning human teams: it used the leaked seeds to fit a custom layout to the exact demand and submitted an offline-searched schedule. it scored 984. it almost beat a room of 48 humans wielding infinite API credits.
The performance of all six autonomous agents compared to the human collective. Fable 5, unavailable on the day of the event, was the only agent that managed to match the human trajectory by discovering and weaponizing the same exploits.
Final warehouse layouts generated by the six autonomous agents. Fable 5 exploited the seed targets to fit a highly optimized geometry.
## conclusion
I think this little experiment was successful: humans actually improved what LLMs can do in their own, although the competition is fierce (check Fable). The cost also is not comparable; the human run was way more expensive than any of the individual runs. Each team had 150$ credits granted by the people at Cursor and unlimited credits granted by the people at Google. Many competitors used their own subs. It's difficult to estimate how much money was spent in API tokens during that morning, but I estimate at least 2k$. For comparison, the most expensive run was Fable and it costed around 200$ in Cursor API credits
LLMs are incredible at execution, but left entirely on their own in vast, complex search spaces, they will still fall into local optima. I think they still lack the high-entropy intuition, what we in a researcher we call intuition or taste. I explain better why this happens in the full report
the best formula for the foreseeable future will be hybrid: with humans and their intuition steering the direction, paired with AI agents to do the cheap, fast execution of ideas
We need now to focus right now on building systems that force transparent disclosure so we can build on each other's breakthroughs instead of doing siloed, repeated work
if you want to read the full technical details, the exact scoring mechanics, and more thoughts on the future of research, you can read the full whitepaper, link in the comments of this article
thank you to all the participants who gave up their Saturday to be part of this experiment. Thank you to the sponsors who supported REFUGIO: Maisa AI, Acurio Ventures, Cursor, BackFund, Samaipata, Google, tinybird, Kibo Ventures and Manfred.
and finally, a special thank you to the team at Cursor for helping to set up the infrastructure and for providing me with the API credits that allowed me to run the autonomous agent experiments and complete this report
the future is bright if we build it right. let's fucking go
## 相关链接
- [Josep Ferrer reposted](https://x.com/iamjosepferrer)
- [Mihura](https://x.com/XMihura)
- [@XMihura](https://x.com/XMihura)
- [33K](https://x.com/XMihura/status/2079862285221113938/analytics)
- [Upgrade to Premium](https://x.com/i/premium_sign_up)
- [5:33 PM · Jul 22, 2026](https://x.com/XMihura/status/2079862285221113938)
- [33.6K Views](https://x.com/XMihura/status/2079862285221113938/analytics)
- [View quotes](https://x.com/XMihura/status/2079862285221113938/quotes)
---
*导出时间: 2026/7/23 14:48:00*
---
## 中文翻译
# 速通进步
**作者**: Mihura
**日期**: 2026-07-22T09:33:05.000Z
**来源**: [https://x.com/XMihura/status/2079862285221113938](https://x.com/XMihura/status/2079862285221113938)
---
我相信技术正处于一个拐点。
人工智能开启了一个历史性的机遇窗口：它将取代整个行业，创造新的权力真空，并重新分配经济。但更重要的是，它将彻底改变我们发现新知识的方式，并加速进步的步伐。
我真的认为当前时刻至关重要，不作为将带来巨大的机会成本。REFUGIO 是我在力所能及的范围内，为我的家乡社区所做的一次微小尝试。
上个月，我在马德里召集了我能找到的 48 位最聪明的人（通过这个账号的触达能力），并将他们锁在一个房间里四个小时。
他们主要是计算机科学家、物理学家、数学家、工程师……
目标很简单：我想看看一个由高能力人类组成的房间，在 AI 编程代理的辅助下，是否能在复杂的优化任务中击败最优秀的全自动前沿模型。
我刚刚发布了完整的技术报告（链接在这篇文章的评论中，以避免算法的限流），但我知道你们大多数人的多巴胺系统都被算法信息流炸得太厉害，读不了一篇 20 页的 PDF，所以这里是简短版本。
作为进步模型的速通
如果你想了解如何加速科学和技术进步，不要看学术界。传统的同行评审缓慢、官僚，且越来越充斥着 AI 生成的垃圾内容。相反，看看电子游戏速通玩家。
> 速通是指以尽可能快的速度通关电子游戏的过程。速通通常在有明确终点的游戏进行，例如击败最终 BOSS 或完成所有关卡。这通常是一个定义明确的任务，有一个清晰的指标来衡量进度：时间。
如果你从未熬夜到凌晨 2 点看 Summoning Salt 的纪录片，你可能会困惑，不明白我在说什么，让我解释一下。
Summoning Salt 是一个很棒的 YouTube 频道，制作关于复古电子游戏世界纪录历史的时长纪录片。他详尽地记录了数十年间，成千上万名互不相识的游戏玩家如何形成一个集体蜂巢思维，发现漏洞，利用内存限制，并为了完成《超级马力欧 64》等游戏而削减零点几秒的时间。
看他的视频就是在观看人类优化的绝对巅峰。我真的认为速通社区是探索极其巨大、高度非凸搜索空间的终极搜索引擎，我们可以从它们身上学到很多东西。
他们实际上是怎么做到的？基本上归结为几个关键要素：
第一，先验知识。
速通玩家不是从随机初始化权重开始的白板强化学习智能体。他们是配备高度校准世界模型的通用智能体。例如，想想经典的 80 年代游戏《蒙特祖玛的复仇》。如果你让一个 7 岁的小孩玩它，他们会立刻知道像素化的梯子是用来上下攀爬的。为什么？因为他们在现实生活中见过梯子，并成功地将现实世界梯子的属性映射到游戏中。这是因为他们与游戏开发者对梯子的概念及其表现形式有着共同的理解。然而，一个普通的 RL 智能体只看到像素阵列，必须在数百万次尝试中盲目随机探索，才能学会梯子是什么。从某种意义上说，这是归纳推理。
《蒙特祖玛的复仇》中的一个房间。人类立即将梯子、绳子、钥匙和骷髅识别为功能对象；而白板 RL 智能体在探索之前只看到像素。
为什么这种归纳推理有效？这里有一点点算法信息论的内容：所罗门诺夫归纳，基本上是奥卡姆剃刀的数学形式化。宇宙（以及延伸开来的，电子游戏的物理引擎）是由高度紧凑的规则集生成的。它具有低科尔莫戈罗夫复杂性。因此，如果你在搜索空间的一个小角落观察到结构模式，那么在数学上它极有可能是系统的基本规则，而不是随机巧合。
人类凭直觉理解这一点。我们使用这些先验知识来修剪搜索空间。如果你对此感兴趣，请阅读完整报告。
第二，竞争。
没有什么比知道地球另一端的某个自闭症患者正试图夺走你的第一名更能推动极限了。
第三，也是最重要的一点：强制性的沟通与协作。
当一个速通玩家发现了一个破坏游戏的漏洞，比如在挥杆中途取出 Wii 遥控器电池来击打高尔夫球两次时，他们被迫发布游戏过程的视频证据来认领纪录。瞬间，与他们竞争的整个全球社区都学到了这一发现，他们就像蜂巢思维一样，从这个新前沿开始他们的搜索。
基本上，这与开放科学的原则相同：你需要发布你的结果才能获得认可。
## 进步即搜索
稍微拉远视角，你会看到更大的图景：进步就是搜索。推进任何任务的最先进水平，实际上就是在巨大的搜索空间中找到一个胜过之前所有候选点的点。如果进步是搜索，那么加速进步就不再是一个哲学问题，而变成了一个工程问题：如何配置搜索者群体（他们的激励机制、沟通渠道等），以便他们尽可能高效地探索搜索空间？
我们正在进入一个时代，AI 智能体可以自行提议、实施、运行和评估实验，成本仅仅是算力。科学的瓶颈正从执行转向搜索策略。我们需要认真思考如何工程化这些人类和 AI 智能体的混合群体，我相信这将推动下一阶段的人类研究。
这次黑客马拉松是思考这个问题的一个玩具示例：一个微小的、受控的沙盒，用来观察这些动态在四个小时内如何展开。
公平地说，REFUGIO 并不是一个严肃的科学实验，它主要是一个连接马德里技术人才与其生态系统的社交活动。我对此的思考，以及关于马德里技术圈现状的思考，写在了另一篇文章中，你可以在我的 X 文章中找到。
## 挑战
仓库的表示：红点是机器人，黄色方块是包裹，黑色方块是货架，蓝色方块是送货站。
我给团队一张白纸：一个 52x52 的仓库网格。他们必须为 960 个货架规划布局，并编写 Python 策略来路由 96 个机器人，以便在 300 个时间刻度内完成尽可能多的送货。
激励机制受到了上述速通动态的启发：
- 只有打破全球最高分才能获得黑客马拉松积分。
- 一旦你的代码通过验证并取得领先，你的整个源代码将立即开源并发布给房间里的其他所有团队，以便他们在此基础上迭代。
我还设法为活动的获胜团队搞到了 3 台 Mac Mini。
黑客马拉松的早晨：参与者正在工作，公共排行榜在屏幕上实时更新。
黑客马拉松期间的房间：16 支三人团队，每个团队都在使用 AI 编程代理解决同一个问题。
## 基础设施：无人工干预的安全性
现在，让 16 个团队每几分钟向服务器发送不受信任的 AI 生成的 Python 代码，简直是 DevOps 的噩梦。如果你在共享机器上运行这个，肯定会有人尝试 `os.system("rm -rf /")`。
因为我和妻子要独自管理整个活动，我需要一个零运维平台。
在 Cursor 朋友们的帮助下，我在 Vercel 上构建了一个流水线：使用 Supabase Postgres 作为队列和排行榜，Vercel Blob 存储代码和回放，Vercel Workflows 进行编排。我们使用了 Vercel Sandbox。每一次提交都被启动到它自己隔离的 Firecracker 微型虚拟机中，运行模拟，然后被销毁。
因为我将有 48 个人在他们的电脑上盲目执行由“氛围编程”生成的 Python（有时是由其他参与者生成的），所以我需要格外小心安全性。在代码接触到 CPU 之前，它必须通过一个两级安全过滤器。
- 第一，一个静态过滤器，阻止明显的脚本小子导入（如 socket 或 os）。
- 第二，一个 LLM 审查者。我将 GPT-5.5 通过 OpenAI API 接入流水线，并给出一个严格指令：“拒绝任何看起来像恶意软件、数据窃取或沙箱逃逸的内容，但批准任何游戏策略，无论多么奇怪。”
这很简单，但实际上非常有效。该平台处理了 93 次提交，零崩溃。7 次被拒绝。静态过滤器捕获了基本的东西，而 LLM 捕获了那些高智商的操作：团队最终试图猴子补丁内部的 Python 裁判模块以强制生成有利的种子，或者只是打印一个虚假分数，这些是标准正则表达式没有捕捉到的东西。
一次提交的生命周期，作为一个顺时针循环：提交，在大门口接受审查，排队，被编排器认领，在沙箱信任边界内运行，记录在存储层，发布在前端。
## 人类如何“破坏”游戏
当你强制开源发现时会发生什么？进步非常快。
黑客马拉松最早提交之一的视频。你可以看到机器人被困在拥堵和未解决的碰撞中。
起初，团队使用简单的寻路。这行得通，但导致了巨大的交通拥堵。然后，一个团队（或他们的代理）发现了我有意留下的一个漏洞：因为评估器在同一个 Python 模块中顺序运行机器人的函数，他们可以使用全局变量让机器人进行通信。他们构建了一个集中的时空规划器。轰。新纪录。代码发布了，几分钟内，每个团队都吸收了这一架构飞跃。
然后他们开始优化仓库布局，为交通流量优化过道。
但绝对的高光时刻出现在两个小时后。一个团队意识到平台在前端 JSON 载荷中泄露了确切的评估种子。这是我的一个错误，我太信任 Codex 的平台“氛围编程”了。然而，回想起来这很有趣，因为它给搜索空间增加了另一个维度。所以至少有四个团队，没有编写通用算法，而是编写了一个脚本来识别种子指纹，并执行针对该特定场景定制的硬编码、离线优化路径。
黑客马拉松期间发现的最佳策略视频。机器人没有碰撞，布局形状过度拟合泄露的隐藏种子。
他们彻底摧毁了这个问题，最终得分：1008 次送货。
在四小时活动期间人类-AI 集体的前沿。编号标记是运行的里程碑，从第一次前沿推进到获胜的实例拟合布局。
仓库布局沿人类前沿的演变。从最初的长货架到获胜的实例拟合布局。
## 自主 AI 独自运行
活动结束后，我想看看机器自己能做得怎么样。我选取了当天可用的 3 个最佳模型，给了它们完全相同的问题、相同的评估器和相似的条件（我无法给出完全相同的算力预算，因为这很难，房间里有 48 个人）。但有一条规则是明确的：零人类干预。
Gemini 3.1 Pro 和 GPT-5.5 错过了所有漏洞。它们陷入了局部最优，无休止地微调转向惩罚和变量权重，从未做出结构性飞跃。
Opus 4.8 是最好的，达到了 805 次送货，尽管它花了大约 40 分钟才做出第一次提交。这是因为它立即发现了泄露的种子，并在提交前利用它们进行本地搜索。它实际上发现了全局状态漏洞，但太“自闭”了，没能将其演变成完整的中央规划器。即使上下文窗口中有泄露的种子，它也没意识到可以利用它们。
活动当天可用的三个自主智能体的性能与人类集体的对比。
## 新的前沿模型
但是技术资本机器从不睡觉。几周后，新一代模型发布了。所以我用 Fable 5（好吧，Fable 5 在黑客马拉松之前就可用，但感谢美国政府，在黑客马拉松期间不可用）、Grok 4.5 和 OpenAI 的 GPT-5.6 Sol 重新运行了实验。
结果很有趣。
GPT-5.6 Sol（超高）在这方面确实很糟糕。它得分 488，错过了所有漏洞，并陷入了无用的参数扫描。OpenAI 真的需要搞定这个 v2 子代理架构。
Grok 4.5 做得……很棒（876 分）。它发现了全局通信漏洞并构建了中央规划器，但错过了种子泄露。
但是 Fable 5……兄弟。Fable 5 很可怕。
Fable 5 是唯一一个独立发现了全局通信通道和种子泄露的自主模型。它走上了与获胜人类团队完全相同的黑暗道路：它利用泄露的种子将自定义布局拟合到确切的需求，并提交了一个离线搜索的时间表。它得分 984。它几乎击败了一个拥有无限 API 额度的 48 人房间。
所有六个自主智能体的性能与人类集体的对比。Fable 5 在活动当天不可用，是唯一一个通过发现并利用相同漏洞来匹配人类轨迹的智能体。
六个自主智能体生成的最终仓库布局。Fable 5 利用种子目标拟合了高度优化的几何形状。
## 结论
我认为这个小实验是成功的：人类实际上改进了 LLM 独立能做的事情，尽管竞争非常激烈（看看 Fable）。成本也不可同日而语；人类运行的成本比任何一次单独运行都要高得多。每个团队获得了 Cursor 提供的 150 美元额度和 Google 提供的无限额度。许多竞争对手使用了自己的订阅。很难估算那天早上在 API 令牌上花了多少钱，但我估计至少 2000 美元。相比之下，最昂贵的是 Fable，大约花费了 200 美元的 Cursor API 额度。
LLM 在执行方面令人难以置信，但如果将它们完全留在这个巨大、复杂的搜索空间中独自行动，它们仍然会陷入局部最优。我认为它们仍然缺乏高熵直觉，也就是我们研究人员所说的直觉或品味。我在完整报告中更好地解释了为什么会发生这种情况。
在可预见的未来，最佳公式将是混合式的：人类及其直觉把控方向，配合 AI 智能体进行廉价、快速的想法执行。
我们现在需要专注于构建强制透明披露的系统，这样我们可以建立在彼此的突破之上，而不是进行孤立的、重复的工作。
如果你想阅读完整的技术细节、确切的评分机制以及对研究未来的更多思考，你可以阅读完整白皮书，链接在这篇文章的评论中。
感谢所有放弃周六参加这个实验的参与者。感谢支持 REFUGIO 的赞助商：Maisa AI, Acurio Ventures, Cursor, BackFund, Samaipata, Google, tinybird, Kibo Ventures 和 Manfred。
最后，特别感谢 Cursor 团队帮助设置基础设施，并提供给我 API 额度，使我能够运行自主智能体实验并完成这份报告。
如果我们建造得当，未来是光明的。让我们 fucking go。