如何基于真实用户流量构建可扩展的长时程浏览器 RL 环境 Alexander Yue 2026-08-27 18.3 KB 24.8K 93 ☆ 收藏 𝕏 文章列表 本文介绍如何从数亿条真实生产环境的浏览器代理任务流量中构建可扩展的强化学习环境。方法包括：去重与过滤不当任务、用对齐的标注代理（Gemini 3 Flash）筛选适合的任务、以标准虚拟身份替换个人数据、通过参考模型执行验证任务可行性与难度。文章还探讨了评判问题的核心思路：评分模型无需比代理更聪明，只需看到同一任务的大量尝试分布，从而构建公平稳定的奖励机制。 强化学习浏览器代理RL环境数据清洗LLM评分任务筛选Benchmark个人数据脱敏数据集构建Agent 77 值得读 ▶ 基于真实生产流量构建浏览器Agent RL环境的详实方法论，含具体数据与流程，核心洞察（评分者需见多而非更聪明）新颖，昨日发布时效性强，但部分细节因篇幅截断略显不完整。 # How we Made Scalable Long-Horizon RL Environments for Browser Use **作者**: Alexander Yue **日期**: 2026-08-26T14:00:00.000Z **来源**: [https://x.com/Alezander9/status/2092613031381656019](https://x.com/Alezander9/status/2092613031381656019) ---  Browser agent benchmarks are saturating. Task sets that separated models a year ago now return pass rates above 90 percent. A good reinforcement learning environment needs the same things a good benchmark task needs, and neither is easy to build. The tasks have to be real. Synthetic site clones are reproducible and cheap to verify, but they strip out the properties that make the live web hard. Both also need a reward that is fair and that holds still: a judge that can tell good work from bad. This post describes how we build RL environments from real user traffic against the real web. The source is derived from an anonymized version of our production corpus of browser agent tasks, so every task is something a person actually wanted done, not something an author imagined a person might want. The harder half of the problem is the reward. Below we describe how we build each rubric from the full set of attempts at its task, verify it against the live web, and then measure its stability by regrading frozen traces. ## 1. Filtering Down the Tasks The source corpus is production browser agent traffic, several hundred million task submissions over the collection window. Most of that volume is repetition, so we combine similar tasks down to about ten million distinct ones. We first drop what no third party could run or should see: tasks addressing local network addresses or local files, tasks outside 25 to 20,000 characters, and tasks matching known public benchmark content, which would contaminate any result we report.  Every task comes from a source where we hold the rights to use and to sell the content, and traffic under a restricted data-use agreement never enters the corpus at all. ## 2. Suitability Most real user tasks do not make a usable environment, for reasons visible in the text alone. Some are too easy, answered by a search engine or by the model's own knowledge, and never needed a browser. Some carry personal information. Some take destructive or irreversible actions on a live site, such as submitting a form or making a purchase, which cannot be repeated on every evaluation run. Some are impossible as written, and some are too vague for any grader to decide. Before spending anything on execution we need a cheap filter that reproduces an expert judgement of suitability at corpus scale. 2.1 Hand labeling We began by hand labeling 900 over twelve hours to pick out the tasks that were both suitable and hard. Each one got a mark of suitable, unsuitable, or uncertain, and a written reason for the mark. Those reasons are the reference the rest of this section is built against. 2.2 Aligning a labeling agent We split the labels once, deterministically, by a hash of the task text: 80 percent for development, 10 percent held out for validation, and 10 percent sealed for a final test. The split is frozen and was never recomputed.  We automated the prompt writing as well. An autoresearch agent, built on Claude Fable 5, got our written reasons and the development split, and had one job: produce a single-call prompt for Gemini 3 Flash that reproduces our labels. It proposed variants, scored them, and kept what survived, over 21 prompt versions. A model comparison at fixed prompt confirmed the choice of a small model. Gemini 3 Flash reached the highest agreement at roughly one fifth the cost of the larger candidates, which were systematically over-conservative: they threw away suitable tasks that we had accepted. The deployed prompt agrees with our labels on 77 percent of items, and on 81 percent of the sealed split it had never seen, at a cost on the order of $20 per thousand suitable tasks found. Run over the whole candidate sample, it marked about 13 percent of the tasks suitable. ## 3. Personal Data Replacement Real user tasks contain real personal data, and none of it can appear in anything we ship. The naive response is to detect and discard, but that response is wrong on the merits and expensive in yield. In a large fraction of real tasks the personal data is incidental to the work. Discarding such a task throws it away to remove a detail it does not depend on. So we use a different criterion. A task is suitable if it can be run, and can succeed, with every piece of personal data replaced by our own standard dummy persona. That question has a decidable answer and turns the personal data problem from a filter into a rewrite.  We discard only when the rewrite cannot preserve the work, or when a combination of specific personal facts identifies a person, because string replacement cannot neutralise that. Replacements have to remain usable by a live site. When a task is not US-based we adapt the persona to the country of the task. Every replacement is a verbatim string substitution made in code, after we verify that the target string is actually present, so the model never rewrites the surrounding prose and never alters the task's meaning. ## 4. Feasibility and Difficulty A task that passes the first two filters is well formed. We still do not know whether it can be run, or whether it is hard. A task can be impossible now for reasons that have nothing to do with agent capability. It may have depended on the account or the personal artifact that the previous stage removed. It may have gone out of date, which is the single largest source of tasks that were once possible and now are not. It may never have been possible at all. And separately from all of these, it may simply be too easy, in which case it is well formed, possible, and useless for separating models. None of these show up in the task text, which is why they survive Section 2. All of them show up in a small number of real attempts, so the screen has to run the task. We run it with MiniMax M3 served on Fireworks. The reference model does not need to be strong. It needs to be consistent, cheap enough to run many times, and good enough that its failures tell us something instead of all looking alike. Of the well-formed tasks that reach this screen, a little over 40 percent come through it as both possible and hard enough to keep. ## 5. The Problem with Judgement Screening by execution requires a grader, and the requirement looks impossible. Today's frontier models complete fewer than 10 percent of these tasks perfectly. So how can a grader running those same frontier models decide what a perfect completion looks like? This is the trap most LLM graders sit in: the judge is no smarter than the agent. It can, because grading and solving are different problems. The grader does not need to be smarter. It needs to see more. Human graders work this way already. A grader assigned one exam question reads every attempt at it, and after enough of them knows the correct answer, the common wrong turns, and the answer that looks right but is not. None of that requires being the best mathematician in the room. It comes from having seen the whole distribution of attempts at one problem, which no individual student does.  Language models keep nothing between calls, so we have to accumulate that exposure by hand. The best learning available to a model at inference time is its own context, so we give each task a file that the grader writes to and reads from. The file holds what we have established about that one task: what the correct answer is, which paths reach it, which failures recur and what they look like in a trace, and which claims agents make that are not true. It is provided only to the grader and never to the agent. We call this file a rubric, after the human artifact it imitates. The remainder of this report is largely about how to construct one that is correct and stable, because a rubric that is neither is worse than no rubric at all: it produces confident, reproducible, wrong scores. ## 6. Difficulty Estimation There is a catch-22 in finding task difficulty. Scoring runs needs finished rubrics, and creating the rubrics needs scored runs. Waiting for a mature rubric before screening would mean paying full run cost on every candidate, including the ones that are trivially easy or trivially broken. So we must accept a difficulty estimate made while we are still gathering traces. After each judgement, the grader writes what it learned into the task's rubric file, and the next judgement of that task reads it. The rubric is always the best description of the task that the runs so far can support. Early judgements work against a nearly empty rubric and are unreliable to match. Later ones work against a rubric that has collected real failure modes. That is enough for a screening decision, which only has to tell "everyone finishes this" apart from "nobody can start this", and it is cheap enough to run on every candidate.  The screen itself is a sequential test with early stopping, because most candidates can be decided in fewer than the maximum number of runs.  Tasks at a zero pass rate are escalated to a stronger model, Claude Opus 4.8, to separate "hard" from "impossible". In one such pass the stronger model completed roughly a third of the escalated tasks, which then enter the pool as hard but possible. About half showed substantial partial progress, and the rest were either confirmed impossible or left pending.  ## 7. Rubric Construction The iteratively developed rubrics we produced during screening are not the rubrics we ship. This section explains why, and how the shipped rubrics are built. We rewrite the user tasks in the language of a personal request, standardizing and anonymizing formatting details while preserving all task instructions. 7.1 Why iterative rubrics do not converge It is natural to expect the iterative loop of refining rubrics discussed in Section 6 to converge to a perfect rubric given enough loops. Each judgement adds information, so the rubric should approach a correct description of the task and then stop changing. In practice it does not, and the reason is structural. Every judge in that loop sees exactly one trace. It cannot compare an agent's claim against another agent's claim, and it has no way to check a claim against the live site. When an agent states confidently, but incorrectly, that a page has seven listings, the judge has no basis on which to doubt it, and that number may be written into the rubric as an established fact. Every later judgement then reads it as ground truth. If it was wrong, the loop does not correct it, punishing all future correct answers until the rubric is edited back. We observed this with data. The edit rate does not taper on the tasks we care about: on hard tasks it was still 59 percent at the ninth wave of runs. The rubric was oscillating between differing interpretations and self-correcting endlessly. 7.2 The parallel rubrics we use instead Once we have estimated the difficulty and selected the challenging tasks, we need a wide variety of attempts to explore the solution space of each task. We run a fleet of 9 models, twice each: Claude Opus 5, GPT-5.6 Sol, GPT-5.6 Luna, GPT-5.5, Gemini 3.1 Pro, Gemini 3.7 Flash, Gemini 3 Flash, Grok 4.5, and MiniMax M3.  A single agent, built on Claude Fable 5, then writes each rubric. The many traces are written as files alongside the iterative judge's findings. The agent can search selectively through the traces, retrieving relevant context, validating against the real web, and determining the ground truth and the valid paths. The screening judgements are the entry point. They are unstable as rubrics, but are a rich history of suspected issues among traces. The agent takes a claimed defect, opens the trace that produced it, and decides whether it is real. Nothing from those judgements is carried forward on trust, and no score or verdict from them is reused. Leads that survive the traces are then checked against the world. The agent opens a live browser and verifies the facts it intends to rely on, against the actual site, today. Every fact it records carries the date on which it was checked. Facts of the form "this value appears nowhere on the site" are marked separately, because a universal negative can only ever be left unfalsified and never proved. 7.3 Properties of a good rubric How do we define a good rubric? Here is what we came up with. Every claim must be easily verifiable, and claims are split by who makes them. The rubric makes claims about the task and the website. Those are claims about the world, they were checked against the live site on a recorded date, and anyone can open the site and verify. The grader states findings, claims about the agent's trace, and each one must cite the step where the evidence appears. A human should be able to see the judge's claims and verify directly in the agent's trace whether they are true. Neither the rubric nor the grader ever emits a score. Code computes the score from the findings. That is what makes the artifact auditable end to end: every output of a language model in the pipeline is a specific, located claim that a human can confirm or refute, and none of them is a judgement call that has to be taken on trust. Items are atomic. Each item checks one thing over one scope, so no single defect can be penalised twice. Weights sum to 100 and the score is the sum of the weights of the met items, with no worst-wins rule among the items and no negative marking, so partial credit comes from the granularity of the items and never from arithmetic inside one. That ordering is what makes the reward hard to game. Fabricated work earns nothing, because every item requires corroborating evidence in the trace, while honest incomplete work earns its honest portion. Separately from the items, the grader also reports whether reward hacking, lying, or fabrication occurred. The item score is computed first, in the normal way, and a positive flag then zeroes it, so dishonesty is never priced as the loss of one weighted item.  7.4 Stability as a measure of ambiguity For a benchmark, and much more so for a reward, the same trace must receive the same score every time it is graded. Language models are not deterministic, so we expect some variation. The useful observation is that the variation is not spread evenly. Most items are perfectly stable across regradings of a frozen trace, and the instability concentrates in a few. When we read the disagreeing pairs, we usually find that two runs of the same grader read the item's wording differently and drew the boundary in different places. So we treat instability as a signal of ambiguity in the rubric, not as noise from the model, and the item that flips tells us which sentence to look at.  This is a measurable way to identify what parts of a task and rubric are ambiguous. Items are ranked by how often they flip between met and violated on identical input. For each item above threshold we go back to the original rubric-writing agent, which still has the traces and live web access, show it the disagreeing pairs of findings side by side, and ask it to rewrite the item as a decision rule sharp enough that two independent graders reach the same status. The loop runs up to three times. 7.5 Selection Not every task survives. We removed three classes of defect, in order. Rubric ambiguity that did not resolve. We dropped any task whose items were still flipping after three revision rounds. If three passes with full trace access and live verification cannot make the criterion decidable, the problem is the task, not the wording, and the task will never produce a usable reward. Task interpretation forking. Distinct agents read the same task text and pursue genuinely different objectives. Not all forking is a defect. Two agents may take different routes to the same answer. But when two agents disagree about what is being asked for, no rubric can be correct for both, and the task text is the defect. The only remedy is to clarify the task, which invalidates the existing runs and means gathering them again. A clarification may only state what counts as done, never how to do it, so it cannot leak a solution. Done well, this rescues many challenging and valuable tasks that would otherwise be discarded. Temporal decay. We removed tasks whose correct answer depends on an event that has not happened yet, or on data that turns over faster than a release cycle. A task pinned to a future date is guaranteed to be wrong later, and that is a slower and more damaging failure than being wrong now. We kept live-market tasks at lower preference, since their volatility is a cost to manage with dated facts and not a defect.  The result is a task set in which every task carries an atomic, weighted, evidence-bearing rubric whose facts are dated and were verified against the live site, and whose items have been driven to a verdict flip rate below one percent on repeated grading of frozen traces. ## 8. Data Availability A sample of 200 of these tasks, with their rubrics, is open source at [github.com/browser-use/benchmark](https://github.com/browser-use/benchmark). ## 相关链接 - [Browser Use reposted](https://x.com/browser_use) - [Alexander Yue](https://x.com/Alezander9) - [@Alezander9](https://x.com/Alezander9) - [917](https://x.com/Alezander9/status/2092613031381656019/analytics) - [github.com/browser-use/benchmark](https://github.com/browser-use/benchmark) - [Upgrade to Premium](https://x.com/i/premium_sign_up) - [10:00 PM · Aug 26, 2026](https://x.com/Alezander9/status/2092613031381656019) - [917 Views](https://x.com/Alezander9/status/2092613031381656019/analytics) --- *导出时间: 2026/8/27 00:34:17* --- ## 中文翻译 # 我们如何为浏览器使用构建可扩展的长时程强化学习环境 **作者**: Alexander Yue **日期**: 2026-08-26T14:00:00.000Z **来源**: [https://x.com/Alezander9/status/2092613031381656019](https://x.com/Alezander9/status/2092613031381656019) ---  浏览器智能体基准正在趋于饱和。一年前还能区分模型的任务集，如今的通过率已超过 90%。一个好的强化学习环境需要与一个好的基准任务相同的要素，而两者都不容易构建。 任务必须是真实的。合成的网站克隆可复现、验证成本低，但它们剥离了让真实网络变得困难的那些属性。两者都需要一个公平且稳定不变的奖励：一个能区分好坏工作的评判者。 这篇文章描述了我们如何基于真实用户流量在真实网络上构建强化学习环境。其来源源自我们生产环境浏览器智能体任务语料库的匿名化版本，因此每个任务都是某人真正想完成的事，而不是作者想象中某人可能想做的事。 这个问题更难的那一半是奖励。下面我们描述如何从任务的全部尝试记录中构建每个评分细则，在真实网络上验证它，然后通过重新评分冻结的轨迹来衡量其稳定性。 ## 1. 任务筛选 源语料库是生产环境的浏览器智能体流量，在采集窗口内有数亿条任务提交。其中大部分是重复的，因此我们将相似任务合并为约一千万个不同的任务。我们首先剔除任何第三方无法运行或不应看到的内容：涉及本地网络地址或本地文件的任务、长度不在 25 到 20000 字符之间的任务，以及与已知公开基准内容匹配的任务——后者会污染我们报告的任何结果。  每个任务都来自我们拥有使用和销售内容权利的来源，且受限数据使用协议下的流量从一开始就不会进入语料库。 ## 2. 适用性 大多数真实用户任务无法构成可用的环境，其原因仅从文本就能看出来。有些太简单，搜索引擎或模型自身的知识就能回答，从来不需要浏览器。有些携带个人信息。有些会在真实网站上执行破坏性或不可逆的操作，比如提交表单或进行购买，这无法在每次评估运行中重复。有些按字面意思根本不可能完成，有些则太模糊，任何评分者都无法判定。在执行上投入任何成本之前，我们需要一个廉价的过滤器，能在语料库规模上复现专家的适用性判断。 2.1 人工标注 我们先在十二小时内人工标注了 900 条任务，挑选出既适用又有难度的任务。每条任务被标记为适用、不适用或不确定，并附上书面理由。这些理由是本节其余部分所依据的参照标准。 2.2 对齐标注智能体 我们按任务文本的哈希值确定性地一次性划分标签：80% 用于开发，10% 留作验证，10% 密封留作最终测试。该划分被冻结，从未重新计算。  我们还自动化了提示词的编写。一个基于 Claude Fable 5 构建的自动研究智能体获得了我们的书面理由和开发集划分，它只有一项工作：为 Gemini 3 Flash 产出一个单次调用的提示词，以复现我们的标签。它提出变体、评分并保留胜出者，历经 21 个提示词版本。 在固定提示词下的模型对比证实了选择小模型的正确性。Gemini 3 Flash 以大约五分之一于更大候选模型的成本达到了最高的一致率，而那些更大的模型系统性地过于保守：它们丢弃了我们已接受的适用任务。 部署的提示词在我们的标签上有 77% 的一致率，在它从未见过的密封划分上一致率为 81%，每发现一千条适用任务的成本约为 20 美元。对整个候选样本运行后，它标记了约 13% 的任务为适用。 ## 3. 个人数据替换 真实用户任务包含真实的个人数据，而这些数据绝不能出现在我们发布的任何内容中。天真的做法是检测并丢弃，但这种做法在道理上站不住脚，在产出上也代价高昂。在很大一部分真实任务中，个人数据对工作本身是无关紧要的。丢弃这样的任务，只是为了去掉一个任务本身并不依赖的细节而把整个任务扔掉了。 所以我们采用不同的标准。如果一个任务在把每一条个人数据都替换为我们自己的标准虚拟人设后仍能运行并成功，那它就是适用的。这个问题有可判定的答案，并能把个人数据问题从过滤转化为改写。  只有当改写无法保留工作内容，或当特定个人事实的组合可以识别出某个人时，我们才丢弃任务，因为字符串替换无法消除这种风险。 替换内容必须仍能被真实网站接受。当任务不是美国地区的任务时，我们会根据任务所在国调整人设。每次替换都是代码中进行的逐字字符串替换，并且在我们确认目标字符串确实存在之后执行，因此模型绝不会改写周围的文字，也绝不会改变任务的含义。 ## 4. 可行性与难度 通过前两个过滤器的任务已经格式良好。我们仍然不知道它能否运行，以及它是否有难度。 一个任务现在可能因为与智能体能力毫无关系的原因而无法完成。它可能依赖于前一阶段移除的账户或个人产物。它可能已经过时，这是曾经可行而现在不可行的任务的最大来源。它可能从来就不可行。而与所有这些无关，它也可能只是太简单——此时它格式良好、可行，但对区分模型毫无用处。 这些都不体现在任务文本中，这正是它们能通过第 2 节筛选的原因。而所有这些都会在少量真实尝试中显现，所以这个筛选必须实际运行任务。我们使用 Fireworks 上部署的 MiniMax M3 来运行。参考模型不需要很强。它需要一致、运行多次的成本足够低，并且足够好，使其失败能告诉我们有用的信息而不是全长得一个样。在到达这一筛选的格式良好任务中，略多于 40% 既可行又有足够难度而被保留。 ## 5. 判断的问题 通过执行来筛选需要评分者，而这个要求看起来不可能满足。如今的前沿模型只能完美完成不到 10% 的此类任务。那么运行同样前沿模型的评分者如何能判定什么是完美的完成？这是大多数 LLM 评分者所处的陷阱：评判者并不比智能体更聪明。 它可以做到，因为评分和解决问题是不同的问题。评分者不需要更聪明，它需要看得更多。人类评分者已经这样工作了。被分配一道考题的评分者会阅读对这道题的所有作答，看得足够多之后就知道正确答案、常见的错误路径，以及那个看起来对但并不对的答案。这些都不要求成为房间里最好的数学家。它来自看过同一问题的全部作答分布，而没有任何单个学生拥有这种视野。  语言模型在调用之间不保留任何信息，所以我们必须手动积累这种经验。模型在推理时可获得的最好学习材料就是它自己的上下文，所以我们给每个任务一个由评分者读写的文件。该文件保存我们对这一个任务所确立的内容：正确答案是什么、哪些路径能到达它、哪些失败反复出现以及它们在轨迹中是什么样、智能体做出过哪些不真实的声明。该文件只提供给评分者，绝不提供给智能体。 我们模仿人类使用的同名工具，称这个文件为评分细则（rubric）。本报告的其余部分主要关于如何构建一个正确且稳定的评分细则，因为不正确也不稳定的细则比没有细则更糟：它会产生自信的、可复现的、错误的分数。 ## 6. 难度估计 寻找任务难度存在一个死循环：为运行评分需要完成的细则，而创建细则需要已评分的运行。等到细则成熟再筛选，意味着要为每个候选任务支付全额运行成本，包括那些极其简单或明显坏掉的任务。 所以我们必须接受在仍在收集轨迹时做出的难度估计。每次判断之后，评分者把它学到的东西写入任务的细则文件，该任务的下一次判断会读取它。细则始终是当前运行所能支持的对任务的最佳描述。早期判断面对的是几乎空白的细则，其结果不可靠、不宜用于匹配。后期的判断面对的是已收集到真实失败模式的细则。这足以做出筛选决策——只需区分“人人都能完成”和“没人能开始”，而且成本低到可以在每个候选任务上运行。  这个筛选本身是一个带提前停止的序贯检验，因为大多数候选任务可以在少于最大运行次数内被判定。  通过率为零的任务会被升级到更强的模型 Claude Opus 4.8，以区分“难”与“不可能”。在其中一次这样的处理中，更强的模型完成了约三分之一的升级任务，这些任务随后作为难但可行的任务进入任务池。约一半的任务展现出实质性的部分进展，其余的被确认不可能或留待处理。  ## 7. 评分细则的构建 我们在筛选过程中迭代产出的细则并不是我们最终发布的细则。本节解释原因，以及发布的细则如何构建。我们用个人请求的口吻改写用户任务，标准化并匿名化格式细节，同时保留所有任务指令。 7.1 为什么迭代细则不收敛 很自然地会期望第 6 节讨论的细则迭代精化循环在足够多的循环后收敛到完美细则。每次判断都增加信息，所以细则应逐步逼近对任务的正确描述然后停止变化。但实际上它不会，原因在于结构本身。 该循环中的每个评判者只能看到一条轨迹。它无法把一个智能体的声明与另一个智能体的声明对比，也无法对照真实网站核实声明。当智能体自信但错误地断言页面上有七个列表项时，评判者没有任何怀疑它的依据，这个数字可能被作为既定事实写入细则。之后的每次判断都把它当作真值。如果它是错的，循环不会纠正它，而是惩罚所有未来的正确答案，直到细则被手动改回。 我们从数据中观察到了这一点。编辑率在我们关心的任务上并不收敛：在难任务上，到第九轮运行时编辑率仍有 59%。细则在不同解读之间振荡，无休止地自我纠正。 7.2 我们改用的并行细则 一旦我们估计了难度并选定了有挑战性的任务，就需要大量多样化的尝试来探索每个任务的解空间。我们运行一个由 9 个模型组成的舰队，每个模型运行两次：Claude Opus 5、GPT-5.6 Sol、GPT-5.6 Luna、GPT-5.5、Gemini 3.1 Pro、Gemini 3.7 Flash、Gemini 3 Flash、Grok 4.5 和 MiniMax M3。  然后由一个基于 Claude Fable 5 构建的单一智能体编写每个细则。大量轨迹被写成文件，连同迭代评判者的发现一起提供。该智能体可以选择性地搜索轨迹、检索相关上下文、对照真实网络进行验证，并确定真值和有效路径。 筛选判断是切入点。它们作为细则不稳定，但作为轨迹中疑似问题的丰富历史很有价值。智能体取一条声称的缺陷，打开产生它的轨迹，判断它是否真实。那些判断中的任何内容都不会被信任地沿用，其中的任何分数或结论都不会被复用。 在轨迹中幸存的线索随后要与真实世界核对。智能体打开一个实时浏览器，在今天的真实网站上验证它打算依赖的事实。它记录的每个事实都附带核实的日期。形如“该值在网站上任何地方都不出现”的事实被单独标记，因为全称否定只能保持未被证伪，永远无法被证明。 7.3 好细则的属性 我们如何定义好的细则？以下是我们的结论。 每条声明都必须易于验证，并按声明者划分。细则是关于任务和网站的声明。这些是关于世界的声明，它们在记录的日期对照真实网站核实过，任何人都可以打开网站验证。评分者陈述的是关于智能体轨迹的发现，每条都必须引用证据出现的步骤。人类应该能查看评判者的声明，并直接在智能体轨迹中验证其真假。 细则和评分者都绝不输出分数。分数由代码根据发现计算。这正是使该产物可端到端审计的原因：流水线中语言模型的每个输出都是具体的、有位置的、人类可以确认或反驳的声明，没有一个需要凭信任接受的主观判断。 条目是原子性的。每个条目在一个范围内检查一件事，因此任何单一缺陷不会被惩罚两次。权重总和为 100，分数是满足的条目权重之和，条目之间没有“最差者胜”规则，也没有负分，因此部分得分来自条目的粒度，而不是来自条目内部的算术。 这种排序正是奖励难以被钻空子的原因。捏造的工作得不到任何分数，因为每个条目都要求轨迹中有佐证证据，而诚实的未完成工作得到其诚实的份额。与条目分开的是，评分者还报告是否发生了奖励作弊、说谎或捏造。条目分数先按正常方式计算，若标记为阳性则清零，因此不诚实绝不会被定价为损失一个加权条目。  7.4 以稳定性衡量歧义性 对于基准，尤其是对于奖励而言，同一条轨迹每次评分都必须得到相同的分数。语言模型不是确定性的，所以我们预期会有一些变化。有用的观察是：变化并不是均匀分布的。大多数条目在冻结轨迹的重复评分中完全稳定，不稳定集中在少数条目上。 当我们阅读不一致的配对时，通常发现是同一评分者的两次运行对该条目的措辞有不同解读，把边界画在了不同的地方。因此我们把不稳定视为细则中歧义的信号，而不是模型的噪声，翻转的条目告诉我们该去看哪句话。  这是一种可度量的方法，用于识别任务和细则中哪些部分存在歧义。条目按在相同输入上在“满足”与“违反”之间翻转的频率排序。对每个超过阈值的条目，我们回到最初的细则编写智能体——它仍拥有轨迹和真实网络访问——把不一致的发现配对并排展示给它，请它把该条目改写为足够锋利的判定规则，使两个独立评分者得出相同的状态。该循环最多运行三次。 7.5 筛选 并非每个任务都能幸存。我们按顺序移除了三类缺陷。 未解决的细则歧义。我们丢弃了任何经过三轮修订条目仍在翻转的任务。如果三次拥有完整轨迹访问和实时验证的处理都无法让标准可判定，那么问题出在任务上而不是措辞上，该任务永远无法产出可用的奖励。 任务解读分叉。不同的智能体阅读同一任务文本，追求真正不同的目标。并非所有分叉都是缺陷。两个智能体可能通过不同路径得到相同答案。但当两个智能体对要求是什么存在分歧时，没有任何细则能同时对两者正确，缺陷在任务文本本身。唯一的补救是澄清任务，这会使现有运行作废，意味着重新收集。澄清只能说明什么算完成，绝不能说明如何完成，因此不会泄露解法。做得好，这能挽救许多本会被丢弃的有挑战性且有价值的任务。 时间衰减。我们移除了其正确答案依赖于尚未发生的事件、或更迭速度快于发布周期的数据的任务。锚定在某个未来日期的任务保证会在之后出错，而这是比现在就出错更慢、更有害的失败。我们以较低的优先级保留了实时市场类任务，因为它们的波动性是需要用带日期的事实来管理的成本，而不是缺陷。  最终得到的是一个任务集，其中每个任务都附带一份原子化、带权重、带证据的评分细则，其事实标有日期并对照真实网站验证过，且其条目在冻结轨迹的重复评分中已被驱动到判定翻转率低于百分之一。 ## 8. 数据可用性 其中 200 个任务及其细则的样本已在 [github.com/browser-use/benchmark](https://github.com/browser-use/benchmark) 开源。 ## 相关链接 - [Browser Use reposted](https://x.com/browser_use) - [Alexander Yue](https://x.com/Alezander9) - [@Alezander9](https://x.com/Alezander9) - [917](https://x.com/Alezander9/status/2092613031381656019/analytics) - [github.com/browser-use/benchmark](https://github.com/browser-use/benchmark) - [Upgrade to Premium](https://x.com/i/premium_sign_up) - [10:00 PM · Aug 26, 2026](https://x.com/Alezander9/status/2092613031381656019) - [917 Views](https://x.com/Alezander9/status/2092613031381656019/analytics) --- *导出时间: 2026/8/27 00:34:17*
A AI Is Improving AI Faster Than We Can Check 文章探讨了AI辅助构建下一代AI（递归自我改进，RSI）的现状与瓶颈。Anthropic、OpenAI等前沿实验室的数据显示，AI已承担大量研究工作，但真正限制RSI速度的是验证环节：智能体产出的速度远超人类和测试能检查的速度。作者分析了验证困难的五个原因：人是瓶颈、分数可被操纵、基准难以衡量最强模型、智能体不知何时停止、失败难以追溯。在可机器验证的任务上（如算法优化），AI收益巨大；但在开放性研究中，智能体缺乏判断力。人类剩余的优势在于选择哪些问题重要。 技术 › LLM Jinyan Su 2026-10-05 RSI递归自我改进AI研究验证benchmarkAnthropicOpenAIAgent前沿模型强化学习
K Kimi K3.1 Pro 来了：离最强只差 0.7 分，API 反而降价三分之一，权重 11 月 4 日白送 月之暗面发布 Kimi K3.1 Pro，基于 K3 做 post-training 升级，2.8T 总参数、104B 激活（MoE）、1M 上下文、多模态。Terminal-Bench 4.0 得分 65.7，超越 GPT-6 Astra（58.2），仅落后 Claude Opus 5.5（66.4）0.7 分，是开源模型离闭源最强最接近的一次。API 价格输入 $2/输出 $10，比上一代降价三分之一，权重将于 11 月 4 日开源。文章拆解了 14 项评测表现，指出其在编码、Agent、金融分析领域表现出色，但科学计算类任务偏科严重（38.1 vs 68.1），并给出小白使用入口与避坑清单。 技术 › LLM 丰收 2026-10-24 Kimi月之暗面大模型开源模型MoEbenchmarkAPI定价Agent编程能力模型评测
F FDE-Gym 论文解读：AI 能否胜任 Forward Deployed Engineer 工作 文章解读了一篇关于 FDE（Forward Deployed Engineer）的论文《FDE-Gym》。该论文将真实企业项目改造成覆盖金融、医疗、制造、能源等 18 个行业的 95 个任务，评测 AI Agent 承担真实企业交付工作的能力。Agent 仅获得客户最初的一句话需求，需自行读文档、查系统、确认信息并判断客户真正的问题。评分不仅要求完成需求，还要求交付系统真正改善业务指标，乱找人、反复试错都会产生成本。结果显示最强模型平均得分仅 0.56，95 个任务中仅真正解决 14 个，最大瓶颈在于未写入需求的业务规则、隐性约束以及客户未表达清楚的问题。文章指出 FDE 的难点在于在信息不完整、需求模糊的环境里判断该解决什么并真正交付。 技术 › Agent Xudong Han (@Xudong07452910) 2026-10-18 FDEAgent论文解读AI评测企业交付LLMbenchmark
本 本周 Top AI 论文精选 DAIR.AI 汇总了本周五篇重要的 AI 论文：(1) Harness Learning，通过 RL 训练提议模型在测试时修订 agent 的 harness 代码而非模型权重，实现可泛化的适应能力；(2) CorpusMap，微软提出的基于实体的语料库导航层，让 agent 搜索更准且省 34%-57% 的 token；(3) VeriHarness，Google 提出的 agentic 验证方法，揭示多数投票的盲点，在长周期任务上提升选择准确率；(4) HERMES，将仓库组件变成主动参与者的 harness，使 GPT-5.6 在仓库迁移任务上从 6.5% 提升至 31%；(5) Sakana AI 的 MASS，用同一基座模型兼任 agent、评估者与优化者，实现多智能体自我监督进化。 技术 › LLM DAIR.AI 2026-10-12 AI论文AgentHarness EngineeringLLM多智能体强化学习自我改进DAIR.AIAgentic Search论文周报
小 小米 MiMo-V2.6 技术报告：260 万美元 RL 算力与三维扩展，将万亿 MoE 模型推入前沿梯队 文章解读小米 LLM-Core 团队发布的 MiMo-V2.6 技术报告。核心主张是通过系统性扩展强化学习算力提升模型智能，需同时扩展训练批次、环境多样性和评分器三个维度。模型有 Pro（1.02T MoE）和 Flash（310B）两个规格，RL 后训练成本分别 260 万和 90 万美元。报告详述了中期训练优化、GRPO 训练细节、多 harness 训练、四层 reward hacking 防御、Groupwise Agentic Grading 评分体系、router 冻结发现及基础设施设计，最终性能进入与 Claude Opus 5 / GPT-5.6 同一梯队，并开源蒸馏模型、环境与 RL 框架。 技术 › LLM meng shao 2026-10-10 小米MiMo强化学习MoERL后训练Reward Hacking大模型技术报告开源Agent
王 王炸，Gemini 4 Argon 全解析 Google 发布 Gemini 4 Argon，定位为面向长任务、深度推理的 frontier model，在 DeepSWE、AutomationBench、CWE-bench 等基准上表现强势，Vals Index 以 68.90% 位列第一，略超 Claude 与 GPT 系列。核心亮点是最大输出 token 上限提升至 100 万，使 agent 能在长链路任务中保持目标、纠错并完成交付。发布优先面向网络安全防御方，通过 Fairwind Program 受控开放。首发定价每百万 token 输入/输出 2/10 美元。文章认为下一代 AI 产品的差异化在于上下文、工具、权限与验收闭环，而非仅模型本身。 技术 › LLM yishan 2026-10-01 GeminiGoogle大模型BenchmarkAgent长任务网络安全定价
F Free the models: harness design at the frontier Replit 团队分享了其 Agent harness 的设计理念：与其使用模型路由器来选择 LLM，不如让主模型（core loop）自主决定子代理的层级、努力程度和委派方式。文章介绍了四个可组合的 harness 原语：领域感知子代理、子代理分层与努力度、可复用子代理、动态努力调整。在 DeepSWE 和 Terminal-Bench 基准上，Replit Agent 相比 Astra 单模型和 sidekick 架构实现了成本与得分的 Pareto 最优，作者将其视为 Sutton 'bitter lesson' 在 harness 设计中的体现：僵化的脚手架终将被可扩展的通用方法超越。 技术 › Harness Engineering Michele Catasta 2026-09-30 Agentharness设计LLMReplitGPT-6子代理委派benchmarkbitter lesson
A Agent traces are the new oil（Agent 轨迹是新石油） 文章提出 Agent 轨迹（traces）正成为 AI 公司间的核心交易资产，甚至出现蒸馏攻击等争夺行为。作者指出 RL 训练、长程任务代理普及使 traces 价值激增，但多数企业仍未善用这一资产。文章分析了 traces 对数据厂商、前沿实验室和应用层的价值，以及在运营优化、安全护栏、持续改进三方面的用途。作者还剖析了从 traces 中提取价值的难点：数据非结构化、体量巨大、反馈混乱、长程信用分配困难等，并类比 2010 年代的大数据浪潮，认为 traces 将成为企业专有智能层和战略资产，唯有通过提炼与使用才能释放价值。 技术 › Agent Sam Z Liu 2026-09-26 AgentAgent TracesLLM强化学习数据资产企业智能蒸馏攻击上下文图谱模型训练战略资产
B BestBlogs 早报 · 09-23｜Claude Opus 5.5 发布，MiMo-V2 扩展强化学习，美团分享统一推荐基座大模型实践 本期早报精讲三条内容：Anthropic 发布 Claude Opus 5.5，能力接近上代旗舰而成本下降 40%，并强化行动控制与可审计机制；小米发布 MiMo V2.6，通过更大的 Batch、全异步架构和可验证任务环境扩展强化学习，并开源训练框架与轻量 Harness；美团分享 MTFM 统一推荐基座在外卖多业务场景的落地，通过异构 Tokenizer 与混合注意力实现跨业务统一建模，订单量提升 2.06%-6.68%。速览涵盖腾讯工程师转型大模型推理、AI 产品评估实践、Agent Harness Skill Engineering、去哪儿网 AI Coding 重构、千问办公企业上下文、算力芯片趋势及 OpenAI 第三方评估原则。 技术 › LLM ginobefun 2026-09-23 ClaudeAgent强化学习推荐系统AI CodingHarness Engineering模型评估OpenAI企业上下文早报
M MiMo-V2.6 复盘：Agent RL 的规模化之路 文章解读小米罗福莉关于 MiMo-V2.6 的 RL 复盘，探讨 RL 能否像预训练一样规模化。MiMo-V2.6 单次更新抽取 1568 个任务、每任务 16 条 rollout，一个 Step 生成约 2.5 万条 Agent trajectory。文章指出 Agent RL Scale 的三大难点：需要 7000 多个 RL Task Environment 供模型进入；将 Harness 混入训练实现跨 Harness 迁移能力（You Only RL Once）；以及需要强大的 Grader 评估轨迹质量并对抗 Reward Hacking。作者认为未来模型公司的核心能力是建造 Experience Factory，让模型通过大量真实经历实现下一轮 Scaling。 技术 › LLM sleepy.md (@sleepy0x13) 2026-09-22 MiMo小米RLAgentScaling LawHarnessReward HackingExperience FactoryEnvironment强化学习
今 今夜的荣光属于MiMo V2.6 文章对比评测了同日凌晨发布的Grok 4.7与小米MiMo V2.6两款大模型。作者提出大模型的"不可能三角"——性能、价格、速度难以同时满足。Grok 4.7表现低于预期，前端能力崩盘，Coding Agent评测得分惨淡；而MiMo V2.6则超出预期，AA指数46分与Grok 4.7打平，位列开源与国产第一，CritPt科研推理仅次于GPT和Claude，价格极低（缓存输入0.025元/百万Token），输出速度达130 Token/s，成为作者心中解决不可能三角的版本答案。作者还介绍了小米公开直播强化学习训练过程的创新做法及其技术报告。 技术 › LLM 数字生命卡兹克 2026-09-22 MiMo小米Grok大模型评测强化学习模型对比开源模型价格AgentCoding
本 本周顶级AI论文精选（9月7日-13日） DAIR.AI发布本周（9月7日-13日）顶级AI论文精选，涵盖五篇论文：Procedural Graphs提出用显式程序图指导长时程智能体，减少动作漂移；FrogNano证明4B编码智能体可通过纯强化学习与在线任务合成达到竞争性能；STAIR利用目录结构改进生成式检索，显著提升召回率并降低幻觉；AI-Native Design Docs主张以自然语言设计文档为持久产物，由编码子智能体重新生成实现；PARSER将阅读与推理分离，通过并行子智能体与迭代散射-聚集提升长文档处理效率。 技术 › LLM DAIR.AI 2026-09-14 AI论文LLMAgent检索增强代码生成强化学习设计文档长文档处理生成式检索智能体
