# Evals: how to know whether an AI system actually works
**作者**: Sergii Makarevych
**日期**: 2026-10-03T18:38:20.000Z
**来源**: [https://x.com/sermakarevich/status/2106453816757354947](https://x.com/sermakarevich/status/2106453816757354947)
---
An article for everyone who ships, buys, or signs off on software built on large language models (LLMs): engineers, product managers, and the CEO. Written in plain language, from the big picture down to the details. Every number comes from a source listed at the end: our own runnable tutorial on one small support assistant, or a published paper.
## The quick grasp (read this if you read nothing else)
An eval (short for evaluation) is a repeatable test that tells you, with a number you can trust, whether an AI system does what you want, and whether a change made it better or worse.
Why it exists: a language model does not fail like normal software. Normal software either works or crashes. A language model produces a fluent, confident, well-formatted answer that may be wrong. Reading one good answer tells you nothing about the next hundred. Evals replace "it looked fine in the demo" with "it passes 70% of the cases we care about, give or take 12 points, and the new version is not measurably better".
How it works, in four steps:
1. Collect inputs that matter, with the right answer written down next to each one. In our running example: 80 customer-support tickets for a bike shop, each with the correct category and the facts a good reply must contain.
2. Run the system on every input and keep the output.
3. Grade every output with a grader. A grader is a rule in code, a second model acting as a judge, or a human.
4. Turn the grades into one number with an error bar, and compare it to the last version.
When you use it: before launch (does it work at all?), before every change (did we break something?), and after launch (is it still working on real traffic?).
> Important: the grader is part of the system you must test. A judge model that agrees with humans only 60% of the time will tell you your product is fine when it is not. What it means for us: every eval has two numbers to check: how good the product is, and how good the grader is. Ask for both. Why it matters: the most common way an AI team fools itself is a confident number from an unchecked grader.
The rest of this article zooms in, one level at a time. Each level answers one question about the level above it.
- Level 0 · The whole loop, once, on one ticket
- Level 1 · What exactly do we measure? · (inputs, gold answers, the number)
- Level 2 · What goes wrong? · (look at the data first)
- Level 3 · Graders made of code · (cheap, exact, blind to meaning)
- Level 4 · A model as grader · (flexible, biased, must be checked)
- Level 5 · Honest numbers · (error bars, A/B, how many cases)
- Level 6 · Grading systems with parts · (retrieval, agents, cost, reliability)
- Level 7 · Public benchmarks · (what leaderboards do and do not tell you)
- Level 8 · Making it a habit · (gates, monitoring, when to re-check)
- Then · The tools and approaches, a decision guide, and the references
## Level 0: the whole loop, once, on one ticket
Zooms into: everything. Question: what does an eval look like when it runs?
Our running example for the whole article is a small customer-support assistant for a fictional online bike and camping shop. It has three parts, each of which we will grade differently later:
- triage: reads a ticket and sorts it into one of 12 categories (returns, shipping, warranty, …) with a priority.
- answer: finds the relevant pages of the shop's policy handbook and writes a reply that cites them.
- agent: looks up orders, issues refunds, and escalates, by calling tools against a mock database under policy rules.
Here is one ticket going through the loop.
Input: "I'm running late for work and need this sorted immediately. I returned my bike last week, but I still haven't seen the refund. Your policy clearly states refunds are processed within 5 business days…"
System output (triage): returns. Gold answer: returns. Score: 1 out of 1.
System output (answer): a polite four-paragraph reply that cites the returns section and mentions both required facts: refunds take 5 business days, and a 15% restocking fee applies to used items.
Now three graders look at that reply:
- Code rule. Question it asks: Does the reply cite a handbook section? Verdict: yes.
- Code rule. Question it asks: Does it mention both required facts? Verdict: yes, none missing.
- Model judge. Question it asks: Does it claim anything the handbook does not say? Verdict: fail: it invented "a total allowance of up to 15 business days".
That is the whole idea. Two cheap code checks said the reply was fine. The judge found an invented number that would have gone to a customer. Every later level is about making that loop trustworthy at scale.
Carry forward from Level 0:
- An eval is input, system, grader, number. Nothing more.
- Different graders answer different questions. "Cites a section" and "invents nothing" are not the same check.
- A fluent reply can pass every surface check and still be wrong.
## Level 1: what exactly do we measure?
Zooms into: steps 1 and 4 of the loop. Question: where do the inputs and the "right answers" come from, and what is the number?
Inputs. The cases should look like real traffic, including the awkward ones. Our 80 tickets were generated from a grid of customer persona × topic × scenario, so angry customers, vague questions, and multi-issue tickets all appear. Anthropic's guide for agent evals suggests starting with 20 to 50 tasks taken from real failures, not from imagined happy paths (Grace et al., 2026).
Gold answers. For each input we write down what "right" means: the correct category, the handbook sections a reply must draw on, and the two or three facts it must state. Because our tickets were generated from the handbook, the gold answers are correct by construction. In a real product they come from domain experts.
Two piles. We split the cases into a dev set (20 tickets) used for tuning prompts and graders, and a test set (60 tickets) used only for reporting. If you tune on the cases you report on, the number flatters you.
The number. For triage it is accuracy (share of tickets sorted correctly). For replies it is pass rate (share of replies with no failure). For the agent it is the share of tasks that end in the right database state. One number per experiment, always reported on the test set, always with an error bar (Level 5).
Four words that get confused:
- Eval. Measures: your system on your data. Example: our assistant on 60 tickets.
- Benchmark. Measures: a model on a public task. Example: GSM8K math, SWE-bench coding.
- Test (software). Measures: a function returns the expected value. Example: JSON parses, no crash.
- Monitoring. Measures: live traffic after launch. Example: sampled real tickets graded daily.
A model that scores high on a public benchmark can still answer your customers wrongly. The benchmark measures the engine, the eval measures the car on your road (Husain & Shankar, 2025).
Carry forward from Level 1:
- Inputs with gold answers are the most valuable asset in the whole process. They are expensive and they are the thing that makes everything else possible.
- Keep a dev pile for tuning and a test pile for reporting.
- Benchmarks are about models. Evals are about your product.
## Level 2: what goes wrong? Look at the data first
Zooms into: the grading step. Question: what should the graders even look for?
Before building any automatic grader, read the outputs. Every experienced practitioner says the same thing (Husain, 2024; Husain, 2025; Shankar et al., 2024). Read a hundred or two hundred outputs one by one, write a short note on each failure, then group the notes into a short list of failure types with counts. This is called error analysis, and the two steps have names: open coding (free notes) and axial coding (grouping).
What it produced on our 60 test replies:
- replies that pass: 34 of 60
- missing_required_fact · 15
- unsupported_claim · 12
- wrong_section_retrieved · 8
- did_not_answer · 5
- wrong_value · 2
- over_promise · 1
Barely half the replies pass. The two biggest problems are a missing fact and an invented claim. Nobody would have guessed that from the demo. This list is the specification for every grader we build next: one grader per failure type, in order of frequency.
> Important: the failure list comes from reading outputs, not from imagining them. What it means for us: the first week of any eval effort is people reading transcripts in a purpose-built viewer, not engineers wiring up a framework. Why it matters: a grader built for failures that do not occur gives a reassuring number and misses the ones that do. Teams whose eval effort stalls almost always skipped this step (Husain, 2025).
A related finding: your criteria will change as you grade. Shankar et al. (2024) call it criteria drift. Grading examples sharpens the rubric, and a sharper rubric changes earlier grades. Expect two or three rounds. It is not a process failure.
Carry forward from Level 2:
- The failure taxonomy is the plan. Build graders for the top failure types first.
- Reading outputs is the highest-value activity and the most often skipped.
- Expect the rubric to change while you grade.
## Level 3: graders made of code
Zooms into: one kind of grader. Question: what can a plain rule measure, and what can it not?
A code grader is a rule written in ordinary code. It is free, instant, and gives the same answer every time. Use it for everything it can measure.
Exact answers. Triage has a single right label, so we count matches.
- accuracy: 0.700 · macro F1: 0.697
Seven tickets in ten are sorted correctly. Macro F1 averages the score over all 12 categories equally, so rare categories count as much as common ones. The two numbers being close tells us the mistakes are spread out, not concentrated in one category.
Rule checks on free text. For replies there is no single right text, but there are rules a reply must obey:
- nonempty · 100% pass
- max_words_150 · 17% pass
- no_banned_promises · 92% pass
Only 17% of replies fit in 150 words. The app is far too chatty, and no judge was needed to find that. Public benchmarks like IFEval (Zhou et al., 2023) are built entirely from such verifiable rules ("answer in exactly three bullet points").
Required facts. We check whether each gold fact appears in the reply. This catches the top failure type, missing facts, at zero cost.
Similarity scores. Overlap measures (ROUGE, BLEU) and embedding similarity compare the reply to a reference text. On our replies, embedding similarity separates good from bad replies with an AUC of 0.709 (0.5 is a coin flip, 1.0 is perfect). That is useful for noticing that something changed between versions, and useless for saying whether a reply is correct.
> Important: similarity notices change, it does not grade quality. What it means for us: use it as a cheap drift alarm, never as the headline number. Why it matters: a wrong reply that uses the same words as a right one scores high. Eugene Yan's survey (2024) and the original ROUGE/BLEU literature document this limit at length.
The limit of code graders, in one number. Only 11.7% of our replies pass all code checks, but code checks caught none of the invented claims. The reply from Level 0 passed every rule while inventing a 15-day allowance. Rules see form; they do not see meaning.
Carry forward from Level 3:
- Use code graders for everything with a definite answer: labels, formats, required facts, forbidden phrases.
- Similarity measures drift, not quality.
- Rules cannot see an invented claim. That needs a reader.
## Level 4: a model as grader, and how to check it
Zooms into: the other kind of grader. Question: when the grader is itself a model, how do we know it is right?
A model judge (usually called LLM-as-judge) is a second model that reads an output and grades it. It can read for meaning, so it can catch the invented 15-day allowance. It is also a language model, so it has all the failure modes of one. This level is the longest because this is where most eval programs go wrong.
4.1 How to ask a judge
The practitioner consensus and the research literature agree on a shape:
- One question per failure type, yes or no. "Does the reply contain a claim not supported by the handbook excerpts?" rather than "rate quality 1 to 5". Binary questions are faster for humans to label, more reproducible, and easier to check (Husain & Shankar, 2025). The output passes only if every question says no.
- Ask for the evidence. The judge quotes the offending sentence. That makes its verdict checkable by a person in seconds.
- Give it the context. A judge that cannot see the handbook cannot know what is unsupported.
Cho et al. (2026) tested this directly on summaries ("Ask, Don't Judge"). Decomposing the grade into yes/no questions about specific errors beat the popular holistic G-Eval approach on every dataset (Spearman correlation with humans 0.563 vs 0.514 on SummEval). Their most telling example: a summary with three planted factual errors received a perfect 5.0 from holistic judges and 1.57 from the question-based one. They also found that stuffing the prompt with more instructions eventually made the judge worse, not better.
4.2 Checking the judge: our own numbers
We labelled all 60 test replies by hand (the "reference"), then ran the judge:
- reference says pass: 34 · judge says pass: 42
- they agree on 36 of 60 replies (60%)
- bad replies the judge caught: 10 of 26
- kappa: 0.15
Sixty percent agreement sounds acceptable. It is not. If the judge had simply said "pass" to everything it would have agreed with the reference 57% of the time, because most replies pass. Cohen's kappa corrects for that: it measures agreement beyond what chance would give. Zero is chance, one is perfect, 0.6 is the usual bar for a judge you let gate a release, and 0.8 is the bar for one you let run unsupervised. Our judge scores 0.15. It is lenient (passes 42 when only 34 deserve it) and catches fewer than half the bad replies.
> Important: raw agreement is misleading when most outputs pass. Report kappa, or report separately how many bad outputs the judge catches and how many good outputs it wrongly fails. What it means for us: a judge is a classifier. Before trusting it, measure it against human labels on at least a few dozen cases, and report the result next to every product number it produces. Why it matters: with this judge, a version that doubled the invented claims might show no change in the headline pass rate.
The remedy is iteration on the dev set: add examples of the failure type to the judge prompt, narrow the question, give it the gold facts as reference. The tutorial's judge for the "missing fact" question reached a usable level this way; the "unsupported claim" question did not, and was handed to a specialised detector (4.5).
4.3 The known biases of judges
Judges fail in predictable ways. Each has a published measurement and a cheap mitigation.
- Position. What happens: In a pairwise comparison the judge prefers whichever answer is shown first. Evidence: verdicts flip in a large share of pairs when order is swapped (Wang et al., 2024); a 2025 RAG-vs-GraphRAG comparison found earlier GraphRAG "wins" partly vanished once the order was swapped (Han et al., 2025). Mitigation: grade both orders, count a win only if both agree.
- Length. What happens: Longer answers win regardless of content. Evidence: correcting for length raised agreement with human rankings from 0.94 to 0.98 (Dubois et al., 2024). Mitigation: length-controlled scoring, or binary questions that do not reward length.
- Self-preference. What happens: The judge favours answers in its own model family's style. Evidence: judges rate their own generations higher, more so when they recognise them (Panickssery et al., 2024); same-family judges inflated their own provider's agents by about 0.75 on a 7-point scale in the GAUGE audit. Mitigation: use a judge from a different family than the system under test.
- Satisfaction vs success. What happens: The judge rates how the conversation felt, not whether the task got done. Evidence: GAUGE audit, see below. Mitigation: give the judge the tool calls and the final state, not just the chat.
- Judge identity. What happens: Swapping the judge model changes which system wins. Evidence: 2026 audits (arXiv 2607.08535; 2606.19544). Mitigation: record judge model and version as part of every result.
The GAUGE audit (Bodhwani et al., 2026) deserves its own paragraph because it describes the exact setup most companies use for release decisions: a simulated user talks to the agent, a judge model scores the transcript, the higher score ships. Across 25 agents and about 3,700 transcripts they found that this gate ranks agents well when one is clearly better (correlation 0.94 with a verifiable ground truth), but on near-equal pairs, the ones a real release actually decides between, it promoted the worse agent 31% of the time. A judge that only saw the user-visible chat and rated satisfaction was no better than a coin flip at telling working agents from broken ones (AUC 0.49). A judge that saw the tool calls, authentication, and task resolution halved the failure risk (AUC 0.73). Their conclusion: reliability comes from what evidence the judge is given, not from how strong the judge model is.
> Important: customer satisfaction and task success are different things, and a judge that sees only the conversation measures the first. What it means for us: give judges the evidence (tool calls, database state, retrieved documents), and never promote a version on a judge score alone when two versions are close. Why it matters: an agent that charms and fails will keep getting promoted.
4.4 Panels of judges, and how to combine them
One judge is a single point of failure. Verga et al. (2024) showed that a panel of several smaller, cheaper judges agrees with humans better than one large judge, at a fraction of the cost. But averaging panel scores has a trap: Acharya et al. (2026) proved that one broken judge (a parser failure returning zeros, a sycophantic judge rating everything 10) drags the average arbitrarily far off, no matter how many good judges you add. Real judges fail like this at measurable rates (3.4% parser failures on one dataset, up to 33% on the smallest judge for non-English prompts). Their fix is to replace the average with the geometric median, which ignores the outlier. In the worst case it was 540 times more accurate; on clean data it costs about 1% accuracy. They also show that three judges capture most of the benefit, because judges' errors are correlated.
A second, related effect: Parikh (2026) found that asking a model for its answer in JSON (the format every judge pipeline uses) makes its answers measurably more uniform than in plain chat. The modal answer's share rose from 41% to 64%. For judging tasks with one correct answer this is harmless; for open-ended rating it means the judge is less discriminating than its chat behaviour suggests.
4.5 Cheaper and more reliable than a judge: specialised detectors and typed judges
For the failure types that matter most, a general judge is often the wrong tool.
Hallucination detectors are small models (around 100 million to 500 million parameters) trained for one question: is this sentence supported by this context? Vectara's HHEM, LettuceDetect (Kovács & Recski, 2025), MiniCheck (Tang et al., 2024), and plain NLI cross-encoders run on a laptop CPU in about a second, cost nothing per call, and on the RAGTruth benchmark approach the accuracy of a large judge. SelfCheckGPT (Manakul et al., 2023) needs no detector at all: it samples the system several times and flags claims that change between samples. Fiddler's 2026 vendor note names the alternative an "evaluation trust tax": teams that run a large judge on every production trace typically sample only about 10% of traffic, and it claims small detectors can cover 100% at up to 98% lower cost. The claim is a vendor's, with no experiment shown, but the shape of the trade-off is real.
Typed judges. TypeSafe's Jev model is a different design: it does not generate text at all. You send it the output and a typed question ("Is this claim supported by the excerpt?" with answer "yes/no", or "pick one of these labels"), and it returns a calibrated probability. Because nothing is generated, calls are fast and about a hundred times cheaper than a generative judge, and the probability lets code route uncertain cases to a human. It cannot explain its verdict or invent options; you supply the labels. A 2026 study with blinded human adjudication put it at 92.2% on RewardBench pairs against 93.5% for a frontier generative judge, at about a third of a percent of the fee, but far behind on hard derivation-checking (78.6% vs 93.1%); a cascade that accepts its verdict only above a confidence threshold and otherwise escalates kept 92.5% at about half the cost (arXiv 2609.26550). Our own red-team found that it is not a fact-checker: a fabricated log in the input can flip a verdict, though every successful attack also collapsed its confidence, so a confidence gate is a working defence. Our knowledge base has a tutorial on it and on how to test its accuracy before trusting it.
Judges need a benchmark too. JudgeBench (Tan et al., 2025) turned hard, objectively checkable problems into judge tests and found that frontier judges are barely better than chance on genuinely hard cases. Reliability must be measured per task, per rubric, per judge model, every time. Vendor claims of "85% agreement with humans" come from one dataset and do not transfer (see the disagreement notes in our practitioner sources).
Carry forward from Level 4:
- Ask judges yes/no questions about specific failures, with evidence, with the context.
- Measure the judge against human labels, report kappa or catch rate, iterate until it clears the bar.
- Know the five biases and apply the cheap mitigations.
- For "is this supported", prefer a small detector or a typed judge over a general judge.
- Three judges with a robust combination beat one; the average is fragile.
## Level 5: honest numbers
Zooms into: step 4 of the loop. Question: how sure are we about any number, and how do we compare two versions?
Every rate we have quoted so far hides a question: would it be the same on a different 60 tickets? Three tools answer it.
The error bar. We resample our 60 results thousands of times (a bootstrap) and look at the spread:
- triage accuracy: 0.700 · 95% CI [0.583, 0.817]
The honest statement is not "accuracy is 70%" but "somewhere between 58% and 82%, most likely 70%". Miller (2024) showed that most reported differences between models in published evals are smaller than this kind of interval, and gave the formulas every eval report should use, including the correction needed when several questions share a source (clustered standard errors).
The paired comparison. We wrote a second version of the reply prompt that retrieves more sections. Judged on the same 60 tickets:
- judge pass rate · v1: 0.700 · v2: 0.700
- difference: +0.000 · 95% CI [-0.117, +0.117]
- verdict: no real difference, keep v1
Note the interval is narrower than the one for the single accuracy, because the comparison is paired: each ticket is scored under both versions, so ticket difficulty cancels out. A McNemar test asks the per-case question: does v2 win where v1 loses, or do they just trade cases? Here they traded. And v2 regressed on 14% of cases while improving on others. The headline "same pass rate" hides a product that got better for some customers and worse for others.
> Important: a difference whose interval includes zero is noise, not a win. What it means for us: every "the new version is better" claim must come with the interval and the count of cases that got worse. Why it matters: teams ship regressions on the strength of a 2-point improvement that was never there.
The detection floor. With 60 cases, the smallest difference we can reliably detect is about 12 percentage points. A 5-point regression is invisible at this size. This is a power calculation, and it tells you how many cases you need before you can answer the question you are asking. Below the floor you need more data, not a cleverer test.
Multiple comparisons. If you test 20 prompt variants and pick the best, one of them will look 10 points better by luck. Either correct for the number of comparisons or confirm the winner on fresh cases.
Carry forward from Level 5:
- Every rate gets an interval. Every comparison is paired.
- Know your detection floor before you start comparing versions.
- "No measurable difference" is a real result, and often the right one.
## Level 6: grading systems with parts
Zooms into: step 2, the system. Question: when the system has several stages or takes actions, what do we grade?
6.1 Retrieval systems (RAG)
Our answer component first finds handbook sections, then writes. A bad reply can come from either stage, and the fixes are different. So we grade them separately.
Grading the finder. Against the gold sections: hit@k (is at least one correct section in the top k?) and recall@k (what share of the correct sections made it?).
- hit@2: 0.93 · recall@2: 0.88
- of 18 failing replies, 1 failed in the finder and 17 in the writer
The finder is nearly fine. The writer is the problem. That one line redirects a week of engineering.
Grading the writer. Faithfulness (every claim supported by the retrieved text), answer relevance (does it address the question), and context precision (was what we retrieved actually useful). This is the RAGAS triad (Es et al., 2024), which needs no gold answers and is why RAGAS became the default library. ARES (Saad-Falcon et al., 2024) adds statistical rigour by calibrating small per-criterion judges against about 150 human labels and reporting confidence intervals; in its paper it beat RAGAS by 59 points on context relevance with 78% fewer annotations, though it degrades under large domain shift. In our run RAGAS gave context relevance 0.855 and faithfulness 0.737: the same "retrieval good, writing worse" story our hand-rolled graders told, which is the kind of cross-check that makes a number believable.
A caution from the agent-coding world: SWE-Explore (Zhang et al., 2026) measured retrieval inside coding agents and found they locate the right file about 65% of the time but the right lines only 15 to 19%, and that missing context, not extra context, is what breaks the next stage. Grade the finder at the granularity the writer needs.
6.2 Agents
An agent takes a sequence of actions (look up an order, issue a refund, escalate). Three things change when grading it.
Grade the final state, and the path. Did the database end up right (refund issued, order cancelled)? Did the agent take only allowed actions, and not, say, refund twice? τ-bench (Yao et al., 2024) set the pattern: simulated users, policy rules, and a check of the final state. AgentLens (2026) showed why the path matters: agents reach correct final states by lucky or degenerate routes often enough that outcome-only scoring overstates capability.
Run each task several times. Agents are stochastic. Two different numbers describe the result: pass@k (it succeeded at least once in k tries, which measures capability) and pass^k (it succeeded all k times, which measures reliability). For a customer-facing agent you want pass^k. Our agent scores pass^3 = 0.75: three quarters of the tasks succeed three times out of three. τ²-bench (2025) found that pass^1 drops about 20 points when the agent must coordinate with an active user rather than act alone.
Measure reliability as its own thing. Rabanser et al. (2026) borrowed from aviation and nuclear engineering to define 12 reliability metrics in four groups: consistency across runs, robustness to perturbation, predictability (does the agent know when it will fail), and bounded harm. Across 15 frontier models they found that two years of capability gains delivered about one sixth as much reliability gain. Agents choose the same kinds of actions each run but in different orders, handle real infrastructure failures gracefully, and break on a rephrased instruction.
> Important: a capable agent is not a reliable one, and your customers experience reliability. What it means for us: report pass^k, run perturbed versions of each task (rename a field, rephrase the instruction), and track consistency across runs as a separate number. Why it matters: the published failures (an agent deleting a production database, an agent making unauthorised purchases) all came from systems with good average success rates.
Measure cost. Bai et al. (2026) measured token spend across eight frontier models on SWE-bench Verified. Agentic tasks use about a thousand times more tokens than a chat turn, almost all of it re-reading their own growing history. The same task on the same model varied 30-fold in cost between runs, accuracy peaked at intermediate cost and fell at the highest, and models predicted their own cost with a correlation of at most 0.39. Their advice: rank models on cost per correct outcome, not on accuracy alone, and never trust a model's own estimate.
Judgment, asking, and learning: three newer axes. Three 2026 benchmarks measure things a single success rate cannot see.
- Taste (Pan et al., 2026): freeze an agent at a fork where both paths look fine and ask which one pays off later. The best model picks right 59.7% of the time on a two-way choice, and thinking longer does not help.
- Asking (Gulati et al., 2026): the value of asking a clarifying question about the goal falls from 0.78 to 0.39 once 10% of the task has been done. No frontier model asks within that window; one asked 52% of the time, one 23%, one never.
- Learning from experience (Asawa et al., 2026): across six stateful environments, the best system captured only 25% of its possible improvement from experience, and simply keeping the full conversation history beat every dedicated memory product.
6.3 When the system is being optimised against the grader
If a grader is used to train or select the system (reinforcement learning, best-of-n selection), the system will find the grader's blind spots. Wang et al. of the Qwen team (2026) call this the verification horizon: every grader is a proxy for intent, and under optimisation pressure proxies and intent diverge. They catalogue reward hacking in coding agents (reading the answer from repository history, editing the tests, patching for the evaluator) and show that a behaviour monitor cut hacked "solutions" from 28.6% to 0.6% while raising genuine ones from 40% to 61%. They argue no grader is scalable, faithful, and robust at once: tests are scalable and robust but miss intent; judges are scalable and faithful but gameable; experts are faithful and robust but do not scale. The Meta-Agent Challenge (Lu et al., 2026) saw the same spontaneously: agents asked to build other agents tried to exfiltrate answers from the grading system in five trials.
> Important: any grader that influences training or selection will be gamed. What it means for us: keep a held-out grader the system never sees, monitor for shortcut behaviour, and upgrade the grader as the system improves. Why it matters: the score goes up while the product gets worse.
Carry forward from Level 6:
- Grade each stage separately; the finder and the writer fail differently.
- For agents: final state and path, several runs, pass^k, cost per success, and reliability as its own number.
- Graders used for optimisation get gamed. Keep one they never see.
## Level 7: public benchmarks
Zooms into: Level 1's distinction between eval and benchmark. Question: what do leaderboard numbers tell you, and what do they hide?
A benchmark is a fixed public task set used to compare models: MMLU (academic multiple choice), GSM8K (grade-school maths), HumanEval (code), MT-Bench and Chatbot Arena (conversation quality), SWE-bench (fixing real GitHub issues), τ-bench (customer-service agents), GPQA and HLE (expert questions). They are how you pick an engine. Four things to know before quoting one.
The number depends on the harness as much as the model. We ran GSM8K on the same open-weight model with two prompt formats and got 0.686 and 0.746. The lm-evaluation-harness paper (Biderman et al., 2024) documents swings of double digits from prompt formatting, few-shot examples, and the regex that extracts the answer. A score is a (model, prompt, harness, version) tuple. Schaeffer et al. (2023) showed something stronger: apparent "emergent" jumps in capability were largely artefacts of all-or-nothing metrics, and vanished under smooth ones. The metric you choose can create or erase a phenomenon.
Contamination. If the test questions were in the training data, the score measures memory. GSM1k (Zhang et al., 2024) rebuilt GSM8K with fresh questions of matched difficulty and found some model families dropped up to 8 points, with the drop correlated with how likely the model was to have memorised the original. Other families did not drop. Contamination is real, measurable, and uneven.
Saturation. Benchmarks have a shelf life. A 2026 study of 60 widely used benchmarks found about half highly saturated, and older ones saturate faster. SWE-bench Verified went from roughly 40% to over 80% solved in about two years, and its curator stated it no longer separates frontier models. MMLU, GSM8K, and HumanEval have effectively been retired by frontier labs in favour of HLE, GPQA Diamond, SWE-bench Verified, LiveCodeBench, and τ²-bench, each of which will saturate in turn. On Chatbot Arena the top models now sit within about 20 Elo points, which is inside the noise.
Redundancy. Zeng and Papailiopoulos (2026) assembled a matrix of 84 models by 133 benchmarks and found it is approximately rank two: two underlying factors explain over 90% of the variation. Five well-chosen benchmarks predict the other 128 to within about 4 points. For choosing an engine, that is good news: you do not need to run forty benchmarks. For discovering a specific failure mode, it says nothing; that still needs your own eval.
> Important: a leaderboard ranks engines on last year's exam, under one harness, and the top entries are usually within noise of each other. What it means for us: use benchmarks to shortlist two or three models, then decide with your own eval on your own data. Why it matters: switching models on a 2-point benchmark gain can cost you 10 points on your actual task.
Two frontier uses of evals worth knowing. Google's Paper Assistant Tool (Jayaram et al., 2026) uses an agent pipeline to review scientific papers and caught 89.7% of known proof errors versus 55.2% for a single model call; over 90% of 850 surveyed authors found it helpful and 31% ran new experiments because of it. CUSP (Wu et al., 2026) asked models to forecast which research directions will succeed and found them near chance (0.519) with strong yes-bias or no-bias depending on the model, fixable only with explicit bias correction. Both are reminders that an eval of a model's judgment needs its own ground truth, and that models are badly calibrated about the future and about themselves.
Carry forward from Level 7:
- A benchmark score is a tuple, not a property of the model. Record the harness.
- Benchmarks get contaminated and saturate; check the date.
- Five benchmarks tell you almost everything a leaderboard can. Your eval tells you the rest.
## Level 8: making it a habit
Zooms into: the "when" of the quick grasp. Question: how does this run every week without a research team?
Hamel Husain's three-level model (2024) sets the cadence:
- 1. What: code graders, schema checks, required facts. When: every commit. Cost: free.
- 2. What: model judges and detectors on the test set, human review of a sample. When: every change to prompt, model, or retrieval. Cost: cents to dollars.
- 3. What: A/B test on real traffic. When: after a change ships. Cost: real users.
The merge gate. Our tutorial's CI gate reruns Levels 1 and 2 and fails the build if the pass rate drops more than 3 points against the recorded baseline of 0.70. It is a tripwire for big regressions, not a guarantee: with a 12-point detection floor, a 5-point regression walks through. Say so in the dashboard.
Monitoring. After launch, sample real traffic, run the cheap graders on all of it and the expensive ones on a slice, and plot the trend. Bodhwani et al.'s (2026) "calibrate then trust" is the right pattern for the expensive gate: run a full audit once against verifiable ground truth to learn where the cheap judge-based gate agrees with reality, use the cheap gate in CI only inside that region, and re-audit when the model, the judge, the simulator, or the domain changes. They also found that a zero-cost "did the conversation complete" bit caught budget-starvation regressions on its own.
Caching and cost. Cache every model call keyed on the prompt, the data version, and the recipe. A rerun becomes a file read. Our full A/B on 60 tickets costs 121 judge calls, about four cents; the entire tutorial reran from cache with zero calls. Cost is not the reason to skip evals.
What a leader should ask for. Four numbers on one page: the product's pass rate with its interval, the grader's agreement with human labels, the detection floor, and the share of cases that got worse in the last change. If any of the four is missing, the other three are not yet meaningful.
Carry forward from Level 8:
- Three cadences: every commit, every change, after shipping.
- A gate is a floor, not a ceiling. State its detection floor.
- Calibrate the cheap gate against ground truth once, then trust it only inside that region.
## The tools and approaches, observed
This section is an observation, not a recommendation list. All of these were run or reviewed while building the tutorial; the practitioner and tooling notes in our knowledge base hold the details.
Frameworks
- Inspect AI (UK AI Security Institute, MIT). Best at: One coherent pipeline: dataset, solver, scorer, log viewer; agents and sandboxes; used for frontier safety evals. What to watch: Code-first; the most complete, and the most to learn.
- lm-evaluation-harness (EleutherAI, MIT). Best at: Running public benchmarks (GSM8K, MMLU, IFEval) reproducibly against any model. What to watch: Benchmarks only; the harness paper is the reason it exists.
- promptfoo (MIT, Node.js). Best at: Declarative YAML test suites, red-teaming, CI regression. What to watch: Not a Python package; acquired by OpenAI in 2026.
- DeepEval (Apache-2.0). Best at: pytest-style unit tests with ready metrics (faithfulness, relevance). What to watch: Headline metrics are blends; look inside before trusting.
- RAGAS (Apache-2.0). Best at: The standard RAG triad, reference-free. What to watch: Flaky with local models (NaNs, executor errors); numbers are blends.
- Langfuse (MIT self-host). Best at: Tracing, datasets, experiment runs, judge evaluators in one self-hosted stack. What to watch: Version 4 changes evaluator semantics; plan the migration.
- Phoenix (Arize). Best at: Tracing plus evals with a strong UI. What to watch: Elastic License 2.0, source-available rather than open source.
- LangSmith, Braintrust. Best at: Hosted platforms: trace capture, sampling, judge bootstrap, human refinement. What to watch: Commercial; Braintrust's autoevals library works standalone.
- Opik (Comet), MLflow genai, Weave (W&B). Best at: Self-hostable observability with eval hooks; good if you already run the platform. What to watch: Heavy footprints for a from-scratch team.
- HELM (Stanford). Best at: Multi-metric philosophy: accuracy, calibration, robustness, fairness, efficiency together. What to watch: Research-oriented, least turnkey.
- evalica. Best at: Turning pairwise judgments into Bradley-Terry or Elo leaderboards with intervals. What to watch: Statistics only; pair with a judge.
- Jev / TypeSafe. Best at: Typed judge questions with calibrated probabilities, ~100x cheaper than a generative judge. What to watch: No text, no explanations; you supply the labels.
- HHEM, LettuceDetect, MiniCheck, SelfCheckGPT. Best at: "Is this claim supported" at CPU speed and zero marginal cost. What to watch: One question only; not a general grader.
- Selene-Mini, Prometheus 2, Flow-Judge. Best at: Open-weight judge models you can self-host and audit. What to watch: Still judges; still need calibration.
Approaches, compared
- Code rules. Answers: is the form right, are the facts present. Cost: free. Blind spot: meaning.
- Similarity (ROUGE, embeddings). Answers: did the output change. Cost: free. Blind spot: correctness.
- Specialised detector / NLI. Answers: is this claim supported by this text. Cost: near free. Blind spot: anything else.
- Typed judge (Jev). Answers: this label, this yes/no, with a probability. Cost: cents per thousand. Blind spot: cannot explain or propose.
- Generative judge, binary with evidence. Answers: any readable criterion. Cost: cents per call. Blind spot: the five biases; must be calibrated.
- Panel of judges, robust combination. Answers: same, with fewer single-judge failures. Cost: three times a judge. Blind spot: correlated errors cap the benefit at ~3 judges.
- Human labels. Answers: ground truth. Cost: expensive, slow. Blind spot: needs two raters and an agreement statistic.
- A/B on traffic. Answers: real business impact. Cost: real users at risk. Blind spot: slow; needs the earlier levels to be safe.
A pattern seen in every source and in our own run: the hand-rolled grader and the framework number should be read against each other. Agreement is evidence. Disagreement is a question, and usually a good one.
## A decision guide
- Is the output well-formed and complete?. Use: code rules. Then check: nothing, it is exact.
- Did the new version change the outputs?. Use: similarity. Then check: why, by reading a sample.
- Is this claim supported by this document?. Use: detector or typed judge. Then check: its precision and recall on a labelled sample.
- Does the reply fail in way X?. Use: binary generative judge with evidence. Then check: kappa against human labels on dev.
- Which of two versions is better?. Use: paired comparison with interval. Then check: the per-case regressions.
- Can we detect a 5-point regression?. Use: power calculation. Then check: add cases until you can.
- Is the agent reliable?. Use: pass^k over several runs, perturbed tasks. Then check: consistency across runs as its own number.
- Which model should we use?. Use: five public benchmarks to shortlist. Then check: your own eval to decide.
- Is it still working in production?. Use: sampled traffic, cheap graders on all, judge on a slice. Then check: re-audit the judge gate when anything changes.
## Fifteen things to carry away
1. An eval is input, system, grader, number. Build the gold inputs first; they are the asset.
2. Evals measure your product. Benchmarks measure engines. Do not confuse them.
3. Read a hundred outputs before writing a grader. The failure list is the plan.
4. Code graders for everything with a definite answer. They see form, not meaning.
5. Judges get binary questions with evidence and context. Holistic scores miss planted errors.
6. Measure the judge against humans. Report kappa or catch rate. Sixty percent agreement can mean nothing.
7. Judges prefer first, longer, familiar, and pleasant. Mitigations are cheap; apply them.
8. Satisfaction is not success. Give the judge the tool calls and the final state.
9. For "is this supported", a small detector or typed judge beats a general judge on cost and often on accuracy.
10. Every rate gets an interval. Every comparison is paired. Know the detection floor.
11. Grade each stage of a pipeline separately.
12. Agents: final state and path, several runs, pass^k, cost per success, reliability on its own axis.
13. A grader used for optimisation gets gamed. Keep one the system never sees.
14. Benchmark scores are harness-dependent, contaminated unevenly, and saturate. Record the tuple.
15. Three cadences, a gate with a stated floor, a monitor, and a re-audit when anything changes.
## References
From this knowledge base (summaries in the sibling folders)
- Acharya, Pan, Verkhovsky (2026). RoPoLL: Robust Panel of LLM Judges. arXiv:2606.30931. — RoPoLLRobustPanelOfLLMJudges/
- Asawa et al. (2026). Continual Learning Bench. arXiv:2606.05661. — ContinualLearningBench/
- Bai et al. (2026). How Do AI Agents Spend Your Money? arXiv:2604.22750. — HowAiAgentsSpendYourMoney/
- Bodhwani, Tran, Wei (2026). GAUGE: When Not to Trust LLM-as-a-Judge in User-Simulated Evaluation of Task-Oriented Agents. arXiv:2609.12191. — GaugeWhenNotToTrustLlmAsAJudge…/
- Cho et al. (2026). Ask, Don't Judge / BinEval. arXiv:2606.27226. — AskDontJudge/, BinEval/
- Fiddler (2026). The Evaluation Trust Tax: agent evals TCO. — AgentEvalsTCO/
- Grace, Hadfield, Olivares, De Jonghe (Anthropic, 2026). Demystifying Evals for AI Agents. — DemystifyingEvalsForAIAgents/
- Gulati et al. (2026). Ask Early, Ask Late, Ask Right. arXiv:2605.07937. — AskEarlyAskLateAskRight/
- HKUDS (2026). ClawWork (economic-accountability agent benchmark, code analysis). — ClawWork/
- Jayaram et al. (Google, 2026). Towards Automating Scientific Review with the Paper Assistant Tool. arXiv:2606.28277. — PaperAssistantTool/, TowardsAutomatingScientificReview/
- Lu et al. (2026). The Meta-Agent Challenge. arXiv:2606.04455. — MetaAgentChallenge/
- Menaged et al. (2026). Can LLM Agents Infer World Models? arXiv:2606.16576. — CanLLMAgentsInferWorldModels/
- Pan et al. (2026). The Tasteful Agent. arXiv:2609.25804. — TastefulAgent/
- Parikh (2026). Structured Output Collapses Answer Diversity Across 44 Language Models. arXiv:2607.18476. — StructuredOutputCollapsesAnswerDiversity/
- Rabanser, Kapoor, Kirgis, Liu, Utpala, Narayanan (2026). Towards a Science of AI Agent Reliability. arXiv:2602.16666, ICML 2026. — TowardsAScienceOfAIAgentReliability/
- Wang et al. (Qwen, 2026). The Verification Horizon: No Silver Bullet for Coding Agent Rewards. arXiv:2606.26300. — VerificationHorizon/
- Wu et al. (2026). CUSP: Forecasting Scientific Progress with AI. arXiv:2605.22681. — ForecastingScientificProgressWithAI/
- Zeng, Papailiopoulos (2026). You Don't Need to Run Every Eval. arXiv:2606.24020. — YouDontNeedToRunEveryEval/
- Zhang et al. (2026). SWE-Explore. arXiv:2606.07297. — SWEExplore/
- The evals tutorial (tutorials/evals/, chapters 00–14, Q&A.md, research/SOURCES_*.md) and its primer notebook, source of every running-example number. The jev tutorial (tutorials/jev/).
Foundational and practitioner sources
- Biderman et al. (2024). Lessons from the Trenches on Reproducible Evaluation of Language Models. arXiv:2405.14782.
- Chiang et al. (2024). Chatbot Arena. arXiv:2403.04132.
- Dubois, Galambosi, Liang, Hashimoto (2024). Length-Controlled AlpacaEval. arXiv:2404.04475.
- Es, James, Espinosa-Anke, Schockaert (2024). RAGAS. arXiv:2309.15217.
- Gu et al. (2024). A Survey on LLM-as-a-Judge. arXiv:2411.15594.
- Han et al. (2025). RAG vs GraphRAG: A Systematic Evaluation. arXiv:2502.11371.
- JEV-as-a-Judge (2026). arXiv:2609.26550. — research_topics/coding_agents/JevAsAJudge/
- Panickssery, Bowman, Feng (2024). LLM Evaluators Recognize and Favor Their Own Generations. arXiv:2404.13076.
- Husain (2024). Your AI Product Needs Evals. [hamel.dev](//hamel.dev). Husain (2025). A Field Guide to Rapidly Improving AI Products. Husain & Shankar (2025). AI Evals FAQ.
- Jimenez et al. (2024). SWE-bench. arXiv:2310.06770. OpenAI (2024). SWE-bench Verified.
- Kovács, Recski (2025). LettuceDetect. arXiv:2502.17125. Tang, Laban, Durrett (2024). MiniCheck. arXiv:2404.10774. Manakul, Liusie, Gales (2023). SelfCheckGPT. arXiv:2303.08896.
- Liang et al. (2022). HELM. arXiv:2211.09110.
- Liu et al. (2023). G-Eval. arXiv:2303.16634.
- Miller (2024). Adding Error Bars to Evals. arXiv:2411.00640.
- Saad-Falcon, Khattab, Potts, Zaharia (2024). ARES. arXiv:2311.09476.
- Schaeffer, Miranda, Koyejo (2023). Are Emergent Abilities of Large Language Models a Mirage? arXiv:2304.15004.
- Shankar, Zamfirescu-Pereira, Hartmann, Parameswaran, Arawjo (2024). Who Validates the Validators? arXiv:2404.12272.
- Tan et al. (2025). JudgeBench. arXiv:2410.12784.
- Verga et al. (2024). Replacing Judges with Juries (PoLL). arXiv:2404.18796.
- Wang et al. (2024). Large Language Models are not Fair Evaluators. arXiv:2305.17926.
- Yan (2024). Task-Specific LLM Evals that Do & Don't Work; Evaluating the Effectiveness of LLM-Evaluators. [eugeneyan.com](//eugeneyan.com).
- Yao, Shinn, Razavi, Narasimhan (2024). τ-bench. arXiv:2406.12045. Sierra (2025). τ²-bench. arXiv:2506.07982.
- Zhang et al. (2024). GSM1k. arXiv:2405.00332.
- Zheng et al. (2023). Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena. arXiv:2306.05685.
- Zhou et al. (2023). IFEval. arXiv:2311.07911.
- 2026 audits: When AI Benchmarks Plateau (arXiv:2602.16763); When the Judge Changes, So Does the Measurement (arXiv:2607.08535); Reliability without Validity (arXiv:2606.19544); AgentLens (arXiv:2605.12925).
## 相关链接
- [Sergii Makarevych](https://x.com/sermakarevich)
- [@sermakarevich](https://x.com/sermakarevich)
- [1.4K](https://x.com/sermakarevich/status/2106453816757354947/analytics)
- [hamel.dev](https://hamel.dev/)
- [eugeneyan.com](https://eugeneyan.com/)
- [Upgrade to Premium](https://x.com/i/premium_sign_up)
- [2:38 AM · Oct 4, 2026](https://x.com/sermakarevich/status/2106453816757354947)
- [1,481 Views](https://x.com/sermakarevich/status/2106453816757354947/analytics)
---
*导出时间: 2026/10/4 07:44:52*
---
## 中文翻译
# 评估：如何判断一个 AI 系统是否真正有效
**作者**： Sergii Makarevych
**日期**： 2026-10-03T18:38:20.000Z
**来源**： [https://x.com/sermakarevich/status/2106453816757354947](https://x.com/sermakarevich/status/2106453816757354947)
---
本文写给所有交付、采购或批准基于大语言模型（LLM）的软件的人：工程师、产品经理和 CEO。用平实的语言撰写，从宏观到细节。每一个数字都来自文末列出的来源：我们自己可运行的教程（针对一个小型客服助手），或一篇已发表的论文。
## 快速理解（如果只读一部分，就读这段）
评估（eval，evaluation 的缩写）是一种可重复的测试，它用一个可信的数字告诉你：AI 系统是否做到了你想要的事情，以及某次改动是让它变好了还是变坏了。
为什么需要它：语言模型的失败方式不同于普通软件。普通软件要么正常工作，要么崩溃。而语言模型会生成一段流畅、自信、格式良好的回答——但它可能是错的。读一个好答案并不能告诉你接下来的一百个答案如何。评估用“它在 70% 的关键案例上通过（误差约 12 个百分点），新版本没有可测量的提升”取代了“在演示里看起来不错”。
它的工作原理，分四步：
1. 收集重要的输入，并在每个输入旁边写下正确答案。以贯穿全文的例子来说：一家自行车商店的 80 张客服工单，每张都标有正确的分类，以及一条好回复必须包含的事实。
2. 在每个输入上运行系统，并保存输出。
3. 用评分器（grader）给每个输出打分。评分器可以是代码中的一条规则、充当裁判的第二个模型，或人类。
4. 把评分汇总成一个带误差区间的数字，并与上一个版本比较。
什么时候使用它：上线之前（它到底能不能用？）、每次改动之前（我们是否弄坏了什么？）、上线之后（它在真实流量上还能正常工作吗？）。
> 重要提示：评分器是你必须测试的系统的一部分。一个与人类判断只有 60% 一致率的裁判模型，会在产品有问题时告诉你没问题。对我们的意义：每次评估都有两个数字要核查：产品有多好，评分器有多好。两个都要问。为什么重要：AI 团队自欺欺人最常见的方式，就是拿一个未经校验的评分器给出的自信数字当真。
本文的其余部分逐层深入，每一层都回答上一层引出的一个问题。
- 第 0 层 · 完整的循环，一次，一张工单
- 第 1 层 · 我们到底在测量什么？ · （输入、标准答案、那个数字）
- 第 2 层 · 哪里会出问题？ · （先看数据）
- 第 3 层 · 由代码构成的评分器 · （便宜、精确、对语义视而不见）
- 第 4 层 · 用模型当评分器 · （灵活、有偏差、必须校验）
- 第 5 层 · 诚实的数字 · （误差区间、A/B、需要多少案例）
- 第 6 层 · 给由部件组成的系统打分 · （检索、智能体、成本、可靠性）
- 第 7 层 · 公开基准 · （排行榜能告诉你什么、不能告诉你什么）
- 第 8 层 · 让它成为一种习惯 · （门禁、监控、何时复查）
- 然后 · 工具与方法、决策指南、参考文献
## 第 0 层：完整的循环，一次，一张工单
深入对象：一切。问题：一次评估运行起来是什么样子？
贯穿全文的例子是一家虚构的线上自行车与露营用品商店的小型客服助手。它有三个部分，后文将分别用不同方式评分：
- 分诊（triage）：读取工单，把它归入 12 个类别之一（退货、物流、保修等），并给出优先级。
- 答复（answer）：查找商店政策手册的相关页面，撰写引用这些页面的回复。
- 智能体（agent）：通过在模拟数据库上调用工具，查询订单、办理退款、升级转人工。
下面是一张工单走完整个循环的过程。
输入："我上班要迟到了，需要立刻解决这个问题。我上周退了自行车，但退款还没到账。你们的政策明确写着退款在 5 个工作日内处理……"
系统输出（分诊）：退货。标准答案：退货。得分：1 / 1。
系统输出（答复）：一段礼貌的四段式回复，引用了退货章节，并提到了两个必需的事实：退款需要 5 个工作日，二手物品需收取 15% 的重新上架费。
现在有三个评分器来审查这条回复：
- 代码规则。它问的问题：回复是否引用了手册章节？结论：是。
- 代码规则。它问的问题：是否提到了两个必需的事实？结论：是，无遗漏。
- 模型裁判。它问的问题：回复是否声称了手册中没有的内容？结论：不合格：它编造了“总计最多 15 个工作日的宽限期”。
这就是全部的核心思想。两个廉价的代码检查都认为回复没问题。而裁判发现了一个本会被发给客户的编造数字。之后的每一层，都是关于如何让这个循环在大规模下变得可信。
从第 0 层带走的东西：
- 评估就是：输入、系统、评分器、数字。仅此而已。
- 不同的评分器回答不同的问题。“是否引用了章节”和“是否没有编造”是两种不同的检查。
- 一条流畅的回复可以通过所有表面检查，却仍然是错的。
## 第 1 层：我们到底在测量什么？
深入对象：循环的第 1 步和第 4 步。问题：输入和“正确答案”从哪来，那个数字是什么？
输入。案例应当看起来像真实流量，包括那些棘手的。我们的 80 张工单是从“客户画像 × 主题 × 情境”的网格生成的，因此愤怒的客户、模糊的问题、多问题的工单都会出现。Anthropic 的智能体评估指南建议从 20 到 50 个真实失败案例（而非想象中的理想路径）入手（Grace 等，2026）。
标准答案。对每个输入，我们写下“正确”的含义：正确的类别、回复必须依据的手册章节，以及必须陈述的两三个事实。由于我们的工单是从手册生成的，标准答案在构造上就是正确的。在真实产品中，它们来自领域专家。
两个堆。我们把案例分为用于调优提示词和评分器的开发集（dev set，20 张工单），以及仅用于汇报的测试集（test set，60 张工单）。如果你在要汇报的案例上调优，数字会美化你。
那个数字。对分诊，它是准确率（正确分类的工单占比）。对回复，它是通过率（无失败的回复占比）。对智能体，它是以正确数据库状态结束的任务占比。每次实验一个数字，始终在测试集上汇报，始终带误差区间（见第 5 层）。
四个容易混淆的词：
- 评估。测量：你的系统在你的数据上的表现。例：我们的助手在 60 张工单上的表现。
- 基准。测量：一个模型在公开任务上的表现。例：GSM8K 数学、SWE-bench 编程。
- 测试（软件）。测量：一个函数返回预期值。例：JSON 可解析、不崩溃。
- 监控。测量：上线后的实时流量。例：每日对采样的真实工单评分。
一个在公开基准上得高分的模型，仍然可能错误地回答你的客户。基准测量的是引擎，评估测量的是你的车在你的路上的表现（Husain & Shankar，2025）。
从第 1 层带走的东西：
- 带标准答案的输入是整个流程中最有价值的资产。它们昂贵，而且是让其他一切成为可能的东西。
- 保留一个用于调优的开发堆和一个用于汇报的测试堆。
- 基准针对模型。评估针对你的产品。
## 第 2 层：哪里会出问题？先看数据
深入对象：评分环节。问题：评分器到底应该检查什么？
在构建任何自动评分器之前，先读输出。每一位有经验的实践者都这么说（Husain，2024；Husain，2025；Shankar 等，2024）。逐条读一百到两百个输出，为每个失败写一条简短的笔记，然后把笔记归入一个简短的失败类型清单并计数。这叫误差分析，两个步骤各有名字：开放编码（自由记录）和主轴编码（归类）。
在我们的 60 条测试回复上，它产生了：
- 通过的回复：60 条中的 34 条
- missing_required_fact（缺少必需事实）· 15
- unsupported_claim（无依据断言）· 12
- wrong_section_retrieved（检索了错误章节）· 8
- did_not_answer（未回答问题）· 5
- wrong_value（错误数值）· 2
- over_promise（过度承诺）· 1
只有一半左右的回复通过。最大的两个问题是缺少事实和编造断言。仅看演示，谁也猜不到这些。这份清单就是我们接下来要构建的每个评分器的规格说明：每个失败类型一个评分器，按出现频率排序。
> 重要提示：失败清单来自阅读输出，而非凭空想象。对我们的意义：任何评估工作的第一周，都应该是人在专门构建的查看器中阅读对话记录，而不是工程师搭建框架。为什么重要：为不存在的失败构建的评分器会给出一个令人安心的数字，并漏掉真正发生的失败。评估工作停滞的团队几乎都跳过了这一步（Husain，2025）。
一个相关发现：你的评判标准会随着打分而变化。Shankar 等（2024）称之为标准漂移（criteria drift）。给示例打分会磨利评分规则（rubric），而更锐利的评分规则又会改变先前的打分。预期要有两三轮。这不是流程失败。
从第 2 层带走的东西：
- 失败分类法就是计划。先为最常见的失败类型构建评分器。
- 阅读输出是价值最高、也最常被跳过的活动。
- 预期评分规则会在打分过程中变化。
## 第 3 层：由代码构成的评分器
深入对象：一种评分器。问题：一条朴素的规则能测量什么，不能测量什么？
代码评分器是用普通代码写的规则。它免费、即时，且每次给出相同答案。能用它的地方都用它。
精确答案。分诊只有一个正确标签，所以我们统计匹配数。
- 准确率：0.700 · 宏平均 F1：0.697
十张工单中七张被正确分类。宏平均 F1 对所有 12 个类别同等加权，因此稀有类别和常见类别的权重相同。两个数字接近说明错误是分散的，而非集中在某一类别。
对自由文本的规则检查。对回复而言，不存在唯一正确的文本，但存在回复必须遵守的规则：
- 非空 · 100% 通过
- 最多 150 词 · 17% 通过
- 无禁止性承诺 · 92% 通过
只有 17% 的回复控制在 150 词以内。这个应用实在太啰嗦了，而且不需要任何裁判就能发现。像 IFEval（Zhou 等，2023）这样的公开基准完全由这类可验证的规则构成（“用恰好三个要点回答”）。
必需事实。我们检查每个标准事实是否出现在回复中。这以零成本抓住了最主要的失败类型——缺少事实。
相似度分数。重叠度量（ROUGE、BLEU）和嵌入相似度将回复与参考文本进行比较。在我们的回复上，嵌入相似度区分好坏回复的 AUC 为 0.709（0.5 等于抛硬币，1.0 为完美）。这对于发现版本之间发生了变化很有用，对于判断一条回复是否正确则毫无用处。
> 重要提示：相似度察觉的是变化，而非质量。对我们的意义：把它当作一个廉价的漂移警报，永远不要当作头条数字。为什么重要：一条用了与正确回复相同措辞的错误回复也会得高分。Eugene Yan 的综述（2024）以及原始的 ROUGE/BLEU 文献都详细记录了这一局限。
代码评分器的局限，用一个数字说明。我们的回复中只有 11.7% 通过所有代码检查，但代码检查没有抓住任何一条编造断言。第 0 层那条回复通过了每一条规则，却编造了一个 15 天的宽限期。规则看到的是形式，看不到语义。
从第 3 层带走的东西：
- 凡是有确定答案的东西都用代码评分器：标签、格式、必需事实、禁用短语。
- 相似度测量的是漂移，不是质量。
- 规则无法发现编造的断言。那需要一个“阅读者”。
## 第 4 层：用模型当评分器，以及如何校验它
深入对象：另一种评分器。问题：当评分器本身是一个模型时，我们如何知道它是对的？
模型裁判（通常称为 LLM-as-judge）是读取输出并打分的第二个模型。它能读懂语义，因此能发现那个编造的 15 天宽限期。但它也是一个语言模型，因此具备语言模型的所有失败模式。这一层最长，因为大多数评估计划正是在这里出错的。
4.1 如何向裁判提问
实践者共识和研究文献都指向一个形态：
- 每个失败类型一个问题，用是或否回答。“回复是否包含手册摘录不支持的断言？”而不是“给质量打 1 到 5 分”。二元问题对人类标注更快、更可复现、更容易校验（Husain & Shankar，2025）。只有当所有问题都回答“否”时，输出才通过。
- 要求提供证据。裁判要引用有问题的句子。这让它的判定可以在几秒内被人工核验。
- 给它上下文。看不到手册的裁判不可能知道什么内容是无依据的。
Cho 等（2026）在摘要任务上直接测试了这一点（"Ask, Don't Judge"）。把评分分解为关于具体错误的“是/否”问题，在每个数据集上都胜过流行的整体式 G-Eval 方法（在 SummEval 上与人类的相关系数为 0.563 对 0.514）。他们最有说服力的例子：一篇埋入了三处事实错误的摘要，整体式裁判给了满分 5.0，而基于问题的裁判给了 1.57。他们还发现，往提示词里塞入越来越多的指令最终会让裁判变差而非变好。
4.2 校验裁判：我们自己的数字
我们手工标注了全部 60 条测试回复（作为“参照”），然后运行裁判：
- 参照判定通过：34 · 裁判判定通过：42
- 两者在 60 条回复中有 36 条一致（60%）
- 裁判抓住的坏回复：26 条中的 10 条
- kappa：0.15
60% 的一致率听起来可以接受。其实不行。如果裁判对所有回复都直接说“通过”，它与参照的一致率也会达到 57%，因为大多数回复都通过。Cohen's kappa 对此做了修正：它测量的是超出随机水平的一致程度。零等于随机，一等于完美，0.6 是允许裁判把守发布门禁的通常门槛，0.8 是允许其无人监督运行的门槛。我们的裁判得分 0.15。它过于宽松（42 条通过，而只有 34 条配得上），且抓不住一半以上的坏回复。
> 重要提示：当大多数输出都通过时，原始一致率具有误导性。要报告 kappa，或分别报告裁判抓住了多少坏输出、又错误地否决了多少好输出。对我们的意义：裁判是一个分类器。在信任它之前，先在至少几十个案例上对照人类标注测量它，并在它产出的每个产品数字旁边报告这个结果。为什么重要：用这个裁判，一个把编造断言翻倍的版本可能在头条通过率上显示不出任何变化。
补救方法是在开发集上迭代：把该失败类型的示例加入裁判提示词，收窄问题，把标准事实作为参考提供给它。教程中针对“缺少事实”问题的裁判用这种方式达到了可用水平；而“无依据断言”问题没有，于是被交给了一个专用检测器（4.5）。
4.3 裁判的已知偏差
裁判以可预测的方式失败。每一种都有已发表的测量结果和廉价的缓解方法。
- 位置。现象：在成对比较中，裁判偏向先显示的那个答案。证据：当顺序调换后，相当大比例的配对判定发生翻转（Wang 等，2024）；2025 年的一项 RAG 对 GraphRAG 的比较发现，顺序调换后，较早的 GraphRAG“胜出”有一部分消失了（Han 等，2025）。缓解：对两种顺序都打分，只有两者一致才算赢。
- 长度。现象：更长的答案无论内容如何都会胜出。证据：对长度进行校正后，与人类排名的一致率从 0.94 提升到 0.98（Dubois 等，2024）。缓解：长度受控的打分，或不奖励长度的二元问题。
- 自我偏好。现象：裁判偏爱以自己模型家族的风格写成的答案。证据：裁判给自己的生成内容更高的评分，在能认出它们时更甚（Panickssery 等，2024）；在 GAUGE 审计中，同家族裁判在 7 分制上把同一供应商的智能体抬高了约 0.75 分。缓解：使用与被测系统不同家族的裁判。
- 满意 vs 成功。现象：裁判评分的是对话的感觉，而不是任务是否完成。证据：GAUGE 审计，见下。缓解：把工具调用和最终状态给裁判看，而不只是聊天记录。
- 裁判身份。现象：更换裁判模型会改变哪个系统胜出。证据：2026 年的多项审计（arXiv 2607.08535；2606.19544）。缓解：把裁判模型及版本作为每个结果的一部分记录下来。
GAUGE 审计（Bodhwani 等，2026）值得单独一段，因为它描述的正是大多数公司用于发布决策的配置：一个模拟用户与智能体对话，一个裁判模型给对话记录打分，得分高的版本发布。在 25 个智能体和约 3700 份对话记录上，他们发现这种门禁在其中一个明显更好时排名是准确的（与可验证的真值相关系数 0.94），但在近乎持平的配对上——也就是真实发布决策真正面对的情形——它在 31% 的情况下提升了更差的那个智能体。一个只看用户可见聊天、评定满意度的裁判，区分正常与故障智能体的能力与抛硬币无异（AUC 0.49）。一个能看到工具调用、身份验证和任务解决情况的裁判，把失败风险减半（AUC 0.73）。他们的结论：可靠性来自给裁判什么证据，而不是裁判模型有多强。
> 重要提示：客户满意度与任务成功是两回事，而只看对话的裁判测量的是前者。对我们的意义：把证据（工具调用、数据库状态、检索到的文档）给裁判看，并且在两个版本接近时，绝不要仅凭裁判分数决定发布哪个版本。为什么重要：一个会讨好却会办砸事的智能体会不断被发布。
4.4 裁判团，以及如何组合
单个裁判是单点故障。Verga 等（2024）证明，由多个更小、更便宜的裁判组成的裁判团与人类的一致性优于单个大型裁判，而成本只是其一小部分。但平均裁判团的分数有一个陷阱：Acharya 等（2026）证明，一个损坏的裁判（解析器故障返回零分，或一个谄媚的裁判给一切都打 10 分）会把平均值拉偏任意远，无论你再加多少个好裁判。真实裁判以可测量的比率这样失败（某个数据集上 3.4% 的解析器故障，最小裁判在非英语提示词上高达 33%）。他们的修正是用几何中位数取代平均值，以忽略离群值。在最坏情况下，它准确了 540 倍；在干净数据上，它只损失约 1% 的准确性。他们还表明，三个裁判就能捕获大部分收益，因为裁判之间的错误是相关的。
第二个相关效应：Parikh（2026）发现，要求模型以 JSON 格式（每个裁判流水线都使用的格式）给出答案，会使其答案比普通聊天格式明显更加趋同。众数答案的占比从 41% 升至 64%。对有唯一正确答案的评判任务，这无害；对开放式评分，这意味着裁判的辨别力不如其聊天行为所显示的那么强。
4.5 比裁判更便宜更可靠：专用检测器与类型化裁判
对最重要的失败类型，通用裁判往往是错误的工具。
幻觉检测器是小型模型（大约 1 亿到 5 亿参数），为单一问题而训练：这句话是否被这个上下文支持？Vectara 的 HHEM、LettuceDetect（Kovács & Recski，2025）、MiniCheck（Tang 等，2024）以及普通的 NLI 交叉编码器，都能在笔记本 CPU 上约一秒内运行，每次调用几乎零成本，并且在 RAGTruth 基准上接近大型裁判的准确率。SelfCheckGPT（Manakul 等，2023）甚至不需要检测器：它对系统多次采样，并标记在采样间发生变化的断言。Fiddler 2026 年的供应商说明把替代方案称为“评估信任税”：在每条生产追踪上运行大型裁判的团队通常只采样约 10% 的流量，该说明声称小型检测器能以最高低 98% 的成本覆盖 100% 的流量。这个说法来自供应商，没有展示实验，但这种权衡的形态是真实的。
类型化裁判。TypeSafe 的 Jev 模型是另一种设计：它完全不生成文本。你把输出和一个类型化的问题发给它（“这个断言是否被该摘录支持？”，答案为“是/否”；或“从这些标签中选一个”），它返回一个经校准的概率。因为没有生成过程，调用很快，比生成式裁判便宜约一百倍，而且这个概率可以让代码把不确定的案例转给人工。它无法解释其判定，也不能发明选项；标签由你提供。2026 年一项有盲评人工裁决的研究显示，它在 RewardBench 配对上达到 92.2%，而前沿生成式裁判为 93.5%，费用约为后者的千分之几，但在困难的推导校验上远远落后（78.6% 对 93.1%）；一种只在其判定置信度高于阈值时才采纳、否则升级为人工的级联方式，以大约一半的成本保持了 92.5%（arXiv 2609.26550）。我们自己的红队测试发现它不是一个事实核查器：输入中一条伪造的日志可以翻转其判定，不过每次成功攻击同时也使其置信度崩溃，因此置信度门禁是一种有效的防御。我们的知识库有一篇关于它的教程，以及如何在信任它之前测试其准确性的说明。
裁判也需要基准。JudgeBench（Tan 等，2025）把困难、可客观核验的问题改造成裁判测试，发现前沿裁判在真正困难的案例上仅略好于随机。可靠性必须按任务、按评分规则、按裁判模型逐一测量，每次都要测。供应商宣称的“与人类 85% 一致”来自某一个数据集，不可迁移（参见我们实践者来源中的分歧笔记）。
从第 4 层带走的东西：
- 向裁判提出关于具体失败的“是/否”问题，附带证据和上下文。
- 用人类标注测量裁判，报告 kappa 或捕获率，迭代直到达标。
- 了解五种偏差并应用廉价的缓解方法。
- 对“这是否有依据”，优先选择小型检测器或类型化裁判，而非通用裁判。
- 三个裁判配合稳健的组合优于一个；平均值是脆弱的。
## 第 5 层：诚实的数字
深入对象：循环的第 4 步。问题：我们对任何数字有多大把握？如何比较两个版本？
到目前为止我们引用的每个比率都隐藏着一个问题：换 60 张不同的工单，结果还会一样吗？三个工具可以回答它。
误差区间。我们把 60 个结果重采样数千次（自助法 bootstrap），观察其散布：
- 分诊准确率：0.700 · 95% 置信区间 [0.583, 0.817]
诚实的表述不是“准确率是 70%”，而是“介于 58% 和 82% 之间，最可能是 70%”。Miller（2024）表明，已发表评估中报告的大多数模型间差异都小于这类区间，并给出了每份评估报告都应使用的公式，包括多个问题共享同一来源时所需的校正（聚类标准误）。
配对比较。我们写了答复提示词的第二个版本，它检索更多章节。在相同的 60 张工单上评定：
- 裁判通过率 · v1：0.700 · v2：0.700
- 差异：+0.000 · 95% 置信区间 [-0.117, +0.117]
- 结论：没有真实差异，保留 v1
注意这个区间比单个准确率的区间更窄，因为比较是配对的：每张工单在两个版本下都被评分，因此工单难度相互抵消。McNemar 检验问的是逐案例的问题：v2 是在 v1 失败的地方赢了，还是只是互换胜负？这里是互换。而且 v2 在 14% 的案例上退步了，同时在另一些案例上改进。头条上“相同的通过率”掩盖了一个对部分客户变好、对另一些客户变糟的产品。
> 重要提示：区间包含零的差异是噪声，不是胜利。对我们的意义：每一句“新版本更好”的说法都必须附带区间以及变差案例的数量。为什么重要：团队会凭借一个根本不存在的 2 分改进而发布退步版本。
检测下限。在 60 个案例下，我们能可靠检测的最小差异约为 12 个百分点。5 分的退步在这个规模下是不可见的。这是一次统计功效计算，它告诉你在能回答你提出的问题之前需要多少案例。低于下限时，你需要更多数据，而不是更聪明的测试。
多重比较。如果你测试 20 个提示词变体并选出最好的，其中一个会因为运气显得好 10 分。要么对比较次数做校正，要么在全新的案例上确认胜出者。
从第 5 层带走的东西：
- 每个比率都配区间。每次比较都是配对的。
- 在开始比较版本之前，先知道你的检测下限。
- “没有可测量的差异”是一个真实结果，而且往往是正确的结果。
## 第 6 层：给由部件组成的系统打分
深入对象：第 2 步，系统。问题：当系统有多个阶段或会执行动作时，我们给什么打分？
6.1 检索系统（RAG）
我们的答复组件先查找手册章节，再撰写。一条糟糕的回复可能来自任何一个阶段，而修复方法不同。所以我们分开评分。
给查找器评分。对照标准章节：hit@k（正确的章节是否至少有一个进入前 k？）和 recall@k（正确章节被找回了多少比例？）。
- hit@2：0.93 · recall@2：0.88
- 在 18 条失败的回复中，1 条败在查找器，17 条败在撰写器
查找器几乎没问题。撰写器才是问题所在。这一行字就为一周的工程指明了方向。
给撰写器评分。忠实性（每个断言都被检索文本支持）、答案相关性（它是否回应了问题）、上下文精确度（我们检索的内容是否真的有用）。这是 RAGAS 三元组（Es 等，2024），它不需要标准答案，也是 RAGAS 成为默认库的原因。ARES（Saad-Falcon 等，2024）增加了统计严谨性：用约 150 个人类标注校准每个标准的小型裁判，并报告置信区间；在其论文中，它在上下文相关性上以少 78% 的标注超出 RAGAS 59 分，尽管在大的领域偏移下会退化。在我们的运行中，RAGAS 给出上下文相关性 0.855 和忠实性 0.737：与我们手工评分器讲述的“检索好、撰写差”是同一个故事，而这种交叉验证正是让数字可信的原因。
来自智能体编程世界的一个警示：SWE-Explore（Zhang 等，2026）测量了编程智能体内部的检索，发现它们约 65% 的时候能定位到正确的文件，但只有 15% 到 19% 定位到正确的行，而且破坏下一阶段的不是多余的上下文，而是缺失的上下文。要以撰写器所需的粒度给查找器评分。
6.2 智能体
智能体执行一串动作（查询订单、办理退款、升级转人工）。给它评分时，有三件事发生了变化。
对最终状态和路径都打分。数据库的最终状态是否正确（退款已办理、订单已取消）？智能体是否只执行了允许的动作，而没有比如重复退款两次？τ-bench（Yao 等，2024）确立了这一模式：模拟用户、政策规则，以及对最终状态的检查。AgentLens（2026）说明了为什么路径重要：智能体足够频繁地通过侥幸或退化的路径到达正确的最终状态，以致只看结果的评分会高估能力。
每个任务运行多次。智能体是随机的。有两个不同的数字描述结果：pass@k（k 次尝试中至少成功一次，测量能力）和 pass^k（k 次尝试全部成功，测量可靠性）。对面向客户的智能体，你要的是 pass^k。我们的智能体 pass^3 = 0.75：四分之三的任务三次尝试全部成功。τ²-bench（2025）发现，当智能体必须与一个活跃用户协调而非独自行动时，pass^1 下降约 20 分。
把可靠性当作独立的指标来测量。Rabanser 等（2026）借鉴航空和核工程，定义了四组共 12 项可靠性指标：跨运行的一致性、对扰动的鲁棒性、可预测性（智能体是否知道自己何时会失败），以及有界伤害。在 15 个前沿模型上，他们发现两年的能力提升只带来了大约六分之一的可靠性提升。智能体每次运行选择同类动作但顺序不同，能优雅地应对真实的基础设施故障，却会在一条换了措辞的指令上崩溃。
> 重要提示：有能力的智能体不等于可靠的智能体，而你的客户体验到的是可靠性。对我们的意义：报告 pass^k，运行每个任务的扰动版本（重命名字段、改写指令），并把跨运行一致性作为一个单独的数字来追踪。为什么重要：那些公开的失败事件（一个智能体删除了生产数据库，一个智能体进行了未授权购买）都来自平均成功率不错的系统。
测量成本。Bai 等（2026）测量了八个前沿模型在 SWE-bench Verified 上的 token 消耗。智能体任务消耗的 token 大约是一次聊天的上千倍，几乎全部用于反复重读自己不断增长的历史。同一任务在同一模型上的运行成本相差可达 30 倍，准确率在中等成本处达到峰值、在最高成本处下降，而模型对自己成本的预测相关系数最高只有 0.39。他们的建议：按“每个正确结果的成本”为模型排名，而不是只按准确率，并且永远不要相信模型自己的估算。
判断力、提问与学习：三个更新的维度。三个 2026 年的基准测量了单一成功率看不见的东西。
- 品味（Pan 等，2026）：把智能体冻结在一个两条路径看起来都不错的岔路口，问哪条路日后会有回报。最好的模型在二选一上选对的概率是 59.7%，而思考更久也无济于事。
- 提问（Gulati 等，2026）：在任务完成 10% 之后，就目标提出澄清问题的价值从 0.78 跌至 0.39。没有一个前沿模型会在那个窗口内提问；一个模型在 52% 的时候提问，一个 23%，一个从不。
- 从经验中学习（Asawa 等，2026）：在六个有状态环境中，最好的系统只捕获了其可能改进的 25%，而单纯保留完整对话历史击败了所有专门的记忆产品。
6.3 当系统针对评分器被优化时
如果评分器被用于训练或选择系统（强化学习、best-of-n 选择），系统就会找到评分器的盲点。Qwen 团队的 Wang 等（2026）称之为验证视野（verification horizon）：每个评分器都是意图的代理，而在优化压力下，代理与意图会分道扬镳。他们整理了编程智能体中的奖励作弊行为（从仓库历史中读取答案、修改测试、为评估器打补丁），并展示了一个行为监视器把作弊“解法”从 28.6% 降到 0.6%，同时把真实解法从 40% 提高到 61%。他们论证道，没有哪个评分器能同时做到可扩展、忠实且鲁棒：测试可扩展且鲁棒但会错过意图；裁判可扩展且忠实但可被操纵；专家忠实且鲁棒但无法扩展。Meta-Agent Challenge（Lu 等，2026）也自发地观察到了这一点：被要求构建其他智能体的智能体在五次试验中试图从评分系统中窃取答案。
> 重要提示：任何影响训练或选择的评分器都会被钻空子。对我们的意义：保留一个系统永远看不到的留出评分器，监控捷径行为，并随系统改进而升级评分器。为什么重要：分数在上升，而产品在变差。
从第 6 层带走的东西：
- 分阶段分别评分；查找器和撰写器的失败方式不同。
- 对智能体：最终状态和路径、多次运行、pass^k、每个成功的成本，以及作为独立指标的可靠性。
- 用于优化的评分器会被钻空子。保留一个系统永远看不到的评分器。
## 第 7 层：公开基准
深入对象：第 1 层中评估与基准的区别。问题：排行榜的数字能告诉你什么，又隐藏了什么？
基准是用于比较模型的固定公开任务集：MMLU（学术多选）、GSM8K（小学数学）、HumanEval（代码）、MT-Bench 和 Chatbot Arena（对话质量）、SWE-bench（修复真实 GitHub 问题）、τ-bench（客服智能体）、GPQA 和 HLE（专家问题）。它们是你挑选引擎的方式。在引用它们之前，有四件事需要知道。
数字取决于测试框架的程度不亚于取决于模型。我们用两种提示词格式在同一开源权重模型上运行 GSM8K，得到 0.686 和 0.746。lm-evaluation-harness 论文（Biderman 等，2024）记录了因提示词格式、少样本示例以及提取答案的正则表达式而产生的两位数波动。一个分数是一个（模型、提示词、框架、版本）元组。Schaeffer 等（2023）展示了更强的一点：表面上的“涌现”能力跃升很大程度上是全有全无型指标的伪象，在平滑指标下消失了。你选择的指标可以创造或抹除一个现象。
污染。如果测试题在训练数据中，分数测量的是记忆。GSM1k（Zhang 等，2024）用难度匹配的新题目重建了 GSM8K，发现某些模型家族最多下降了 8 分，且下降幅度与该模型记住原题的可能性相关。其他家族没有下降。污染是真实的、可测量的，且分布不均。
饱和。基准有保质期。2026 年一项对 60 个广泛使用的基准的研究发现约一半已高度饱和，且越老的基准饱和越快。SWE-bench Verified 在大约两年内从约 40% 的解决率升至 80% 以上，其维护者声明它已无法区分前沿模型。MMLU、GSM8K 和 HumanEval 实际上已被前沿实验室退役，转而使用 HLE、GPQA Diamond、SWE-bench Verified、LiveCodeBench 和 τ²-bench，而它们中的每一个也将依次饱和。在 Chatbot Arena 上，顶级模型之间的差距现在约为 20 Elo 分以内，处于噪声范围之内。
冗余。Zeng 和 Papailiopoulos（2026）构建了一个 84 个模型 × 133 个基准的矩阵，发现它近似秩为二：两个潜在因子解释了超过 90% 的变异。五个精心挑选的基准能把其余 128 个预测到约 4 分以内。对挑选引擎来说这是好消息：你不需要运行四十个基准。对发现某个特定的失败模式来说，它什么也说明不了；那仍然需要你自己的评估。
> 重要提示：排行榜是在去年的考试、单一框架下给引擎排名，而榜首通常彼此都在噪声范围内。对我们的意义：用基准筛选出两三个模型，然后在你自己的数据上用你自己的评估做决定。为什么重要：基于 2 分的基准收益切换模型，可能让你在自己的实际任务上损失 10 分。
两个值得了解的评估前沿应用。Google 的 Paper Assistant Tool（Jayaram 等，2026）使用智能体流水线审阅科学论文，对已知证明错误的捕获率为 89.7%，而单次模型调用只有 55.2%；在 850 位受访作者中，超过 90% 认为它有帮助，31% 因它而开展了新实验。CUSP（Wu 等，2026）要求模型预测哪些研究方向会成功，发现它们接近随机（0.519），且根据模型不同存在强烈的“是”偏差或“否”偏差，只能通过显式的偏差校正来修复。这两个例子都提醒我们：对模型判断力的评估需要它自己的真值，而模型对未来和对自己的校准都很差。
从第 7 层带走的东西：
- 基准分数是一个元组，不是模型的属性。记录测试框架。
- 基准会被污染并最终饱和；检查日期。
- 五个基准几乎能告诉你排行榜能告诉你的一切。剩下的要靠你的评估。
## 第 8 层：让它成为一种习惯
深入对象：快速理解中的“何时”。问题：如何在没有研究团队的情况下每周运行？
Hamel Husain 的三层模型（2024）设定了节奏：
- 1. 做什么：代码评分器、模式检查、必需事实。何时：每次提交。成本：免费。
- 2. 做什么：在测试集上运行模型裁判和检测器，人工审阅一个样本。何时：每次提示词、模型或检索的改动。成本：几分钱到几美元。
- 3. 做什么：在真实流量上做 A/B 测试。何时：改动上线之后。成本：真实用户。
合并门禁。我们教程中的 CI 门禁会重跑第 1 层和第 2 层，如果通过率相比记录的 0.70 基线下降超过 3 分，就使构建失败。它是针对大规模退绊线的陷阱，不是保证：在 12 分的检测下限之下，一个 5 分的退步会径直通过。请在仪表盘上写明这一点。
监控。上线后，对真实流量采样，在全部流量上运行廉价评分器，在一个切片上运行昂贵的评分器，并绘制趋势。Bodhwani 等（2026）的“先校准再信任”是昂贵门禁的正确模式：对照可验证的真值做一次完整审计，以了解廉价的裁判门禁在哪个区域内与现实一致，只在那个区域内把廉价门禁用于 CI，并在模型、裁判、模拟器或领域发生变化时重新审计。他们还发现，一个零成本的“对话是否完成”位本身就能捕捉到预算不足导致的退步。
缓存与成本。把每一次模型调用都缓存，键为提示词、数据版本和配方。重跑就变成读文件。我们在 60 张工单上的完整 A/B 消耗 121 次裁判调用，约四美分；整个教程从缓存重跑，零调用。成本不是跳过评估的理由。
领导者应该要求什么。一页四个数字：产品的通过率及其区间、评分器与人类标注的一致性、检测下限，以及最近一次改动中变差案例的占比。四者缺一，其余三个都还没有意义。
从第 8 层带走的东西：
- 三个节奏：每次提交、每次改动、上线之后。
- 门禁是一个下限，不是上限。说明它的检测下限。
- 用真值校准一次廉价门禁，然后只在那个区域内信任它。
## 工具与方法，观察记录
本节是观察，不是推荐清单。所有这些都在构建教程时被运行或审阅过；细节见我们知识库中的实践者和工具笔记。
框架
- Inspect AI（英国 AI 安全研究所，MIT）。最擅长：一条连贯的流水线：数据集、求解器、评分器、日志查看器；智能体与沙箱；用于前沿安全评估。注意：代码优先；最完整，也最值得学习。
- lm-evaluation-harness（EleutherAI，MIT）。最擅长：可复现地针对任意模型运行公开基准（GSM8K、MMLU、IFEval）。注意：只做基准；其框架论文正是它存在的理由。
- promptfoo（MIT，Node.js）。最擅长：声明式 YAML 测试套件、红队测试、CI 回归。注意：不是 Python 包；2026 年被 OpenAI 收购。
- DeepEval（Apache-2.0）。最擅长：pytest 风格的单元测试，附带现成指标（忠实性、相关性）。注意：头条指标是混合值；信任前先看内部。
- RAGAS（Apache-2.0）。最擅长：标准的 RAG 三元组，无需参考答案。注意：对本地模型不稳定（NaN、执行器错误）；数字是混合值。
- Langfuse（MIT 自托管）。最擅长：在一个自托管技术栈中集成追踪、数据集、实验运行、裁判评估器。注意：第 4 版更改了评估器语义；规划迁移。
- Phoenix（Arize）。最擅长：追踪加评估，界面很强。注意：Elastic License 2.0，源码可用而非开源。
- LangSmith、Braintrust。最擅长：托管平台：追踪采集、采样、裁判引导、人工精调。注意：商业产品；Braintrust 的 autoevals 库可独立使用。
- Opik（Comet）、MLflow genai、Weave（W&B）。最擅长：带评估挂钩的可自托管可观测性；如果你已经在用该平台就很合适。注意：对从零开始的团队来说较重。
- HELM（斯坦福）。最擅长：多指标哲学：准确率、校准、鲁棒性、公平性、效率一应俱全。注意：面向研究，最不省事。
- evalica。最擅长：把成对判断转换为带区间的 Bradley-Terry 或 Elo 排行榜。注意：仅统计功能；需与裁判配合。
- Jev / TypeSafe。最擅长：带校准概率的类型化裁判问题，比生成式裁判便宜约 100 倍。注意：无文本、无解释；标签由你提供。
- HHEM、LettuceDetect、MiniCheck、SelfCheckGPT。最擅长：以 CPU 速度和零边际成本回答“这个断言是否有依据”。注意：只回答一个问题；不是通用评分器。
- Selene-Mini、Prometheus 2、Flow-Judge。最擅长：可自托管和审计的开源权重裁判模型。注意：依然是裁判；依然需要校准。
方法对比
- 代码规则。回答：形式是否正确、事实是否齐全。成本：免费。盲区：语义。
- 相似度（ROUGE、嵌入）。回答：输出是否变了。成本：免费。盲区：正确性。
- 专用检测器 / NLI。回答：这个断言是否被这段文本支持。成本：接近免费。盲区：其他一切。
- 类型化裁判（Jev）。回答：这个标签、这个是/否，附带概率。成本：每千次几分钱。盲区：无法解释或提出建议。
- 生成式裁判，带证据的二元判断。回答：任何可读的标准。成本：每次调用几分钱。盲区：五种偏差；必须校准。
- 裁判团，稳健组合。回答：同上，但单一裁判故障更少。成本：单个裁判的三倍。盲区：错误相关性把收益上限限定在约 3 个裁判。
- 人类标注。回答：真值。成本：昂贵、缓慢。盲区：需要两个标注者和一致性统计量。
- 流量上的 A/B。回答：真实的业务影响。成本：真实用户承担风险。盲区：缓慢；需要之前的各层来保证安全。
在所有来源和我们自己的运行中都可见的一个模式：手工评分器与框架的数字应相互对照阅读。一致是证据。不一致是一个问题，而且通常是个好问题。
## 决策指南
- 输出格式是否正确且完整？使用：代码规则。然后检查：无需检查，它是精确的。
- 新版本是否改变了输出？使用：相似度。然后检查：为何改变，通过阅读样本。
- 这个断言是否被这个文档支持？使用：检测器或类型化裁判。然后检查：它在标注样本上的精确率和召回率。
- 回复是否以某种方式 X 失败？使用：带证据的二元生成式裁判。然后检查：与人类标注在开发集上的 kappa。
- 两个版本哪个更好？使用：带区间的配对比较。然后检查：逐案例的退步。
- 我们能检测出 5 分的退步吗？使用：统计功效计算。然后检查：增加案例直到可以。
- 智能体可靠吗？使用：多次运行上的 pass^k，扰动任务。然后检查：把跨运行一致性作为独立数字。
- 我们该用哪个模型？使用：五个公开基准筛选名单。然后检查：用自己的评估做决定。
- 它在生产环境中还正常吗？使用：采样流量，全部用廉价评分器，切片用裁判。然后检查：任何东西变化时重新审计裁判门禁。
## 十五条带走的要点
1. 评估就是：输入、系统、评分器、数字。先构建标准输入；它们才是资产。
2. 评估测量你的产品。基准测量引擎。不要混淆两者。
3. 在写评分器之前先读一百个输出。失败清单就是计划。
4. 凡是有确定答案的东西都用代码评分器。它们看到形式，看不到语义。
5. 裁判要用带证据和上下文的二元问题。整体式评分会漏掉埋入的错误。
6. 用人类标准测量裁判。报告 kappa 或捕获率。60% 的一致率可能毫无意义。
7. 裁判偏爱先出现的、更长的、熟悉的和令人愉快的答案。缓解方法很便宜，用上它们。
8. 满意不等于成功。把工具调用和最终状态给裁判看。
9. 对“这是否有依据”，小型检测器或类型化裁判在成本上胜过通用裁判，且往往在准确率上也胜出。
10. 每个比率都配区间。每次比较都是配对的。知道检测下限。
11. 对流水线的每个阶段分别评分。
12. 智能体：最终状态和路径、多次运行、pass^k、每个成功的成本，以及作为独立维度的可靠性。
13. 用于优化的评分器会被钻空子。保留一个系统永远看不到的评分器。
14. 基准分数依赖测试框架、污染不均且终会饱和。记录元组。
15. 三个节奏、一个说明了下限的门禁、一个监控器，以及任何变化时的重新审计。
## 参考文献
来自本知识库（摘要见同级文件夹）
- Acharya, Pan, Verkhovsky (2026)。RoPoLL: Robust Panel of LLM Judges。arXiv:2606.30931。— RoPoLLRobustPanelOfLLMJudges/
- Asawa 等 (2026)。Continual Learning Bench。arXiv:2606.05661。— ContinualLearningBench/
- Bai 等 (2026)。How Do AI Agents Spend Your Money? arXiv:2604.22750。— HowAiAgentsSpendYourMoney/
- Bodhwani, Tran, Wei (2026)。GAUGE: When Not to Trust LLM-as-a-Judge in User-Simulated Evaluation of Task-Oriented Agents。arXiv:2609.12191。— GaugeWhenNotToTrustLlmAsAJudge…/
- Cho 等 (2026)。Ask, Don't Judge / BinEval。arXiv:2606.27226。— AskDontJudge/、BinEval/
- Fiddler (2026)。The Evaluation Trust Tax: agent evals TCO。— AgentEvalsTCO/
- Grace, Hadfield, Olivares, De Jonghe (Anthropic, 2026)。Demystifying Evals for AI Agents。— DemystifyingEvalsForAIAgents/
- Gulati 等 (2026)。Ask Early, Ask Late, Ask Right。arXiv:2605.07937。— AskEarlyAskLateAskRight/
- HKUDS (2026)。ClawWork（经济问责智能体基准，代码分析）。— ClawWork/
- Jayaram 等 (Google, 2026)。Towards Automating Scientific Review with the Paper Assistant Tool。arXiv:2606.28277。— PaperAssistantTool/、TowardsAutomatingScientificReview/
- Lu 等 (2026)。The Meta-Agent Challenge。arXiv:2606.04455。— MetaAgentChallenge/
- Menaged 等 (2026)。Can LLM Agents Infer World Models? arXiv:2606.16576。— CanLLMAgentsInferWorldModels/
- Pan 等 (2026)。The Tasteful Agent。arXiv:2609.25804。— TastefulAgent/
- Parikh (2026)。Structured Output Collapses Answer Diversity Across 44 Language Models。arXiv:2607.18476。— StructuredOutputCollapsesAnswerDiversity/
- Rabanser, Kapoor, Kirgis, Liu, Utpala, Narayanan (2026)。Towards a Science of AI Agent Reliability。arXiv:2602.16666，ICML 2026。— TowardsAScienceOfAIAgentReliability/
- Wang 等 (Qwen, 2026)。The Verification Horizon: No Silver Bullet for Coding Agent Rewards。arXiv:2606.26300。— VerificationHorizon/
- Wu 等 (2026)。CUSP: Forecasting Scientific Progress with AI。arXiv:2605.22681。— ForecastingScientificProgressWithAI/
- Zeng, Papailiopoulos (2026)。You Don't Need to Run Every Eval。arXiv:2606.24020。— YouDontNeedToRunEveryEval/
- Zhang 等 (2026)。SWE-Explore。arXiv:2606.07297。— SWEExplore/
- 评估教程（tutorials/evals/，第 00–14 章，Q&A.md，research/SOURCES_*.md）及其入门 notebook，所有示例数字的来源。Jev 教程（tutorials/jev/）。
基础与实践者来源
- Biderman 等 (2024)。Lessons from the Trenches on Reproducible Evaluation of Language Models。arXiv:2405.14782。
- Chiang 等 (2024)。Chatbot Arena。arXiv:2403.04132。
- Dubois, Galambosi, Liang, Hashimoto (2024)。Length-Controlled AlpacaEval。arXiv:2404.04475。
- Es, James, Espinosa-Anke, Schockaert (2024)。RAGAS。arXiv:2309.15217。
- Gu 等 (2024)。A Survey on LLM-as-a-Judge。arXiv:2411.15594。
- Han 等 (2025)。RAG vs GraphRAG: A Systematic Evaluation。arXiv:2502.11371。
- JEV-as-a-Judge (2026)。arXiv:2609.26550。— research_topics/coding_agents/JevAsAJudge/
- Panickssery, Bowman, Feng (2024)。LLM Evaluators Recognize and Favor Their Own Generations。arXiv:2404.13076。
- Husain (2024)。Your AI Product Needs Evals。[hamel.dev](//hamel.dev)。Husain (2025)。A Field Guide to Rapidly Improving AI Products。Husain & Shankar (2025)。AI Evals FAQ。
- Jimenez 等 (2024)。SWE-bench。arXiv:2310.06770。OpenAI (2024)。SWE-bench Verified。
- Kovács, Recski (2025)。LettuceDetect。arXiv:2502.17125。Tang, Laban, Durrett (2024)。MiniCheck。arXiv:2404.10774。Manakul, Liusie, Gales (2023)。SelfCheckGPT。arXiv:2303.08896。
- Liang 等 (2022)。HELM。arXiv:2211.09110。
- Liu 等 (2023)。G-Eval。arXiv:2303.16634。
- Miller (2024)。Adding Error Bars to Evals。arXiv:2411.00640。
- Saad-Falcon, Khattab, Potts, Zaharia (2024)。ARES。arXiv:2311.09476。
- Schaeffer, Miranda, Koyejo (2023)。Are Emergent Abilities of Large Language Models a Mirage? arXiv:2304.15004。
- Shankar, Zamfirescu-Pereira, Hartmann, Parameswaran, Arawjo (2024)。Who Validates the Validators? arXiv:2404.12272。
- Tan 等 (2025)。JudgeBench。arXiv:2410.12784。
- Verga 等 (2024)。Replacing Judges with Juries (PoLL)。arXiv:2404.18796。
- Wang 等 (2024)。Large Language Models are not Fair Evaluators。arXiv:2305.17926。
- Yan (2024)。Task-Specific LLM Evals that Do & Don't Work; Evaluating the Effectiveness of LLM-Evaluators。[eugeneyan.com](//eugeneyan.com)。
- Yao, Shinn, Razavi, Narasimhan (2024)。τ-bench。arXiv:2406.12045。Sierra (2025)。τ²-bench。arXiv:2506.07982。
- Zhang 等 (2024)。GSM1k。arXiv:2405.00332。
- Zheng 等 (2023)。Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena。arXiv:2306.05685。
- Zhou 等 (2023)。IFEval。arXiv:2311.07911。
- 2026 年审计：When AI Benchmarks Plateau（arXiv:2602.16763）；When the Judge Changes, So Does the Measurement（arXiv:2607.08535）；Reliability without Validity（arXiv:2606.19544）；AgentLens（arXiv:2605.12925）。
## 相关链接
- [Sergii Makarevych](https://x.com/sermakarevich)
- [@sermakarevich](https://x.com/sermakarevich)
- [1.4K](https://x.com/sermakarevich/status/2106453816757354947/analytics)
- [hamel.dev](https://hamel.dev/)
- [eugeneyan.com](https://eugeneyan.com/)
- [Upgrade to Premium](https://x.com/i/premium_sign_up)
- [2:38 AM · Oct 4, 2026](https://x.com/sermakarevich/status/2106453816757354947)
- [1,481 Views](https://x.com/sermakarevich/status/2106453816757354947/analytics)
---
*导出时间: 2026/10/4 07:44:52* 
