# How to Build a Coding Agent That Costs 80% Less: Jev + Opus 5.5 (Full Architecture)
**作者**: CyrilXBT
**日期**: 2026-10-09T22:58:08.000Z
**来源**: [https://x.com/cyrilXBT/status/2108693524900110515](https://x.com/cyrilXBT/status/2108693524900110515)
---
Your coding agent is paying frontier prices for multiple choice questions.
Which notes should I load? Should this step go to the big model or a small one? The tool just failed. Retry, fix, or give up? Which tests should run first?
Every one of those is a question with four or five possible answers. And in most agent setups today, every one of them gets answered by the most expensive model you have, with thinking switched on, reading the whole context, writing a paragraph before it commits.
That's the leak.
This article shows you how to plug it. Opus 5.5 keeps the hard reasoning: planning, writing code, debugging. Jev takes the four decisions that happen on almost every step. Your harness prepares the options, checks the answers, and makes the final call.
In the worked example below, one feature session drops from $6.42 to $2.00. That's 69% cheaper, and every line of the math is in this article so you can check it against your own numbers.
This is written for anyone running a coding agent on real work: Claude Code, your own harness on the API, or anything in between. You don't need to rewrite your agent. Every piece in this article slots into an existing loop as a function call with a flag to switch it off. If you've read my Jev + Claude Code article, this is the upgraded version: same idea, a newer reasoning model, and four specific decisions instead of a general principle.
You'll need a Jev API key, an Anthropic API key, and about a week of your own agent logs to measure against.
Let's build it.
opus reasons jev decides.
## The Problem: One Model Doing Two Jobs
Watch a coding agent work for ten minutes and you'll see it doing two very different kinds of work.
Job one is reasoning. Planning a feature. Writing a function. Reading a stack trace and figuring out why the date parser breaks on leap years. This work is open ended. The answer isn't on a list. You want the smartest model you can afford here, and Opus 5.5 is very good at it.
Job two is deciding. Picking the three project notes that matter out of forty. Deciding a rename can go to a cheap model. Choosing what to do after npm test times out. Picking which 12 test files to run before the full 1,400. These answers are on a list. The list is short. And your code can act on the answer directly.
Most agents run both jobs through the same model, because that's the default. The loop calls Opus, Opus thinks, Opus answers, the loop continues.
It works. It's also slow and expensive in a way that's easy to miss, because the decision calls don't look expensive one at a time. A single "which notes do I need" call is maybe a cent. But it happens on every step, and it carries the full context with it every time.
And there's a second cost that's even easier to miss: when the agent can't decide what to load, it loads everything. Every project note, every convention doc, every past decision, stuffed into every call. Opus 5.5 made cache reads cheap at $0.20 per million tokens, but "cheap times sixty steps times your whole notes folder" still adds up.
## The Fix: Opus Reasons, Jev Decides, Your Harness Rules
Jev is the model from TypeSafe built for exactly job two. It doesn't write text. You give it context and typed questions with a fixed set of options, and it hands back a probability for every option. It answers in milliseconds, input costs $0.042 per million tokens, and output is free.
That makes the architecture simple to describe:
Opus 5.5 does the reasoning. Planning, code, debugging, anything where the answer isn't on a list.
Jev makes the small decisions. It picks from options your harness prepared.
Your harness stays in charge. It builds the option lists, reads the probabilities, applies thresholds, and falls back to Opus when Jev isn't sure.
That third line is the one people skip, and it's the one that makes the whole thing safe. Jev never invents an option. Jev never gets the final word on anything you can't undo. Your code decides what "sure enough" means.
Here's the helper everything else in this article is built on:
from dataclasses import dataclass
@dataclass
class Decision:
answer: str
prob: float
probs: dict
confident: bool
def jev_choice(context: str, question: str, options: list[str]):
# Wire this to your Jev client: the TypeSafe SDK, Vercel AI Gateway,
# or the LangChain package. It must return {option: probability}
# for every option you passed in.
raise NotImplementedError
def decide(context: str, question: str, options: list[str], threshold: float):
probs = jev_choice(context, question, options)
answer = max(probs, key=probs.get)
return Decision(answer, probs[answer], probs, probs[answer] >= threshold)
Notice what's not in there: no question asking Jev how confident it is. You read the probability it returns. Asking any model "are you sure?" gets you a "yes" almost every time. The probability is the honest signal.
Now the four decisions.
The fourt decisions inside agent step
## Decision 1: Pick the Relevant Project Notes Before Loading Context
This is the biggest single lever, and it's the one almost nobody has built.
Your agent has a notes folder. An AGENTS.md, architecture docs, conventions, past decisions, "never touch the billing module without a migration" warnings. Maybe 30,000 tokens of it. Most agents load all of it on every call, because nobody wants to be the one who left out the important note.
So instead of loading everything, you ask Jev a simple question about each note, all in parallel: does the agent need this note for this task?
First, split your notes into chunks by heading. Each section of AGENTS.md becomes its own chunk. Each architecture doc becomes one or more chunks.
import re
def chunk_notes(markdown: str, source: str):
parts = re.split(r"\n(?=## )", markdown)
return [{"source": source, "text": p.strip()} for p in parts if p.strip()]
Then score every chunk against the current task:
from concurrent.futures import ThreadPoolExecutor
def pick_notes(task: str, notes: list[dict], keep: int = 6, floor: float = 0.55):
def score(note):
probs = jev_choice(
context=f"TASK:\n{task}\n\nNOTE ({note['source']}):\n{note['text']}",
question="Would an engineer need this note to do the task correctly?",
options=["needed", "not_needed"],
)
return probs["needed"], note
with ThreadPoolExecutor(max_workers=16) as pool:
scored = sorted(pool.map(score, notes), key=lambda s: s[0], reverse=True)
picked = [n for p, n in scored[:keep] if p >= floor]
return picked or [n for _, n in scored[:2]]
Three details matter here.
Always load the non negotiables. Some notes should be in every call no matter what: security rules, "never force push," the one line that says which database is production. Tag those always and skip scoring them.
Keep a floor and a fallback. If nothing clears 0.55, you still load the top two. An agent with zero context is worse than one with slightly too much.
Log what got dropped. When the agent makes a mistake, the first thing you check is whether the note that would have prevented it was scored out. That log is how you tune keep and floor.
In the worked example, this one change takes the notes portion of each call from 30,000 tokens to 6,000. On its own, that cuts the session cost by about 24%.
## Decision 2: Route Suitable Tasks to a Faster Worker
Not every step needs Opus.
Renaming a function across eight files doesn't. Adding a field to a form that already has nine fields just like it doesn't. Updating an import path doesn't. These are edits that follow a pattern already sitting in the codebase.
So before each step runs, Jev classifies the step, and your harness maps the class to a worker:
ROUTES = {
"rename_or_move": "fast",
"small_edit_following_existing_pattern": "fast",
"new_logic": "opus",
"debugging": "opus",
"design_decision": "opus",
}
def route(step: str, recent_diff: str):
d = decide(
context=f"STEP:\n{step}\n\nRECENT CHANGES:\n{recent_diff}",
question="What kind of work is this step?",
options=list(ROUTES),
threshold=0.8,
)
if not d.confident:
return "opus"
return ROUTES[d.answer]
Look at how the options are written. They aren't "easy" and "hard," and they definitely aren't "can a small model handle this?" They're concrete kinds of work that your code can map to an action. That's the rule for every Jev question: every option has to be something your harness knows how to act on.
And notice the default. Below 0.8, the step goes to Opus. Routing mistakes only go in one direction here: a hard task sent to the fast worker costs you a failed step and a retry, while an easy task sent to Opus just costs a little extra. So when in doubt, send it up.
For the fast worker I use Haiku 4.5 at $1 per million input tokens and $5 per million output. In the worked example, 12 of the 36 real work steps go to it.
## Decision 3: Choose a Recovery Path When a Tool Fails
Tools fail constantly. Tests time out. A file path is wrong. A package isn't installed. A shell command hits a permission error. A linter complains about something unrelated.
Most agents handle this in the most expensive way possible: they hand the whole error back to the big model and let it think about what to do. Sometimes that's right. Often the answer is obvious, and Opus spends 1,500 tokens rediscovering that ENOENT means the file isn't there.
Worse, agents get stuck in loops. Retry, fail, retry, fail, each time paying for a full reasoning step.
So you give the failure to Jev with a fixed set of recovery paths:
from collections import deque
def tail(text: str, n: int = 4000):
return "".join(deque(text, maxlen=n)) # last n characters of the error
def recover(tool: str, command: str, error: str, attempt: int):
if attempt >= 3:
return "stop_and_report"
d = decide(
context=f"TOOL: {tool}\nCOMMAND: {command}\nATTEMPT: {attempt}\nERROR:\n{tail(error)}",
question="What is the best next move after this failure?",
options=RECOVERY,
threshold=0.8,
)
return d.answer if d.confident else "escalate_to_opus"
```
RECOVERY = [
"retry_unchanged", # flaky network or timeout
"retry_after_install", # missing package or tool
"fix_path_and_retry", # wrong file path or cwd
"switch_tool", # this tool can't do it, another can
"escalate_to_opus", # needs real debugging
"stop_and_report", # needs a human
]
```
The hard cap on attempts lives in code, not in the model. No model, however cheap, should be allowed to retry forever.
Each recovery path maps to a small function in your harness. retry_after_install reads the missing package from the error with a regex and installs it. fix_path_and_retry checks the paths that exist and suggests the closest match. Only escalate_to_opus spends real money, and it only fires when the failure actually needs thinking.
In the worked example, this removes 6 wasted retry steps from the session.
## Decision 4: Run Focused Checks Before the Full Test Suite
This one saves time more than money, and time is what makes an agent feel fast.
After every change, most agents run the whole test suite. If your suite takes four minutes, your agent spends four minutes waiting after every edit, and then reads through hundreds of lines of output looking for the one failure that matters.
The fix is the same pattern as the notes: score each test file against the diff, run the most relevant ones first, and only run the full suite when the focused checks pass.
```
def pick_tests(diff: str, test_files: list[str], keep: int = 12):
def score(path):
probs = jev_choice(
context=f"DIFF:\n{diff[:12000]}\n\nTEST FILE: {path}",
question="Could this change break tests in this file?",
options=["likely", "unlikely"],
)
return probs["likely"], path
with ThreadPoolExecutor(max_workers=16) as pool:
scored = sorted(pool.map(score, test_files), key=lambda s: s[0], reverse=True)
return [p for _, p in scored[:keep]]
def check(diff: str, test_files: list[str]):
focused = pick_tests(diff, test_files)
if not run_tests(focused):
return False # fail fast, fix before anything else
return run_tests(test_files) # full suite still gates the commit
```
Read that last line twice. The full suite still runs before anything gets committed. Focused checks are an early warning, not a replacement. You catch most breaks in seconds instead of minutes, and you never ship something the full suite would have caught.
Add the cheap checks first too: type check and lint on the changed files only. They take a second and catch a surprising share of mistakes before any test runs.
## Putting It Together: The Harness Loop
Here's how the four decisions sit inside one step of the loop:
def run_step(task: str, step: str, state):
# 1. context: load only the notes this step needs
notes = state.always_notes + pick_notes(step, state.notes)
# 2. routing: pick the worker for this step
worker = state.workers[route(step, state.recent_diff())]
result = worker.run(step, notes=notes, history=state.history)
# 3. recovery: handle any tool failures in this step
for failure in result.failures:
action = recover(failure.tool, failure.command, failure.error, failure.attempt)
state.apply_recovery(action, failure)
# 4. checks: focused tests first, full suite before commit
if result.diff:
state.log_check(check(result.diff, state.test_files))
And here's the Opus side: it's just your normal Opus 5.5 call. Put your stable context first (system prompt, tool definitions, the "always" notes) so it stays cached across steps, and put the step specific material last.
import os
import anthropic
client = anthropic.Anthropic()
OPUS = os.environ["OPUS_MODEL"] # your Opus 5.5 model id
def opus_step(system: str, notes: list[dict], history: list, step: str):
notes_text = "\n\n".join(n["text"] for n in notes)
return client.messages.create(
model=OPUS,
max_tokens=8000,
system=[{"type": "text", "text": system, "cache_control": {"type": "ephemeral"}}],
messages=history + [{"role": "user", "content": f"NOTES:\n{notes_text}\n\nSTEP:\n{step}"}],
)
Nothing exotic. The savings come from what you stop sending to it, not from a clever prompt.
## One Step, Traced
Here's what a single step looks like in the log once all four decisions are live. The task: add rate limiting to the public signup endpoint. This trace is illustrative, but it's the shape you'll see in your own logs.
step 14 "add a per IP rate limit to POST /signup"
[notes] scored 41 chunks in parallel
kept: api/middleware.md (0.94), conventions/errors.md (0.88),
infra/redis.md (0.81), decisions/2025_auth.md (0.62)
dropped: 37 chunks, including billing, email templates, CI
[route] new_logic (0.91) → opus
[opus] plans the change, writes middleware + config + handler edit
[tool] npm test FAILED "Cannot find module 'bottleneck'"
[recover] retry_after_install (0.97) → npm install, rerun
[checks] typecheck changed files: pass
focused tests (12 of 1,388): pass
full suite: pass
Read it from top to bottom and count how many of those lines used to be a full Opus call. Picking notes: one big call, or no picking at all and 30,000 tokens of notes loaded blind. Routing: nothing, every step went to Opus. The missing package: a full reasoning step to work out that a missing module means "install the module." Choosing tests: nothing, the full suite on every edit.
Now only one line in that trace is Opus, and it's the line that actually needed a frontier model: planning and writing the rate limiter.
That's the whole idea in one picture. The expensive model spends its time on the part of the job only it can do.
## The Worked Example: Where the 69% Comes From
Here's the full math for one feature session: 60 agent steps plus the 6 wasted retries a typical agent racks up along the way. All prices are the current public list prices.
Before: Opus 5.5 does everything
Every step loads 70,000 tokens of context: 15,000 for the system prompt and tools, 30,000 for the full notes folder, and 25,000 for the conversation so far. I assume 80% of that is a cache hit. Every step produces about 1,500 output tokens, because thinking is always on in Opus 5.5.
cache reads: 56,000 × $0.20/M = $0.0112
fresh input: 14,000 × $4.00/M = $0.0560
output: 1,500 × $20.00/M = $0.0300
per step = $0.0972
66 steps × $0.0972 = $6.42
After: the decision layer
Context drops to 46,000 tokens per step, because Jev picks 6,000 tokens of notes instead of loading all 30,000. The 24 pure decision steps move to Jev. The 6 retry loops disappear. And 12 of the remaining 36 work steps go to Haiku 4.5.
Opus 5.5: 24 steps × $0.0742 = $1.78
Haiku 4.5: 12 steps × $0.0169 = $0.20
Jev: 102 calls × 5,000 tokens
× $0.042/M, output free = $0.02
total = $2.00
From $6.42 to $2.00. That's 68.8% cheaper for the same feature.
set the threshold by what a wrong answer costs
And here's how much each lever contributes, stacked in order:
notes selection only $4.89 23.7% saved
+ decisions moved to Jev $3.13 51.2% saved
+ recovery paths $2.69 58.1% saved
+ routing to a fast worker $2.00 68.8% saved
Two honest notes about this math.
First, these are modeled numbers built on explicit assumptions, not a measurement of your stack. Your context size, cache hit rate and decision count will be different. The point of showing every line is so you can swap in your own numbers. Once you've measured your real before and after, use your number, not mine.
Second, look at where the money actually goes. Jev's 102 calls cost two cents in total. The savings don't come from Jev being cheap per call. They come from everything that stops happening once Jev makes the decision: bloated context, full reasoning steps for multiple choice questions, retry loops, and big model calls on rename jobs.
## Why Not Just Use Haiku for the Decisions?
Fair question. Haiku 4.5 is cheap and fast. Why add a new model at all?
Three reasons.
It still writes. Haiku answers in text. You have to prompt it to answer with one of your options, then parse the answer, then handle the times it adds an explanation, picks something that isn't on the list, or wraps the answer in a sentence. Jev returns a probability for every option you passed in. There's nothing to parse and nothing off the list.
You get the full distribution, not just an answer. When Jev says needed: 0.52, not_needed: 0.48, you know it's a coin flip and your harness can escalate. When a text model says "needed," you have no idea whether it was sure. Thresholds only work when you can see the probabilities.
The cost math is different. Jev's output is free and its input is $0.042 per million tokens. Scoring 41 note chunks every step means 41 small calls per step. With Jev that's a fraction of a cent. With any text model, even a cheap one, that's 41 generations with their own output cost and latency.
Use Haiku for small edits, where it actually has to write code. Use Jev for picking from lists. Use Opus for thinking. Each model does the job it's built for.
## How to Roll It Out Without Breaking Anything
Don't flip all four decisions on at once. Here's the order I'd use.
Week 1: shadow mode. Keep your current agent exactly as it is. For every decision it makes, also ask Jev the same question and log both answers plus Jev's probability. Don't act on Jev's answer at all.
```
def shadow(decision_name, context, question, options, current_answer):
probs = jev_choice(context, question, options)
jev_answer = max(probs, key=probs.get)
log_row(
decision=decision_name,
current=current_answer,
jev=jev_answer,
prob=probs[jev_answer],
agree=(jev_answer == current_answer),
)
```
Week 2: read the disagreements. Look only at the rows where Jev and your current agent disagreed. Sort them by Jev's probability. You'll usually find a clean line: above some probability, Jev is right about as often as Opus. That line is your threshold. Those disagreement rows also become your test set, built for free out of your real traffic.
Week 3: switch on notes selection. It's the biggest lever and the easiest to undo, since a missed note shows up fast in your logs.
Week 4: switch on recovery and tests. Both have hard caps and fallbacks in code, so the worst case is falling back to what you do today.
Last: routing. Routing mistakes are the most visible to users, because a hard task sent to the fast worker produces bad code. Turn it on last, with a conservative threshold, and watch the fail rate on fast worker steps.
Put every decision behind an on/off flag so you can undo it in one line.
## Setting Thresholds by Risk
One threshold for everything is a mistake. Set them by what happens when the answer is wrong.
One feature session, same result $6.42 to $2.00
Low stakes, easy to undo: 0.55 to 0.65. Picking notes. Ordering tests. A wrong answer costs a bit of context or a few seconds.
Medium stakes: 0.8. Routing and recovery. A wrong answer costs a failed step, which your harness catches.
High stakes, hard to undo: 0.9 or never. Deleting files. Force pushing. Touching production config. Running a migration. My honest advice: don't let any model make these calls alone. Route them to a human, or at minimum to Opus with a confirmation step.
Anything below the threshold goes up the ladder: to Opus if it needs reasoning, to a human if it needs judgment.
## Where This Doesn't Help
This architecture is not a magic multiplier on every workload.
Pure generation workloads barely change. If your agent spends most of its time writing large amounts of new code from a clear spec, there aren't many decisions to move. You'll still get the notes selection savings, but not much else.
Tiny projects don't need it. If your whole notes folder is 2,000 tokens and your suite runs in eight seconds, the overhead of building this isn't worth it yet.
Jev isn't great at making the final call alone. It's excellent at noticing things and picking from a short list. It's not the model you want deciding "is this pull request ready to merge?" as a single question. Break big questions into small ones, score each, and combine the results in your own code with your own weights.
If you already have labelled data for a fixed question, a small model you host yourself can beat Jev on price. Jev shines when the questions keep changing and you don't have training data, which describes most coding agents.
## The Mistakes I'd Avoid
Letting the model write the option list. The whole architecture depends on options your code can act on. If a model generates the options, you're back to parsing free text.
Asking Jev if it's confident. Read the probability instead. A self report is almost always "yes."
No fallback. Every Jev decision needs a path for "not sure," and that path is Opus or a human, never a guess.
Retry limits in the prompt instead of in code. A model told "only retry 3 times" will occasionally retry 30 times. Code doesn't.
Dropping the full test suite. Focused checks speed you up. The full suite keeps you honest. Keep both.
Measuring nothing. If you don't log cost per session before you start, you'll never know whether you saved 20% or 70%. Pull your usage for a week of normal work first. It takes ten minutes and it's the only number that matters.
## What to Do This Week
Here's the whole thing as a checklist:
1. Log one week of agent sessions: steps, tokens, cost, failures.
2. Find the decisions hiding in your loop: notes, routing, recovery, tests.
3. Write each one as a question with a fixed list of options.
4. Run Jev in shadow mode next to your current agent for a week.
5. Set thresholds from the disagreement log, by risk.
6. Switch on notes selection first, then recovery and tests, then routing.
7. Compare cost per session before and after. That's your number.
Opus 5.5 is the best reasoning model I've used for code. The mistake is making it do everything else too.
Let it think. Let Jev decide. Let your code rule.
If this was useful, follow @cyrilXBT. I break down how to build with the newest AI models every week, with the real numbers.
Bookmark this so you have it when you start building.
## 相关链接
- [CyrilXBT](https://x.com/cyrilXBT)
- [@cyrilXBT](https://x.com/cyrilXBT)
- [515K](https://x.com/cyrilXBT/status/2108693524900110515/analytics)
- [@dataclass](https://x.com/@dataclass)
- [pool.map](https://pool.map/)
- [worker.run](https://worker.run/)
- [@cyrilXBT](https://x.com/@cyrilXBT)
- [Upgrade to Plus](https://x.com/i/premium_sign_up)
- [6:58 AM · Oct 10, 2026](https://x.com/cyrilXBT/status/2108693524900110515)
- [515.6K Views](https://x.com/cyrilXBT/status/2108693524900110515/analytics)
- [View quotes](https://x.com/cyrilXBT/status/2108693524900110515/quotes)
---
*导出时间: 2026/10/26 10:09:24*
---
## 中文翻译
# 如何构建一个成本降低 80% 的编程智能体：Jev + Opus 5.5（完整架构）
**作者**: CyrilXBT
**日期**: 2026-10-09T22:58:08.000Z
**来源**: [https://x.com/cyrilXBT/status/2108693524900110515](https://x.com/cyrilXBT/status/2108693524900110515)
**阅读**: 515678
**点赞**: 385
**转发**: 46
**回复**: 15
---
你的编程智能体正在用前沿模型的价格去做多项选择题。
该加载哪些笔记？这一步应该交给大模型还是小模型？工具刚刚失败了，该重试、修复还是放弃？哪些测试应该先跑？
每一个都是有四五个可能答案的问题。而在如今大多数智能体的设置中，每一个问题都是由你最贵的模型来回答的——开着思考模式，读着全部上下文，在提交答案之前还要写上一段话。
这就是漏损所在。
本文将告诉你如何堵住它。Opus 5.5 保留困难推理：规划、写代码、调试。Jev 负责几乎每一步都会出现的四个决策。你的编排层（harness）准备好选项、校验答案，并做最终裁决。
在下面的实际案例中，一个功能会话的成本从 $6.42 降到 $2.00——便宜了 69%，而且每一行计算都在本文中，你可以对照自己的数字进行核验。
本文写给所有在真实工作中运行编程智能体的人：Claude Code、你在 API 上自建的编排层，或介于两者之间的任何东西。你不需要重写你的智能体。本文的每个组件都可以作为一个带开关标志的函数调用，插入到现有的循环中。如果你读过我的《Jev + Claude Code》一文，这篇是升级版：同样的思路、更新的推理模型，以及四个具体决策，而不是一个泛泛的原则。
你需要一个 Jev API key、一个 Anthropic API key，以及大约一周的智能体日志用于对比测量。
开始构建吧。
opus 负责推理，jev 负责决策。
## 问题：一个模型干两份活
观察一个编程智能体工作十分钟，你会看到它在做两种截然不同的工作。
第一种是推理。规划一个功能。写一个函数。阅读堆栈跟踪，弄清楚为什么日期解析器在闰年会出错。这类工作是开放式的，答案不在选项列表里。在这方面你想要能负担得起的最聪明的模型，而 Opus 5.5 非常擅长。
第二种是决策。从四十条笔记中挑出重要的三条。决定某个重命名操作可以交给便宜模型。选择 npm test 超时后该做什么。挑选在跑全部 1,400 个测试文件之前先跑哪 12 个。这些答案都在列表里，列表很短，而且你的代码可以直接根据答案行动。
大多数智能体用同一个模型处理这两种工作，因为这就是默认做法。循环调用 Opus，Opus 思考，Opus 回答，循环继续。
它能工作。但它又慢又贵，而且这种浪费很容易被忽略，因为决策调用单独看并不贵。一次“我需要哪些笔记”的调用可能只要一分钱。但它每一步都会发生，而且每次都携带完整上下文。
还有一个更容易忽略的隐性成本：当智能体决定不了该加载什么时，它就全部加载。每条项目笔记、每份规范文档、每个历史决策，塞进每一次调用。Opus 5.5 让缓存读取变得便宜了，每百万 token $0.20，但“便宜 × 六十步 × 你的整个笔记文件夹”加起来依然可观。
## 解决方案：Opus 推理，Jev 决策，你的编排层掌控规则
Jev 是 TypeSafe 出品的模型，专为第二种工作而生。它不写文本。你给它上下文和带固定选项集的类型化问题，它返回每个选项的概率。它在毫秒内回答，输入成本为每百万 token $0.042，输出免费。
这让架构变得简单可述：
Opus 5.5 负责推理。规划、代码、调试，任何答案不在选项列表里的工作。
Jev 做小决策。它从你的编排层准备好的选项中进行选择。
你的编排层保持掌控。它构建选项列表、读取概率、应用阈值，并在 Jev 不确定时回退到 Opus。
第三点最容易被忽视，但它正是让整套方案安全的关键。Jev 永远不会发明一个选项。Jev 永远不会对无法撤销的事情拥有最终决定权。“足够确定”的含义由你的代码来定义。
以下是本文所有内容赖以构建的辅助函数：
from dataclasses import dataclass
@dataclass
class Decision:
answer: str
prob: float
probs: dict
confident: bool
def jev_choice(context: str, question: str, options: list[str]):
# 将此接入你的 Jev 客户端：TypeSafe SDK、Vercel AI Gateway
# 或 LangChain 包。它必须为你传入的每个选项
# 返回 {选项: 概率}。
raise NotImplementedError
def decide(context: str, question: str, options: list[str], threshold: float):
probs = jev_choice(context, question, options)
answer = max(probs, key=probs.get)
return Decision(answer, probs[answer], probs, probs[answer] >= threshold)
注意这里没有的东西：没有让 Jev 自评有多自信的问题。你直接读取它返回的概率。问任何模型“你确定吗？”，几乎每次得到的都是“确定”。概率才是诚实的信号。
下面是四个决策。
智能体步骤内的四个决策
## 决策 1：在加载上下文之前挑选相关项目笔记
这是单个最大的杠杆，也是几乎没人做过的。
你的智能体有一个笔记文件夹。AGENTS.md、架构文档、编码规范、历史决策、“没有迁移就永远不要动计费模块”之类的警告。大概 30,000 token。大多数智能体每次调用都加载全部内容，因为没人想成为漏掉重要笔记的那个人。
所以与其全部加载，不如就每条笔记向 Jev 问一个简单的问题，全部并行执行：智能体做这个任务需要这条笔记吗？
首先，按标题把笔记切分成块。AGENTS.md 的每个部分成为一个独立的块。每份架构文档成为一个或多个块。
import re
def chunk_notes(markdown: str, source: str):
parts = re.split(r"\n(?=## )", markdown)
return [{"source": source, "text": p.strip()} for p in parts if p.strip()]
然后针对当前任务给每个块打分：
from concurrent.futures import ThreadPoolExecutor
def pick_notes(task: str, notes: list[dict], keep: int = 6, floor: float = 0.55):
def score(note):
probs = jev_choice(
context=f"TASK:\n{task}\n\nNOTE ({note['source']}):\n{note['text']}",
question="Would an engineer need this note to do the task correctly?",
options=["needed", "not_needed"],
)
return probs["needed"], note
with ThreadPoolExecutor(max_workers=16) as pool:
scored = sorted(pool.map(score, notes), key=lambda s: s[0], reverse=True)
picked = [n for p, n in scored[:keep] if p >= floor]
return picked or [n for _, n in scored[:2]]
这里有三个重要细节。
始终加载不可协商的笔记。有些笔记无论如何每次调用都应该包含：安全规则、“永远不要强制推送”、标明哪个数据库是生产环境的那一行。给这些打上 always 标签，跳过评分。
设置下限和回退。如果没有任何块超过 0.55，你依然加载得分最高的两条。一个零上下文的智能体比上下文略多的智能体更糟。
记录被丢弃的内容。当智能体犯错时，你首先要检查的就是那条本可以避免错误的笔记是否被评分淘汰了。这份日志就是你调优 keep 和 floor 的依据。
在实际案例中，仅这一项改动就把每次调用的笔记部分从 30,000 token 降到 6,000。仅此一项就能把会话成本降低约 24%。
## 决策 2：把合适的任务路由到更快的执行者
不是每一步都需要 Opus。
跨八个文件重命名一个函数不需要。给一个已有九个同类的表单添加一个字段不需要。更新导入路径不需要。这些都是遵循代码库中已有模式的编辑。
所以在每一步运行之前，Jev 对步骤进行分类，你的编排层把类别映射到执行者：
ROUTES = {
"rename_or_move": "fast",
"small_edit_following_existing_pattern": "fast",
"new_logic": "opus",
"debugging": "opus",
"design_decision": "opus",
}
def route(step: str, recent_diff: str):
d = decide(
context=f"STEP:\n{step}\n\nRECENT CHANGES:\n{recent_diff}",
question="What kind of work is this step?",
options=list(ROUTES),
threshold=0.8,
)
if not d.confident:
return "opus"
return ROUTES[d.answer]
注意选项是怎么写的。它们不是“简单”和“困难”，更不是“小模型能处理吗？”它们是你的代码可以映射为具体动作的工作类型。这就是每个 Jev 问题的规则：每个选项都必须是你的编排层知道如何执行的东西。
再注意默认值。低于 0.8 时，该步骤交给 Opus。这里路由错误只朝一个方向发生代价：把困难任务发给快速执行者会浪费一步并需要重试，而把简单任务发给 Opus 只是稍微贵一点。所以拿不准时，就往上发。
快速执行者我用 Haiku 4.5，每百万输入 token $1、输出 $5。在实际案例中，36 个实际工作步骤中有 12 个交给了它。
## 决策 3：工具失败时选择恢复路径
工具 constantly 失败。测试超时。文件路径错误。包没安装。Shell 命令遇到权限错误。Linter 抱怨不相关的问题。
大多数智能体用最昂贵的方式处理：把整个错误交回大模型，让它思考该怎么办。有时这是对的。但通常答案显而易见，Opus 却花 1,500 个 token 重新发现 ENOENT 意味着文件不存在。
更糟的是，智能体会陷入循环。重试、失败、重试、失败，每次都为一个完整的推理步骤买单。
所以你把失败交给 Jev，配上固定的恢复路径集：
from collections import deque
def tail(text: str, n: int = 4000):
return "".join(deque(text, maxlen=n)) # 错误的最后 n 个字符
def recover(tool: str, command: str, error: str, attempt: int):
if attempt >= 3:
return "stop_and_report"
d = decide(
context=f"TOOL: {tool}\nCOMMAND: {command}\nATTEMPT: {attempt}\nERROR:\n{tail(error)}",
question="What is the best next move after this failure?",
options=RECOVERY,
threshold=0.8,
)
return d.answer if d.confident else "escalate_to_opus"
```
RECOVERY = [
"retry_unchanged", # 网络抖动或超时
"retry_after_install", # 缺少包或工具
"fix_path_and_retry", # 文件路径或工作目录错误
"switch_tool", # 这个工具做不了，换一个
"escalate_to_opus", # 需要真正的调试
"stop_and_report", # 需要人类介入
]
```
尝试次数的硬上限写在代码里，而不是模型里。任何模型，无论多便宜，都不应被允许无限重试。
每条恢复路径映射到编排层中的一个小函数。retry_after_install 用正则从错误中读取缺失的包并安装它。fix_path_and_retry 检查存在的路径并建议最接近的匹配。只有 escalate_to_opus 才花真金白银，而且只在失败确实需要思考时才触发。
在实际案例中，这一项从会话中去掉了 6 次浪费的重试步骤。
## 决策 4：在完整测试套件之前跑聚焦检查
这一项节省的更多是时间而非金钱，而时间正是让智能体感觉快的关键。
每次改动之后，大多数智能体会跑完整的测试套件。如果你的套件需要四分钟，你的智能体每次编辑后就要等四分钟，然后还要通读数百行输出，寻找那一个真正重要的失败。
修复方法和笔记一样：针对 diff 给每个测试文件打分，先跑最相关的几个，只有聚焦检查通过后才跑完整套件。
```
def pick_tests(diff: str, test_files: list[str], keep: int = 12):
def score(path):
probs = jev_choice(
context=f"DIFF:\n{diff[:12000]}\n\nTEST FILE: {path}",
question="Could this change break tests in this file?",
options=["likely", "unlikely"],
)
return probs["likely"], path
with ThreadPoolExecutor(max_workers=16) as pool:
scored = sorted(pool.map(score, test_files), key=lambda s: s[0], reverse=True)
return [p for _, p in scored[:keep]]
def check(diff: str, test_files: list[str]):
focused = pick_tests(diff, test_files)
if not run_tests(focused):
return False # 快速失败，先修复再继续
return run_tests(test_files) # 完整套件仍然是提交前的关卡
```
把最后一行读两遍。任何东西提交之前，完整套件依然会运行。聚焦检查是预警，不是替代。你在几秒内而不是几分钟内捕获大多数破坏，而且你永远不会发布一个完整套件本可拦住的东西。
同样先加上廉价的检查：只对改动的文件做类型检查和 lint。它们只要一秒钟，能在任何测试运行之前抓住相当比例的错误。
## 组合起来：编排层循环
下面是四个决策如何嵌入循环中的一步：
def run_step(task: str, step: str, state):
# 1. 上下文：只加载本步骤需要的笔记
notes = state.always_notes + pick_notes(step, state.notes)
# 2. 路由：为本步骤挑选执行者
worker = state.workers[route(step, state.recent_diff())]
result = worker.run(step, notes=notes, history=state.history)
# 3. 恢复：处理本步骤的任何工具失败
for failure in result.failures:
action = recover(failure.tool, failure.command, failure.error, failure.attempt)
state.apply_recovery(action, failure)
# 4. 检查：先聚焦测试，提交前跑完整套件
if result.diff:
state.log_check(check(result.diff, state.test_files))
Opus 这一侧很简单：就是你普通的 Opus 5.5 调用。把稳定的上下文放在前面（系统提示词、工具定义、“始终加载”的笔记），让它在各步骤间保持缓存命中，把步骤相关的内容放在最后。
import os
import anthropic
client = anthropic.Anthropic()
OPUS = os.environ["OPUS_MODEL"] # 你的 Opus 5.5 模型 id
def opus_step(system: str, notes: list[dict], history: list, step: str):
notes_text = "\n\n".join(n["text"] for n in notes)
return client.messages.create(
model=OPUS,
max_tokens=8000,
system=[{"type": "text", "text": system, "cache_control": {"type": "ephemeral"}}],
messages=history + [{"role": "user", "content": f"NOTES:\n{notes_text}\n\nSTEP:\n{step}"}],
)
没有任何花哨的东西。节省来自于你不再发给它的内容，而不是一个聪明的提示词。
## 一步的完整追踪
当四个决策全部上线后，日志中单步的样子如下。任务是：给公开注册端点加限流。这条追踪是示意性的，但这就是你会在自己的日志中看到的形态。
step 14 "add a per IP rate limit to POST /signup"
[notes] scored 41 chunks in parallel
kept: api/middleware.md (0.94), conventions/errors.md (0.88),
infra/redis.md (0.81), decisions/2025_auth.md (0.62)
dropped: 37 chunks, including billing, email templates, CI
[route] new_logic (0.91) → opus
[opus] plans the change, writes middleware + config + handler edit
[tool] npm test FAILED "Cannot find module 'bottleneck'"
[recover] retry_after_install (0.97) → npm install, rerun
[checks] typecheck changed files: pass
focused tests (12 of 1,388): pass
full suite: pass
从上往下读，数一数这些行里有多少过去是一整次 Opus 调用。挑笔记：一次大调用，或者干脆不挑、盲目加载 30,000 token 的笔记。路由：什么都没有，每一步都给 Opus。缺失的包：一整个推理步骤，只为搞清楚“模块缺失意味着安装模块”。选测试：什么都没有，每次编辑都跑完整套件。
现在整条追踪里只有一行用了 Opus，而且它正是真正需要前沿模型的那一行：规划并编写限流器。
整张图的核心思想就在这里。昂贵的模型把时间花在只有它能做的那部分工作上。
## 实际案例：69% 从哪来
下面是一个功能会话的完整计算：60 个智能体步骤，加上典型智能体一路积累的 6 次浪费重试。所有价格均为当前公开定价。
之前：Opus 5.5 干所有事
每步加载 70,000 token 上下文：系统提示词和工具 15,000，完整笔记文件夹 30,000，迄今对话 25,000。我假设其中 80% 是缓存命中。每步产出约 1,500 输出 token，因为 Opus 5.5 的思考模式始终开启。
cache reads: 56,000 × $0.20/M = $0.0112
fresh input: 14,000 × $4.00/M = $0.0560
output: 1,500 × $20.00/M = $0.0300
per step = $0.0972
66 steps × $0.0972 = $6.42
之后：决策层
上下文降到每步 46,000 token，因为 Jev 只挑 6,000 token 的笔记，而不是加载全部 30,000。24 个纯决策步骤交给 Jev。6 次重试循环消失。剩余 36 个工作步骤中，12 个交给 Haiku 4.5。
Opus 5.5: 24 steps × $0.0742 = $1.78
Haiku 4.5: 12 steps × $0.0169 = $0.20
Jev: 102 calls × 5,000 tokens
× $0.042/M, output free = $0.02
total = $2.00
从 $6.42 到 $2.00。同样的功能，便宜了 68.8%。
根据错误答案的代价来设定阈值
下面是每个杠杆的贡献，按顺序叠加：
仅笔记筛选 $4.89 节省 23.7%
+ 决策转移给 Jev $3.13 节省 51.2%
+ 恢复路径 $2.69 节省 58.1%
+ 路由到快速执行者 $2.00 节省 68.8%
关于这些数字，有两点诚实的说明。
第一，这些是基于明确假设的建模数字，不是对你技术栈的实测。你的上下文大小、缓存命中率和决策数量都会不同。展示每一行计算的意义就是让你能代入自己的数字。一旦你实测了自己的前后对比，用你的数字，别用我的。
第二，看看钱实际省在哪里。Jev 的 102 次调用总共只花了 2 美分。节省并不是因为 Jev 单次调用便宜，而是因为一旦 Jev 做了决策，一大堆事情就不再发生：臃肿的上下文、为多项选择题跑的完整推理步骤、重试循环、以及在重命名这类小事上调用大模型。
## 为什么不直接用 Haiku 做决策？
合理的问题。Haiku 4.5 又便宜又快，为什么还要引入一个新模型？
三个原因。
它还是要写文本。Haiku 用文本回答。你得提示它从你的选项中选一个，然后解析答案，还要处理它偶尔补充解释、选了不在列表上的东西、或者把答案包在一句话里的情况。Jev 为你传入的每个选项返回概率。没有东西需要解析，也没有列表之外的选择。
你拿到的是完整的概率分布，而不只是一个答案。当 Jev 说 needed: 0.52、not_needed: 0.48 时，你知道这几乎是抛硬币，你的编排层可以升级处理。当文本模型说“需要”时，你完全不知道它是不是确定。阈值只有在你能看到概率时才有效。
成本结构不同。Jev 输出免费，输入每百万 token $0.042。每步给 41 个笔记块打分意味着每步 41 次小调用。用 Jev 这只是几分之一美分。用任何文本模型，哪怕便宜的，也是 41 次带有输出成本和延迟的生成。
小编辑用 Haiku，那里它确实要写代码。从列表中挑选用 Jev。思考用 Opus。每个模型干它擅长的活。
## 如何在不破坏任何东西的前提下上线
不要一次性打开全部四个决策。以下是我推荐的顺序。
第 1 周：影子模式。你的智能体完全保持现状。它每做一个决策，就同时问 Jev 同样的问题，记录两个答案以及 Jev 的概率。完全不要根据 Jev 的答案行动。
```
def shadow(decision_name, context, question, options, current_answer):
probs = jev_choice(context, question, options)
jev_answer = max(probs, key=probs.get)
log_row(
decision=decision_name,
current=current_answer,
jev=jev_answer,
prob=probs[jev_answer],
agree=(jev_answer == current_answer),
)
```
第 2 周：分析分歧。只看 Jev 和你当前智能体不一致的行。按 Jev 的概率排序。你通常会发现一条清晰的界线：在某个概率之上，Jev 的正确率和 Opus 差不多。那条线就是你的阈值。这些分歧行还构成了你的测试集，从真实流量中免费获得。
第 3 周：打开笔记筛选。它是最大的杠杆，也最容易撤销，因为漏掉的笔记会很快在你的日志中暴露。
第 4 周：打开恢复和测试。两者在代码中都有硬上限和回退，所以最坏情况也只是退回到你今天的做法。
最后：路由。路由错误对用户来说最显眼，因为发给快速执行者的困难任务会产生糟糕的代码。最后开启它，用保守的阈值，并密切关注快速执行者步骤的失败率。
把每个决策都放在开/关标志后面，这样你可以一行代码撤销。
## 按风险设定阈值
所有事情共用一个阈值是错误的。按答案出错时的后果来设定。
一个功能会话，同样结果 $6.42 到 $2.00
低风险、易撤销：0.55 到 0.65。挑笔记。排序测试。错误答案的代价只是少量上下文或几秒钟。
中风险：0.8。路由和恢复。错误答案的代价是一步失败，而你的编排层能捕获它。
高风险、难撤销：0.9 或干脆不用。删除文件。强制推送。动生产配置。跑数据库迁移。我的诚实建议：不要让任何模型单独做这些决定。路由给人类，或者至少给 Opus 并加一个确认步骤。
任何低于阈值的情况都往梯子上走：需要推理的给 Opus，需要判断的给人类。
## 哪些场景帮助不大
这套架构不是所有工作负载的万能倍增器。
纯生成类工作负载几乎没有变化。如果你的智能体大部分时间在根据清晰的规格编写大量新代码，就没有多少决策可以转移。你依然能获得笔记筛选的节省，但其他就不多了。
小项目不需要它。如果你的整个笔记文件夹只有 2,000 token，测试套件八秒跑完，搭建这套东西的开销还不值得。
Jev 不擅长单独做最终裁决。它极其擅长发现问题和从短列表中选择。它不是你想要的把“这个 PR 可以合并吗？”作为单个问题来回答的模型。把大问题拆成小问题，逐个打分，然后在你的代码里用你自己的权重组合结果。
如果你已经为某个固定问题积累了带标签的数据，你自己托管的小模型可能在价格上胜过 Jev。Jev 的优势在于问题不断变化、又没有训练数据的场景——而这正是大多数编程智能体的写照。
## 我会避免的错误
让模型自己写选项列表。整个架构依赖于你的代码能够执行的选项。如果选项由模型生成，你就又回到了解析自由文本的老路。
问 Jev 它自不确定。直接读概率。自我报告几乎总是“确定”。
没有回退。每个 Jev 决策都需要一条“不确定”的路径，那条路径是 Opus 或人类，永远不是瞎猜。
重试上限写在提示词里而不是代码里。被告知“只重试 3 次”的模型偶尔会重试 30 次。代码不会。
丢掉完整测试套件。聚焦检查让你更快，完整套件让你诚实。两个都要。
什么都不测量。如果开始之前不记录每次会话的成本，你永远不知道自己省了 20% 还是 70%。先拉取一周正常工作的用量数据。只需要十分钟，而它是唯一重要的数字。
## 本周可以做什么
把整个方案浓缩为一份清单：
1. 记录一周的智能体会话：步骤、token、成本、失败。
2. 找出藏在循环里的决策：笔记、路由、恢复、测试。
3. 把每一个写成带固定选项列表的问题。
4. 让 Jev 以影子模式在你当前智能体旁边运行一周。
5. 根据分歧日志、按风险设定阈值。
6. 先打开笔记筛选，然后是恢复和测试，最后是路由。
7. 对比前后的每会话成本。那就是你的数字。
Opus 5.5 是我用过的最好的代码推理模型。错误在于让它把其他事情也全干了。
让它思考。让 Jev 决策。让你的代码掌权。
如果这篇文章对你有用，请关注 @cyrilXBT。我每周拆解如何用最新的 AI 模型构建，带着真实数据。
收藏本文，以便你开始构建时可以查阅。
## 相关链接
- [CyrilXBT](https://x.com/cyrilXBT)
- [@cyrilXBT](https://x.com/cyrilXBT)
- [515K](https://x.com/cyrilXBT/status/2108693524900110515/analytics)
- [@dataclass](https://x.com/@dataclass)
- [pool.map](https://pool.map/)
- [worker.run](https://worker.run/)
- [@cyrilXBT](https://x.com/@cyrilXBT)
- [Upgrade to Plus](https://x.com/i/premium_sign_up)
- [6:58 AM · Oct 10, 2026](https://x.com/cyrilXBT/status/2108693524900110515)
- [515.6K Views](https://x.com/cyrilXBT/status/2108693524900110515/analytics)
- [View quotes](https://x.com/cyrilXBT/status/2108693524900110515/quotes)
---
*导出时间： 2026/10/26 10:09:24* 
