# Model-Harness-Fit
**作者**: Nicolas Bustamante
**日期**: 2026-05-04T02:48:49.000Z
**来源**: [https://x.com/nicbstme/status/2051131906327212298](https://x.com/nicbstme/status/2051131906327212298)
---
Is it best to use an LLM with its native harness (like Claude Code or Codex), or a generic harness that swaps models on demand?
OpenAI engineer Ryan (@_lopopolo) tweeted this hilarious meme this morning, so I decided to dig deeper by looking at the harness implementations of Codex, Claude Code, and Github sdk. Does the harness really matter that much?
I keep three coding agents alive on the same workstation. Claude Code in one terminal. Codex CLI in another. GitHub Copilot CLI in a third. Same files. Same git tree. Same bash. Three different harnesses that look indistinguishable.
I ran the same prompt through all three and the behavior was visibly different in ways that went well past the surface differences of style and speed that I had expected to see across vendors.
The hand wave answer is that "models behave differently because they are different models." but here I tested the same models and different harness.
Models are post trained against the harness, not just the API. The tool names they expect, the input schemas they emit, the citation tags they wrap around remembered facts, the file structure of skills they invoke, the planning protocol they follow when the harness says "make a plan first" (none of these are generic capabilities of the model).
They are byte level conventions baked into the post training of one specific model against one specific harness. Pull the model out of its harness and you give up performance you cannot get back without rewriting either side.
This has a direct consequence that anyone who has tried to ship a "model agnostic" agent has run into. You cannot just swap a model.
Supporting BYOK and multi model (which is the responsible posture, since relying on a single provider is risky!) adds real engineering complexity.
To swap a model cleanly, you have to swap the harness with it: the tool surface, the schema shapes, the skill bodies that name those tools, the citation contract, the memory ritual, the system prompt structure, sometimes the planning protocol.
Everything above the model has to move when the model moves. That is why every agent vendor that supports multiple providers ends up either
(a) running a degraded variant of every model they support, or
(b) maintaining a separate full stack per model and exposing the choice to the user as "you are picking a product, not just a model." Option (b) is the path that wins on quality, and it is worth the engineering cost to avoid being locked into one lab.
Swapping orchestrators is not a cosmetic change. It is a model swap in disguise. The frontier lab spent the last year shaping the model's instincts to a particular tool surface, a particular memory ritual, a particular skill format. When you mix and match, you spend that work.
I think this is the single most underrated constraint in agent design today, and it has a clean name. Call it Model-Harness-Fit.
## So how does this actually show up under the hood?
I dug into three open implementations that ship today: Codex CLI (OpenAI, fully open source at github.com/openai/codex, Claude Code (Anthropic, closed binary, but a Rust port called claw-code at github.com/ultraworkers/claw-code tracks upstream behavior closely enough), and GitHub Copilot CLI, where the SDK is fully open source licensed at github.com/github/copilot-sdk.
Here is what I will cover:
- The Evidence: Terminal-Bench 2.0 : what the leaderboard actually shows about model harness pairs
- Three Harnesses, Three Bets : SQ/EQ vs typed conversation loop vs JSON RPC supervisor
- The Tool Surface : where post training is most visible
- Skills Carry Tool Specs : why "same SKILL.md format" does not mean "interchangeable"
- The Memory Layer : synchronous live writes vs deferred batch vs server side, and why the citation tag matters
- The Citation Discipline : how the model talks back to the harness
- The System Prompt Skeleton : ten section IDs is a contract
- The Routing Reality : what GitHub Copilot CLI is actually doing about all this
- Mid-Chat Model Switching : the cleanest concrete failure mode
- What the Labs Are Saying : Cursor, Anthropic, and LangChain all converging on the same framing
- The Identity File Convention : CLAUDE.md, AGENTS.md, SOUL.md, USER.md, and what each one is for
- What This Means : the model is no longer the moat alone, and the matched pair shifts as the model matures
I covered the memory layer in detail at Agent Memory Engineering. This article is about everything else, with memory revisited only where it intersects orchestration. If you want the bottom up tour of how MEMORY.md indexes, system reminder injection, age in days warnings, and signal gates work, read that one first.
## The Evidence: Terminal-Bench 2.0
Before any argument about architecture, look at the leaderboard. Terminal-Bench 2.0 evaluates agents on bash heavy multi step tasks, and it ranks by harness plus model pair, not by model alone. From April 30, 2026:
```
TERMINAL-BENCH 2.0 TOP PAIRS PASS RATE
======================================
Codex + GPT-5.5 82.0%
ForgeCode + GPT-5.4 81.8%
TongAgents + Gemini 3.1 Pro 80.2%
ForgeCode + Claude Opus 4.6 79.8%
SageAgent + GPT-5.3-Codex 78.4%
ForgeCode + Gemini 3.1 Pro 78.4%
Droid + GPT-5.3-Codex 77.3%
Capy + Claude Opus 4.6 75.3%
Simple Codex + GPT-5.3-Codex 75.1%
Terminus-KIRA + Gemini 3.1 Pro 74.8%
```
Two things jump out.
First, Claude Opus 4.6 paired with ForgeCode hits 79.8%, while the same model paired with Capy hits 75.3%. Same weights, different harness, and a 4.5 percentage point spread between them on a benchmark where every entry is fighting for a tenth of a point. Interesting!
Second, the upper rankings are not dominated by the labs that trained the models. ForgeCode is a third party harness that lands three of the top six entries by routing across model families. Stanford's IRIS Lab paired Opus 4.6 with an automated harness evolution system called Meta-Harness and pushed the same model to 76.4% on the same benchmark, well past the best baseline they started from. The harness is moving the score by more than the model upgrades are moving it.
Cursor's research team makes the point even sharper. In their April 30 post on harness engineering, they note that they took their own coding agent from "Top 30 to Top 5 on Terminal Bench 2.0 by only changing the harness." Same model. Same benchmark. Different scaffolding. A 25-position jump on a public leaderboard, attributable to the harness alone. Bravo Cursor!
LangChain's Vivek Trivedy puts the same observation in one sentence: "Opus 4.6 in Claude Code scores far below Opus 4.6 in other harnesses." Anthropic's flagship model in Anthropic's flagship harness loses to the same weights in third party scaffolding. If you only saw the model name on the spec sheet, you would not predict that.
This is the empirical case for model harness fit. Hold the model fixed and swap the harness, and the pass rate moves by enough to outweigh a model generation upgrade.
So why? What exactly does the harness do that lets two implementations of the same model produce different scores?
## Three Harnesses, Three Bets
Each harness picks a different orchestration protocol. The model was trained on that protocol's exact wire format.
These are not three implementations of the same idea. They are three different contracts between model and runtime.
Codex is a typed asynchronous protocol. The model emits a Submission with an Op and gets back a stream of typed Event messages. The protocol is defined at codex-rs/protocol/src/protocol.rs with explicit #[serde(tag = "type", rename_all = "snake_case")] enums. There is a second protocol layered on top: app-server-protocol/src/protocol/v2.rs is 10,721 lines of JSON RPC for cross process clients (IDE plugin, desktop app), where v1 (245 lines) is frozen and all new RPCs go to v2. Methods are named <resource>/<method> with singular resource names, camelCase wire format. The two protocols stack: agent layer for in process, JSON RPC layer for cross process. The model was trained to emit submissions and consume events.
Claude Code is a direct typed conversation loop. The runtime's ConversationRuntime::run_turn consumes a Vec<AssistantEvent> per turn from ApiClient::stream. AssistantEvent variants are TextDelta(String), ToolUse { id, name, input }, Usage(TokenUsage), PromptCache(PromptCacheEvent), and MessageStop. There is no separate submission queue. The protocol is the Anthropic Messages API plus a tight in process tool dispatcher. The model was trained to emit tool calls inside an assistant message and respond to tool results in the next turn.
GitHub Copilot CLI is a supervisor protocol. The host app does not run the agent loop. It spawns the bundled @github/copilot binary as a subprocess, opens a vscode-jsonrpc channel over stdio, and sends session.create with the full configuration: model, system message, tools, MCP servers, custom agents, skill directories, hook flags. The agent loop runs inside the child process. The host gets session.event notifications back. It runs inside this supervisor and emit JSON RPC events that the supervisor can route.
You can see the architectural commitment harden in each design. Codex's AGENTS.md literally polices crate growth: "Resist adding code to codex-core. The largest crate is explicitly off limits for new features." A 500 line soft cap, 800 line hard cap per Rust module. New features pay rent in the form of a new crate. This is a compiler toolchain attitude applied to an agent harness, and the model was trained to operate inside it.
Claude Code's port enforces a different rule: "one agent loop, not a fan out of specialized agents," which is why subagents in Claude Code start with a fresh context and cannot recurse. Copilot CLI's supervisor model is what lets a single binary serve three surfaces (terminal, cloud agent, third party hosts). Each surface gets the same model behavior because the model is always running inside the same supervisor.
Now imagine you swap models. Take a model trained to emit Submission { id, op: UserTurn, trace } and feed it Claude Code's Vec<AssistantEvent> stream. The model has been taught one wire shape. The harness expects another. The mismatch shows up not as an outright failure but as a quiet degradation: missed tool calls, wrong reasoning effort levels, inconsistent compaction triggers, citation tags that the harness never parses. The wire format is part of the model.
# The Tool Surface
This is where post training is most visible.
Every harness has a tool registry. The names look similar at the top: read, write, bash, grep, glob. But once you go past the first six, the surfaces diverge in ways that the model has been taught to exploit.
## Codex tool surface
Codex's tools/src/lib.rs exposes a particular vocabulary:
- apply_patch — Codex's custom diff format. Two flavors: a freeform Lark grammar at tool_apply_patch.lark and a JSON variant. The model was trained to emit patches in this format. It is not interchangeable with Claude Code's edit_file (which takes old_string / new_string).
- local_shell — the bash family. Plus exec_command_tool and write_stdin_tool for long lived processes that the model can drive with stdin writes after the fact.
- update_plan_tool — the plan/todo tool. A model not trained on this tool will use a different convention to track work.
- request_permissions_tool — model can request expanded permissions mid turn. Codex is the only harness with this exact verb.
- agent_tool — multi agent orchestration with spawn_agent_v1, spawn_agent_v2, wait_agent_v1, wait_agent_v2, send_message, close_agent_v1, close_agent_v2, resume_agent. Eight verbs. The model knows all eight.
- tool_search, tool_suggest — tools that find other tools. Codex's answer to deferred tool loading.
- goal_tool — create_goal, get_goal, update_goal. Tied to migration 0029_thread_goals.sql.
## Claude Code tool surface
Claude Code's port enumerates 40 specs in mvp_tool_specs():
- read_file, write_file, edit_file — lower case names internally, surfaced to the model as CamelCase (Read, Write, Edit). The model was trained on the CamelCase variant.
- Edit requires old_string, new_string, optional replace_all. Not the same shape as Codex's apply_patch.
- Bash has the deepest sandbox surface: command, timeout, description, run_in_background, dangerouslyDisableSandbox, namespaceRestrictions, isolateNetwork, filesystemMode. The model knows when to set run_in_background: true and pair it with the Monitor tool.
- Skill and ToolSearch — the lazy load primitives.
- Agent — single tool for subagent dispatch. Takes description, prompt, optional subagent_type, optional model. The post training has the model emit short imperative descriptions for these.
- EnterPlanMode / ExitPlanMode — both WorkspaceWrite permission. Toggles a worktree local override.
- EnterWorktree / ExitWorktree — wrap git worktree add for subagent isolation.
- Monitor — streams stdout from a background process. Pairs with Bash run_in_background: true. The model knows this pattern; Codex does not have it.
- TodoWrite — the workflow scaffolding tool. The model writes {content, activeForm, status} triplets in a particular pattern.
## GitHub Copilot CLI tool surface
Copilot CLI bundles a different default, drawn from the public changelog:
- grep, glob (bundled ripgrep), view, view_range — file reading with explicit range params.
- web_fetch — built in (v0.0.374). Rejects file:// URLs.
- read_bash, write_bash, stop_bash — three verb interactive shell control.
- task — subagent dispatch with depth and concurrency limits.
- read_agent, write_agent — multi turn subagent control. A different shape from Codex's six verb agent surface.
- ask_user — interactive clarification.
- store_memory — persistent memory tied to a remote backend. Memory is not local files here.
- apply_patch — included specifically when serving Codex models. A different patch toolchain than Codex's own.
- create_pull_request, sql, exit_plan_mode, list_copilot_spaces, show_file.
A model trained on Codex's eight verb subagent surface knows how to send a message to a running subagent. A model trained on Claude Code's Agent tool does not have that verb in its instinct set.
> The harness can paper over this with a router, but the router cannot give the model an instinct it does not have.
Cursor's harness team puts the underlying mechanic plainly. From their April 30 research post: "OpenAI's models are trained to edit files using a patch-based format, while Anthropic's models are trained on string replacement. Either model could use either tool, but giving it the unfamiliar one costs extra reasoning tokens and produces more mistakes. So in our harness, we provision each model with the tool format it had during training." This is the single cleanest description of model harness fit I have seen from any vendor, and it is not a hand wave about model preferences but a specific measurable cost in reasoning tokens paired with an observable increase in error rate, recorded at scale across millions of agent turns in production.
This is where model harness fit shows up most visibly. The tool surface is the model's vocabulary for the world. Cross train on a different vocabulary and you lose precision in every interaction.
## Skills Carry Tool Specs
Skills look interchangeable on the surface. All three harnesses use a SKILL.md file with YAML frontmatter (name, description, optional metadata). Codex even baked in cross compat: core-skills parses Claude style markdown skills. Copilot CLI explicitly reads .claude/ config. The format is so similar that the same SKILL.md body would parse in all three.
But skills are not just markdown. A skill carries an implicit contract about which tools it expects to call. That contract is not in the frontmatter. It is embedded in the body, in the form of imperative instructions that name specific tools by name, with specific argument shapes, and with specific verbs the model must emit.
Look at what each harness ships as a system skill.
Codex's bootstrap skills, baked in via include_dir! and extracted to $CODEX_HOME/skills/.system on first launch, are five: imagegen, openai-docs, plugin-creator, skill-creator, skill-installer. The skill-installer body invokes list-skills.py and install-skill-from-github.py as scripts (codex-rs/skills/src/assets/samples/skill-installer/SKILL.md). It assumes the model can call bash to run a Python script. It assumes the model knows that scripts in scripts/ of a skill folder are invokable. It assumes a sparse checkout fallback for private repos. None of that is in the frontmatter. All of it is in the body.
Claude Code's skills are different. The superpowers plugin ships superpowers:test-driven-development, superpowers:writing-plans, superpowers:verification-before-completion, superpowers:requesting-code-review, superpowers:dispatching-parallel-agents, plus many more. The bodies invoke Claude's specific tools: Skill to bootstrap into a workflow, TodoWrite to track steps, Agent to dispatch parallel subagents, Read / Edit for file changes, Grep / Glob for search. The skills also encode hard process rules: "Use this BEFORE any creative work," "Use when about to claim work is complete." These rules anchor on the harness's <system-reminder> injection model, which Codex does not have in the same form.
Copilot CLI's skills are part of the plugin marketplace ecosystem, and the changelog reveals a different posture. v1.0.5 added "Embedding based dynamic retrieval of MCP and skill instructions per turn" as experimental. The model was trained to consume skill instructions delivered as a per turn injection chosen by an embedding ranker, rather than as a description match. A skill body that assumes "you will see all skills in the system reminder" does not behave the same way when the harness ranks skills via embedding and only injects the top three.
```
SAME SKILL.MD, DIFFERENT BEHAVIORS
===================================
A skill body that says:
"Step 1: call TodoWrite with a list of three subtasks.
Step 2: dispatch each subtask via Agent with subagent_type='Explore'.
Step 3: wait for results, then verify with Bash before claiming done."
In Claude Code:
TodoWrite is a built in. Agent is a built in. subagent_type='Explore'
has a specific allowed tool set the harness enforces. Bash returns
truncated output the model knows how to parse.
✅ Works as designed.
In Codex:
TodoWrite does not exist. The closest is update_plan_tool, but the
schema differs. Agent does not exist with that exact shape.
Codex has spawn_agent_v1 / v2. The Bash equivalent is local_shell.
❌ Skill silently fails or executes a degraded version.
In Copilot CLI:
TodoWrite as a verb does not exist. Agent dispatch is via task or
read_agent / write_agent. The Bash equivalent is the three verb
read_bash / write_bash / stop_bash. Plus skills may not even be
loaded into the prompt this turn (embedding ranker decided).
❌ Skill is invisible or executes against a non existent surface.
```
This is why "we both use SKILL.md" is misleading. The format is identical; the contract underneath is not. Skills carry tool specs implicitly, and the implicit specs are pinned to the harness that authored them.
The same applies to plugin manifests. Copilot CLI's v1.0.22 explicitly added: "Plugins using .claude-plugin/ or .plugin/ manifest directories now load their MCP and LSP servers correctly." That is GitHub treating Claude Code's plugin format as a substrate to interoperate with at the file level. But the skills inside those plugins still bring assumptions about Claude Code's tool surface. Loading the file does not give the model the right vocabulary.
The lesson generalizes. A skills marketplace that claims to be cross harness is a routing problem, not just a parsing problem. Each skill needs to either declare its target harness explicitly, or get rewritten per harness, or run inside a router that translates tool calls between dialects. None of these are free.
## The Memory Layer
I covered memory in detail in Agent Memory Engineering, so I will keep this section to the parts that matter for harness fit.
Three memory architectures, three different bets:
```
HERMES CLAUDE CODE CODEX
========== ========== ==========
Synchronous live writes, Synchronous live writes, Deferred batch writes,
frozen snapshot at session one .md file per memory, two phase pipeline
start, no decay, char cap always loaded MEMORY.md triggered by 6+ hour
index, body read on idle, gpt 5.4 mini
demand with age in days extracts, gpt 5.4
<system-reminder> consolidates against
git baseline
GITHUB COPILOT CLI
==========
Server side memory backend, store_memory tool, per repo,
remote retrieval, hangs the agent if the backend is unavailable
(v1.0.23 specifically fixed: "Agent no longer hangs on the first
turn when the memory backend is unavailable")
```
The architectural choices already differ. But the harness fit story is sharper than that. Each model was trained to write memory using a specific tool with a specific schema, and to cite memory using a specific tag with a specific format.
Codex's model writes a structured raw memory artifact via Phase 1 extraction with a strict JSON schema:
```
---
description: concise but information dense description
task: <primary_task_signature>
task_group: <cwd_or_workflow_bucket>
task_outcome: <success|partial|fail|uncertain>
cwd: <single best primary working directory>
keywords: k1, k2, k3
---
```
The Phase 2 consolidation prompt is 841 lines. additionalProperties: false. Schema validation rejects malformed output at parse time. The model citations are wrapped in <oai-mem-citation> blocks. The harness has a parser at citations.rs:6-43 that increments usage_count in the SQLite state DB whenever a citation arrives. This is the model's memory ritual.
> Strip the citation tag and the harness loses its decay signal.
Claude Code's model writes memory using the standard Write and Edit tools, into one file per memory under ~/.claude/projects/<encoded-cwd>/memory/. There is no separate memory tool. The model picks one of four types (user, feedback, project, reference) by file name prefix. The body uses a **Why:** / **How to apply:** convention for behavioral rules. The harness wraps every body read in a <system-reminder> block with the dynamic age in days and a verification reminder. The model was trained to read memory through that wrapper, weight it accordingly, and skip stale claims.
Copilot CLI's model invokes store_memory as a dedicated tool. The body of the memory goes to a remote backend. Cross session memory was added in v0.0.412 as experimental. The retrieval surface is a server side query, not a local grep. The model expects the backend to be there. When the backend is unavailable (v1.0.23 fix), the agent used to hang on the first turn. That is a load bearing dependency.
Now mix and match. Run a Codex trained model on Claude Code's harness. The model will look for a memory write tool, find Write, and write a file — but it will write a file in Codex's structured format, with task_group: headers and cwd: annotations, into a directory that Claude Code does not auto load on the next session. The harness does not know to inject the index. The next session does not see the memory.
And critically, the model will emit <oai-mem-citation> blocks that Claude Code never parses. Memory effectively does not exist on the next turn.
Run a Claude trained model on Codex's harness. The model will not emit citation tags. Codex's usage_count decay signal stops incrementing. Memories that were used silently rank below memories that were not used, because the harness sees zero citations. Within a few weeks, the wrong memories are getting evicted.
Run either on Copilot CLI's harness with the remote backend. The model's local file instincts do not transfer. The store_memory tool is the only path, the schema is different, and the cross session retrieval is keyword search against a server, not the always loaded index plus on demand body read pattern that the model was trained on. The first turns will look fine because the model has memory shaped instincts. The retention will be different.
The memory layer is the densest collision surface for model harness fit. Tools, schemas, citation tags, decay signals, retrieval rituals — all of these are coupled, all of these were learned together during post training, and none of them transfer cleanly when you swap one side.
## The Citation Discipline
The <oai-mem-citation> tag is a microcosm of the larger problem.
Codex's model emits a small XML block at the end of an assistant message whenever it pulled in memory:
```
<oai-mem-citation thread_id="xyz" raw_memory_id="abc">
this entry: <description string>
</oai-mem-citation>
```
The harness has a parser that strips the block before showing the assistant message to the user, and uses the parsed thread_id to bump usage_count and last_usage columns in stage1_outputs. The parser is at citations.rs:6-43. The SQL is in migration 0016_memory_usage.sql:
```
ALTER TABLE stage1_outputs ADD COLUMN usage_count INTEGER;
ALTER TABLE stage1_outputs ADD COLUMN last_usage INTEGER;
UPDATE stage1_outputs
SET
usage_count = COALESCE(usage_count, 0) + 1,
last_usage = ?
WHERE thread_id = ?
```
This is the model's contract with the harness. Cite what you used. The harness will reward what you cited by keeping it alive. The Phase 2 consolidator ranks memories by usage_count and decays anything with no citations and no fresh source_updated_at after 30 days.
Claude Code's model has no equivalent citation tag. The harness does not need one because memory is read via the standard Read tool, and the agent's verification grep is what doubles as the "I used this" signal. The reminder text in front of every body read explicitly tells the model: "Records can become stale over time. Verify before recommending." There is no decay loop because the harness assumes the user will prune or the verification will fail in place.
Copilot CLI's model talks to a remote memory backend. The store, retrieve, and rank logic is server side. The model does not need a citation tag because the backend tracks reads on its own.
Now look at what happens in a cross harness run.
```
TRAINING DISTRIBUTION VS RUNTIME
Codex model emitted 12,000 example ↓ runtime parses
turns during post training that <oai-mem-citation>,
include <oai-mem-citation> blocks bumps usage_count.
✅ memory ranked, decayed
correctly.
Same model on Claude Code ↓ runtime ignores tag,
harness emits <oai-mem-citation> leaves it in the
in the assistant text. assistant text shown
to the user.
❌ user sees raw XML.
❌ no decay signal in
Claude Code's memory.
Claude trained model on Codex ↓ no citation tag,
harness uses memory inline, usage_count never bumps.
never wraps it in a tag. ❌ Codex's decay loop
evicts good memories
because they look unused.
```
A six character XML tag becomes the difference between a memory system that improves with use and one that degrades silently.
This is what I mean by "the wire format is part of the model." The citation tag is not a feature on a roadmap. It is a habit the model picked up during post training, and that habit only pays off inside the harness that taught it.
## The System Prompt Skeleton
The Copilot CLI SDK exposes its system prompt as a structured object with ten section IDs. Hosts can override each section, replace it, or take full control. From the open source TypeScript at github.com/github/copilot-sdk/nodejs/src/types.ts:636:
```
const SYSTEM_PROMPT_SECTIONS = {
identity: "Agent identity preamble and mode statement",
tone: "Response style, conciseness rules, output formatting",
tool_efficiency: "Tool usage patterns, parallel calling, batching",
environment_context: "CWD, OS, git root, directory listing, available tools",
code_change_rules: "Coding rules, linting/testing, ecosystem tools, style",
guidelines: "Tips, behavioral best practices",
safety: "Environment limitations, prohibited actions, security",
tool_instructions: "Per-tool usage instructions",
custom_instructions: "Repository and organization custom instructions",
last_instructions: "End of prompt: parallel tool calling, persistence"
};
```
This is not just a documentation but more the public contract of the model's training distribution. Each section has a specific role, and the model was trained to read each section as a particular kind of instruction. The safety section is harder than guidelines. The tool_instructions section is consulted when the model is mid tool call. The last_instructions section is what the model reads right before emitting a turn.
Codex has its own equivalent, less explicit. The developer prompt is assembled in this order:
```
CODEX DEVELOPER PROMPT
======================
- permission instructions
- base developer instructions
- memory_summary.md (5K tokens, always)
- collaboration mode
- realtime updates
- personality
- apps
```
Memory comes after policy and identity, before behavioral overrides. The model was trained to read this exact order.
Claude Code's static prefix:
```
CLAUDE CODE STATIC PREFIX
==========================
<base agent system prompt>
<environment block: cwd, platform, OS>
# claudeMd ← project CLAUDE.md content
# auto memory ← MEMORY.md index, capped at 200 lines
<types block describing user/feedback/project/reference>
<when to save guidance>
<verification rule before acting on memory>
<full MEMORY.md contents>
# userEmail
# currentDate
```
A different shape, a different ordering, and a different set of precedence claims about what the model should treat as binding.
The Claude trained model knows that # auto memory instructions "OVERRIDE any default behavior and you MUST follow them exactly as written." That phrase lives inside the harness rather than inside the model itself, but the model has been trained to recognize the heading and treat its contents as binding. A model trained against this prefix will hunt for # auto memory and react accordingly, while a model trained against a different prefix simply will not see the heading the same way and will give it the weight of any other piece of context.
This is the same lesson as the citation tag, scaled up. The system prompt is not generic. It is a structured artifact with section conventions that the model was taught to read in a specific way. Swap harnesses and you keep the model's reading habits but lose the structure they apply to.
## The Routing Reality: What GitHub Copilot CLI Is Doing
GitHub Copilot CLI is the most interesting harness in the comparison because it explicitly tries to route across model families. Sonnet is the default. The picker exposes Sonnet, Opus, Haiku, and the GPT 5.x family. v1.0.32 added an auto mode that selects per session.
How does Copilot CLI handle the model harness fit problem? Looking at the changelog, the strategy has three legs.
1. Per model tool inclusion
The apply_patch tool is included only when the active model is from the Codex family. v0.0.366: "Codex specific patch toolchain." The harness knows which models were trained on apply_patch and only exposes it to those models. Anthropic models get the Edit and Write shape they were trained on.
This is not a translation layer. It is a per model tool surface. The router does not pretend apply_patch and Edit are the same operation. It serves the right tool to the right model.
2. Tool search per model
v1.0.13: "Tool search for Claude models." The implication: Claude trained models expect a deferred tool loading pattern via ToolSearch. The harness only exposes the discovery loop to those models.
OpenAI trained models do not get the same loop. They get the full tool list up front because that is what they were trained on.
3. Critic agent with complementary model
v1.0.18: "New Critic agent automatically reviews plans and complex implementations using a complementary model to catch errors early (available in experimental mode for Claude models)."
The Critic is a different model than the main agent. Plans get reviewed by the complementary model. This is multi model orchestration baked into the harness, and the routing is explicit.
```
COPILOT CLI'S MULTI MODEL ROUTING
==================================
User prompt
↓
Active model (Sonnet 4.6 or GPT 5.4 or Opus 4.7)
↓
For Claude models: For OpenAI models:
expose Edit, Write, expose apply_patch,
Bash with full sandbox local_shell, exec_command,
ToolSearch deferred full tool list up front
loading enabled
↓ ↓
Critic agent (different No Critic
model) reviews plan
↓ ↓
emit assistant message emit assistant message
```
This is what a real router looks like. Not "translate everything to a common dialect," but "serve the right dialect to each model." It is more code, more state, more telemetry. It is also the only way to get top performance from each model.
The cost of this approach is honesty. The harness has to admit that "Claude on Copilot CLI" and "GPT on Copilot CLI" are different products. The user picks one or the other and gets different behavior. There is no neutral common denominator.
This is the right honest answer to model harness fit, and Copilot CLI is the only harness in the open or semi open set that actually ships it. Multi model is the crucial bet for any serious agent platform in 2026
Most customers are running multi model workflows whether their vendor admits it or not, and the only way to give every model its best performance is to build the per model routing surface inside the harness itself.
## Mid-Chat Model Switching: The Cleanest Failure Mode
The single sharpest concrete demonstration of model harness fit comes from what happens when a user switches models mid conversation. Cursor's research team describes this carefully in their April 30 post, and the failure surface is worth walking through because every assumption that breaks here is an assumption a single model harness pair quietly relies on.
Three things break at the moment of a model switch.
First, the conversation history itself is now out of distribution. The previous model produced tool calls in its native vocabulary: apply_patch blocks, <oai-mem-citation> tags, six or eight verb subagent dispatches. The new model was trained against a different vocabulary and now has to reason about a transcript full of tool calls it would not have emitted. Cursor handles this by injecting a custom instruction explicitly telling the model "you are taking over mid chat from another model" (which I find very smart!) plus steering it away from the prior model's tools.
> That mitigates but does not eliminate the cost. The model is still reading a transcript that does not match its instincts.
Second, the prompt cache breaks (painful!) Caches are provider and model specific, which means a switch is a guaranteed cache miss. For a long session, this turns the first turn after the switch into a full price re entry of every byte of system prompt and conversation history. Cursor's mitigation is to summarize the conversation at switch time, which yields a shorter clean transcript that costs less to re cache, at the price of losing details that the summary did not preserve.
Third, the tools themselves change shape. The new model's harness loads its native tool set. If the user was deep into a subagent dispatch flow with one set of verbs, the next turn presents a different set. The model has to figure out whether the prior tools are still valid (they are not) and which of its own tools maps to the user's apparent intent.
> Cursor's recommendation, after building the mitigations, is honest: "we generally recommend staying with one model for the duration of a conversation, unless you have a reason to switch."
The cleanest workaround they describe is to spawn a subagent with a different model rather than switch the main conversation. A subagent starts with a fresh context window, no transcript bias, no cache to break, and the new model's native tool surface from the first turn.
Each of these failure modes maps directly back to the thesis. The transcript, the cache prefix, and the tool surface are all parts of the wire format the model was trained against. Change the model and you change the contract on all three sides at once. A model switch is not a model swap. It is a harness swap, a tool swap, and a cache invalidation, all at once.
## What the Labs Are Saying
Cursor's Stefan Heule and Jediah Katz (@jediahkatz) describe their harness work as "obsessively stacking small optimizations" specifically because a step change is rare and the gains compound only inside a matched pair. Their team builds in custom prompting per provider and per model version, citing OpenAI's literal precision versus Claude's tolerance for imprecise instructions as concrete differentiators that flow back into prompt design.
They report driving unexpected tool call errors down by an order of magnitude in one focused sprint.
> Tool call reliability is not a model property. It is a harness property, and one that compounds every turn the agent stays alive.
Anthropic's @rgb_prithvi ran a related experiment in his March 24 post on long running application development. The architecture: a planner, a generator, and an evaluator agent, modeled on Generative Adversarial Networks.
The evaluator uses Playwright MCP to actually click through the running application as a user would, then grades against a rubric. Out of the box, Rajasekaran reports, "Claude is a poor QA agent" like it identifies legitimate issues and then talks itself into approving the work anyway. Tuning the evaluator prompt over multiple rounds is what turns it into a reliable judge.
> The harness creates the judgment surface; the model alone does not.
The deeper lesson from Rajasekaran's work is about how harnesses should evolve as models improve. He built one harness against Claude Sonnet 4.5, which exhibited "context anxiety" (read about it, it's quite funny!) strongly enough that compaction alone was not sufficient. The harness needed full context resets between sessions, with structured handoff artifacts to carry state across the boundary.
When Opus 4.6 shipped, that behavior was largely gone. Rajasekaran dropped the entire context reset machinery and ran one continuous session for over two hours. Every component in a harness encodes an assumption about what the model cannot do on its own. Those assumptions go stale.
The matched pair is not static. It moves as the model matures, and the harness has to retire scaffolding that is no longer load bearing which is the lesson I wrote about in "LLMs eat scaffolding for breakfast"
https://x.com/nicbstme/status/2015795605524901957?s=20
LangChain's Vivek Trivedy has the cleanest framing : "Agent = Model + Harness. If you're not the model, you're the harness." The harness in this view is every piece of code, configuration, and execution logic that is not the weights themselves. System prompts, tool descriptions, bundled infrastructure, orchestration logic, hooks, middleware. Working backwards from the desired agent behavior, every harness primitive earns its place by patching a specific model gap. Filesystems for durable state, bash for arbitrary action, sandboxes for safe execution, memory for continual learning, planning and self verification for long horizons. Each primitive started life as a workaround for a specific deficiency the model had at training time. Some of those primitives will get absorbed back into the model over time. Others will compound.
Trivedy also names the mechanism that makes model harness fit so durable: a co-evolution feedback loop. "Useful primitives are discovered, added to the harness, and then used when training the next generation of models. As this cycle repeats, models become more capable within the harness they were trained in." This is very very important to understand.
This is the pipeline that hardens the matched pair over generations. A new harness primitive ships in week one. By month three, it shows up in millions of agent traces. By month six, those traces are training data for the next model. By month twelve, the next model has the primitive baked into its instincts and the harness can lean on it. The loop is what makes "swap to a foreign harness" not just clumsy but compounding clumsy. The model's habits got shaped by the previous generation of its own harness, which itself was shaped by the generation before. Move sideways and you skip every cycle of that compounding.
Trivedy is honest about the cost of this loop, and I want to flag the counter argument cleanly. Quoting him: "A truly intelligent model should have little trouble switching between patch methods, but training with a harness in the loop creates this overfitting." If the model's tool format preference is overfit to its training harness, you could argue that the right long term move is to train against a more diverse set of harnesses so the model generalizes. That argument has merit. The labs that ship one model and one harness as a pair are buying near term performance at the cost of the model's portability. Whether that trade is the right one depends on whether portability is something the customer values, and right now the customer mostly values the leaderboard.
Three independent posts published within weeks of each other, all converging on a single thesis: the model is only half of the system, the harness is the other half, the matched pair is the proper unit of analysis, and the vendors that ship the matched pair as a single product are the ones currently sitting at the top of the leaderboards.
## The Identity File Convention
The harness side of the contract has converged on a markdown file per concern, and the file names are now load bearing across the ecosystem. A model trained on one harness recognizes the file names and knows which one carries which kind of authority.
```
THE MARKDOWN-FILE-PER-CONCERN CONVENTION (April 2026)
=====================================================
CLAUDE.md Static project instructions for the Claude Code session.
Anthropic's proprietary format. Loaded at session start.
The de facto standard since 2024. Closely paired with
Anthropic post training.
AGENTS.md Cross tool standard for procedural rules. "What do you
do and how." Adopted by Codex CLI, OpenClaw, Cursor.
Claude Code does not natively read AGENTS.md as of
April 2026 (community feature request open). The
GitHub Copilot CLI changelog explicitly added support
for AGENTS.md as a context source.
SOUL.md Personality, voice, worldview. From github.com/
aaronjmars/soul.md. Used by OpenClaw and the Aeon
framework. "An AI that thinks and speaks as you" vs
"a chatbot that talks about you." Opt in convention,
not yet baked into the major harnesses.
USER.md Who the user is. Hermes splits its memory into
MEMORY.md (what the agent learned) and USER.md (who
the user is). The split lets the model treat user
facts as authoritative and project facts as evolving.
MEMORY.md Auto memory index. Claude Code loads this on every
turn under # auto memory and treats it as overriding
default behavior. Codex calls its index memory_summary.md
with a 5K token cap and lazy loads the bodies.
STYLE.md Voice and writing patterns (SOUL.md ecosystem).
SKILL.md Per skill folder, with YAML frontmatter. Cross harness
adopted: Claude Code, Codex CLI, GitHub Copilot CLI all
read this format. Body conventions still differ.
IDENTITY.md Lightweight public facing card with name, role, metadata.
OpenClaw convention.
```
The key observation: the file names are now part of the wire format. A model that has been trained to look for a # auto memory block under a MEMORY.md heading will hunt for that exact heading on a turn. A model trained against AGENTS.md will look for AGENTS.md and miss CLAUDE.md. A model trained against SOUL.md will load personality from SOUL.md and ignore the same content if you put it in STYLE.md.
This is why the AGENTS.md feature request against Anthropic's repo matters. It is not a docs migration. It is a request for the model's training distribution to expand its file recognition vocabulary. Until Anthropic post trains Claude to read AGENTS.md, that file is invisible to Claude Code even if it sits next to CLAUDE.md in the repo.
The SOUL.md ecosystem is a stress test of this thesis. SOUL.md is not yet recognized by any major harness's default loader. So the SOUL.md repo's installation instructions are revealing: copy your soul/ directory into the project, then add a few lines to CLAUDE.md pointing the model at it. That is a manual bridge from a non-recognized convention to a recognized one. The SOUL.md authors understand that the bytes do not work unless the model knows where to look, and "where to look" is a habit fixed in post training.
The same routing problem shows up in the open. GitHub Copilot CLI v1.0.4 added: "Read .claude/settings.json and .claude/settings.local.json as additional repo config sources." v1.0.36 walked some of it back: "Custom agents, skills, and commands from ~/.claude/ are no longer loaded by the Copilot CLI." That is a router that tried to be permissive about file names, then narrowed when the user surface got confusing. The lesson sits underneath the changelog: even the harness that runs Claude models cannot treat .claude/ files as authoritative without negotiating with the user about which conventions count.
Pick the convention. Ship the post training to match. Or ship a router that explicitly maps each file to the model that recognizes it. The middle path of "be permissive and load anything that looks plausible" loses every time.
## What This Means
The harness is no longer a wrapper around the model. The harness is part of the model's effective parameters. The post training process embeds the harness's tool surface, schema shapes, memory rituals, citation contracts, and system prompt structure into the model's instinct set. You can take the weights to a different harness, but you cannot take the instincts. The instincts only fire when the harness presents the world the way the post training presented it.
This has three consequences worth naming.
For agent platform builders: pick a harness, pick a model, ship them as a pair. Do not pretend the model is portable. Do not pretend the harness is neutral. The frontier labs are publishing model harness pairs whether they say so or not, and the per pair performance is the only number that matters. Copilot CLI's "different tools for different models" approach is the honest version of this. The dishonest versions ship a common denominator and underperform on every model they serve.
For model labs: the harness is product strategy, not infrastructure. The harness is where the lab's post training investment compounds. Anthropic's <system-reminder> injection model, the typed memory taxonomy, the verification on every body read, are not infrastructure choices. They are the surface the model was sculpted against, and they are the moat that makes the model less interchangeable than it would otherwise be. Same for Codex's two phase memory pipeline, the citation tag, the strict JSON schema. Same for Copilot CLI's ten section system prompt skeleton. The harness is where the model becomes irreplaceable.
For users: the cost of switching is higher than it looks, and lower than vendors would like you to think. Higher because the model and the harness fused over months of training and you cannot pull them apart cleanly. Lower because the simple stack underneath is shared, and the conventions on top are documentable. A honest port — replicate the tool surface, replicate the citation contract, replicate the system prompt structure, replicate the memory ritual — would close most of the gap. It just costs as much as the original post training did to set up.
The matched pair is not static. It shifts as the model matures. This is the most useful nuance from Rajasekaran's Anthropic post. A harness component that was load bearing for Sonnet 4.5 (context resets, sprint decomposition, aggressive compaction) became dead weight on Opus 4.6 because the model started doing that work natively. The right harness for a model in March is not the right harness for that model's successor in October. The discipline is to read the traces, identify which components are still earning their place, and retire the ones that are now patches over solved problems. Cursor's blog says the same thing in different words: "Every component in a harness encodes an assumption about what the model cannot do on its own, and those assumptions go stale."
So back to the question I started with. Why does the same prompt produce visibly different output across three harnesses running the same model?
Because the model running on three harnesses is effectively three different models, even though the weights on disk are byte for byte identical. The instincts that fire at runtime are not stored only in the weights, they are conditioned by the harness the weights were trained against, and the instincts turn out to be most of what shows up in the assistant's output on any given turn.
> The interesting design move now is not a better model. It is not a better harness either. It is the matched pair, designed end to end, where the post training and the runtime reinforce each other turn after turn until the model becomes legibly better at the things this specific harness rewards.
You can see the major builders converging on this idea from three different starting points. Anthropic shipped Claude Code as the canonical Claude harness, with the post training and the runtime co-designed as a single product. OpenAI shipped Codex CLI as the canonical Codex harness, with the same vertical integration on the OpenAI side of the house.
The frontier work in 2026 is not about new model architectures. It is about new harness primitives. Ralph Loops, where a hook intercepts the model's exit attempt and reinjects the original prompt in a clean context window, forcing the agent to keep grinding against the goal. Just-in-time harness assembly, where the tool surface and the system prompt get composed per task instead of pre-configured per session. Self-tracing agents that read their own logs to find harness-level failure modes and patch them without human intervention. Each one of these is a primitive that some model will eventually be post trained against, and that pairing will show up at the top of the next leaderboard.
Once again, if you want to stay at the edge, you have to delete most of your code when a new model is released. LLMs eat scaffolding for breakfast.
## 相关链接
- [Nicolas Bustamante](https://x.com/nicbstme)
- [@nicbstme](https://x.com/nicbstme)
- [@_lopopolo](https://x.com/@_lopopolo)
- [github.com/openai/codex](https://github.com/openai/codex)
- [github.com/ultraworkers/claw-code](https://github.com/ultraworkers/claw-code)
- [github.com/github/copilot-sdk](https://github.com/github/copilot-sdk)
- [Agent Memory Engineering](https://nicolasbustamante.com/blog/agent-memory-engineering)
- [@github/copilot](https://x.com/@github/copilot)
- [$CODEX](https://x.com/search?q=%24CODEX&src=cashtag_click)
- [list-skills.py](https://list-skills.py/)
- [install-skill-from-github.py](https://install-skill-from-github.py/)
- [Agent Memory Engineering](https://x.com/nicbstme/status/2050301124314563025?s=20)
- [citations.rs](https://citations.rs/)
- [citations.rs](https://citations.rs/)
- [github.com/github/copilot-sdk/nodejs/src/types.ts:636](https://github.com/github/copilot-sdk/nodejs/src/types.ts:636)
- [@jediahkatz](https://x.com/@jediahkatz)
- [@rgb_prithvi](https://x.com/@rgb_prithvi)
- [https://x.com/nicbstme/status/2015795605524901957?s=20](https://x.com/nicbstme/status/2015795605524901957?s=20)
- [Upgrade to Premium](https://x.com/i/premium_sign_up)
- [10:48 AM · May 4, 2026](https://x.com/nicbstme/status/2051131906327212298)
- [5,082 Views](https://x.com/nicbstme/status/2051131906327212298/analytics)
---
*导出时间: 2026/5/4 13:08:30*
---
## 中文翻译
# 模型-框架契合度（Model-Harness-Fit）
**作者**：Nicolas Bustamante
**日期**：2026-05-04T02:48:49.000Z
**来源**：[https://x.com/nicbstme/status/2051131906327212298](https://x.com/nicbstme/status/2051131906327212298)
---
使用 LLM 时，是搭配其原生框架（如 Claude Code 或 Codex）更好，还是使用可以按需切换模型的通用框架更好？
OpenAI 工程师 Ryan（@_lopopolo）今天早上发了一条非常搞笑的表情包，所以我决定深入研究一下 Codex、Claude Code 和 GitHub SDK 的框架实现。框架真的有那么重要吗？
我在同一台工作站上同时运行三个编程代理。一个终端运行 Claude Code，另一个运行 Codex CLI，第三个运行 GitHub Copilot CLI。相同的文件、相同的 git 树、相同的 bash。三个看起来毫无差别的不同框架。
我对这三个代理运行了相同的提示词，行为差异非常明显，远远超出了我原本预期的供应商之间在风格和速度上的表面差异。
敷衍的回答是“模型表现不同，因为它们本来就是不同的模型”。但在这里，我测试的是相同的模型和不同的框架。
模型是针对框架进行后训练的，而不仅仅是针对 API。它们期望的工具名称、它们输出的输入模式（schema）、它们包裹已记忆事实的引用标签、它们调用的技能的文件结构、当框架说“先制定计划”时它们遵循的规划协议——这些都不是模型的通用能力。
它们是字节级别的约定，在某个特定模型针对某个特定框架的后训练中被固化下来。把模型从它的框架中抽出来，你就放弃了那些不重写任何一方就无法挽回的性能。
这直接带来了一个任何尝试发布“模型无关”代理的人都遇到过的问题：你不能简单地替换模型。
支持 BYOK（自带密钥）和多模型（这是负责任的做法，因为依赖单一供应商是有风险的！）会增加实实在在的工程复杂性。
要干净地替换一个模型，你必须连同框架一起替换：工具接口、模式结构、命名这些工具的技能内容、引用契约、记忆仪式、系统提示词结构，有时还包括规划协议。
模型一动，模型之上的一切都得跟着动。这就是为什么每个支持多个供应商的代理厂商最终要么
(a) 运行其支持的每个模型的降级变体，要么
(b) 为每个模型维护一套独立的全栈，并把选择权交给用户，即“你选择的是一个产品，而不仅仅是一个模型”。选项 (b) 是在质量上胜出的路径，为了避免被锁定在单一实验室，付出这个工程成本是值得的。
更换编排器不是表面上的改动。它是伪装起来的模型更换。前沿实验室在过去一年里将模型的直觉塑造成特定的工具接口、特定的记忆仪式、特定的技能格式。当你混搭时，这些工作就白费了。
我认为这是当今代理设计中最被低估的约束条件，而且它有一个清晰的名字。就叫它“模型-框架契合度”（Model-Harness-Fit）。
## 那么这实际上在底层是如何体现的？
我深入研究了三个目前已在使用的开源实现：Codex CLI（OpenAI，在 github.com/openai/codex 完全开源）、Claude Code（Anthropic，闭源二进制，但在 github.com/ultraworkers/claw-code 有一个叫 claw-code 的 Rust 移植版，对上游行为的追踪足够紧密），以及 GitHub Copilot CLI，其 SDK 在 github.com/github/copilot-sdk 完全开源。
以下是我将要讲的内容：
- 证据：Terminal-Bench 2.0——排行榜实际展示了关于“模型+框架”组合的什么信息
- 三个框架，三种押注：SQ/EQ 对比类型化对话循环对比 JSON RPC 监管器
- 工具接口：后训练最明显的地方
- 技能承载工具规格：为什么“相同的 SKILL.md 格式”不意味着“可以互换”
- 记忆层：同步实时写入对比延迟批处理对比服务器端，以及为什么引用标签很重要
- 引用规范：模型如何与框架对话
- 系统提示词骨架：十个区块 ID 就是一份契约
- 路由现实：GitHub Copilot CLI 在这一切中究竟在做什么
- 对话中途切换模型：最干净的具体失败模式
- 实验室们怎么说：Cursor、Anthropic 和 LangChain 都在向同一个框架靠拢
- 身份文件约定：CLAUDE.md、AGENTS.md、SOUL.md、USER.md，以及各自用途
- 这意味着什么：模型本身不再是唯一的护城河，而且匹配的组合会随着模型的成熟而变化
我在《Agent Memory Engineering》一文中详细介绍了记忆层。本文讨论其余的一切，仅在记忆与编排交叉的地方重新提及记忆。如果你想自底向上了解 MEMORY.md 索引、系统提醒注入、按天计龄警告和信号门控是如何工作的，请先读那篇文章。
## 证据：Terminal-Bench 2.0
在任何关于架构的讨论之前，先看看排行榜。Terminal-Bench 2.0 在以 bash 为主的多步骤任务上评估代理，它按“框架+模型”组合排名，而不是按模型单独排名。截至 2026 年 4 月 30 日：
```
TERMINAL-BENCH 2.0 TOP PAIRS PASS RATE
======================================
Codex + GPT-5.5 82.0%
ForgeCode + GPT-5.4 81.8%
TongAgents + Gemini 3.1 Pro 80.2%
ForgeCode + Claude Opus 4.6 79.8%
SageAgent + GPT-5.3-Codex 78.4%
ForgeCode + Gemini 3.1 Pro 78.4%
Droid + GPT-5.3-Codex 77.3%
Capy + Claude Opus 4.6 75.3%
Simple Codex + GPT-5.3-Codex 75.1%
Terminus-KIRA + Gemini 3.1 Pro 74.8%
```
有两个现象格外引人注目。
首先，Claude Opus 4.6 搭配 ForgeCode 达到了 79.8%，而同一模型搭配 Capy 达到了 75.3%。相同的权重，不同的执行框架，在一个每零点一分都竞争激烈的基准测试中，两者之间相差 4.5 个百分点。有意思！
其次，排行榜上游的位置并不是由训练这些模型的实验室所主导。ForgeCode 是一个第三方执行框架，通过跨模型家族路由，占据了前六名中的三个席位。斯坦福大学的 IRIS 实验室将 Opus 4.6 与一个名为 Meta-Harness 的自动化执行框架进化系统结合，将同一模型在同一基准测试上推到了 76.4%，远超他们最初的基线。执行框架带来的分数提升超过了模型升级带来的提升。
Cursor 的研究团队把这一点说得更尖锐。在他们 4 月 30 日关于执行框架工程的帖子中，他们指出，他们仅通过更改执行框架，就将自己的编码智能体从“Terminal Bench 2.0 的前 30 名提升到前 5 名”。相同的模型。相同的基准测试。不同的脚手架。在公开排行榜上跃升 25 个名次，完全归因于执行框架。干得漂亮，Cursor！
LangChain 的 Vivek Trivedy 用一句话概括了同样的观察："Opus 4.6 在 Claude Code 中的得分远低于它在其他执行框架中的得分。" Anthropic 的旗舰模型在 Anthropic 的旗舰执行框架中，输给了运行在第三方脚手架中的相同权重。如果你只看规格表上的模型名称，你不会预料到这一点。
这就是模型与执行框架匹配的实证案例。固定模型，更换执行框架，通过率的变化足以超越一次模型代际升级。
那么为什么呢？执行框架究竟做了什么，让同一模型的两种实现产生不同的分数？
## 三个执行框架，三种赌注
每个执行框架都选择了不同的编排协议。模型正是在该协议的确切线路格式上进行训练的。
这不是同一个想法的三种实现。它们是模型与运行时之间的三份不同契约。
Codex 是一个带类型的异步协议。模型发出一个包含 Op 的 Submission，并接收一串带类型的 Event 消息。该协议定义于 codex-rs/protocol/src/protocol.rs，使用了显式的 #[serde(tag = "type", rename_all = "snake_case")] 枚举。其上还叠加了第二个协议：app-server-protocol/src/protocol/v2.rs 是 10,721 行用于跨进程客户端（IDE 插件、桌面应用）的 JSON RPC，其中 v1（245 行）已冻结，所有新 RPC 都进入 v2。方法命名为 <resource>/<method>，资源名用单数形式，线路格式采用 camelCase。两个协议叠加：进程内使用智能体层，跨进程使用 JSON RPC 层。模型被训练为发出 submission 并消费 event。
Claude Code 是一个直接的类型化对话循环。运行时的 ConversationRuntime::run_turn 每回合从 ApiClient::stream 消费一个 Vec<AssistantEvent>。AssistantEvent 的变体包括 TextDelta(String)、ToolUse { id, name, input }、Usage(TokenUsage)、PromptCache(PromptCacheEvent) 和 MessageStop。没有单独的提交队列。该协议就是 Anthropic Messages API 加上一个紧凑的进程内工具调度器。模型被训练为在 assistant 消息内发出工具调用，并在下一回合响应工具结果。
GitHub Copilot CLI 是一个监督者协议。宿主应用并不运行智能体循环。它将打包的 @github/copilot 二进制文件作为子进程启动，通过 stdio 打开一个 vscode-jsonrpc 通道，并发送包含完整配置的 session.create：模型、系统消息、工具、MCP 服务器、自定义智能体、技能目录、钩子标志。智能体循环在子进程内部运行。宿主接收 session.event 通知。它在这个监督者内部运行，并发出监督者可以路由的 JSON RPC 事件。
你可以在每个设计中看到架构承诺的固化。Codex 的 AGENTS.md 字面上管束着 crate 的增长："抵制向 codex-core 添加代码。最大的 crate 明确禁止新功能。" 每个 Rust 模块有 500 行的软上限和 800 行的硬上限。新功能必须以新建 crate 的形式"交房租"。这是将编译器工具链的态度应用于智能体执行框架，而模型正是被训练在这个环境中运作的。
Claude Code 的移植版执行另一条规则："一个智能体循环，而不是专业化智能体的扇出"，这就是为什么 Claude Code 中的子智能体以全新上下文启动且不能递归。Copilot CLI 的监督者模型使得单个二进制文件可以服务三个界面（终端、云端智能体、第三方宿主）。每个界面获得相同的模型行为，因为模型始终在同一个监督者内部运行。
现在想象你更换模型。拿一个被训练为发出 Submission { id, op: UserTurn, trace } 的模型，喂给它 Claude Code 的 Vec<AssistantEvent> 流。模型只学过一种线路格式，而执行框架期望的是另一种。这种不匹配不会表现为彻底失败，而是表现为悄然退化：漏掉的工具调用、错误的推理强度级别、不一致的压缩触发，以及执行框架永远不会解析的引用标签。线路格式是模型的一部分。
# 工具界面
这是训练后改动最明显的地方。
每个执行框架都有一个工具注册表。最上层看起来名称相似：read、write、bash、grep、glob。但一旦越过前六个，这些工具界面就会分化，而模型已经被训练去利用这些差异。
## Codex 的工具界面
Codex 的 tools/src/lib.rs 暴露了一套特定的词汇表：
- apply_patch — Codex 自定义的差异格式。有两种形式：位于 tool_apply_patch.lark 的自由格式 Lark 语法，以及一种 JSON 变体。模型经过训练，会以这种格式输出补丁。它与 Claude Code 的 edit_file（接受 old_string / new_string 参数）不可互换。
- local_shell — bash 家族工具。外加 exec_command_tool 和 write_stdin_tool，用于长时运行的进程，模型可以在事后通过写入 stdin 来驱动它们。
- update_plan_tool — 计划/待办工具。未在该工具上训练过的模型会采用不同的约定来跟踪工作。
- request_permissions_tool — 模型可以在对话中途请求扩展权限。Codex 是唯一具备这一确切操作的高层框架。
- agent_tool — 多智能体编排，包含 spawn_agent_v1、spawn_agent_v2、wait_agent_v1、wait_agent_v2、send_message、close_agent_v1、close_agent_v2、resume_agent。共八个操作。模型全部知晓。
- tool_search、tool_suggest — 用于查找其他工具的工具。这是 Codex 对延迟加载工具的解决方案。
- goal_tool — create_goal、get_goal、update_goal。与迁移脚本 0029_thread_goals.sql 绑定。
## Claude Code 工具面
Claude Code 的移植在 mvp_tool_specs() 中枚举了 40 个规格：
- read_file、write_file、edit_file — 内部使用小写名称，以 CamelCase 形式（Read、Write、Edit）呈现给模型。模型是在 CamelCase 变体上训练的。
- Edit 需要 old_string、new_string、可选的 replace_all。与 Codex 的 apply_patch 形状不同。
- Bash 拥有最深的沙箱参数面：command、timeout、description、run_in_background、dangerouslyDisableSandbox、namespaceRestrictions、isolateNetwork、filesystemMode。模型知道何时设置 run_in_background: true，并将其与 Monitor 工具配合使用。
- Skill 和 ToolSearch — 懒加载原语。
- Agent — 用于子智能体调度的单一工具。接受 description、prompt、可选的 subagent_type、可选的 model。后训练使模型会为这些参数输出简短的祈使式描述。
- EnterPlanMode / ExitPlanMode — 两者均为 WorkspaceWrite 权限。切换一个 worktree 本地覆盖。
- EnterWorktree / ExitWorktree — 封装 git worktree add，用于子智能体隔离。
- Monitor — 流式读取后台进程的 stdout。与 Bash 的 run_in_background: true 配对。模型熟悉这种模式；Codex 没有它。
- TodoWrite — 工作流脚手架工具。模型会以特定模式写入 {content, activeForm, status} 三元组。
## GitHub Copilot CLI 工具面
Copilot CLI 打包了一套不同的默认工具，取自公开的更新日志：
- grep、glob（捆绑 ripgrep）、view、view_range — 带有显式范围参数的文件读取。
- web_fetch — 内置（v0.0.374）。拒绝 file:// URL。
- read_bash、write_bash、stop_bash — 三操作交互式 shell 控制。
- task — 带有深度和并发限制的子智能体调度。
- read_agent、write_agent — 多轮子智能体控制。与 Codex 的六操作智能体面形状不同。
- ask_user — 交互式澄清。
- store_memory — 绑定到远程后端的持久记忆。这里的记忆不是本地文件。
- apply_patch — 仅在服务 Codex 模型时包含。与 Codex 自己的补丁工具链不同。
- create_pull_request、sql、exit_plan_mode、list_copilot_spaces、show_file。
在 Codex 的八操作子智能体面上训练的模型，知道如何向一个正在运行的子智能体发送消息。在 Claude Code 的 Agent 工具上训练的模型，其本能集合中没有那个操作。
> 高层框架可以用一个路由器来掩盖这一点，但路由器无法赋予模型它所不具备的本能。
Cursor 的框架团队直白地描述了这一底层机制。在他们 4 月 30 日的研究文章中写道："OpenAI 的模型经过训练，使用基于补丁的格式编辑文件，而 Anthropic 的模型则在字符串替换上训练。任何一个模型都可以使用任何一种工具，但给它不熟悉的那个工具会消耗额外的推理 token，并产生更多错误。因此在我们的框架中，我们为每个模型提供它在训练时所使用的工具格式。"这是我在任何厂商那里见过的关于模型与框架契合度的最清晰的描述，它不是对模型偏好的敷衍了事，而是一种具体可衡量的推理 token 成本，伴随着可观察到的错误率上升，并在生产环境中数百万次智能体轮次中大规模记录在案。
这正是模型与框架契合度最明显体现之处。工具面就是模型理解世界的词汇表。用不同的词汇表交叉训练，你在每一次交互中都会损失精确度。
## 技能承载工具规格
技能在表面上看起来可以互换。三个框架都使用带 YAML frontmatter（name、description、可选元数据）的 SKILL.md 文件。Codex 甚至内置了跨兼容性：core-skills 可以解析 Claude 风格的 markdown 技能。Copilot CLI 明确读取 .claude/ 配置。格式如此相似，以至于同一个 SKILL.md 正文可以在三者中都被解析。
但技能不仅仅是 markdown。一项技能承载着关于它期望调用哪些工具的隐式契约。这个契约不在 frontmatter 中。它嵌入在正文中，以祈使式指令的形式存在，这些指令指名道姓地提到特定工具，带有特定的参数形状，以及模型必须输出的特定操作。
看看每个框架作为系统技能发布了什么。
Codex 的引导技能，通过 include_dir! 内置并在首次启动时解压到 $CODEX_HOME/skills/.system，共有五个：imagegen、openai-docs、plugin-creator、skill-creator、skill-installer。skill-installer 的正文将 list-skills.py 和 install-skill-from-github.py 作为脚本调用（codex-rs/skills/src/assets/samples/skill-installer/SKILL.md）。它假设模型可以调用 bash 来运行 Python 脚本。它假设模型知道技能文件夹 scripts/ 目录中的脚本是可调用的。它假设对私有仓库有稀疏检出（sparse checkout）的回退机制。这些都不在 frontmatter 中，全都在正文里。
Claude Code 的技能则不同。superpowers 插件提供了 superpowers:test-driven-development、superpowers:writing-plans、superpowers:verification-before-completion、superpowers:requesting-code-review、superpowers:dispatching-parallel-agents 等等许多技能。这些技能的正文会调用 Claude 的特定工具：用 Skill 来引导进入工作流，用 TodoWrite 来跟踪步骤，用 Agent 来派发并行子代理，用 Read / Edit 来修改文件，用 Grep / Glob 来搜索。这些技能还固化了严格的流程规则：“在进行任何创造性工作之前使用本技能”、“在即将宣称工作完成时使用”。这些规则建立在 harness 的 `<system-reminder>` 注入机制之上，而 Codex 并没有相同形式的机制。
Copilot CLI 的技能则属于插件市场生态的一部分，其变更日志透露出另一种姿态。v1.0.5 将“基于嵌入的 MCP 与技能指令逐轮动态检索”作为实验性功能加入。模型被训练为消费由嵌入排序器逐轮选择注入的技能指令，而不是通过描述匹配。一个假设“你会在系统提醒中看到所有技能”的技能正文，在 harness 通过嵌入排序且只注入前三个技能的情况下，行为就会不一样。
```
SAME SKILL.MD, DIFFERENT BEHAVIORS
===================================
A skill body that says:
"Step 1: call TodoWrite with a list of three subtasks.
Step 2: dispatch each subtask via Agent with subagent_type='Explore'.
Step 3: wait for results, then verify with Bash before claiming done."
In Claude Code:
TodoWrite is a built in. Agent is a built in. subagent_type='Explore'
has a specific allowed tool set the harness enforces. Bash returns
truncated output the model knows how to parse.
✅ Works as designed.
In Codex:
TodoWrite does not exist. The closest is update_plan_tool, but the
schema differs. Agent does not exist with that exact shape.
Codex has spawn_agent_v1 / v2. The Bash equivalent is local_shell.
❌ Skill silently fails or executes a degraded version.
In Copilot CLI:
TodoWrite as a verb does not exist. Agent dispatch is via task or
read_agent / write_agent. The Bash equivalent is the three verb
read_bash / write_bash / stop_bash. Plus skills may not even be
loaded into the prompt this turn (embedding ranker decided).
❌ Skill is invisible or executes against a non existent surface.
```
这就是为什么“我们都用 SKILL.md”这种说法具有误导性。格式完全相同，但底层的契约并不相同。技能隐含地携带工具规范，而这些隐含规范是与其创作所在的 harness 绑定的。
同样的道理也适用于插件清单。Copilot CLI 的 v1.0.22 明确加入：“使用 .claude-plugin/ 或 .plugin/ 清单目录的插件现在能正确加载其 MCP 和 LSP 服务器。”这是 GitHub 将 Claude Code 的插件格式当作可以在文件层面互操作的底层格式。但这些插件内部的技能仍然带着对 Claude Code 工具面的假设。加载文件并不能让模型获得正确的词汇表。
这个教训具有普遍性。一个声称跨 harness 的技能市场，是一个路由问题，而不仅仅是一个解析问题。每个技能要么显式声明其目标 harness，要么针对每个 harness 重写，要么运行在一个能在不同方言之间翻译工具调用的路由器中。这些选项没有一个是零成本的。
## 记忆层
我在《Agent Memory Engineering》中详细讨论过记忆，所以本节只保留与 harness 适配相关的部分。
三种记忆架构，三种不同的押注：
```
HERMES CLAUDE CODE CODEX
========== ========== ==========
Synchronous live writes, Synchronous live writes, Deferred batch writes,
frozen snapshot at session one .md file per memory, two phase pipeline
start, no decay, char cap always loaded MEMORY.md triggered by 6+ hour
index, body read on idle, gpt 5.4 mini
demand with age in days extracts, gpt 5.4
<system-reminder> consolidates against
git baseline
GITHUB COPILOT CLI
==========
Server side memory backend, store_memory tool, per repo,
remote retrieval, hangs the agent if the backend is unavailable
(v1.0.23 specifically fixed: "Agent no longer hangs on the first
turn when the memory backend is unavailable")
```
架构选择已经各不相同。但 harness 适配的故事比这更尖锐。每个模型都被训练成使用具有特定模式的特定工具来写入记忆，并使用具有特定格式的特定标签来引用记忆。
Codex 的模型通过第一阶段提取写入一个结构化的原始记忆工件，采用严格的 JSON 模式：
```
---
description: concise but information dense description
task: <primary_task_signature>
task_group: <cwd_or_workflow_bucket>
task_outcome: <success|partial|fail|uncertain>
cwd: <single best primary working directory>
keywords: k1, k2, k3
---
```
第二阶段的整合提示词有 841 行。additionalProperties: false。模式验证会在解析时拒绝格式错误的输出。模型的引用被包裹在 `<oai-mem-citation>` 块中。harness 在 citations.rs:6-43 处有一个解析器，每当有引用到达时就在 SQLite 状态数据库中将 usage_count 递增。这就是模型的记忆仪式。
> 去掉引用标签，harness 就失去了它的衰减信号。
Claude Code 的模型使用标准的 Write 和 Edit 工具来写入记忆，每个记忆对应一个文件，存放在 ~/.claude/projects/<encoded-cwd>/memory/ 目录下。没有单独的记忆工具。模型通过文件名前缀从四种类型（user、feedback、project、reference）中选择一种。正文采用 **Why:** / **How to apply:** 的约定来记录行为规则。框架在每次读取正文时都会将其包裹在一个 <system-reminder> 块中，其中包含以天为单位的动态时效以及一条验证提醒。模型经过训练，会通过这个包装层读取记忆、据此调整权重，并跳过过时的内容。
Copilot CLI 的模型通过调用 store_memory 作为专用工具来存储记忆。记忆内容会被发送到远程后端。跨会话记忆是在 v0.0.412 中作为实验性功能加入的。检索界面是服务器端的查询，而不是本地的 grep。模型期望后端始终存在。当后端不可用时（在 v1.0.23 中修复），智能体会在第一轮对话时挂起。这是一个承重型依赖。
现在来做混搭实验。在 Claude Code 的框架上运行一个 Codex 训练的模型。模型会寻找一个记忆写入工具，找到 Write，然后写入一个文件——但它会以 Codex 的结构化格式写入文件，带有 task_group: 标头和 cwd: 注解，写入到一个 Claude Code 在下次会话时不会自动加载的目录中。框架不知道要注入索引。下次会话看不到这些记忆。
而且关键的是，模型会输出 Claude Code 从不解析的 <oai-mem-citation> 块。记忆在下一轮实际上就等于不存在。
在 Codex 的框架上运行一个 Claude 训练的模型。模型不会输出引用标签。Codex 的 usage_count 衰减信号停止递增。被使用过的记忆会默默排在未使用记忆的后面，因为框架看到的引用次数为零。几周之内，错误的记忆就会被清除。
在带有远程后端的 Copilot CLI 框架上运行其中任何一个模型。模型的本地文件直觉无法迁移。store_memory 工具是唯一的路径，模式不同，跨会话检索是对服务器进行的关键词搜索，而不是模型训练中所学到的"始终加载的索引加按需读取正文"模式。最初几轮看起来会正常，因为模型具有与记忆相关的直觉。但记忆保持效果会不同。
记忆层是模型与框架匹配度最密集的碰撞面。工具、模式、引用标签、衰减信号、检索仪式——所有这些都是耦合的，都是在后训练期间一起学习的，而当你把其中一边换掉时，它们都无法干净地迁移。
## 引用规约
<oai-mem-citation> 标签是更大问题的一个缩影。
每当 Codex 的模型在回答中调用了记忆时，它就会在助手消息末尾输出一个小的 XML 块：
```
<oai-mem-citation thread_id="xyz" raw_memory_id="abc">
this entry: <description string>
</oai-mem-citation>
```
框架有一个解析器，在向用户展示助手消息之前会剥离该块，并使用解析出的 thread_id 来递增 stage1_outputs 中的 usage_count 和 last_usage 字段。解析器位于 citations.rs:6-43。SQL 位于迁移文件 0016_memory_usage.sql 中：
```
ALTER TABLE stage1_outputs ADD COLUMN usage_count INTEGER;
ALTER TABLE stage1_outputs ADD COLUMN last_usage INTEGER;
UPDATE stage1_outputs
SET
usage_count = COALESCE(usage_count, 0) + 1,
last_usage = ?
WHERE thread_id = ?
```
这是模型与框架之间的契约：引用你用到的内容。框架会通过让你引用的内容保持存活来奖励你的引用。Phase 2 整合器按 usage_count 对记忆排序，并衰减那些在 30 天后既无引用又无新鲜 source_updated_at 的记忆。
Claude Code 的模型没有对应的引用标签。框架也不需要，因为记忆是通过标准 Read 工具读取的，而智能体的验证 grep 正好充当了"我用过这个"的信号。每次读取正文前的提醒文本会明确告诉模型："记录会随时间过时。推荐之前请先验证。"这里没有衰减循环，因为框架假设用户会手动清理，或者验证会当场失败。
Copilot CLI 的模型与远程记忆后端通信。存储、检索和排序逻辑都在服务器端。模型不需要引用标签，因为后端会自行跟踪读取。
现在看看跨框架运行时会发生什么。
```
训练分布 vs 运行时
Codex 模型在后训练期间 ↓ 运行时解析
输出了 12,000 个包含 <oai-mem-citation>，
<oai-mem-citation> 块的 递增 usage_count。
示例轮次。 ✅ 记忆被正确排序和衰减。
同一模型在 Claude Code ↓ 运行时忽略标签，
框架上在助手文本中输出 将其留在展示给
<oai-mem-citation>。 用户的助手文本中。
❌ 用户看到原始 XML。
❌ Claude Code 的记忆
中没有衰减信号。
Claude 训练的模型在 ↓ 没有引用标签，
Codex 框架上内联使用记忆， usage_count 永远不会递增。
从不将其包裹在标签中。 ❌ Codex 的衰减循环
会清除好记忆，
因为它们看起来未被使用。
```
一个六个字符的 XML 标签，成为一个记忆系统是越用越好还是悄悄退化的分水岭。
这就是我所说的"线格式是模型的一部分"的含义。引用标签不是路线图上的一个功能。它是模型在后训练期间养成的一种习惯，而这种习惯只在教会它的那个框架内才能发挥作用。
## 系统提示词骨架
Copilot CLI SDK 将其系统提示词暴露为一个包含十个部分 ID 的结构化对象。宿主可以覆盖每个部分、替换它，或者完全接管控制。来自开源 TypeScript 代码 github.com/github/copilot-sdk/nodejs/src/types.ts:636：
```
const SYSTEM_PROMPT_SECTIONS = {
identity: "Agent identity preamble and mode statement",
tone: "Response style, conciseness rules, output formatting",
tool_efficiency: "Tool usage patterns, parallel calling, batching",
environment_context: "CWD, OS, git root, directory listing, available tools",
code_change_rules: "Coding rules, linting/testing, ecosystem tools, style",
guidelines: "Tips, behavioral best practices",
safety: "Environment limitations, prohibited actions, security",
tool_instructions: "Per-tool usage instructions",
custom_instructions: "Repository and organization custom instructions",
last_instructions: "End of prompt: parallel tool calling, persistence"
};
```
这不仅是文档，更像是模型训练分布的公开契约。每个部分都有特定的角色，模型经过训练后会以特定类型指令的方式阅读每个部分。safety 部分的约束力比 guidelines 更强。tool_instructions 部分是在模型进行工具调用过程中被参考的。last_instructions 部分则是模型在发出一轮回复之前最后读取的内容。
Codex 有自己的对应实现，只是不那么显式。开发者提示词按以下顺序组装：
```
CODEX DEVELOPER PROMPT
======================
- permission instructions
- base developer instructions
- memory_summary.md (5K tokens, always)
- collaboration mode
- realtime updates
- personality
- apps
```
记忆位于策略和身份之后、行为覆盖之前。模型就是按照这个确切顺序训练出来的。
Claude Code 的静态前缀：
```
CLAUDE CODE STATIC PREFIX
==========================
<base agent system prompt>
<environment block: cwd, platform, OS>
# claudeMd ← project CLAUDE.md content
# auto memory ← MEMORY.md index, capped at 200 lines
<types block describing user/feedback/project/reference>
<when to save guidance>
<verification rule before acting on memory>
<full MEMORY.md contents>
# userEmail
# currentDate
```
不同的形态、不同的排序，以及关于模型应将什么视为约束的不同优先级主张。
经过训练的 Claude 模型知道 # auto memory 指令"OVERRIDE any default behavior and you MUST follow them exactly as written."（覆盖任何默认行为，且你必须严格照做）。这个短语存在于 harness 中而非模型本身之内，但模型已被训练去识别该标题并将其内容视为约束。针对这个前缀训练的模型会搜寻 # auto memory 并做出相应反应，而针对不同前缀训练的模型则不会以相同方式看待这个标题，只会把它当作普通上下文赋予同等权重。
这与引用标签的教训相同，只是规模更大。系统提示词并非通用的。它是一个结构化的产物，具有模型被教导以特定方式阅读的部分约定。更换 harness，你保留了模型的阅读习惯，却失去了它们所作用的结构。
## 路由的现实：GitHub Copilot CLI 在做什么
GitHub Copilot CLI 是这场对比中最有趣的 harness，因为它明确尝试跨模型家族进行路由。Sonnet 是默认选项。选择器提供 Sonnet、Opus、Haiku 以及 GPT 5.x 家族。v1.0.32 增加了一种按会话选择的自动模式。
Copilot CLI 如何处理模型与 harness 的契合问题？从更新日志来看，其策略有三条支柱。
1. 按模型包含工具
apply_patch 工具仅在活跃模型来自 Codex 家族时才被包含。v0.0.366："Codex specific patch toolchain。"（Codex 专用的补丁工具链。）harness 知道哪些模型是针对 apply_patch 训练的，并只将该工具暴露给这些模型。Anthropic 模型则获得它们训练时所用的 Edit 和 Write 形态。
这不是一个翻译层。这是按模型的工具表面。路由器并不假装 apply_patch 和 Edit 是同一操作。它为正确的模型提供正确的工具。
2. 按模型的工具搜索
v1.0.13："Tool search for Claude models."（为 Claude 模型提供工具搜索。）这暗示着：经训练的 Claude 模型期望通过 ToolSearch 实现延迟工具加载模式。harness 只向这些模型暴露发现循环。
OpenAI 训练的模型不会得到同样的循环。它们直接获得完整的工具列表，因为那就是它们训练时的方式。
3. 使用互补模型的 Critic 智能体
v1.0.18："New Critic agent automatically reviews plans and complex implementations using a complementary model to catch errors early (available in experimental mode for Claude models)."（新的 Critic 智能体使用互补模型自动审查计划和复杂实现，以尽早发现错误——在 Claude 模型的实验模式下可用。）
Critic 是与主智能体不同的模型。计划由互补模型审查。这是内置于 harness 中的多模型编排，且路由是显式的。
```
COPILOT CLI'S MULTI MODEL ROUTING
==================================
User prompt
↓
Active model (Sonnet 4.6 or GPT 5.4 or Opus 4.7)
↓
For Claude models: For OpenAI models:
expose Edit, Write, expose apply_patch,
Bash with full sandbox local_shell, exec_command,
ToolSearch deferred full tool list up front
loading enabled
↓ ↓
Critic agent (different No Critic
model) reviews plan
↓ ↓
emit assistant message emit assistant message
```
这才是真正的路由器该有的样子。不是"把一切都翻译成一种通用方言"，而是"为每个模型提供正确的方言"。这需要更多代码、更多状态、更多遥测数据。这也是从每个模型中获得最佳性能的唯一途径。
这种方法的代价是诚实性。harness 必须承认"Copilot CLI 上的 Claude"和"Copilot CLI 上的 GPT"是不同的产品。用户选择其中一个，就会得到不同的行为。不存在中立的公共标准。
这是对模型与 harness 适配问题的正确且诚实的回答，而在开放或半开放集合中，Copilot CLI 是唯一真正做到了这一点的 harness。多模型是 2026 年任何严肃的智能体平台必须下的关键赌注。
无论供应商是否承认，大多数客户实际上都在运行多模型工作流，而让每个模型都发挥出最佳性能的唯一方法，就是在 harness 内部构建针对每个模型的路由界面。
## 对话中途切换模型：最干净的失败模式
关于模型与 harness 适配问题，最尖锐的具体例证来自用户在对话中途切换模型时发生的情况。Cursor 的研究团队在 4 月 30 日的帖子中对此做了细致描述，其失败面值得仔细梳理，因为在此处失效的每一个假设，都是单一模型-harness 配对所默默依赖的假设。
切换模型的那一刻，有三样东西会失效。
首先，对话历史本身已经超出了分布范围。前一个模型用它自己的原生词汇产生了工具调用：apply_patch 块、<oai-mem-citation> 标签、六到八个动词形式的子智能体调度。新模型是用不同的词汇训练的，现在必须面对一份充满它自己绝不会发出的工具调用的对话记录。Cursor 的处理方式是注入一条自定义指令，明确告诉模型"你正在从另一个模型手中接管对话中途"（我觉得这非常聪明！），同时引导它远离前一个模型的工具。
> 这只能缓解但不能消除代价。模型仍在阅读一份与它的直觉不匹配的对话记录。
其次，提示词缓存失效了（很痛苦！）。缓存是特定于供应商和模型的，这意味着切换必然导致缓存未命中。对于一个长会话来说，这会让切换后的第一轮对话以全价重新写入系统提示词和对话历史的每一个字节。Cursor 的缓解措施是在切换时对对话进行摘要，这会产生一份更短、更干净的对话记录，重新缓存成本更低，代价是丢失摘要未保留的细节。
第三，工具本身的形态发生了变化。新模型的 harness 会加载它原生的工具集。如果用户之前正在用一组动词深入进行子智能体调度流程，下一轮呈现的却是另一组。模型必须弄清楚先前的工具是否仍然有效（它们无效），以及自己的哪些工具对应用户的明显意图。
> Cursor 在构建了这些缓解措施后的建议很诚实："我们通常建议在一次对话中始终使用同一个模型，除非你有切换的理由。"
他们所描述的最干净的变通方案是派生一个使用不同模型的子智能体，而不是切换主对话。子智能体从一个全新的上下文窗口开始，没有对话记录偏差，没有缓存可失效，并且从第一轮起就使用新模型的原生工具界面。
这些失败模式中的每一个都直接印证了前文的论点。对话记录、缓存前缀和工具界面，都是模型训练时所对应的线上格式的组成部分。更换模型，就等于同时改变了这三方面的契约。切换模型不是简单的模型替换。它是 harness 替换、工具替换和缓存失效，三者同时发生。
## 实验室在说什么
Cursor 的 Stefan Heule 和 Jediah Katz（@jediahkatz）将他们的 harness 工作描述为"痴迷地堆叠小的优化"，正是因为阶跃式的改进很罕见，而收益只在匹配的配对内部才能复利累积。他们的团队针对每个供应商、每个模型版本进行定制化提示词设计，并援引 OpenAI 对字面精确性的坚持与 Claude 对不精确指令的容忍度作为反哺提示词设计的具体差异点。
他们报告称，在一个专注的冲刺周期内，将意外的工具调用错误降低了一个数量级。
> 工具调用可靠性不是模型的属性，而是 harness 的属性，而且是随着智能体运行时间增加而逐轮复利累积的属性。
Anthropic 的 @rgb_prithvi 在他 3 月 24 日关于长时间运行的应用开发的帖子中进行了一个相关实验。其架构是：一个规划器、一个生成器和一个评估器智能体，以生成对抗网络（GAN）为原型。
评估器使用 Playwright MCP 真实地像用户一样点击运行中的应用，然后根据评分标准进行打分。Rajasekaran 报告说，开箱即用时，"Claude 是一个糟糕的 QA 智能体"——它能识别出真实的问题，然后却说服自己批准了这项工作。通过多轮调整评估器提示词，才把它变成了一个可靠的裁判。
> 判断界面是 harness 创造的；仅靠模型本身做不到。
从 Rajasekaran 的工作中得出的更深层教训是关于 harness 应当如何随着模型的进步而演进。他针对 Claude Sonnet 4.5 构建了一个 harness，该模型表现出了强烈的"上下文焦虑"（值得一读，非常有趣！），以至于仅靠压缩并不足够。这个 harness 需要在会话之间进行完全的上下文重置，并用结构化的交接产物来跨越边界传递状态。
当 Opus 4.6 发布时，这种行为基本消失了。Rajasekaran 去掉了整套上下文重置机制，运行了一个持续超过两小时的连续会话。harness 中的每一个组件都编码着一个关于模型自身无法做到什么的假设。而这些假设会过时。
匹配的配对不是静态的。它随着模型的成熟而移动，harness 必须淘汰不再承重的脚手架——这正是我在《LLM 把脚手架当早餐吃掉》一文中所写的教训。
https://x.com/nicbstme/status/2015795605524901957?s=20
LangChain 的 Vivek Trivedy 给出了最清晰的表述：“Agent = 模型 + 框架（Harness）。如果你不是模型，那你就是框架。”在这个视角下，框架指的是权重本身之外的所有代码、配置和执行逻辑。系统提示词、工具描述、捆绑的基础设施、编排逻辑、钩子、中间件。从期望的 agent 行为倒推，每一个框架原语都是通过修补模型的某个具体缺口而获得其存在的理由。文件系统用于持久状态，bash 用于任意操作，沙箱用于安全执行，记忆用于持续学习，规划与自我验证用于长程任务。每个原语的起点都是对模型在训练时某个具体缺陷的变通方案。其中一些原语会随着时间被吸收回模型中。另一些则会持续复利式增长。
Trivedy 还指出了使模型-框架匹配如此持久的机制：共同进化反馈循环。“有用的原语被发现、加入框架，然后在训练下一代模型时被使用。随着这个循环的重复，模型在其训练所用的框架内变得愈发强大。”理解这一点非常重要。
这正是使匹配成对的组合在一代代中不断固化的流水线。一个新的框架原语在第一周上线。到第三个月，它出现在数百万条 agent 轨迹中。到第六个月，这些轨迹成为下一个模型的训练数据。到第十二个月，下一个模型已将该原语内化为本能，框架可以依赖它。这个循环正是让“换用外来框架”不仅笨拙、而且笨拙会不断累积的原因。模型的习惯是由其前一代框架塑造的，而那一代框架又是由再前一代塑造的。横向迁移会让你跳过这一复利累积的每一个循环。
Trivedy 对这个循环的代价也很坦诚，我想清晰地呈现这个反方论点。引用他的话：“一个真正智能的模型应该能轻松在补丁方法之间切换，但在训练中将框架纳入循环会造成这种过拟合。”如果模型的工具格式偏好过拟合于其训练框架，你可以主张正确的长期策略是针对更多样化的框架集合进行训练，使模型具备泛化能力。这个论点有其道理。那些以“一个模型+一个框架”成对交付的实验室，是用模型的可移植性换取近期性能。这种取舍是否正确，取决于可移植性是否是客户重视的东西——而目前客户更看重排行榜。
三篇相互独立的文章在几周内相继发布，全都汇聚到同一个论点：模型只是系统的一半，框架是另一半，匹配成对的组合才是正确的分析单元，而以单一产品形式交付这种成对组合的厂商，正是目前占据排行榜顶端的那几家。
## 身份文件约定
契约中的框架一侧已经收敛为“每个关注点一个 markdown 文件”，而这些文件名如今在整个生态中具有承重作用。在某个框架上训练的模型会识别这些文件名，并知道哪个文件承载哪种权威。
```
THE MARKDOWN-FILE-PER-CONCERN CONVENTION (April 2026)
=====================================================
CLAUDE.md Static project instructions for the Claude Code session.
Anthropic's proprietary format. Loaded at session start.
The de facto standard since 2024. Closely paired with
Anthropic post training.
AGENTS.md Cross tool standard for procedural rules. "What do you
do and how." Adopted by Codex CLI, OpenClaw, Cursor.
Claude Code does not natively read AGENTS.md as of
April 2026 (community feature request open). The
GitHub Copilot CLI changelog explicitly added support
for AGENTS.md as a context source.
SOUL.md Personality, voice, worldview. From github.com/
aaronjmars/soul.md. Used by OpenClaw and the Aeon
framework. "An AI that thinks and speaks as you" vs
"a chatbot that talks about you." Opt in convention,
not yet baked into the major harnesses.
USER.md Who the user is. Hermes splits its memory into
MEMORY.md (what the agent learned) and USER.md (who
the user is). The split lets the model treat user
facts as authoritative and project facts as evolving.
MEMORY.md Auto memory index. Claude Code loads this on every
turn under # auto memory and treats it as overriding
default behavior. Codex calls its index memory_summary.md
with a 5K token cap and lazy loads the bodies.
STYLE.md Voice and writing patterns (SOUL.md ecosystem).
SKILL.md Per skill folder, with YAML frontmatter. Cross harness
adopted: Claude Code, Codex CLI, GitHub Copilot CLI all
read this format. Body conventions still differ.
IDENTITY.md Lightweight public facing card with name, role, metadata.
OpenClaw convention.
```
关键观察是：这些文件名如今已是线上传输格式的一部分。一个被训练去寻找 MEMORY.md 标题下 # auto memory 块的模型，会在每轮对话中搜寻那个确切的标题。一个基于 AGENTS.md 训练的模型会寻找 AGENTS.md 而错过 CLAUDE.md。一个基于 SOUL.md 训练的模型会从 SOUL.md 加载个性设定，而如果你把同样的内容放进 STYLE.md，它就会忽略。
这就是为什么针对 Anthropic 仓库的 AGENTS.md 功能请求很重要。这不是一次文档迁移。这是要求模型的训练分布扩展其文件识别词汇表。在 Anthropic 对 Claude 做后训练使其能读取 AGENTS.md 之前，即使该文件就放在仓库中 CLAUDE.md 的旁边，Claude Code 也对它视而不见。
SOUL.md 生态正是对这一论点的压力测试。SOUL.md 尚未被任何主流 harness 的默认加载器识别。因此 SOUL.md 仓库的安装说明很能说明问题：把你的 soul/ 目录复制到项目中，然后在 CLAUDE.md 里加几行，把模型指向它。这是一个从不被认可的约定到被认可的约定之间的手动桥接。SOUL.md 的作者明白，除非模型知道去哪里看，否则这些字节不会起作用，而“去哪里看”是后训练中固化下来的习惯。
同样的路由问题也公开地显现出来。GitHub Copilot CLI v1.0.4 添加了："Read .claude/settings.json and .claude/settings.local.json as additional repo config sources."（将 .claude/settings.json 和 .claude/settings.local.json 作为额外的仓库配置来源读取。）v1.0.36 又部分收回了："Custom agents, skills, and commands from ~/.claude/ are no longer loaded by the Copilot CLI."（~/.claude/ 中的自定义智能体、技能和命令不再由 Copilot CLI 加载。）这是一个试图在文件名上保持宽容的路由器，随后在用户界面上变得混乱时又收紧了。教训藏在变更日志之下：即使是运行 Claude 模型的 harness，也无法在不与用户协商哪些约定有效的情况下，把 .claude/ 文件当作权威。
选一个约定。配套发布相应的后训练。或者发布一个显式地将每个文件映射到识别它的模型的路由器。“保持宽容、加载任何看起来合理的文件”这种中间路线每次都会输。
## 这意味着什么
harness 不再是模型外围的包装。harness 是模型有效参数的一部分。后训练过程将 harness 的工具表面、schema 形态、记忆仪式、引用契约和系统提示词结构植入了模型的直觉集合。你可以把权重带到另一个 harness，但带不走那些直觉。只有当 harness 以后训练时呈现世界的方式呈现世界时，直觉才会被触发。
这有三个值得点名的后果。
对于智能体平台构建者：选定一个 harness，选定一个模型，把它们作为一个整体发布。不要假装模型是可移植的。不要假装 harness 是中立的。前沿实验室正在发布模型-harness 组合，不管它们自己这么说与否，而每个组合的性能才是唯一重要的数字。Copilot CLI 的“不同模型用不同工具”的方法是这一点的诚实版本。不诚实的版本发布一个最大公约数，然后在它服务的每个模型上都表现不佳。
对于模型实验室：harness 是产品战略，而不是基础设施。harness 是实验室后训练投资复利累积的地方。Anthropic 的 <system-reminder> 注入模型、类型化的记忆分类法、每次正文读取时的验证，这些都不是基础设施选择。它们是模型在其上被雕琢的表面，也是让模型不那么容易被替代的护城河。Codex 的两阶段记忆管道、引用标签、严格的 JSON schema 也是如此。Copilot CLI 的十节系统提示词骨架也是如此。harness 正是模型变得不可替代的地方。
对于用户：切换的成本比看上去更高，也比供应商想让你以为的更低。更高，是因为模型和 harness 在数月的训练中融合在一起，你无法干净地把它们拆开。更低，是因为底层的简单技术栈是共享的，顶层的约定是可文档化的。一次诚实的移植——复刻工具表面、复刻引用契约、复刻系统提示词结构、复刻记忆仪式——将弥合大部分差距。只是其设置成本和最初的后训练一样高。
匹配的组合不是静态的。它随模型的成熟而变化。这是 Rajasekaran 那篇 Anthropic 文章中最有用的细微之处。一个对 Sonnet 4.5 起支撑作用的 harness 组件（上下文重置、冲刺分解、激进的压缩），在 Opus 4.6 上变成了死重，因为模型开始原生地做这项工作。三月时适合某模型的 harness，未必适合十月该模型的继任者。纪律在于阅读轨迹，识别哪些组件仍在发挥价值，并淘汰那些如今只是对已解决问题打补丁的组件。Cursor 的博客用不同的话说了同一件事："Every component in a harness encodes an assumption about what the model cannot do on its own, and those assumptions go stale."（harness 中的每个组件都编码了一个关于模型自身无法完成什么的假设，而这些假设会过时。）
所以回到我开始的问题。为什么同一个提示词在运行同一模型的三个 harness 上产生明显不同的输出？
因为在三个 harness 上运行的模型实际上是三个不同的模型，即使磁盘上的权重逐字节完全相同。运行时被触发的直觉并不只存储在权重里，它们受权重训练时所在的 harness 的条件制约，而事实证明，直觉占了助手在任何一轮输出中的大部分。
> 如今有趣的设计举措不是更好的模型。也不是更好的 harness。而是匹配的组合，端到端地设计，让后训练和运行时一轮又一轮地相互强化，直到模型在这个特定 harness 所奖励的事情上变得明显更好。
你可以看到主要的构建者正从三个不同的起点向这个理念汇聚。Anthropic 将 Claude Code 作为 Claude 的规范 harness 发布，后训练和运行时作为一个产品共同设计。OpenAI 将 Codex CLI 作为 Codex 的规范 harness 发布，在 OpenAI 这边进行了同样的垂直整合。
2026年的前沿工作不在于新的模型架构，而在于新的执行框架原语。Ralph 循环：用一个钩子拦截模型的退出尝试，并在干净的上下文窗口中重新注入原始提示词，迫使智能体不断朝目标持续磨砺。即时框架组装：针对每个任务动态组合工具表面和系统提示词，而不是按会话预先配置。自我追踪智能体：读取自己的日志来发现框架层面的失败模式，并在没有人工干预的情况下修补它们。这些原语中的每一个，最终都会有某个模型针对其进行后训练，而这种组合将出现在下一个排行榜的榜首。
再次强调，如果你想保持领先，就必须在新模型发布时删掉你的大部分代码。LLM 们把脚手架当早餐吃。
## 相关链接
- [Nicolas Bustamante](https://x.com/nicbstme)
- [@nicbstme](https://x.com/nicbstme)
- [@_lopopolo](https://x.com/@_lopopolo)
- [github.com/openai/codex](https://github.com/openai/codex)
- [github.com/ultraworkers/claw-code](https://github.com/ultraworkers/claw-code)
- [github.com/github/copilot-sdk](https://github.com/github/copilot-sdk)
- [Agent Memory Engineering](https://nicolasbustamante.com/blog/agent-memory-engineering)
- [@github/copilot](https://x.com/@github/copilot)
- [$CODEX](https://x.com/search?q=%24CODEX&src=cashtag_click)
- [list-skills.py](https://list-skills.py/)
- [install-skill-from-github.py](https://install-skill-from-github.py/)
- [Agent Memory Engineering](https://x.com/nicbstme/status/2050301124314563025?s=20)
- [citations.rs](https://citations.rs/)
- [citations.rs](https://citations.rs/)
- [github.com/github/copilot-sdk/nodejs/src/types.ts:636](https://github.com/github/copilot-sdk/nodejs/src/types.ts:636)
- [@jediahkatz](https://x.com/@jediahkatz)
- [@rgb_prithvi](https://x.com/@rgb_prithvi)
- [https://x.com/nicbstme/status/2015795605524901957?s=20](https://x.com/nicbstme/status/2015795605524901957?s=20)
- [Upgrade to Premium](https://x.com/i/premium_sign_up)
- [10:48 AM · May 4, 2026](https://x.com/nicbstme/status/2051131906327212298)
- [5,082 Views](https://x.com/nicbstme/status/2051131906327212298/analytics)
---
*导出时间: 2026/5/4 13:08:30*
