ChatGPT Agent Loop 优化技术解析 ✍ Bytebytego🕐 2026-07-30📦 28.6 KB 🟢 已读 𝕏 文章列表 本文深入解析了 ChatGPT 如何通过 Harness、API 和 Inference 三层架构优化 Agent 循环，重点介绍了持久化 WebSocket、增量 Token 化、KV 缓存管理和推测解码等技术，以降低成本并提升效率。 Agent优化LLM架构ChatGPTOpenAI性能成本控制WebSocketTokenization # How ChatGPT Optimizes its Agent Loop: Harness, API, and Inference **作者**: Bytebytego **日期**: 2026-07-29T15:19:09.000Z **来源**: [https://x.com/bytebytego/status/2082486089642885248](https://x.com/bytebytego/status/2082486089642885248) ---  AI labs are moving faster than ever and releasing the most capable models we have ever seen. Just recently, Anthropic released Fable 5, then OpenAI released the GPT-5.6 family, including GPT-5.6 Sol, their most capable model yet, Kimi released Kimi K3, and Opus 5 came out just a few days ago. But capability is only half of the picture. The other half is how much it costs these models to complete tasks, that is, the cost per successful task. Lower cost makes the model more affordable for users and less costly for the provider. A huge amount of effort inside the labs goes into making every component and layer of their AI applications optimized and more efficient, to reduce the overall cost. For example, GPT 5.6 Sol with max reasoning scores higher than Fable 5 on the Artificial Analysis Coding Agent Index while costing less than half as much.. To understand what techniques are adopted in frontier labs to make AI applications more efficient, we met with the OpenAI engineers who developed and shipped various efficiency techniques into the systems behind Codex and ChatGPT Work. Thanks to Joe, Ahmed, Steve, Matthew, and Philippe for sharing valuable details with u In this article, you’ll learn: - What actually happens when you send a request to an AI agent - How the harness layer cuts repeated work with persistent WebSockets, stable prompt prefixes, deferred tool discovery, and Code Mode - How the API layer tokenizes only the delta and runs safety checks in parallel with inference - How the inference layer gets more out of GPUs with cache-aware routing, KV cache management, speculative decoding, and separating prefill from decode - What OpenAI learned from all of this, and which lessons you can apply to your own systems  Overview of efficiency techniques ## Anatomy of an Agentic AI Application When you give Codex or ChatGPT Work a task, like fixing a bug, the query does not go directly to the LLM. It is more complex than that. An AI application like Codex is not just an LLM. It is a system, with many components. User queries pass through several layers before the LLM ever sees a single token.  An AI application like Codex is made of many components. The LLM is just one of them. But why can’t we send the user query directly to the LLM? To understand why, let’s see what an LLM actually is. An LLM is a neural network trained to predict the next token. It takes a sequence of tokens as input and produces a sequence of tokens as output. It cannot run a shell command, edit a file, or remember anything between calls. But an agent task like “fix this bug and run the tests” is mostly actions. Something has to turn the model’s predicted tokens into real commands, feed the results back, and continue until the task is done. That is the job of the harness layer, a system built on top of the LLM to handle those responsibilities. It takes the user’s task as input, decides which instructions, which tool definitions, and how much history to include in the context, and maintains the conversation history. When the model responds with a tool call, the harness executes it under approval policies in a sandbox environment, appends the result, and sends the conversation back to the LLM.  The harness builds the context and executes tools But even the harness’s request does not go directly to the LLM. In production-ready applications, there is an API layer (an application layer) sitting between the harness and the inference endpoint. This layer exists because a real product needs to handle things that are not covered by the harness layer or the LLM. For example, authenticating the caller, enforcing rate limits, and, importantly, converting text into the token IDs the LLM expects, and converting generated tokens back into text. Once the API layer prepares the context and tokenizes it into a sequence of token IDs, it is time for the LLM to process the input. That is the inference layer. It is a remote endpoint backed by fleets of GPUs, hosting the model. Its job is to run the model’s computation over the prepared tokens and produce the response, whether that is a new tool call or the final answer, as efficiently as possible. Then hand the generated tokens back to the API layer.  The three layers of an agentic AI application that a request travels through. What happens when you send a request to an AI agent? Now that we understand the three main layers of agentic AI applications, let’s walk through a concrete example to see how a request travels between these layers. Suppose you ask Codex to “trace the checkout regression, patch it, and run the tests.” Here is what happens: 1. The harness assembles instructions, tool definitions, and your task into a request and sends it to the API. 2. The API buffers the request into memory, parses the JSON, and validates it: the request is well-formed, and the chosen model supports every feature it asks for. It checks who is calling, applies rate limits, and runs preflight checks. 3. The API renders the conversation into the model’s input format and tokenizes it. 4. The API dispatches the tokens to the inference layer, and starts its safety checks at the same time: classifiers that look for things like cyberattacks and bioweapon content. The safety checks race to finish before the first generated token comes back. 5. The inference layer processes the prompt and begins generating. Its output here is not the final answer; it is a tool call: search the code for “checkout timeout.” 6. The generated tokens flow back to the API 7. The API converts tokens into text, wraps them in API events 8. The API streams them to the harness. 9. The harness recognizes the tool call, runs the search in its sandbox, appends the output to the conversation, and sends the updated conversation back to the API.  From the user’s request to the final summary coming back This is just one iteration. In practice, a task repeats this loop many times until the model produces its final summary and the harness hands control back to the user. Each iteration redoes a lot of the same work described above. For example, the history gets resent, the text gets retokenized, or the prompt gets reprocessed. Removing this repeated work is the main opportunity for optimization. The next three sections cover the optimization techniques OpenAI adopted to make each layer more efficient. ## Harness Optimization The harness is the orchestration layer closest to the user that turns the raw user request into a context and interacts with the LLM in a loop until the task is complete. It has two main responsibilities. First, it is the source of truth for the conversation. It holds the authoritative record of every instruction, message, tool call, and tool result. Second, it runs the agentic loop. It decides what goes into the context sent to the LLM (which instructions, which tool definitions, and how much history). It sends the request, receives the response as a stream, parses it, and watches for tool calls. When a tool call appears, it executes it under approval policies in a sandbox environment, appends the result to the conversation, and sends it back to the LLM. It repeats this loop until the task is done. Heavy tasks could in theory repeat more than 100 times. Each iteration carries overhead. For example, one extra second per model call adds roughly half a minute to a long task.  Agentic loop cycle How to make the harness layer efficient? From the harness layer’s perspective, the latency comes from four sources: the network, prompt processing, context contents, and the loop’s round trips. OpenAI adopted the following four techniques at the harness layer to make it more efficient. 1. Persistent WebSockets and incremental requests The harness runs on the user’s machine, but the model runs in OpenAI’s data centers, so every model call is a network exchange. The standard way to make that exchange is HTTPS. The harness opens a connection, sends a request containing everything the server (the API layer) needs, and receives a response. Because responses from a language model arrive gradually, one token at a time, chat applications typically use Server-Sent Events (SSE) on top of HTTPS, so the client sends one request, and the server streams the answer back in small chunks over the open response. SSE is a one-way street. It is a great fit for the chat era, where one user message produces one model call and one streamed answer. But agents are not one-pass. A single Codex turn can contain many model calls. With HTTPS, each of those calls is a new request with two separate costs. The first cost is connection setup. Opening a fresh HTTPS connection means a TCP handshake followed by a TLS handshake, several network round trips before a byte of useful payload is transmitted. Paying that once per user message is manageable, but paying it many times inside one conversation turn is expensive. The second cost is repetition of the payload itself. HTTP is stateless, so each request must carry everything the server needs. For example, the instructions, tool definitions, and the entire conversation so far, all need to be included in the request. By the twentieth tool call, the harness is resending the original prompt, nineteen tool calls, and nineteen tool results, just to add one new result at the end. This causes the payload to grow with every iteration, so the harness spends more and more time uploading data the server has already seen.  HTTPS vs WebSocket But how do we fix these two sources of cost? The fix for the connection cost is to open one connection and keep it alive, instead of creating a new one for every call. This is what WebSockets are designed for. A WebSocket needs just one initial handshake, and after that, both sides can send messages whenever they want with no per-message setup. Codex opens a single WebSocket to the API and keeps it open across all the model calls in a turn. This eliminates the repeated TCP and TLS setup. The fix for the payload repetition is to stop resending what the server already knows. The harness keeps the previous request and the completed response. If nothing but the conversation input has changed, it sends only the new items along with a reference to the previous response. After a tool call, the next message on the socket can be as small as:  Notice that this message has no instructions, tool schemas, or history. The server will rebuild the full context from the state it kept, so the model still sees everything. The only thing that changed is how much data crossed the network.  The harness sends only the new tool result 2. Stable prompt prefixes LLM providers avoid repeated calculations with a technique called prompt caching. When a prompt arrives, the model reuses the cached internal state for the beginning of the prompt that matches a previous request (the prefix), and only computes the rest. The match is exact, token by token. If you change one token near the front of the prompt, everything after it must be recomputed.  Appending to the end of the prompt keeps the cache valid. For the harness, this means the prompt it builds on every call must start with exactly the same bytes as the one before. That sounds trivial, but the harness rebuilds the request each time from its live in-memory state, so any small difference in how it assembles the prompt may silently break the match. OpenAI shared an example of this. Codex kept MCP tool definitions in a hash map, which does not guarantee ordering, so the same tools could serialize in a different order on each request. It was the same tools in the context, just in a different order. Codex was still completing tasks, just more expensively. The efficiency technique here is to treat the history as append-only, and keep volatile runtime state like approval settings out of the prompt. For example, when a user changes approval settings mid-session, the harness does not edit the tool definitions in the prompt. It applies the new policy itself the next time a tool runs. This way, the prompt stays unchanged so the cache stays valid. 3. Deferred tool discovery Stable prompt prefixes make repeated context cheaper, but they do not keep the prompt small and concise. In agents with access to lots of tools, this can be an issue. For example, the tool schemas alone can take up a huge amount of space, because a Codex session can expose hundreds of tools once you count connected integrations and MCP servers. Every tool schema is a JSON object with names, descriptions, and parameter definitions. Once all of these are in the prompt, the LLM has to process all of them in its calculations, even though many may go unused for the current task. To keep unused tools out of the prompt, OpenAI uses a technique called deferred discovery. The prompt carries only the core tools, say the shell and file editing, plus one tool_search tool. The other hundreds of integration definitions stay out of the prompt entirely. When the model needs a capability, it calls the search tool with keywords like “list deployments”. The harness searches the catalog, and the matching definition gets loaded into the context. The search itself is BM25, a lexical ranking algorithm that finds tools whose descriptions overlap with the generated keywords.  The model searches for a tool when it needs one In addition to deferred discovery, two more techniques can keep the context small. Schema compaction trims oversized tool schemas down to a token budget by stripping descriptions and collapsing nested structures while preserving argument names. Conversation compaction condenses a long history into a short summary. 4. Code Mode In ordinary setups, the model emits one tool call, waits for the result, reasons, and emits the next. In many scenarios, there is no reasoning needed between these tool calls. The model just emits them one by one to collect the results it needs. This is costly because each tool call requires a full model round trip. Also, after each tool call, its result keeps getting added to the context, which wastes context space. With Code Mode, instead of emitting tool calls one by one, the model writes a small program that makes those calls. The harness runs this program in an embedded JavaScript runtime, where every tool is available as a function. The script can fan out independent calls in parallel, filter and join the results in plain code, and return only the compact answer. This way, the intermediate data stays in the runtime, and only the final result enters the context.  Direct tool calls vs Code Mode ## API Optimization The API layer is the service the harness is talking to. It sits between the harness layer and the inference layer, running on ordinary CPUs. The harness sends JSON describing structured conversation items, while the model consumes and produces token IDs. When a request arrives, it goes through the following steps: 1. The request body is buffered into memory. 2. The JSON is parsed and validated against a schema (well-formed fields, arrays within expected bounds, etc.) 3. A second validation round checks semantics. For example, does the chosen model support the requested features? 4. Preflight checks run: authorization and entitlements, rate limits, and checks on any images in the request. 5. The API renders and tokenizes the conversation, then kicks off two jobs at the same time: the inference request to the GPUs and the safety checks on the prompt. 6. As generated tokens flow back from the inference layer, the API converts each one into text, wraps it in an API event, and streams it to the client.  The steps a request goes through inside the API layer These steps are repeated on each iteration of the loop. The inference is out of the API layer’s control. The API cannot make the GPU any faster. All it can do is add as little delay as possible around it. The time the API layer spends parsing, validating, or tokenizing is extra latency the user feels. ## How to make the API layer efficient? OpenAI adopted the following techniques to reduce the overhead coming from different places in the API layer. 1. Tokenizing only the delta Models do not read text. Before inference starts, the prompt must be converted into token IDs. Tokenizers convert the text into token IDs in O(n), linearly going over each token in the prompt and replacing it with the token ID. Under stateless HTTP, the whole conversation gets retokenized on every call. By the twentieth tool call, the API is rereading many thousands of tokens to extract one new result. The GPU only needs the new tool result, but the CPU is rereading the book from page one, and the cost keeps growing with input length. With the WebSocket, the API keeps the tokenized conversation in server memory. The first request tokenizes the full prompt, but every later request sends only the new items. The API tokenizes just that piece and appends it to the stored sequence. This way, per-call tokenization stops depending on conversation length and becomes closer to O(1), depending on the length of the marginal input.  Tokenization with stateless HTTP and WebSocket 2. Running safety checks in parallel with inference Agents have to check each request for safety before showing the output to a user. OpenAI implemented safety checks in its API layer. Images go through their own classifiers, while the prompt text goes to classifier models that look for dangerous content, like cyberattacks or bioweapon instructions. These classifiers take time to run. The simple design is to run them first and start inference only after they pass. The problem is that this delay is added to the time-to-first-token (TTFT) of all requests, while the vast majority of them are completely harmless. The fix is to run the safety checks and inference at the same time. The model takes some time to process the prompt before its first token comes out, so the checks use that window to finish. If a check fails, the API reacts depending on the model. For some models, it streams tokens to the user right away and cuts the stream when a check fails. For more sensitive models, it holds the output until the checks pass and only then releases it. In both cases, the time spent on safety hides inside a wait that was going to happen anyway.  Running safety checks in parallel 3. Routing traffic to newer CPUs When a company runs servers at scale, its fleet ends up with machines bought in different years, and therefore with different generations of CPUs. The scheduling software usually treats machines of the same type as identical. In reality, they are not. OpenAI queried Kubernetes for the actual processor model behind each of its deployments and found a mix of older Broadwell chips and newer Ice Lake chips behind the same machine labels. The older ones were serving the same traffic with roughly 20% worse time to first token, while using about twice as much CPU. To fix this, OpenAI shifted more traffic toward the machines with newer processors and started treating CPU generation as an explicit factor in capacity planning. Sometimes the best software optimization is better hardware. ## Inference Optimization The inference layer is where the model actually runs. There are fleets of GPUs and other accelerators, and the serving software (the engine) that operates them. The model’s core computation is enormous matrix arithmetic, which parallelizes across the thousands of cores an accelerator provides. The layer schedules and batches incoming requests across the fleet, holds the per-conversation state that generation depends on, executes the model’s forward passes, and hands each generated token back to the API layer.  Each machine holds the model weights and the KV state of the conversations it serves. How to make the inference layer efficient? Demand for tokens grows faster than hardware can be added, so at this layer slow and wasteful are the same word. The waste hides in four places: work routed to the wrong machine, memory holding the wrong state, parallel hardware idling through sequential generation, and two mismatched phases sharing the same machines. OpenAI adopted the following techniques to make the inference layer more efficient. 1. Balancing load with cache-aware routing OpenAI serves its models from many GPU machines. Every request has to be sent to one of them. If that routing is uneven, some machines sit idle while others queue up work. Idle GPUs are the most expensive kind of waste in the stack. There is also a second, less obvious problem. Each machine keeps the computed state (the cache) for the conversations it recently served. If the next request of a conversation lands on a different machine, that cache is useless. The new machine must recompute everything from scratch, even though the work already exists somewhere else. So the router has two goals: 1. Spreading load evenly: send the request to the least busy machine. 2. Using the cache: send it back to the machine that already knows this conversation.  Cache-aware routing OpenAI’s routing weighs both for every request, along with basics like geography and available capacity. Improvements in load balancing alone have dramatically reduced the cost of serving its models. 2. Managing the KV cache When a transformer generates a token, it attends over all previous tokens. To avoid recomputing that attention state for every new token, the serving system keeps it in accelerator memory, in what is called the KV cache. The cache grows linearly with conversation length, multiplied by every concurrent conversation the fleet is serving. For long contexts it can rival the size of the weights themselves. If the software evicts the wrong state, the inference layer pays the full processing cost again when that conversation returns. The efficiency technique here is to manage the cache based on real usage. OpenAI analyzes production traces to learn which cached state is likely to be needed again, tests eviction policies against that data instead of intuition, and optimizes how cached state is stored and moved between memory tiers. This ensures the most valuable work stays around longer without running out of memory or wasting time moving data.  The hot conversation state stays on the GPU. The colder state is moved or evicted. 3. Speculative decoding Generation is sequential. Each token depends on the previous one, so a model that processes a huge prompt in one parallel pass must produce its answer one token at a time. In large models, producing these tokens is expensive and slow, even when the next token is easy to guess. For an input like “the capital of France is”, the next token is almost certainly “Paris”, but the model still has to process the entire conversation and perform large matrix multiplications to predict it. Speculative decoding uses a small draft model to propose the next several tokens, and the large model verifies all of them in a single parallel pass. In many cases, the draft proposals are correct and will be accepted by the large model, which saves the time the large model would have taken to produce them one at a time. When a proposal is wrong, the large model discards it and produces its own token, so the output quality does not change. The main metric to evaluate the draft model is the average acceptance length, which is how many proposed tokens the main model accepts in each pass.  Speculative decoding 4. Separating prefill and decode Inference has two phases, prefill and decode. Prefill processes the entire incoming prompt in one massively parallel pass, and it builds the internal state (KV cache) the model needs. Decode then generates the response one token at a time. These two are very different workloads. Prefill is compute-heavy, while decode is memory-heavy, spending most of its time streaming weights and cached state through memory. Running both on the same machines means neither runs on hardware set up for it. The efficiency technique is to separate them. One part of the fleet handles prefill and another handles decode, each configured for its own bottleneck. When a request finishes prefill, its computed state (the KV cache) is shipped to a decode machine, which generates the response token by token.  Summary of efficiency techniques across three layers That covers the efficiency techniques at each layer of an agentic AI application. The harness avoids resending what the server already has, the API avoids reprocessing what it has already processed, and inference avoids recomputing what it has already computed. To close, let’s look at the lessons from this work that apply beyond OpenAI. ## What to Learn from OpenAI’s Push for Efficiency and Lower Cost per Successful Task Zooming out, all these optimization techniques at different layers try to avoid paying for the same work twice. The harness sends only what is new and keeps prefixes stable so caches survive. The API tokenizes only the delta and hides unavoidable work inside windows that had to pass anyway. Inference routes conversations back to their cached state instead of recomputing it. These layers also feed each other. When the harness keeps the cache intact and the API improves the cache hit rate, the savings appear on the GPUs, which means lower costs for users. Beyond the techniques, here are a few lessons that the OpenAI engineers shared with us. Keep the design simple. Prioritize simplicity over complexity. For example, use lexical search instead of embeddings for tool discovery, and one compaction path instead of a menu. An LLM is already quite intelligent, so adding more machinery in front of it often adds complexity without adding value. Build the simplest thing that could work, then scale it. Use the agent to optimize its own stack. Codex wrote much of the migration of the very API that serves Codex, turning years of rewrite work into months for a couple of engineers. The inference team uses it to analyze traces and prototype kernels. When agents write the code, you no longer need a language that is easy for humans to write, so you simply pick the most efficient one to run. Efficiency work is becoming a loop that accelerates itself. Optimize end to end. The inference team told us that every time they picked one favorite technique to focus on, they regretted it. Focusing on one part of the stack made them underinvest in the others. No single optimization is a game changer on its own. The big wins come from chaining many small ones together. They also learned to test with the same traffic shapes that production actually serves, because a change that looks like a win offline can hurt in real traffic. ## 相关链接 - [Alex Xu reposted](https://x.com/alexxubyte) - [Bytebytego](https://x.com/bytebytego) - [@bytebytego](https://x.com/bytebytego) - [25K](https://x.com/bytebytego/status/2082486089642885248/analytics) - [Artificial Analysis Coding Agent Index](https://openai.com/index/gpt-5-6/) - [Upgrade to Premium](https://x.com/i/premium_sign_up) - [11:19 PM · Jul 29, 2026](https://x.com/bytebytego/status/2082486089642885248) - [25.4K Views](https://x.com/bytebytego/status/2082486089642885248/analytics) - [View quotes](https://x.com/bytebytego/status/2082486089642885248/quotes) --- *导出时间: 2026/7/30 10:52:45* --- ## 中文翻译 # ChatGPT 如何优化其 Agent 循环：Harness、API 和推理 **作者**: Bytebytego **日期**: 2026-07-29T15:19:09.000Z **来源**: [https://x.com/bytebytego/status/2082486089642885248](https://x.com/bytebytego/status/2082486089642885248) ---  AI 实验室的节奏比以往任何时候都要快，并发布了我们见过的最强模型。就在最近，Anthropic 发布了 Fable 5，OpenAI 发布了 GPT-5.6 系列，包括其迄今为止最强的模型 GPT-5.6 Sol，Kimi 发布了 Kimi K3，Opus 5 也在几天前刚刚面世。 但能力只是硬币的一面。另一面是这些模型完成任务的成本，即单次成功任务的成本。更低的成本让用户更能负担得起，也能降低提供商的成本。实验室投入了巨大精力来优化 AI 应用程序中的每个组件和层级，以提高整体效率。例如，在 Artificial Analysis Coding Agent Index 上，GPT 5.6 Sol 的最大推理得分高于 Fable 5，但成本却不到后者的一半。 为了了解前沿实验室采用了哪些技术来提高 AI 应用程序的效率，我们采访了开发并部署了各种效率技术到 Codex 和 ChatGPT Work 背后系统的 OpenAI 工程师。感谢 Joe、Ahmed、Steve、Matthew 和 Philippe 与我们分享宝贵的细节。 在本文中，您将了解到： - 当您向 AI agent 发送请求时实际发生了什么 - Harness 层如何通过持久化 WebSocket、稳定的提示前缀、延迟工具发现和代码模式来减少重复工作 - API 层如何仅对增量内容进行分词，并与推理并行运行安全检查 - 推理层如何通过缓存感知路由、KV 缓存管理、推测解码以及分离预填充和解码来充分利用 GPU - OpenAI 从中学到了什么，以及您可以将其中的哪些经验应用到自己的系统中  效率技术概览 ## Agent AI 应用程序剖析 当您给 Codex 或 ChatGPT Work 下达一个任务，比如修复一个 bug 时，查询不会直接发送给 LLM。它比这要复杂得多。像 Codex 这样的 AI 应用程序不仅仅是一个 LLM。它是一个系统，由许多组件组成。用户查询在 LLM 看到一个 token 之前要经过好几层。  像 Codex 这样的 AI 应用程序由许多组件组成。LLM 只是其中之一。 但是，为什么我们不能直接将用户查询发送给 LLM 呢？为了理解这一点，让我们看看 LLM 实际上是什么。 LLM 是一个经过训练以预测下一个 token 的神经网络。它接收一个 token 序列作为输入，并产生一个 token 序列作为输出。它无法运行 shell 命令、编辑文件，或者在调用之间记住任何东西。但是，像“修复这个 bug 并运行测试”这样的 agent 任务主要都是动作。必须有东西将模型预测的 token 转换为真实的命令，将结果反馈回去，并持续循环直到任务完成。 这就是 Harness 层的工作，它是构建在 LLM 之上的系统，用于处理这些职责。它接收用户的任务作为输入，决定将哪些指令、哪些工具定义以及多少历史记录包含在上下文中，并维护对话历史。当模型响应工具调用时，Harness 会根据批准策略在沙箱环境中执行它，追加结果，并将对话发送回 LLM。  Harness 构建上下文并执行工具 但即使是 Harness 的请求也不会直接发送给 LLM。在生产就绪的应用程序中，在 Harness 和推理端点之间有一个 API 层（应用层）。这一层的存在是因为真实产品需要处理 Harness 层或 LLM 未涵盖的事情。例如，验证调用者身份、强制执行速率限制，以及至关重要的是，将文本转换为 LLM 期望的 token ID，并将生成的 token 转换回文本。 一旦 API 层准备好上下文并将其分词为 token ID 序列，就轮到 LLM 处理输入了。这就是推理层。它是一个由成千上万个 GPU 支持的远程端点，用于托管模型。它的工作是尽可能高效地对准备好的 token 运行模型计算并生成响应，无论是新的工具调用还是最终答案，然后将生成的 token 交还给 API 层。  请求通过的 Agent AI 应用程序的三个层级。 当您向 AI agent 发送请求时会发生什么？ 既然我们了解了 Agent AI 应用程序的三个主要层，让我们通过一个具体的例子来看看请求是如何在这些层之间传递的。假设您要求 Codex “追踪结账回归问题，修补它，并运行测试”。过程如下： 1. Harness 将指令、工具定义和您的任务组装成一个请求，并将其发送给 API。 2. API 将请求缓冲到内存中，解析 JSON 并进行验证：请求格式正确，所选模型支持其请求的每个功能。它检查调用者是谁，应用速率限制，并运行飞行前检查。 3. API 将对话渲染为模型的输入格式并对其进行分词。 4. API 将 token 分发给推理层，并同时开始其安全检查：寻找网络攻击和生物武器内容等内容的分类器。安全检查竞相在第一个生成的 token 返回之前完成。 5. 推理层处理提示词并开始生成。这里的输出不是最终答案；它是一个工具调用：在代码中搜索“checkout timeout”。 6. 生成的 token 流回 API。 7. API 将 token 转换为文本，并将它们包装在 API 事件中。 8. API 将它们流式传输给 Harness。 9. Harness 识别出工具调用，在其沙箱中运行搜索，将输出追加到对话中，并将更新后的对话发送回 API。  从用户请求到返回最终摘要 这只是一次迭代。在实践中，一个任务会多次重复这个循环，直到模型生成最终摘要并且 Harness 将控制权交还给用户。每次迭代都会重复上述大量相同的工作。例如，历史记录被重新发送，文本被重新分词，或者提示词被重新处理。消除这种重复工作是优化的主要机会。接下来的三个部分将介绍 OpenAI 采用的使每一层更高效的优化技术。 ## Harness 优化 Harness 是最接近用户的编排层，它将原始用户请求转换为上下文，并与 LLM 循环交互直到任务完成。它有两个主要职责。 首先，它是对话的真实来源。它保存着每条指令、消息、工具调用和工具结果的权威记录。其次，它运行 agent 循环。它决定发送给 LLM 的上下文中包含什么（哪些指令、哪些工具定义以及多少历史记录）。它发送请求，接收流式响应，解析它，并监视工具调用。当出现工具调用时，它会根据批准策略在沙箱环境中执行它，将结果追加到对话中，并将其发送回 LLM。它重复这个循环直到任务完成。繁重的任务理论上可能重复超过 100 次。每次迭代都会带来开销。例如，每次模型调用多出一秒钟，大约会给长任务增加半分钟的时间。  Agent 循环周期 如何让 Harness 层高效？ 从 Harness 层的角度来看，延迟来自四个方面：网络、提示词处理、上下文内容和循环往返。OpenAI 在 Harness 层采用了以下四种技术来提高效率。 1. 持久化 WebSocket 和增量请求 Harness 运行在用户的机器上，而模型运行在 OpenAI 的数据中心，因此每次模型调用都是一次网络交换。进行这种交换的标准方式是 HTTPS。Harness 打开一个连接，发送包含服务器（API 层）所需所有内容的请求，并接收响应。由于语言模型的响应是逐渐到达的，一次一个 token，聊天应用程序通常在 HTTPS 之上使用服务器发送事件 (SSE)，因此客户端发送一个请求，服务器通过开放的响应以小块的形式流式传回答案。SSE 是单行道。它非常适合聊天时代，即一条用户消息产生一次模型调用和一个流式答案。但是 Agent 不是单通的。一个 Codex 轮次可以包含许多模型调用。使用 HTTPS，这些调用中的每一个都是一个新请求，具有两个独立的成本。 第一个成本是连接设置。打开一个新的 HTTPS 连接意味着先进行 TCP 握手，然后进行 TLS 握手，在传输一个字节的有效负载之前需要进行几次网络往返。每条用户消息支付一次是可以管理的，但在一次对话轮次中多次支付则是昂贵的。 第二个成本是有效负载本身的重复。HTTP 是无状态的，因此每个请求必须携带服务器所需的一切。例如，指令、工具定义和迄今为止的整个对话都需要包含在请求中。到第二十个工具调用时，Harness 正在重新发送原始提示词、十九个工具调用和十九个工具结果，仅仅是为了在末尾添加一个新结果。这导致有效负载随着每次迭代而增长，因此 Harness 花费越来越多的时间上传服务器已经看到的数据。  HTTPS 与 WebSocket 但是，我们如何修复这两个成本来源呢？ 修复连接成本的方法是打开一个连接并使其保持活动状态，而不是为每次调用创建一个新连接。这就是 WebSocket 的设计目的。WebSocket 只需要一次初始握手，之后双方都可以随时发送消息，而无需每次进行设置。Codex 打开一个到 API 的单个 WebSocket，并在一轮中的所有模型调用期间保持其打开状态。这消除了重复的 TCP 和 TLS 设置。 修复有效负载重复的方法是停止重新发送服务器已经知道的内容。Harness 保留先前的请求和完成的响应。如果除了对话输入之外没有任何变化，它仅发送新项目以及对先前响应的引用。在工具调用之后，套接字上的下一条消息可能小到：  请注意，这条消息没有指令、工具架构或历史记录。服务器将从其保留的状态重建完整的上下文，因此模型仍然可以看到所有内容。唯一改变的是数据通过网络传输的量。  Harness 仅发送新的工具结果 2. 稳定的提示词前缀 LLM 提供商使用一种称为提示词缓存的技术来避免重复计算。当提示词到达时，模型重用与先前请求（前缀）匹配的提示词开头的缓存内部状态，并仅计算其余部分。匹配是精确的，逐个 token 进行。如果在提示词前部附近更改了一个 token，则其后的所有内容都必须重新计算。  追加到提示词末尾可保持缓存有效。 对于 Harness 来说，这意味着它在每次调用时构建的提示词必须与前一个提示词以完全相同的字节开头。这听起来微不足道，但 Harness 每次都从其实时内存状态重建请求，因此它在组装提示词时的任何细微差异都可能无声地破坏匹配。OpenAI 分享了一个这样的例子。Codex 将 MCP 工具定义保留在哈希映射中，这不保证顺序，因此相同的工具可能会在每个请求上以不同的顺序序列化。上下文中的工具是相同的，只是顺序不同。Codex 仍然在完成任务，只是成本更高。 这里的效率技术是将历史记录视为仅追加，并将易变的运行时状态（如批准设置）排除在提示词之外。例如，当用户在会话中途更改批准设置时，Harness 不会编辑提示词中的工具定义。它在下次运行工具时自行应用新策略。这样，提示词保持不变，因此缓存保持有效。 3. 延迟工具发现 稳定的提示词前缀降低了重复上下文的成本，但它们并不能保持提示词的小巧和简洁。在可以访问大量工具的 agent 中，这可能是一个问题。例如，仅工具架构就可能占用大量空间，因为一旦您计算连接的集成和 MCP 服务器，Codex 会话就可以暴露数百个工具。每个工具架构都是一个 JSON 对象，包含名称、描述和参数定义。一旦所有这些都在提示词中，LLM 就必须在其计算中处理所有这些，即使许多对于当前任务可能未被使用。 为了将未使用的工具排除在提示词之外，OpenAI 使用了一种称为延迟发现的技术。提示词仅携带核心工具，比如 shell 和文件编辑，加上一个 tool_search 工具。其他数百个集成定义完全保留在提示词之外。当模型需要某种功能时，它会使用诸如“list deployments”之类的关键字调用搜索工具。Harness 搜索目录，匹配的定义就会被加载到上下文中。搜索本身是 BM25，一种词汇排序算法，用于查找描述与生成关键字重叠的工具。  模型在需要时搜索工具 除了延迟发现之外，还有两种技术可以保持上下文的小巧。架构压缩通过剥离描述和折叠嵌套结构同时保留参数名称，将过大的工具架构压缩到 token 预算。对话压缩将长历史记录浓缩为简短摘要。 4. 代码模式 在普通设置中，模型发出一个工具调用，等待结果，推理，然后发出下一个。在许多场景中，这些工具调用之间不需要推理。模型只是逐个发出它们以收集它需要的结果。这是昂贵的，因为每个工具调用都需要完整的模型往返。此外，在每个工具调用之后，其结果会不断被添加到上下文中，这浪费了上下文空间。 使用代码模式，模型不是逐个发出工具调用，而是编写一个进行这些调用的小程序。Harness 在嵌入式 JavaScript 运行时中运行此程序，其中每个工具都可用作函数。脚本可以并行分发独立调用，在纯代码中过滤和连接结果，并仅返回紧凑的答案。这样，中间数据保留在运行时中，只有最终结果进入上下文。  直接工具调用与代码模式 ## API 优化 API 层是 Harness 对话的服务。它位于 Harness 层和推理层之间，运行在普通 CPU 上。Harness 发送描述结构化对话项目的 JSON，而模型消费并生成 token ID。当请求到达时，它经过以下步骤： 1. 将请求主体缓冲到内存中。 2. 解析 JSON 并根据架构进行验证（格式正确的字段，在预期范围内的数组等）。 3. 第二轮验证检查语义。例如，所选模型是否支持请求的功能？ 4. 运行飞行前检查：授权和权利、速率限制以及对请求中任何图像的检查。 5. API 渲染并对对话进行分词，然后同时启动两个作业：对 GPU 的推理请求和对提示词的安全检查。 6. 当生成的 token 从推理层流回时，API 将每个 token 转换为文本，将其包装在 API 事件中，并将其流式传输给客户端。  请求在 API 层内部经过的步骤 这些步骤在循环的每次迭代中都会重复。推理不受 API 层的控制。API 无法让 GPU 变得更快。它所能做的就是尽可能少地增加延迟。API 层花费在解析、验证或分词上的时间是用户感受到的额外延迟。 ## 如何让 API 层高效？ OpenAI 采用了以下技术来减少来自 API 层不同部分的开销。 1. 仅对增量内容进行分词 模型不阅读文本。在推理开始之前，提示词必须转换为 token ID。分词器以 O(n) 将文本转换为 token ID，线性遍历提示词中的每个 token 并将其替换为 token ID。在无状态 HTTP 下，整个对话在每次调用时都会被重新分词。到第二十个工具调用时，API 正在重新读取数千个 token 以提取一个新结果。GPU 只需要新的工具结果，但 CPU 正在从头开始重读整本书，并且成本随着输入长度的增长而增长。 使用 WebSocket，API 将分词后的对话保留在服务器内存中。第一个请求对完整的提示词进行分词，但随后的每个请求仅发送新项目。API 仅对该部分进行分词并将其附加到存储的序列中。这样，每次调用的分词不再依赖于对话长度，而是接近 O(1)，取决于边际输入的长度。  使用无状态 HTTP 和 WebSocket 的分词 2. 与推理并行运行安全检查 Agent 必须在向用户显示输出之前检查每个请求的安全性。OpenAI 在其 API 层中实施了安全检查。图像经过它们自己的分类器，而提示词文本发送到分类器模型，寻找危险内容，如网络攻击或生物武器说明。这些分类器需要时间来运行。简单的设计是先运行它们，并仅在它们通过后才开始推理。问题是，这个延迟被添加到所有请求的首个 token 生成时间 (TTFT) 中，而绝大多数请求是完全无害的。 修复方法是同时运行安全检查和推理。模型在第一个 token 出现之前需要一些时间来处理提示词，因此检查利用该窗口来完成。如果检查失败，API 会根据模型做出反应。对于某些模型，它立即将 token 流式传输给用户，并在检查失败时切断流。对于更敏感的模型，它会保持输出直到检查通过，然后才释放它。在这两种情况下，花费在安全性上的时间都隐藏在一个无论如何都会发生的等待中。  并行运行安全检查 3. 将流量路由到更新的 CPU 当一家公司大规模运行服务器时，其机群最终会包含不同年份购买的机器，因此具有不同代的 CPU。调度软件通常将相同类型的机器视为相同的。实际上，它们并不相同。OpenAI 向 Kubernetes 查询其每个部署背后的实际处理器模型，发现在相同的机器标签背后混合了较旧的 Broadwell 芯片和较新的 Ice Lake 芯片。较旧的芯片以大约差 20% 的首个 token 生成时间提供相同的流量，同时使用大约两倍的 CPU。 为了解决这个问题，OpenAI 将更多流量转移到具有更新处理器的机器上，并开始将 CPU 代作为容量规划中的一个显式因素。有时，最好的软件优化是更好的硬件。 ## 推理优化 推理层是模型实际运行的地方。有成千上万的 GPU 和其他加速器，以及操作它们的服务软件（引擎）。模型的核心计算是庞大的矩阵算术，它可以在加速器提供的数千个核心上并行化。该层在机群中调度和批处理传入的请求，保存生成所依赖的每个对话状态，执行模型的前向传递，并将每个生成的 token 交还给 API 层。  每台机器保存模型权重及其服务的对话的 KV 状态。 如何让推理层高效？ 对 token 的需求增长速度快于硬件的增加速度，因此在这一层，缓慢和浪费是同一个词。浪费隐藏在四个地方：工作被路由到错误的机器，内存保存了错误的状态，并行硬件通过顺序生成闲置，以及两个不匹配的阶段共享同一台机器。 OpenAI 采用了以下技术来提高推理层的效率。 1. 利用缓存感知路由平衡负载 OpenAI 从许多 GPU 机器为其模型提供服务。每个请求都必须发送到其中一台。如果这种路由不均匀，有些机器会闲置，而其他机器则会排队工作。闲置的 GPU 是堆栈中最昂贵的浪费。还有第二个不太明显的问题。每台机器都保存其最近服务的对话的计算状态（缓存）。如果对话的下一个请求落在不同的机器上，则该缓存是无用的。新机器必须从头开始重新计算所有内容，即使该工作已经存在于其他地方。 因此，路由器有两个目标： 1. 均匀分配负载：将请求发送到最不忙碌的机器。 2. 使用缓存：将其发送回已经知道该对话的机器。  缓存感知路由 OpenAI 的路由对每个请求都会权衡这两点，以及地理位置和可用容量等基本因素。仅负载平衡的改进就显著降低了提供其模型的成本。 2. 管理 KV 缓存 当 Transformer 生成一个 token 时，它会关注所有先前的 token。为了避免为每个新 token 重新计算该注意力状态，服务系统将其保留在加速器内存中，这称为 KV 缓存。缓存随着对话长度线性增长，乘以机群同时服务的每个对话。对于长上下文，它可以与权重本身的大小相媲美。如果软件驱逐了错误的状态，当该对话返回时，推理层将再次支付完整的处理成本。 这里的效率技术是根据实际使用情况管理缓存。OpenAI 分析生产跟踪以了解哪些缓存状态可能再次需要，根据该数据而不是直觉测试驱逐策略，并优化缓存状态如何在内存层级之间存储和移动。这确保最有价值的工作保留更长时间，而不会耗尽内存或浪费时间移动数据。  热对话状态保留在 GPU 上。较冷的状态被移动或驱逐。 3. 推测解码 生成是顺序的。每个 token 都取决于前一个 token，因此在一个并行传递中处理巨大提示词的模型必须一次生成一个 token 的答案。在大型模型中，生成这些 token 既昂贵又缓慢，即使下一个 token 很容易猜测。对于像“法国的首都是”这样的输入，下一个 token 几乎可以肯定是“Paris”，但模型仍然必须处理整个对话并执行大型矩阵乘法来预测它。 推测解码使用小型草稿模型提出接下来的几个 token，而大型模型在一个并行传递中验证所有这些 token。在许多情况下，草稿提议是正确的，并将被大型模型接受，这节省了大型模型一次生成它们所需的时间。当提议错误时，大型模型会丢弃它并生成自己的 token，因此输出质量不会改变。评估草稿模型的主要指标是平均接受长度，即主模型在每次传递中接受的提议 token 的数量。  推测解码 4. 分离预填充和解码 推理有两个阶段，预填充和解码。预填充在一个大规模并行传递中处理整个传入提示词，并构建模型所需的内部状态 (KV 缓存)。然后，解码一次生成一个 token 的响应。这两者是非常不同的工作负载。预填充是计算密集型的，而解码是内存密集型的，大部分时间用于通过内存流式传输权重和缓存状态。在同一台机器上运行这两者意味着两者都没有在为其设置的硬件上运行。 效率技术是将它们分离。机群的一部分处理预填充，另一部分处理解码，每个都针对其自己的瓶颈进行配置。当请求完成预填充时，其计算状态 (KV 缓存) 被运送到解码机器，后者逐个生成响应 token。  三个层中效率技术的摘要 这涵盖了 Agent AI 应用程序每一层的效率技术。Harness 避免重新发送服务器已有的内容，API 避免重新处理它已经处理过的内容，推理避免重新计算它已经计算过的内容。最后，让我们看看这项工作中适用于 OpenAI 之外的经验教训。 ## 从 OpenAI 对效率的推动和降低成功任务成本中学到什么 放大来看，所有这些不同层的优化技术都试图避免为同一项工作付费两次。Harness 仅发送新的内容并保持前缀稳定，以便缓存得以保留。API 仅对增量内容进行分词，并将不可避免的工作隐藏在必须通过的窗口中。推理将对话路由回其缓存状态，而不是重新计算它。这些层也相互反馈。当 Harness 保持缓存完整且 API 提高缓存命中率时，节省体现在 GPU 上，这意味着用户的成本降低。除了技术之外，以下是 OpenAI 工程师与我们分享的几点经验。 保持设计简单。优先考虑简单性而不是复杂性。例如，使用词汇搜索而不是嵌入进行工具发现，使用一条压缩路径而不是菜单。LLM 已经相当智能，因此在其前面添加更多机械结构通常会增加复杂性而不增加价值。构建最简单可行的东西，然后进行扩展。 使用 agent 优化其自己的堆栈。Codex 编写了为 Codex 提供服务的 API 的迁移的大部分工作，将几年的重写工作变成了几个月的几个工程师的工作。推理团队使用它来分析跟踪和原型化内核。当 agent 编写代码时，您不再需要一种易于人类编写的语言，因此只需选择运行效率最高的一种。效率工作正在成为一个自我加速的循环。 端到端优化。推理团队告诉我们，每当他们挑选一种最喜欢的技术来专注于时，他们都会后悔。专注于堆栈的一部分导致他们对其他部分投资不足。没有单一的优化本身就是一个游戏规则改变者。大的胜利来自于将许多小的胜利链接在一起。他们还学会了使用生产实际服务的相同流量形状进行测试，因为离线看起来像胜利的更改在实际流量中可能会造成损害。 ## 相关链接 - [Alex Xu reposted](https://x.com/alexxubyte) - [Bytebytego](https://x.com/bytebytego) - [@bytebytego](https://x.com/bytebytego) - [25K](https://x.com/bytebytego/status/2082486089642885248/analytics) - [Artificial Analysis Coding Agent Index](https://openai.com/index/gpt-5-6/) - [Upgrade to Premium](https://x.com/i/premium_sign_up) - [11:19 PM · Jul 29, 2026](https://x.com/bytebytego/status/2082486089642885248) - [25.4K Views](https://x.com/bytebytego/status/2082486089642885248/analytics) - [View quotes](https://x.com/bytebytego/status/2082486089642885248/quotes) --- *导出时间: 2026/7/30 10:52:45*
R Recursive Agent Optimization Actually Works 文章介绍了 HALO（分层 Agent 循环优化），一种构建递归自改进 Agent 框架的方法论。通过专门的递归语言模型（RLM）分析追踪数据并诊断问题，利用编码 Agent 修复漏洞并重新部署。在 AppWorld 基准测试中，经过 5 个循环优化，场景目标完成率从 73.7% 提升至 89.5%。该实验证明，利用 AI 审视自身追踪数据的“鸟瞰视角”能有效识别幻觉模式、工具调用错误及配置缺陷，为未来的框架设计提供了新的优化范式。 技术 › Harness Engineering ✍ Amar Singh🕐 2026-05-05 AgentHalo优化LLM自动化调试RecursiveOpenAI方法论效能
B BestBlogs 早报｜实现周期骤缩后，创业者如何重选问题 本期早报探讨了 AI 智能体缩短实现周期后，创业者的机遇与挑战。文章涵盖 Sam Altman 对创业窗口的判断、GPT-5.6 的效率工程实践，以及如何通过 Skill Harness 将模型能力封装为可维护的产品功能。 技术 › Skill ✍ ginobefun🕐 2026-07-30 GPT-5.6Agent创业效率工程ProductHarnessSkillLLMOpenAI
B BestBlogs 早报 · 07-29｜MCP 无状态化与多智能体编排成本 本期早报探讨 MCP 协议的无状态核心变化与 Claude 的生产化接入，分析 Codex 与 ChatGPT Work 共用的执行框架差异，并审视多智能体并行中上下文搬运的隐性成本“编排器的税”。同时涵盖图工程、vLLM 商业化及 Uber 零增长架构等速览内容。 技术 › LLM ✍ ginobefun🕐 2026-07-29 MCPAgentOpenAI架构多智能体上下文Claude早报DevOps工程化
管 管理 AI 员工团队：Ryan Carson 的高效工作系统 Ryan Carson 分享了如何作为唯一员工，通过管理云端 AI Agent 团队日处理 40 个 Pull Request 的经验。文章详细介绍了将工作迁移至云端、建立工作节奏、将重复检查自动化以及控制 Token 成本四个关键步骤，强调在 AI 时代，优秀的工程管理能力比以往任何时候都重要。 技术 › Agent ✍ The Startup Ideas Podcast (SIP)🕐 2026-07-25 AIAgentDevin工程管理自动化云端开发成本控制DevOpsLLM
什 什么是图工程及其走红原因解析 文章解释了从“循环工程”到“图工程”的技术演进。循环是简单的单一代理执行模式，而图（由节点、边和状态组成）通过可视化的流程图处理复杂逻辑和多代理协作。文章介绍了如何使用 LangGraph 构建第一个图，并指出在逻辑变得复杂时应从循环升级到图。 技术 › Agent ✍ Alex Martin🕐 2026-07-21 Graph EngineeringLangGraphAgentLoopsLLMClaudeOpenAI教程
微 微软开源 SkillOpt：不动模型，只训练 Prompt 微软开源了 SkillOpt 项目，引入了“训练 Prompt”的新思路。与传统的 Agent Skill 开发不同，SkillOpt 在不改变模型权重的情况下，将神经网络训练方法（如 epoch、验证集）应用于优化 SKILL.md 文本。系统通过执行任务、打分、反思和筛选修改建议，实现技能文档的自动化迭代。论文表明，该方法在多模型环境下均有显著提升，且优化后的技能具备跨模型迁移能力。 技术 › Agent ✍ sitin🕐 2026-07-13 Prompt工程Agent微软开源LLMAutoML优化技能文档
C ChatGPT Work 新手完整教学 本文详细介绍了 OpenAI 推出的工作型 AI Agent —— ChatGPT Work。文章阐述了其与普通 ChatGPT、Codex 的区别，重点说明了它在会议整理、简报制作、表格分析等职场任务中的应用，并提供了新手入门的五个步骤、实用 Prompt 以及资料安全风险评估。 技术 › Agent ✍ mousepotato🕐 2026-07-10 ChatGPTOpenAI职场效率AI实战Agent工具指南风险提示
为 为什么你学了很多 Agent 工具，还是做不出真正可用的 Agent？ 文章指出许多人学习 AI Agent 时陷入了“工具焦虑”，盲目追逐 LangChain、MCP 等框架，却忽视了 Agent 的本质是一套围绕目标运行的系统。作者强调，真正的难点在于系统设计——理解工具调用、记忆、反馈修正及工程化约束，而非仅仅跑通 Demo。文章推荐 GitHub 项目 agent_learning，主张从 LLM 基础、Prompt 等底层原理开始，逐步构建从基础到工程化的完整知识体系，最终实现 Agent 在真实环境中的稳定运行。 技术 › Agent ✍ Nana🕐 2026-06-15 Agent系统设计LLM学习路线LangChain工程化人工智能方法论LangGraph架构
H Hermes Agent多智能体群聊协作配置教程 这是一份关于如何利用 Hermes 的 Profile 机制构建多智能体协作群聊的实操教程。文章详细介绍了配置 1 个主 Agent（Leader）和 3 个辅助 Agent（数据、技术、市场研究员）的完整流程。内容涵盖了 Profile 创建、共享 API 密钥配置、角色 Prompt 编写、Discord 机器人对接以及发言顺序控制与成本优化策略。 技术 › Agent ✍ DeFi狙击手 | Ai🕐 2026-05-28 Hermes多智能体Agent配置教程提示词群聊协作成本控制LLMDeepSeek
H How to Build LLM Architectures From Scratch 本文是一篇关于从零构建大型语言模型（LLM）架构的深度指南。文章详细解释了 LLM 的工作原理，包括从原始数据收集、清洗、分词，到 Transformer 架构的核心组件（如自注意力机制、位置编码）。同时涵盖了预训练、微调、基于人类反馈的强化学习（RLHF）以及推理优化等关键技术环节，旨在帮助读者理解 ChatGPT 和 Claude 等模型背后的系统构建全流程。 技术 › LLM ✍ Shabnam Parveen🕐 2026-05-25 LLMTransformer架构设计RLHF深度学习人工智能模型训练OpenAIChatGPT技术原理
A Agent Harness 拆解：AI Agent 的工程化基础设施 本文深入探讨了 Agent Harness 的概念，即包裹在 LLM 外部、将无状态模型转化为可用智能体的完整软件基础设施。文章引用了 Anthropic、OpenAI 和 LangChain 的实践，详细拆解了生产级 Harness 的 12 个核心组件（如编排循环、记忆系统、上下文管理、验证循环等），并阐述了如何通过优化这层“操作系统”来解决遗忘、工具调用失败和上下文腐烂等工程难题。 技术 › Harness Engineering ✍ 土豆本豆🕐 2026-05-21 AgentHarnessLLM架构设计LangChainClaudeOpenAI上下文管理工程化Agent拆解
S Shopify 工程师团队的 Claude Code 配置与实战策略 文章详细介绍了 Shopify 23,000 名工程师如何通过 Claude Code 实现 96% 的代码自动化。核心策略包括：构建统一的 LLM 代理层以控制成本，并行运行多个 Agent 处理不同模块，利用 MCP 服务器集成内部文档与 API，以及将 CLAUDE.md 纳入版本控制。工程师角色正从代码编写者转向架构审查者，通过护栏机制保障安全。 技术 › Claude Code ✍ darkzodchi🕐 2026-05-19 AI编程ShopifyAgentMCP工程效率自动化LLM架构最佳实践团队协作