如何在 AI 领域赢得最大市场

✍ Sarah Wang🕐 2026-07-24📦 10.2 KB 🟢 已读 𝕏 文章列表

文章探讨了计算历史上的专业化趋势，指出 AI 推理需求正在爆发，通用 GPU 已无法满足其对内存和能效的特定要求。文中以 Google 的 TPU 为例，介绍了定制化硬件的兴起。重点分析了初创公司 Etched 如何押注专用推理芯片，通过低电压推理和集群级内存技术，试图解决算力与能耗问题，抢占这一巨大的市场份额。

AI推理芯片专用硬件 Etched TPU 能效比计算架构 LLM

# How to Win the Largest Market in AI **作者**: Sarah Wang **日期**: 2026-07-23T15:37:28.000Z **来源**: [https://x.com/sarahdingwang/status/2080316371632808112](https://x.com/sarahdingwang/status/2080316371632808112) --- ![Image](/doc/3389/image/img_63546e1a4e53.jpg) There’s a certain rhythm in the history of computing: we can call it the specialization turn. When a workload is varied and shifting—usually early in the lifecycle of that workflow—people pay for flexibility: nobody knows what tomorrow’s program will look like, so the work goes to a general-purpose system. That works for most things, most of the time. But every so often a workload grows so large, and coheres into a shape so stable, that the economics of the workload can be improved by specialization. The work can migrate into hardware built in its own image: into custom chips and, increasingly, entire custom systems built around them. A machine designed for one job, and only that one job, can do that job faster and cheaper than a machine that has to be ready for everything. That’s what we’ve seen through the history of computing. Graphics migrated from the CPU to a new class of accelerator, the GPU: the “graphics processing unit.” The same turn happened with networking: the internet’s trillions of daily packets long ago stopped moving through CPUs and started moving through custom switch chips from companies like Broadcom. In short: the more important a workload, and the more stable its shape, the more sense it makes to build hardware in the image of the workload. We are witnessing this now for perhaps the most consequential workload of our lifetime: AI inference. # Demand for inference is going vertical When people talk about AI compute, they typically mean training: training has been where all the drama (and dollars) lived these past few years. But training, in accounting terms, is just capex. You pay for it once per model, the way a studio pays to make a film: an enormous cost, typically, but a fixed and amortizable one. Inference, by contrast, is opex: a cost that scales with usage. Every time you ask ChatGPT a question or make an API call or have an agent write some code for you, the company that’s serving the model incurs some cost. To generate a single token, a model needs to read its weights—hundreds of gigabytes of parameters—out of memory, run them through the matrix arithmetic, produce the token, and then do it all over again for the next one. Every token, in other words, has a price in silicon time and electricity. Inference is the COGS of intelligence. And for the companies serving models at scale, the last few years have seen the demand for inference, and the cost of inference, go sharply vertical. In May 2026, Google announced that it was processing 3.2 quadrillion tokens per month across its products, roughly 300 times what it had been processing two years earlier. We see the same story with OpenAI, which now has roughly a billion monthly active users and a rapidly growing coding agent platform in Codex, and with Anthropic, which has seen enormous success with its Claude Code product—not to mention all the companies building on top of these platforms. AI inference is quickly becoming the largest workload computers have ever run; and we believe that it’s only going to grow much further. The machine serving nearly all of that demand today is the GPU. The reason for that is simple: the GPU is flexible. The GPU’s flexibility—the fact that it can support all sorts of demanding workloads—is the reason the AI era exists at all: when nobody knew which architectures would win, a processor that could run anything was exactly the right tool. But if you hold the workload up against the machine, the mismatch is hard to unsee. Generating a token is less about arithmetic and more about memory: to produce each token, the chip must stream the model’s entire weights, plus a growing cache of everything already in the context window, and performing only a couple of simple operations on each byte it reads. The standard patch for this is what’s called “batching”: serving many users’ requests at once, so that each expensive read of the weights from memory is amortized across dozens or hundreds of tokens instead of just one. Batching works well; but it’s not a panacea. It buys throughput, but it does so by selling latency and making every inference workload slower. That’s a trade that the fastest-growing workloads—coding agents in tight loops, long-context reasoning—can’t really afford: customers want things done fast. In short: GPUs are built for everything; inference is now big enough, and differentiated enough, that it requires custom hardware. And this is particularly true for AI, which is so compute- and energy-intensive. Data centers are now gated by watts rather than dollars, and every watt that a general-purpose chip spends on flexibility that the inference workload never touches is a watt not producing tokens. Tokens per watt is the real currency of inference. Over the last few years, the players with the most money at stake have more or less figured this out. For neural networks, this transformation started in the early 2010s, with Google’s development of the TPU, the “tensor processing unit,” and the entire infrastructure around it—custom interconnect, pods, and cooling. Other hyperscalers, like Amazon, Meta, and Microsoft, have followed; and now the frontier labs, like OpenAI, are following as well. This isn’t a mysterious development. It’s just the oldest pattern in computing reasserting itself: when a workload gets big enough and stable enough, it earns specialized hardware of its own. The machines that win a stable and huge market like AI inference are the ones designed as a whole, from the transistor to the data center floor, and that nail production scale to meet the demands of this new work. # The Etched bet In 2022, months before ChatGPT and years before Cursor, Claude Code, and Codex, Gavin Uberti, Robert Wachen, and Chris Zhu dropped out of Harvard to make a bet that looked, at the time, nearly irresponsible: they would build an entire inference system—new chips, boards, interconnects, racks—from scratch. Their thesis was simple. Inference would be the largest market in AI, and the general-purpose GPU wouldn’t be the endgame of that market. Hyperscalers had realized that already. But their custom silicon programs were for their workloads: you can rent a TPU, but you can’t buy one, or rack it in your own data center, or build a business on it. For everyone who wasn’t a hyperscaler—for AI labs without silicon teams, inference clouds, sovereign AI programs, enterprises building their own fleets—there was a huge opportunity for chips designed specifically for inference. Gavin, Robert, and Chris set out to build exactly that. What began as a radical, contrarian wager has matured into a strong bet on the defining challenge of our era. Over the last few years, Etched has hired more than 400 engineers from Nvidia, Google’s TPU group, Broadcom, Apple, SK Hynix, and TSMC. A striking share of them work not on chip design but on production: supply chains, rack assembly, burn-in, and logistics. In a supply-constrained market, the scarce skill is not designing a fast chip; it is shipping complete, working systems by the thousands. As Gavin, Robert, and Chris like to put it: production is the product. Under the hood of that product are two core ideas. The first is low-voltage inference: running the math blocks at roughly half the voltage of a typical AI chip, packing several times more usable compute into the same power envelope—more tokens per watt, the currency that matters. The second is cluster-scale memory: an ultra-low-latency interconnect that pools memory across chips, so that during decode, an entire rack behaves like one enormous machine. Both of these technologies are not possible by only designing a chip: they require technical breakthroughs across the entire system across memory, power delivery, cooling and more—precisely the bottlenecks specialization predicts, in precisely the market where every multiple of tokens per watt converts directly into revenue. But as impressed as we are with the elegance of Etched’s first product, what’s blown us away is their tempo of execution. Etched’s first production chip taped out on TSMC’s N4P process and worked on the first attempt—a“first-pass silicon,” in industry shorthand—rare even for industry incumbents. As we sat in their 2 megawatt lab in their office, we heard stories again and again of the team defying the odds to achieve unprecedented timelines, flying across the world to unblock vendors and running company-wide day and night shifts to bring up their chip in under two months. (The industry norm is between six and nine months.) The result: those first racks ship to customers this summer; a 10 megawatt site is standing up, with customers already running workloads on Etched hardware remotely. Despite the origin story, what Etched has built is not a transformer-only machine, but a full-stack inference system to serve the most demanding workloads, whether many-trillion parameter MoEs, state-space models, or long-context agents. History is unsentimental on this point: accelerated hardware will be built for the most important workloads, but very few can build a machine capable of production at scale. It’s not enough to have a great architecture and demo rack; you have to ship at scale, with the speed and quality that the most demanding customers in the world require amid fierce competition. In a world where production is the product and the machine is the moat, we’re thrilled to partner with Etched. ## 相关链接 - [Sarah Wang](https://x.com/sarahdingwang) - [@sarahdingwang](https://x.com/sarahdingwang) - [38K](https://x.com/sarahdingwang/status/2080316371632808112/analytics) - [Upgrade to Premium](https://x.com/i/premium_sign_up) - [11:37 PM · Jul 23, 2026](https://x.com/sarahdingwang/status/2080316371632808112) - [38.5K Views](https://x.com/sarahdingwang/status/2080316371632808112/analytics) - [View quotes](https://x.com/sarahdingwang/status/2080316371632808112/quotes) --- *导出时间: 2026/7/24 16:08:09* --- ## 中文翻译 # 如何在 AI 领域赢得最大市场 **作者**: Sarah Wang **日期**: 2026-07-23T15:37:28.000Z **来源**: [https://x.com/sarahdingwang/status/2080316371632808112](https://x.com/sarahdingwang/status/2080316371632808112) --- ![Image](/doc/3389/image/img_63546e1a4e53.jpg) 在计算机历史上，存在着一种特定的节奏：我们可以称之为专业化转向。当工作负载多变且不稳定时——通常在该工作生命周期的早期——人们会为灵活性付费：没人知道明天的程序会是什么样子，因此工作会被交给通用系统。这在大多数时候、对大多数事情都有效。但每隔一段时间，某个工作负载就会变得非常庞大，并且凝聚成一种极其稳定的形态，以至于通过专业化可以改善其经济效益。这项工作可以迁移到以其自身形象构建的硬件中：迁移到定制芯片，以及越来越多的围绕这些芯片构建的完整定制系统。一台为单一工作而设计、且仅为该工作而设计的机器，比一台必须随时准备应对所有工作的机器，能更快、更廉价地完成该工作。这就是我们在计算机历史中看到的过程。图形处理从 CPU 迁移到了一类新的加速器，即 GPU：“图形处理单元”。网络领域也发生了同样的转变：互联网每天数万亿的数据包早已不再通过 CPU 传输，而是开始通过来自博通等公司的定制交换芯片进行传输。简而言之：工作负载越重要，其形态越稳定，就越有必要按照工作负载的形象来构建硬件。我们现在正在见证这一过程发生在也许是我们一生中最重要的工作负载上：AI 推理。 # 推理需求正在急剧增长当人们谈论 AI 算力时，他们通常指的是训练：过去几年，所有的戏剧性事件（和资金）都发生在训练领域。但在会计术语中，训练只是资本支出。每个模型只需支付一次，就像工作室支付电影制作费一样：通常成本巨大，但是一笔固定的、可摊销的费用。相比之下，推理是运营支出：一种随使用规模增加而增加的成本。每当你向 ChatGPT 提问、进行 API 调用或让代理为你编写代码时，提供模型服务的公司都要承担一定的成本。为了生成一个 token，模型需要从内存中读取其权重——数百 GB 的参数——通过矩阵运算进行处理，生成 token，然后为下一个 token 再重复这一切。换句话说，每个 token 都有硅基时间（计算时间）和电力的成本。推理是智能的销货成本（COGS）。对于那些大规模提供模型服务的公司来说，过去几年里，推理的需求和成本都急剧攀升。2026 年 5 月，谷歌宣布其产品每月处理 320 亿亿个 token，大约是两年前的 300 倍。我们在 OpenAI 身上也看到了同样的故事，它现在拥有约 10 亿月活跃用户以及快速增长的代码代理平台 Codex；还有 Anthropic，其 Claude Code 产品取得了巨大成功——更不用说所有在这些平台上构建的公司。AI 推理正迅速成为计算机有史以来运行的最大工作负载；而且我们相信它还会进一步大幅增长。目前，服务于几乎所有这些需求的机器是 GPU。原因很简单：GPU 具有灵活性。GPU 的灵活性——即它能够支持各种繁重工作负载这一事实——正是 AI 时代存在的根本原因：当没人知道哪种架构会胜出时，一个能运行任何东西的处理器正是正确的工具。但是，如果你将工作负载与机器进行对比，这种不匹配是显而易见的。生成 token 与其说是算术问题，不如说是内存问题：为了生成每个 token，芯片必须传输模型的全部权重，加上上下文窗口中已有的不断增长的缓存，并且对它读取的每个字节只执行几个简单的操作。针对此问题的标准补丁被称为“批处理”：同时服务多个用户的请求，这样每次从内存读取昂贵权重的成本就可以分摊到几十甚至几百个 token 上，而不仅仅是一个。批处理效果很好；但它不是万能药。它换取了吞吐量，但代价是增加了延迟，使每个推理工作负载都变慢了。对于那些增长最快的工作负载——处于紧密循环中的代码代理、长上下文推理——这是一笔它们负担不起的交易：客户希望事情能快速完成。简而言之：GPU 是为万物而建的；推理现在已经足够庞大，且足够独特，需要定制硬件。这对于算力和能耗极其密集的 AI 来说尤其如此。数据中心的瓶颈现在不再是资金，而是电力，通用芯片花在推理工作负载根本用不到的灵活性上的每一瓦电力，都是未用于生成 token 的一瓦电力。每瓦生成的 token 数才是推理真正的货币。在过去几年里，那些利益攸关的玩家们或多或少都意识到了这一点。对于神经网络，这种转型始于 2010 年代初，当时谷歌开发了 TPU（“张量处理单元”）及其周围的所有基础设施——定制互连、Pod 和冷却系统。其他超大规模企业，如亚马逊、Meta 和微软，也紧随其后；现在，前沿实验室，如 OpenAI，也紧随其后。这并不是什么神秘的发展。这只是计算机历史上最古老的模式在重申自己：当一个工作负载变得足够大、足够稳定时，它就会赢得属于自己的专用硬件。那些能够赢得像 AI 推理这样稳定且巨大市场的机器，是那些作为一个整体来设计的机器——从晶体管到数据中心机房——并且能够精准把控生产规模以满足这一新工作需求的机器。 # Etched 的赌注 2022 年，在 ChatGPT 问世前几个月，在 Cursor、Claude Code 和 Codex 出现前几年，Gavin Uberti、Robert Wachen 和 Chris Zhu 从哈佛辍学，做出了一项在当时看起来几乎是不负责任的赌注：他们将从零开始构建一个完整的推理系统——包括新芯片、电路板、互连、机架。他们的论点很简单。推理将是 AI 领域最大的市场，而通用的 GPU 不会是这个市场的终局。超大规模企业已经意识到了这一点。但他们的定制硅芯片项目是针对他们自己的工作负载的：你可以租用 TPU，但不能购买一个，也不能把它安装在你自己的数据中心里，或者在此基础上建立业务。对于所有不是超大规模企业的实体——对于没有芯片团队的 AI 实验室、推理云服务、主权 AI 计划、以及构建自己机队的企业来说——专门为推理设计的芯片存在巨大的机会。Gavin、Robert 和 Chris 着手构建的正是这种芯片。这一最初激进的、逆向思维的赌注，已经成熟为对我们这个时代决定性挑战的坚定押注。在过去几年里，Etched 聘请了 400 多名来自英伟达、谷歌 TPU 团队、博通、苹果、SK 海力士和台积电的工程师。其中相当大一部分人并非从事芯片设计，而是从事生产工作：供应链、机架组装、老化测试和物流。在一个供应受限的市场中，稀缺的技能不是设计一块快速芯片；而是成千上万地交付完整的、可运行的系统。正如 Gavin、Robert 和 Chris 喜欢说的那样：生产即产品。在该产品的核心有两个想法。第一个是低电压推理：以典型 AI 芯片大约一半的电压运行计算模块，从而在相同的功耗限制下封装数倍的可用算力——即更高的每瓦 token 数，这才是至关重要的货币。第二个是集群级内存：一种超低延迟的互连技术，可以在芯片之间汇聚内存，使得在解码过程中，整个机架的表现就像一台巨大的机器。这两项技术都不可能仅通过设计芯片来实现：它们需要跨越整个系统的技术突破，包括内存、供电、冷却等——这正是专业化所预测的瓶颈，也恰恰出现在每一个每瓦 token 数的倍数都能直接转化为收入的市场中。尽管我们对 Etched 第一款产品的精妙设计印象深刻，但真正让我们震撼的是他们的执行节奏。Etched 的第一款生产芯片在台积电的 N4P 工艺上流片，并且首次尝试就成功运行了——行业术语叫“首轮硅片成功”——即使是行业老手也难以企及。当我们坐在他们办公室那间 2 兆瓦的实验室里时，我们一次又一次地听到这个团队克服重重困难、实现前所未有的时间表的故事：他们飞往世界各地去解除供应商的阻碍，并在公司内部实行日夜轮班制，在不到两个月的时间内让芯片运行起来。（行业标准是六到九个月。）结果是：首批机架将于今年夏天发货给客户；一个 10 兆瓦的设施正在建立中，客户已经可以在 Etched 硬件上远程运行工作负载。尽管有其起源故事，但 Etched 构建的并不是一台仅支持 Transformer 的机器，而是一个全栈推理系统，用于服务最苛刻的工作负载，无论是数万亿参数的混合专家模型、状态空间模型，还是长上下文代理。历史在这一点上是冷酷无情的：加速硬件将是为最重要的工作负载而构建的，但很少有人能够构建出能够进行大规模生产的机器。拥有优秀的架构和演示机架是不够的；你必须能够大规模交付，以世界上最苛刻的客户在激烈竞争中所要求的速度和质量进行交付。在一个生产即产品、机器即护城河的世界里，我们很高兴能与 Etched 合作。 ## 相关链接 - [Sarah Wang](https://x.com/sarahdingwang) - [@sarahdingwang](https://x.com/sarahdingwang) - [38K](https://x.com/sarahdingwang/status/2080316371632808112/analytics) - [升级到 Premium](https://x.com/i/premium_sign_up) - [11:37 PM · Jul 23, 2026](https://x.com/sarahdingwang/status/2080316371632808112) - [38.5K Views](https://x.com/sarahdingwang/status/2080316371632808112/analytics) - [查看引用](https://x.com/sarahdingwang/status/2080316371632808112/quotes) --- *导出时间: 2026/7/24 16:08:09*

🔗 相关文档