存储廉价，注意力昂贵：Agent 内存系统的分层架构设计

✍ Steven (Batman) Batchelor-Manning🕐 2026-05-28📦 13.2 KB 🟢 已读 𝕏 文章列表

本文探讨了Agent内存系统的设计核心原则：利用存储廉价与注意力昂贵的非对称性。文章通过分析MemoryOS、Hindsight和supermemory三个案例，阐述了分层内存架构的必要性。这种架构通常包含热数据、温数据和冷数据三层，通过不同的访问模式、数据形状和提升机制，在降低上下文噪音的同时保留关键信息，从而在有限的注意力窗口内实现最优的检索质量。

# Storage is cheap. Attention is expensive. Are you using the system that exploits the difference? **作者**: Steven (Batman) Batchelor-Manning **日期**: 2026-05-26T20:52:42.000Z **来源**: [https://x.com/S_BatMan/status/2059377206661370009](https://x.com/S_BatMan/status/2059377206661370009) --- ![Image](/doc/2293/image/img_ec4dd969ebf7.jpg) ## The asymmetry that makes tiering pay back If you change one thing about how you design agent memory, change this: stop treating storage cost and attention cost as if they're the same thing. I've been going deep on this across 19 systems, and the finding that keeps surfacing is that the ones that handle memory well aren't necessarily the ones with the most sophisticated retrieval. They're the ones that figured out which memories belong in the prompt at all. That's a different problem, and it has a different solution. Storage is cheap. Attention is expensive. A 128k-context model reading 110k of mediocre context isn't, empirically, a better agent than the same model reading 8k of carefully selected context. The research on this is consistent: retrieval quality degrades as context fills with noise, and the degradation isn't linear. The model doesn't simply ignore the irrelevant material. It processes it, and the processing crowds out the signal. Storage doesn't have this property. A fact sitting in a database costs nothing to keep. The cost only arrives when you retrieve it and load it into the prompt. Which means the question isn't "should I store this?" but "should I retrieve this, and if so, when?" That reframing is where tiering comes from. Different memory items have different access patterns. Some things an agent needs every turn: the user's name, their current project, their stated preferences. Some things it needs often but not always: recent decisions, open questions, episodic facts from the last few sessions. Some things it should be able to find when needed but should never burden every turn: meeting notes from three months ago, completed tasks, raw transcripts, one-off reference material. A flat store treats all three categories identically. Hot items pay the search cost of cold items. Cold items inflate the prompt with noise. There's no mechanism for items to graduate as they become more relevant, and no mechanism for items to age out as they become less relevant. The OS analogy is exact: CPUs have L1, L2, L3, RAM, SSD, and disk not because bytes are different but because frequency of access varies by orders of magnitude. Seven of the 19 systems I went through had already built explicit tiering before I started looking. Two of them are worth understanding in detail. ## MemoryOS as reference implementation MemoryOS is the clearest implementation of tiered memory in the 19 systems. Three tiers, each with a distinct data shape, a distinct latency budget, and a distinct role. The short-term tier is a Python deque with a maximum length of 10 QA pairs. No embeddings, no search index, no heat tracking. It's pure conversational raw material, the last ten exchanges, available at microsecond latency. When the deque fills, the oldest pair drains into the mid-term tier. The mid-term tier holds up to 2000 sessions. Each session carries a summary, an embedding, a keyword set, and heat counters. The tier is indexed with Faiss and searched by cosine similarity. When the tier reaches capacity, the coldest sessions are evicted. When a session gets hot enough, it's promoted to the long-term tier. The long-term tier is a 90-dimension psychology and alignment schema, two knowledge-base deques, one for user facts and one for assistant facts, each capped at 100 entries. This is the persistent layer, the one that survives across sessions and carries the durable model of the user. The heat formula that governs promotion is twelve lines of Python. Three signals: visit frequency, an LFU analogue; interaction depth, a proxy for topical engagement; and recency decay, exponential with a 24-hour half-life. A segment crosses the promotion threshold at 5.0. After promotion, the visit and interaction counters reset to zero, and heat collapses back to roughly 1.0. The design decision that's easy to miss: heat gates promotion, not retrieval. The retriever is purely semantic, cosine similarity over the mid-term embeddings. Heat is a background signal that decides whether a segment should graduate to the long-term tier. The two concerns are decoupled, and that decoupling matters more than the formula itself. What MemoryOS leaves on the table is worth naming. The coefficients are hardcoded at 1.0, there's no mechanism for learning weights from actual usage patterns. There are no demotion paths; once something reaches the long-term tier, it stays. And the formula optimises for frequency over importance. A critical but rare fact, a partner's name, a medical condition, a hard constraint, may never cross the promotion threshold if it only surfaces once. Once the mid-term tier evicts the segment, the fact is gone. ## Hindsight's theory-derived tiers Hindsight arrives at the same three-tier structure from a completely different starting point. Where MemoryOS draws on OS cache theory, Hindsight draws on cognitive science. The three tiers are World, Experience, and Observations. World holds objective claims about the universe, ground truth, always-on, long-lived. Experience holds first-person actions of the system itself, the episodic record. Observations hold consolidated beliefs derived from World and Experience facts, carrying source memory IDs, a proof count, and a history field that tracks how the belief has evolved. All three tiers live in the same database table, differentiated by a fact-type discriminator. Partial HNSW indexes are built per fact type. The schema is unified; the access patterns aren't. Promotion in Hindsight isn't counter-driven. It's batched LLM-driven consolidation. When a new fact is written, it's enqueued into an async operations table. A background worker fetches the new facts alongside existing overlapping observations, builds a batch prompt, and asks the model for creates, updates, and deletes. Source memories are stamped with a consolidated-at timestamp to prevent reprocessing. Every new fact, regardless of how frequently it's been accessed, gets considered for promotion to the Observations tier. That's the key difference from MemoryOS. By tying upper-tier promotion to consolidation rather than heat, Hindsight avoids the blind spot for critical-but-rare facts. A single consolidation pass considers every new fact. Frequency is irrelevant to whether something graduates. Put plainly: two systems, different first principles, different implementation languages, different target use cases, and both land on three tiers with raw material at the bottom, a working layer in the middle, and a synthesised persistent layer at the top. Both use async promotion. Both carry provenance back to lower tiers. That's what convergent evolution looks like in software architecture, and it's the strongest signal I know of that a pattern is load-bearing. ## @supermemory 's genre-conditioned tiering supermemory operates in the managed API deployment shape, which changes the implementation without changing the architecture. The three tiers are static profile, dynamic profile, and document and chunk store. The static profile holds stable long-term facts, the hot tier. It's returned as a static array from the profile endpoint, cached at the edge, with a latency budget of roughly 50ms. The dynamic profile holds recent and episodic context, the warm tier. Many entries carry a forgetAfter field that sets a TTL. The document and chunk store is the cold tier, queried via search endpoints when needed. Tier assignment happens at write time. An extraction LLM classifies each incoming memory with an isStatic boolean and optionally a forgetAfter value. The classification is enforced by a closed extraction prompt, uniform across all consumers. The managed API deployment shape enables three things that an in-process designer can't directly copy but should understand. Cold-tier data can sit on cheaper hardware, object storage for raw bytes, a standard relational store for metadata and chunks, with the hot profile cached separately at the edge. The hot tier gets its own endpoint with its own SLA, separate from the search path. And the extraction prompt is centralised, which means tier assignment is consistent in a way that per-agent classification rarely is. The trade-offs are real. A remote hot tier is only fast if the network is fast. The agent can't override the engine's tier classification. The extraction prompt is a black box. But the architectural pattern, hot tier as its own endpoint, cold tier on cheaper hardware, tier assignment at write time, is worth copying even if the deployment shape isn't. ## Promotion vs. fixed typology mem9 is the contrast case that clarifies what tiering isn't. mem9 has a memory-type column with three values: pinned, insight, and digest. Pinned memories are assigned by explicit content-write paths, manually created, protected from LLM reconciliation. Insights are assigned by every LLM-extracted write, mutable, versionable, supersedable. Both participate in the same hybrid recall with the same RRF scoring. The type field is a write-protection flag, not a retrieval filter and not a tier signal. This is typology, not tiering. The distinction matters because the two are easy to conflate. Typology describes governance: who can mutate this memory, under what conditions. Tiering describes access patterns: how frequently is this memory needed, and what storage representation best serves that frequency. A system can have both. supermemory's isStatic is a tier signal, while a separate isInference flag is closer to a governance class. But conflating them produces the most ambiguity in practice. If you find yourself adding a type field to your memory rows, the question to ask is whether the field describes an access pattern or a governance class. If it's an access pattern, you're building tiering. If it's a governance class, you're building typology. Both are useful. They're not the same thing. ## What flat costs you Four concrete things follow from running a flat memory store. The hot path pays the cold path's search cost. Every retrieval scans the same index over the same items. The agent looking up the user's name pays the same search cost as the agent looking up a meeting note from six months ago. At small scale this is invisible. At scale it's a latency problem. The cold path inflates the prompt with noise. Retrieval returns semantically near items, which includes standing context, superseded facts, and material that's factually correct but irrelevant to the current turn. The model processes all of it. The signal-to-noise ratio in the context window degrades as the store grows. There's no mechanism for items to graduate. Memory isn't static. A fleeting episodic fact from early in a relationship may, over time, become a durable signal about the user's preferences or constraints. A flat store gives no machinery for noticing that transition. The item stays in the same representation it was written in, regardless of how its relevance has changed. There's no mechanism for items to age out. Demotion isn't deletion. A flat store that wants to remove stale material must delete it. A tiered store can move it to a colder representation, still findable, no longer burdening the hot path. The flat store forces a binary choice that the tiered store doesn't. ## The net result 18 of the 19 systems implement tiering, gesture at it, or have explicit recommendations for it. The exception is mem9, which has a typology layer solving a different problem and would benefit from tiering on top of it. If you're building agent memory from scratch, the progression the 19 systems point to is this. Identify what the agent needs every turn, that's your hot tier. Identify what it needs often but not always, that's your warm tier. Identify what it should find when needed but never burden every turn, that's your cold tier. Pick a promotion mechanism: heat-based like MemoryOS, LLM-judgement like Hindsight, or extraction-time classification like supermemory. Pick a demotion mechanism: time decay, TTL, or freshness categories. Keep promotion and retrieval scoring separate at first, the decoupling is easier to add than to unpick. Track provenance from upper tiers back to lower tiers, so you can always answer the question of where a synthesised belief came from. The 19 systems aren't unanimous on much. On this, they are: a single flat memory store is the wrong default for any non-trivial agent memory system. Storage is cheap. Attention is expensive. Build the system that exploits the difference. If you found this Article interesting please share ## 相关链接 - [Steven (Batman) Batchelor-Manning](https://x.com/S_BatMan) - [@S_BatMan](https://x.com/S_BatMan) - [681K](https://x.com/S_BatMan/status/2059377206661370009/analytics) - [@supermemory](https://x.com/@supermemory) - [Upgrade to Premium](https://x.com/i/premium_sign_up) - [4:52 AM · May 27, 2026](https://x.com/S_BatMan/status/2059377206661370009) - [681K Views](https://x.com/S_BatMan/status/2059377206661370009/analytics) - [View quotes](https://x.com/S_BatMan/status/2059377206661370009/quotes) --- *导出时间: 2026/5/28 09:26:21* --- ## 中文翻译 # 存储廉价，注意力昂贵。你正在利用这种差异的系统吗？ **作者**: Steven (Batman) Batchelor-Manning **日期**: 2026-05-26T20:52:42.000Z **来源**: [https://x.com/S_BatMan/status/2059377206661370009](https://x.com/S_BatMan/status/2059377206661370009) --- ![图片](/doc/2293/image/img_ec4dd969ebf7.jpg) ## 让分层存储物有所值的不对称性如果你在设计 Agent 记忆时只能改变一件事，那就改变这一点：不要再把存储成本和注意力成本混为一谈。我深入研究了 19 个系统，一个反复出现的发现是，那些能妥善处理记忆的系统，未必拥有最复杂的检索机制。它们之所以成功，是因为弄清楚了哪些记忆*根本不该*出现在提示词中。这是两个不同的问题，需要不同的解决方案。存储廉价。注意力昂贵。经验表明，一个读取 11 万字符平庸上下文的 128k 上下文模型，并不比同一个模型读取 8k 字符精心挑选的上下文表现得更好。相关研究结论一致：随着上下文填充噪声，检索质量会下降，而且这种下降并非线性。模型并不会简单地忽略无关材料。它会处理这些材料，而这种处理会挤占信号。存储不具备这种属性。一条事实躺在数据库里，保存它的成本为零。只有当你检索它并将其加载进提示词时，成本才会产生。这意味着问题不在于“我该存储这个吗？”，而在于“我该检索这个吗？如果是，什么时候？？” 这种重构正是分层存储的由来。不同的记忆项有不同的访问模式。有些东西 Agent 每一轮都需要：用户的名字、当前项目、明确陈述的偏好。有些东西经常需要但并非总是需要：最近的决策、未解决的问题、过去几次会话中的片段事实。有些东西应该能在需要时找到，但绝不应成为每一轮的负担：三个月前的会议记录、已完成的任务、原始逐字稿、一次性的参考资料。扁平化存储对所有这三类一视同仁。热数据承担着冷数据的搜索成本。冷数据用噪声污染了提示词。没有机制能让数据随着相关性增加而“毕业”，也没有机制能让数据随着相关性降低而“老化”。操作系统的类比非常贴切：CPU 之所以有 L1、L2、L3、RAM、SSD 和硬盘，不是因为字节本身有什么不同，而是因为访问频率在数量级上存在差异。在我研究的 19 个系统中，有 7 个在我开始观察之前就已经构建了明确的分层。其中有两个值得详细了解。 ## 作为参考实现的 MemoryOS 在这 19 个系统中，MemoryOS 是分层记忆最清晰的实现。三个层级，每一层都有独特的数据形态、延迟预算和角色。短期层是一个最大长度为 10 个 QA 对的 Python 双端队列。没有嵌入、没有搜索索引、没有热度追踪。它是纯粹的对话原材料，最近十次交流，以微秒级延迟可用。当队列填满时，最旧的一对会流入中期层。中期层最多容纳 2000 个会话。每个会话携带摘要、嵌入、关键词集和热度计数器。该层使用 Faiss 索引，并通过余弦相似度搜索。当该层达到容量上限时，最冷的会话会被驱逐。当某个会话足够热时，它会被提升到长期层。长期层是一个 90 维的心理和一致性格式，两个知识库双端队列，一个用于用户事实，一个用于助手事实，每个上限为 100 条。这是持久层，是跨越会话存活并承载用户持久模型的层。控制提升的热度公式只有 12 行 Python。三个信号：访问频率（类似 LFU）；交互深度（主题参与度的代理）；以及最近度衰减（半衰期为 24 小时的指数衰减）。片段的提升阈值是 5.0。提升后，访问和交互计数器归零，热度回落到大约 1.0。一个容易被忽略的设计决策是：热度控制的是提升，而不是检索。检索器纯粹是语义的，基于中期嵌入的余弦相似度。热度是一个后台信号，决定片段是否应该毕业进入长期层。这两个关注点是解耦的，这种解耦比公式本身更重要。 MemoryOS 的局限性也值得指出。系数硬编码为 1.0，没有机制从实际使用模式中学习权重。没有降级路径；一旦某样东西进入长期层，它就永远留在那里。而且公式优化的是频率而非重要性。一个关键但罕见的事实——比如伴侣的名字、某种病情、硬性约束——如果只出现一次，可能永远达不到提升阈值。一旦中期层驱逐了该片段，该事实就消失了。 ## Hindsight 的理论衍生层级 Hindsight 从一个完全不同的起点出发，得出了同样的三层结构。MemoryOS 借鉴的是操作系统缓存理论，而 Hindsight 借鉴的是认知科学。这三个层级是 World（世界）、Experience（经验）和 Observations（观察）。World 持有关于宇宙的客观主张，是基本真理，永远在线，长期存在。Experience 持有系统本身的第一人称动作，即片段记录。Observations 持有从 World 和 Experience 事实中综合出来的信念，携带来源记忆 ID、证明计数以及一个追踪信念如何演变的历史字段。所有三层都位于同一个数据库表中，通过事实类型鉴别器进行区分。针对每种事实类型构建了部分 HNSW 索引。模式是统一的，但访问模式不同。 Hindsight 的提升不是由计数器驱动的。它是基于批处理 LLM 的整合。当写入新事实时，它被排入一个异步操作表。后台工作线程获取新事实以及现有的重叠观察，构建批处理提示词，并要求模型进行创建、更新和删除操作。源记忆被打上“整合时间”戳以防止重复处理。每个新事实，无论被访问频率如何，都会被考虑提升到 Observations 层。这是与 MemoryOS 的关键区别。通过将上层提升与整合而非热度挂钩，Hindsight 避免了对关键但罕见事实的盲点。单次整合过程会考虑每一个新事实。频率与某样东西能否毕业无关。简而言之：两个系统，不同的第一性原理，不同的实现语言，不同的目标用例，但都采用了三层结构，底层是原材料，中间是工作层，顶层是综合持久层。都使用异步提升。都将谱系追溯回下层。这就是软件架构中的趋同进化，这是我所知道的最强有力的信号，表明某种模式是承重关键的。 ## @supermemory 的体裁条件分层 supermemory 采用托管 API 部署形态，这改变了实现方式但未改变架构。三个层级是静态档案、动态档案以及文档和块存储。静态档案持有稳定的长期事实，即热层。它作为静态数组从档案端点返回，在边缘缓存，延迟预算约为 50ms。动态档案持有近期的和片段的上下文，即温层。许多条目带有一个 forgetAfter 字段来设置 TTL。文档和块存储是冷层，在需要时通过搜索端点查询。层级分配发生在写入时。一个提取 LLM 用 isStatic 布尔值和可选的 forgetAfter 值对每个传入的记忆进行分类。分类由一个封闭的提取提示词强制执行，在所有消费者中保持一致。托管 API 部署形态实现了三件事，这是进程内设计者无法直接复制但应该理解的。冷层数据可以位于更便宜的硬件上，原始字节用对象存储，元数据和块用标准关系存储，而热档案在边缘单独缓存。热层拥有自己的端点和 SLA，与搜索路径分离。而且提取提示词是中心化的，这意味着层级分配的一致性是单个 Agent 分类难以企及的。权衡是真实存在的。远程热层只有在网络快的时候才快。Agent 无法覆盖引擎的层级分类。提取提示词是个黑盒。但是，这种架构模式——热层作为独立端点，冷层在廉价硬件上，写入时进行层级分配——即使部署形态不同，也值得复制。 ## 提升 vs. 固定分类学 mem9 是一个阐明“什么不是分层”的反例。 mem9 有一个记忆类型列，有三个值：pinned（固定）、insight（洞察）和 digest（摘要）。Pinned 记忆通过显式的内容写入路径分配，手动创建，受 LLM 协调保护。Insights 由每个 LLM 提取写入分配，可变、可版本化、可覆盖。两者都参与相同的混合召回，具有相同的 RRF 评分。类型字段是一个写保护标志，不是检索过滤器，也不是层级信号。这是分类学，不是分层。这种区别很重要，因为两者很容易混淆。分类学描述治理：谁可以改变这个记忆，在什么条件下。分层描述访问模式：这个记忆多久需要一次，什么存储表示最能服务该频率。一个系统可以两者兼备。supermemory 的 isStatic 是层级信号，而一个单独的 isInference 标志更接近治理类别。但在实践中混淆它们会产生最大的歧义。如果你发现自己正在给记忆行添加一个类型字段，要问的问题是该字段描述的是访问模式还是治理类别。如果是访问模式，你正在构建分层。如果是治理类别，你正在构建分类学。两者都有用。它们不是同一回事。 ## 扁平存储让你付出什么代价运行扁平记忆存储会导致四个具体后果。热路径支付冷路径的搜索成本。每次检索都扫描相同索引上的相同条目。查找用户名字的 Agent 支付的搜索成本与查找六个月前会议记录的 Agent 相同。在小规模下这不可见。在规模上，这是一个延迟问题。冷路径用噪声膨胀提示词。检索返回语义相近的条目，其中包括静态上下文、被取代的事实，以及在事实上正确但与当前轮次无关的材料。模型会处理所有这些。随着存储的增长，上下文窗口中的信噪比会恶化。没有机制让条目毕业。记忆不是静态的。一段来自关系早期的短暂片段事实，随着时间的推移，可能会变成关于用户偏好或约束的持久信号。扁平存储没有机制来注意这种转变。该条目保持写入时的表示不变，不管其相关性如何变化。没有机制让条目老化。降级不是删除。想要移除陈旧材料的扁平存储必须将其删除。分层存储可以将其移动到更冷的表示，仍然可查找，不再负担热路径。扁平存储迫使做出二元选择，而分层存储则不需要。 ## 最终结果 19 个系统中有 18 个实施了分层、暗示分层或有明确的建议。例外是 mem9，它有一个解决不同问题的分类学层，如果在上面叠加分层将会受益。如果你从零开始构建 Agent 记忆，这 19 个系统指向的进程是这样的。确定 Agent 每一轮需要什么，那是你的热层。确定它经常需要但并非总是需要什么，那是你的温层。确定它应该在需要时找到但绝不应成为每一轮负担什么，那是你的冷层。选择一个提升机制：像 MemoryOS 那样基于热度，像 Hindsight 那样基于 LLM 判断，或者像 supermemory 那样基于提取时分类。选择一个降级机制：时间衰减、TTL 或新鲜度类别。起初将提升和检索评分分开，这种解耦添加起来比解开容易。追踪从上层到下层的谱系，以便你总能回答一个综合信念来自哪里的问题。这 19 个系统在很多问题上没有共识。但在这一点上，它们是一致的：单一的扁平记忆存储对于任何非平凡的 Agent 记忆系统来说都是错误的默认选项。存储廉价。注意力昂贵。构建利用这种差异的系统。如果你觉得这篇文章有趣，请分享 ## 相关链接 - [Steven (Batman) Batchelor-Manning](https://x.com/S_BatMan) - [@S_BatMan](https://x.com/S_BatMan) - [681K](https://x.com/S_BatMan/status/2059377206661370009/analytics) - [@supermemory](https://x.com/@supermemory) - [升级到 Premium](https://x.com/i/premium_sign_up) - [2026年5月27日上午4:52](https://x.com/S_BatMan/status/2059377206661370009) - [681K 次观看](https://x.com/S_BatMan/status/2059377206661370009/analytics) - [查看引用](https://x.com/S_BatMan/status/2059377206661370009/quotes) --- *导出时间: 2026/5/28 09:26:21*

🔗 相关文档