# Your Ultimate Guide to Attention: Mechanism, QKV, and KV Cache
**作者**: Turing Post
**日期**: 2026-05-14T22:49:00.000Z
**来源**: [https://x.com/TheTuringPost/status/2055057820110725127](https://x.com/TheTuringPost/status/2055057820110725127)
---
Learn how attention in AI works, from queries, keys, and values to KV cache, self-attention, and modern approaches
## The history of attention: from translation to Transformers
Before attention became the center of the Transformer described in one of the most influential papers in AI – Attention is All You Need from Google, it had appeared three years earlier as a solution for a practical problem in neural machine translation.
Early encoder–decoder models encoded an entire source sentence into a single fixed-length vector – a dense numerical representation that captures the sentence’s meaning – and decoded the translation from that compressed summary. Dzmitry Bahdanau, KyungHyun Cho and Yoshua Bengio in their Neural Machine Translation by Jointly Learning to Align and Translate paper (2014) argued that this fixed-length context vector created a bottleneck, because the model had to compress all relevant information into one representation. They proposed a model that could align source and target words during decoding, letting the decoder softly search over different parts of the input sentence while generating each new word. The decoder dynamically computes a context vector as a weighted combination of source annotations, focusing on the most relevant parts of the input for the current prediction. This was the moment when context became adaptive and target-dependent.
Image credit: @karpathy’s post
Then in 2015, Stanford researchers published Effective Approaches to Attention-based Neural Machine Translation by Minh-Thang Luong, Hieu Pham and Christopher D. Manning that extended this idea with practical attention mechanisms. They introduced global attention, where the decoder attends to all source positions, and local attention, where it focuses only on a smaller window of source words at each step. Plus, they proposed input-feeding approach, where the model feeds earlier attention information back into later steps so it can remember what parts of the source it has already focused on.
Image Credit: Global attention mechanism in Effective Approaches to Attention-based Neural Machine Translation
Image Credit: Local attention window Effective Approaches to Attention-based Neural Machine Translation
Then finally came the main breakthrough – the architecture built around attention itself. Yes, it is about Transformers and Attention Is All You Need paper (2017) by A. Vaswani et al. The Google researchers removed recurrence and convolutions and made self-attention layers the core foundation of Transformers. They also introduced the formulation of attention and the language of queries, keys, and values that became the canonical way to describe how models retrieve and combine contextual information.
This is just the story of how attention became the centerpiece of modern models. It’s not enough to understand the basics. Let’s go through every part of the workflow step-by-step.
## How attention works
## From embeddings to contextual representations
The usual Transformer model starts with token embeddings (dense numerical vector representations of tokens that encode semantic and syntactic information in a continuous vector space), combined with positional encodings, because the word order matters a lot in this architecture. At this stage, these vectors are still not deeply contextual. They have information about token identity, and with positional encoding they “know” something about location, but they do not yet define what matters to what.
So these “token embeddings + positional encodings” vectors serve as the inputs to the attention mechanism, entering the Transformer’s attention layers.
Image Credit: Transformer architecture showing self-attention layers and positional encodings, Attention is All You Need
There, each token representation is transformed into queries, keys, and values. These are linear projections of embeddings or hidden states – context-aware vector representations of tokens as they are processed layer by layer inside the model. Embeddings provide the raw material from which attention builds its comparisons. Without embeddings, the mechanism would have nothing meaningful to compare or route through the network.
## Queries, keys, and values: the QKV mechanism
Here is the essential vocabulary that everyone needs to know to understand the full mechanism and attention formulation.
After embeddings and positional information enter the model, each token vector is multiplied by learned weight matrices to produce three different versions of itself:
- A query (Q) is what the current token is looking for. It is the signal used to compare against other tokens and determine which ones may be relevant.
- A key (K) is information that every token exposes about itself so others can decide whether to attend to it. It acts like a tag or description attached to a token.
- A value (V) represents the information a token contributes if attention selects it. It is the actual content being passed along, like a payload. A weighted combination of value vectors becomes the attention output.
But why can’t we just use ordinary embeddings, and why do we need to split them into these three vectors?
Read further here: https://www.turingpost.com/p/your-ultimate-guide-to-attention-mechanism-qkv-and-kv-cache
## 相关链接
- [Turing Post](https://x.com/TheTuringPost)
- [@TheTuringPost](https://x.com/TheTuringPost)
- [Attention is All You Need](https://arxiv.org/abs/1706.03762)
- [Neural Machine Translation by Jointly Learning to Align and Translate](https://arxiv.org/abs/1409.0473)
- [@karpathy](https://x.com/@karpathy)
- [Effective Approaches to Attention-based Neural Machine Translation](https://arxiv.org/abs/1508.04025)
- [Upgrade to Premium](https://x.com/i/premium_sign_up)
- [6:49 AM · May 15, 2026](https://x.com/TheTuringPost/status/2055057820110725127)
- [3,041 Views](https://x.com/TheTuringPost/status/2055057820110725127/analytics)
---
*导出时间: 2026/5/15 12:11:08*
---
## 中文翻译
# 注意力终极指南：机制、QKV 和 KV 缓存
**作者**: Turing Post
**日期**: 2026-05-14T22:49:00.000Z
**来源**: [https://x.com/TheTuringPost/status/2055057820110725127](https://x.com/TheTuringPost/status/2055057820110725127)
---
了解 AI 中的注意力机制是如何工作的，从查询、键和值，到 KV 缓存、自注意力以及现代方法。
## 注意力的历史：从翻译到 Transformer
在注意力成为谷歌极具影响力的论文《Attention is All You Need》中描述的 Transformer 核心之前，它早在三年前就已出现，成为神经机器翻译中一个实际问题的解决方案。
早期的编码器-解码器模型将整个源句子编码成单个固定长度的向量——一种捕捉句子含义的密集数字表示——并从该压缩摘要中解码翻译。Dzmitry Bahdanau、KyungHyun Cho 和 Yoshua Bengio 在他们的论文《通过联合学习对齐和翻译进行神经机器翻译》（2014）中指出，这种固定长度的上下文向量造成了瓶颈，因为模型必须将所有相关信息压缩到一个表示中。他们提出了一种模型，可以在解码期间对源单词和目标单词进行对齐，让解码器在生成每个新单词时对输入句子的不同部分进行软搜索。解码器动态计算一个上下文向量，作为源注释的加权组合，专注于与当前预测最相关的输入部分。这就是上下文变得具有适应性和依赖目标的时刻。
图片出处：@karpathy 的帖子
随后在 2015 年，斯坦福的研究人员发表了 Minh-Thang Luong、Hieu Pham 和 Christopher D. Manning 撰写的《基于注意力的神经机器翻译的有效方法》，通过实用的注意力机制扩展了这一想法。他们引入了全局注意力，即解码器关注所有源位置；以及局部注意力，即解码器在每一步只关注一小部分源单词窗口。此外，他们提出了输入馈送方法，即模型将较早的注意力信息反馈到后续步骤中，以便它可以记住已经关注了源句子的哪些部分。
图片出处：Effective Approaches to Attention-based Neural Machine Translation 中的全局注意力机制
图片出处：Effective Approaches to Attention-based Neural Machine Translation 中的局部注意力窗口
然后终于迎来了主要的突破——围绕注意力本身构建的架构。是的，这就是关于 Transformer 和 A. Vaswani 等人的《Attention Is All You Need》论文（2017）。谷歌的研究人员移除了循环和卷积，使自注意力层成为 Transformer 的核心基础。他们还引入了注意力的公式以及查询、键和值的概念，这成为描述模型如何检索和组合上下文信息的规范方式。
这只是关于注意力如何成为现代模型核心的故事。仅了解基础知识是不够的。让我们逐步深入了解工作流程的每一个部分。
## 注意力是如何工作的
## 从嵌入到上下文表示
通常的 Transformer 模型以词元嵌入开始（对词元进行编码的密集数字向量表示，在连续向量空间中捕获语义和句法信息），并结合位置编码，因为词序在这个架构中非常重要。在这个阶段，这些向量还没有深入地结合上下文。它们拥有关于词元身份的信息，并且通过位置编码，它们“知道”一些关于位置的信息，但它们尚未定义什么对什么重要。
因此，这些“词元嵌入 + 位置编码”向量作为注意力机制的输入，进入 Transformer 的注意力层。
图片出处：展示自注意力层和位置编码的 Transformer 架构，Attention is All You Need
在那里，每个词元表示被转换为查询、键和值。这些是嵌入或隐藏状态的线性投影——词元在模型内部逐层处理时的上下文感知向量表示。嵌入提供了注意力构建其比较的原始材料。没有嵌入，该机制将没有任何有意义的内容可供比较或通过网络路由。
## 查询、键和值：QKV 机制
这是理解完整机制和注意力公式每个人都必须知道的核心词汇。
在嵌入和位置信息进入模型后，每个词元向量都会乘以学习的权重矩阵以生成其自身的三个不同版本：
- 查询是当前词元正在寻找的内容。它是用于与其他词元进行比较以确定哪些词元可能是相关的信号。
- 键是每个词元暴露的关于其自身的信息，以便其他人可以决定是否关注它。它就像附加在词元上的标签或描述。
- 值表示如果注意力选择了某个词元，该词元所贡献的信息。它是被传递的实际内容，就像有效载荷。值向量的加权组合成为注意力输出。
但是，为什么我们不能只使用普通的嵌入，而为什么需要将它们分成这三个向量呢？
请在此处继续阅读：https://www.turingpost.com/p/your-ultimate-guide-to-attention-mechanism-qkv-and-kv-cache
## 相关链接
- [Turing Post](https://x.com/TheTuringPost)
- [@TheTuringPost](https://x.com/TheTuringPost)
- [Attention is All You Need](https://arxiv.org/abs/1706.03762)
- [Neural Machine Translation by Jointly Learning to Align and Translate](https://arxiv.org/abs/1409.0473)
- [@karpathy](https://x.com/@karpathy)
- [Effective Approaches to Attention-based Neural Machine Translation](https://arxiv.org/abs/1508.04025)
- [升级到 Premium](https://x.com/i/premium_sign_up)
- [2026年5月15日上午 6:49](https://x.com/TheTuringPost/status/2055057820110725127)
- [3,041 次观看](https://x.com/TheTuringPost/status/2055057820110725127/analytics)
---
*导出时间: 2026/5/15 12:11:08*