# Exploring GEPA
**作者**: Quarq
**日期**: 2026-04-27T16:30:37.000Z
**来源**: [https://x.com/quarqlabs/status/2048802002877251600](https://x.com/quarqlabs/status/2048802002877251600)
---
There’s been a spike in discussion around Recursive Language Models (RLMs) and GEPA on this platform.
Both of these are trying to address the same underlying limitation: today’s LLM systems don’t manage context well.
This piece focuses on GEPA. What it is, how it works, and where it fits if you’re exploring this space.
If you are curious about RLMs alone we have covered this topic before. You can read about it here
## RLM + GEPA and Context Management
These are two distinct yet complementary research thrusts, both attacking the same fundamental problem — LLMs are passive consumers of context
The classical paradigm is "stuff everything into the context window and hoping that the model pays attention to the right parts".
This breaks in many ways :
- "Lost in the middle" degradation,
- Quadratic attention costs at scale,
- Model having zero ability to reorganize or filter what it's been given.
Context length scaling has been the industry's answer, but longer windows don't fix the fundamental passivity problem.
## GEPA: Evolving the Context You Feed In
Where RLM focuses on runtime behavior, GEPA operates before inference.
It’s about improving the prompts, instructions, and system setup ahead of time, rather than relying on the model to handle everything at runtime.
GEPA (Genetic-Pareto) is a prompt optimizer that learns from trial and error using natural language feedback.
Given a system with one or more LLM prompts, it runs the system end-to-end, sampling reasoning steps, tool calls, and outputs.
It then reflects on these traces to identify failure modes, propose prompt updates, and test improvements.
Over time, it accumulates and combines useful changes, keeping multiple high-performing variants instead of collapsing to a single solution.
Below is an example of how this optimization loop works in practice.
This figure shows an example prompt generated by GEPA for the second-hop document retrieval to be performed in a multi-hop question-answer system, along with the seed prompt it started with.
Traditional optimizers like RL or evolutionary strategies reduce everything to a single number i.e a reward.
They can tell that something failed, but not why it failed.
GEPA takes a different approach. Instead of collapsing execution into a score, it keeps the feedback in natural language.
Evaluators return Actionable Side Information (ASI), a structured diagnostic feedback about what went wrong and what could be improved.
This plays a role similar to a gradient, but expressed in text rather than numbers.
An LLM reads this feedback, identifies failure modes, and proposes targeted fixes.
To avoid converging too early on a single solution, GEPA doesn’t just optimize for one “best” prompt.
It maintains a Pareto front, a set of high-performing prompts that trade off different strengths. Instead of collapsing to a single optimum, it keeps exploring multiple directions.
This diversity helps it generalize better and reduces the risk of getting stuck in local minima.
The efficiency numbers.
Across six tasks, GEPA outperforms GRPO by 6 percentage points on average and up to 19 points, while using up to 35x fewer rollouts. It also outperforms the leading prompt optimizer MIPROv2 by over 10 percentage points, including +12 points on AIME-2025.
In practical terms: GEPA's reflection-based approach means prompts can be optimized with just 20–100 examples, making it practical for production scenarios where labeled data is expensive or domain-specific.
GRPO-style RL methods, by contrast, have consistently required anywhere from 100,000 to 512,000 rollouts in practice.
It also generalizes beyond simple system prompts.
There's an MCP adapter that optimizes tool descriptions and system prompts for Model Context Protocol servers
There's also a DSPy adapter that evolves entire programs including signatures, modules, and control flow (achieving 93% on MATH vs. 67% with basic DSPy), and a generic RAG adapter that optimizes query reformulation, context synthesis, and document reranking. [source]
RLM ♥️ GEPA
These operate at different layers of the same stack.
1. GEPA focuses on what goes into the context i.e instructions, retrieval queries, agent scaffolding. Making each token more intentional and task-specific.
2. RLM focuses on what happens at runtime, letting the model decide what to load, when, and how, instead of consuming a fixed block of text.
Together, they suggest a shift in how context is handled.
The static part of the system is optimized ahead of time (GEPA), while the dynamic part is managed during execution through recursive decomposition (RLM).
Instead of relying on ever-larger context windows, the direction here is toward selectivity — models that are better at deciding what matters and when.
The broader implication is a move away from “add more context” toward systems that actively curate and manage it.
This shift toward better context management isn’t just theoretical. It directly shapes how agent systems need to be built.
Parting note
We’re building a personal agent at @quarqlabs where the harness sits at the center.
The goal is straightforward: an agent that works out of the box, without requiring users to assemble infrastructure around it. That means treating context as a first-class problem. We’ll be opening an early beta soon.
If you want to see how this approach performs in practice, including benchmark results, you can join the waitlist.
References
https://alexzhang13.github.io/blog/2025/rlm/
https://arxiv.org/abs/2507.19457
https://gepa-ai.github.io/gepa/
https://arxiv.org/pdf/2507.19457
https://openreview.net/forum?id=RQm2KQTM5r
https://decagon.ai/blog/optimizing-gepa-for-production
## 相关链接
- [Quarq](https://x.com/quarqlabs)
- [@quarqlabs](https://x.com/quarqlabs)
- [23K](https://x.com/quarqlabs/status/2048802002877251600/analytics)
- [here](https://x.com/quarqlabs/status/2047936241078059067?s=20)
- [[source]](https://github.com/gepa-ai/gepa)
- [@quarqlabs](https://x.com/@quarqlabs)
- [waitlist](https://quarq.io/#waitlist)
- [https://alexzhang13.github.io/blog/2025/rlm/](https://alexzhang13.github.io/blog/2025/rlm/)
- [https://arxiv.org/abs/2507.19457](https://arxiv.org/abs/2507.19457)
- [https://gepa-ai.github.io/gepa/](https://gepa-ai.github.io/gepa/)
- [https://arxiv.org/pdf/2507.19457](https://arxiv.org/pdf/2507.19457)
- [https://openreview.net/forum?id=RQm2KQTM5r](https://openreview.net/forum?id=RQm2KQTM5r)
- [https://decagon.ai/blog/optimizing-gepa-for-production](https://decagon.ai/blog/optimizing-gepa-for-production)
- [Upgrade to Premium](https://x.com/i/premium_sign_up)
- [12:30 AM · Apr 28, 2026](https://x.com/quarqlabs/status/2048802002877251600)
- [23.1K Views](https://x.com/quarqlabs/status/2048802002877251600/analytics)
---
*导出时间: 2026/4/28 21:14:07*
---
## 中文翻译
# 探索 GEPA
**作者**： Quarq
**日期**： 2026-04-27T16:30:37.000Z
**来源**： [https://x.com/quarqlabs/status/2048802002877251600](https://x.com/quarqlabs/status/2048802002877251600)
---
最近，关于递归语言模型（RLM）和 GEPA 的讨论在这个平台上激增。
这两者都在试图解决同一个根本性局限：当今的 LLM 系统无法很好地管理上下文。
本文聚焦于 GEPA：它是什么、如何工作，以及如果你正在探索这个领域，它适用于哪些场景。
如果你只对 RLM 感兴趣，我们之前已经介绍过这个话题。你可以在这里阅读相关内容。
## RLM + GEPA 与上下文管理
这是两个截然不同却又互补的研究方向，都在攻克同一个根本问题——LLM 是上下文的被动消费者。
传统范式是“把所有东西都塞进上下文窗口，指望模型注意到正确的部分”。
这种做法在很多方面都会失效：
- “迷失在中间”退化问题，
- 规模扩大时二次方的注意力成本，
- 模型完全没有能力重组或筛选它接收到的内容。
上下文长度扩展一直是行业的应对方案，但更长的窗口并不能解决根本的被动性问题。
## GEPA：进化你输入的上下文
RLM 关注的是运行时行为，而 GEPA 则在推理之前运作。
它旨在提前改进提示词、指令和系统设置，而不是依赖模型在运行时处理一切。
GEPA（Genetic-Pareto，遗传-帕累托）是一种提示词优化器，它利用自然语言反馈从试错中学习。
给定一个包含一个或多个 LLM 提示词的系统，它会端到端地运行该系统，采样推理步骤、工具调用和输出。
然后，它会对这些轨迹进行反思，以识别失败模式、提出提示词更新建议，并测试改进效果。
随着时间推移，它会积累并组合有用的变更，保留多个高性能变体，而不是收敛到单一解决方案。
下面是一个展示这种优化循环在实践中如何运作的例子。
这张图展示了 GEPA 为多跳问答系统中要执行的第二次跳转文档检索生成的一个示例提示词，以及它最初使用的种子提示词。
传统的优化器（如强化学习或进化策略）会把一切都简化为单一数字，即奖励。
它们能告诉你某件事失败了，却无法说明为什么失败。
GEPA 采取了不同的方法。它不会把执行过程压缩成一个分数，而是以自然语言的形式保留反馈。
评估器返回可操作的辅助信息（Actionable Side Information，ASI），这是一种结构化的诊断反馈，说明哪里出了问题以及可以改进什么。
它的作用类似于梯度，只不过是以文本而非数字的形式表达。
一个 LLM 读取这些反馈，识别失败模式，并提出针对性的修复方案。
为了避免过早收敛到单一解决方案，GEPA 并不仅仅优化一个“最佳”提示词。
它维护一个帕累托前沿——一组权衡不同优势的高性能提示词。它不会坍缩到单一最优解，而是持续探索多个方向。
这种多样性帮助它更好地泛化，并降低了陷入局部最优的风险。
效率数据。
在六项任务中，GEPA 平均比 GRPO 高出 6 个百分点，最高达 19 个百分点，同时使用的 rollout 次数最多可减少 35 倍。它还比领先的提示词优化器 MIPROv2 高出 10 个百分点以上，在 AIME-2025 上高出 12 个百分点。
从实际角度来看：GEPA 基于反思的方法意味着只需 20 到 100 个样本就可以优化提示词，这使其非常适用于标注数据昂贵或领域特定的生产场景。
相比之下，GRPO 风格的强化学习方法在实践中一贯需要 10 万到 51.2 万次 rollout。
它还能泛化到简单的系统提示词之外。
有一个 MCP 适配器，可以为模型上下文协议服务器优化工具描述和系统提示词。
还有一个 DSPy 适配器，可以进化包含签名、模块和控制流的完整程序（在 MATH 上达到 93%，而基础 DSPy 为 67%），以及一个通用 RAG 适配器，可以优化查询重写、上下文综合和文档重排序。[来源]
RLM ♥️ GEPA
它们作用于同一技术栈的不同层面。
1. GEPA 关注进入上下文的内容，即指令、检索查询、智能体脚手架。让每个 token 更具意图性、更贴合任务。
2. RLM 关注运行时发生的事情，让模型决定加载什么、何时加载以及如何加载，而不是消费固定的文本块。
两者结合，预示着上下文处理方式的转变。
系统的静态部分被提前优化（GEPA），而动态部分则在执行过程中通过递归分解来管理（RLM）。
与其依赖越来越大的上下文窗口，这个方向指向的是选择性——模型更擅长判断什么重要、何时重要。
更广泛的含义是，从“增加更多上下文”转向主动策划和管理上下文的系统。
这种向更好的上下文管理的转变不仅仅是理论上的。它直接决定了智能体系统的构建方式。
结语
我们正在 @quarqlabs 构建一个以 harness 为核心的个人智能体。
目标很简单：一个开箱即用的智能体，无需用户围绕它搭建基础设施。这意味着要把上下文当作一等公民问题来对待。我们很快将开放早期测试版。
如果你想看看这种方法在实践中的表现，包括基准测试结果，可以加入等待名单。
参考资料
https://alexzhang13.github.io/blog/2025/rlm/
https://arxiv.org/abs/2507.19457
https://gepa-ai.github.io/gepa/
https://arxiv.org/pdf/2507.19457
https://openreview.net/forum?id=RQm2KQTM5r
https://decagon.ai/blog/optimizing-gepa-for-production
## 相关链接
- [Quarq](https://x.com/quarqlabs)
- [@quarqlabs](https://x.com/quarqlabs)
- [23K](https://x.com/quarqlabs/status/2048802002877251600/analytics)
- [here（这里）](https://x.com/quarqlabs/status/2047936241078059067?s=20)
- [[source（源代码）]](https://github.com/gepa-ai/gepa)
- [@quarqlabs](https://x.com/@quarqlabs)
- [waitlist（等候名单）](https://quarq.io/#waitlist)
- [https://alexzhang13.github.io/blog/2025/rlm/](https://alexzhang13.github.io/blog/2025/rlm/)
- [https://arxiv.org/abs/2507.19457](https://arxiv.org/abs/2507.19457)
- [https://gepa-ai.github.io/gepa/](https://gepa-ai.github.io/gepa/)
- [https://arxiv.org/pdf/2507.19457](https://arxiv.org/pdf/2507.19457)
- [https://openreview.net/forum?id=RQm2KQTM5r](https://openreview.net/forum?id=RQm2KQTM5r)
- [https://decagon.ai/blog/optimizing-gepa-for-production](https://decagon.ai/blog/optimizing-gepa-for-production)
- [Upgrade to Premium（升级至高级版）](https://x.com/i/premium_sign_up)
- [12:30 AM · Apr 28, 2026（2026年4月28日 上午12:30）](https://x.com/quarqlabs/status/2048802002877251600)
- [23.1K Views（2.31万次浏览）](https://x.com/quarqlabs/status/2048802002877251600/analytics)
---
*导出时间：2026/4/28 21:14:07*
