# Exploring GEPA
**作者**: Quarq
**日期**: 2026-04-27T16:30:37.000Z
**来源**: [https://x.com/quarqlabs/status/2048802002877251600](https://x.com/quarqlabs/status/2048802002877251600)
---
There’s been a spike in discussion around Recursive Language Models (RLMs) and GEPA on this platform.
Both of these are trying to address the same underlying limitation: today’s LLM systems don’t manage context well.
This piece focuses on GEPA. What it is, how it works, and where it fits if you’re exploring this space.
If you are curious about RLMs alone we have covered this topic before. You can read about it here
## RLM + GEPA and Context Management
These are two distinct yet complementary research thrusts, both attacking the same fundamental problem — LLMs are passive consumers of context
The classical paradigm is "stuff everything into the context window and hoping that the model pays attention to the right parts".
This breaks in many ways :
- "Lost in the middle" degradation,
- Quadratic attention costs at scale,
- Model having zero ability to reorganize or filter what it's been given.
Context length scaling has been the industry's answer, but longer windows don't fix the fundamental passivity problem.
## GEPA: Evolving the Context You Feed In
Where RLM focuses on runtime behavior, GEPA operates before inference.
It’s about improving the prompts, instructions, and system setup ahead of time, rather than relying on the model to handle everything at runtime.
GEPA (Genetic-Pareto) is a prompt optimizer that learns from trial and error using natural language feedback.
Given a system with one or more LLM prompts, it runs the system end-to-end, sampling reasoning steps, tool calls, and outputs.
It then reflects on these traces to identify failure modes, propose prompt updates, and test improvements.
Over time, it accumulates and combines useful changes, keeping multiple high-performing variants instead of collapsing to a single solution.
Below is an example of how this optimization loop works in practice.
This figure shows an example prompt generated by GEPA for the second-hop document retrieval to be performed in a multi-hop question-answer system, along with the seed prompt it started with.
Traditional optimizers like RL or evolutionary strategies reduce everything to a single number i.e a reward.
They can tell that something failed, but not why it failed.
GEPA takes a different approach. Instead of collapsing execution into a score, it keeps the feedback in natural language.
Evaluators return Actionable Side Information (ASI), a structured diagnostic feedback about what went wrong and what could be improved.
This plays a role similar to a gradient, but expressed in text rather than numbers.
An LLM reads this feedback, identifies failure modes, and proposes targeted fixes.
To avoid converging too early on a single solution, GEPA doesn’t just optimize for one “best” prompt.
It maintains a Pareto front, a set of high-performing prompts that trade off different strengths. Instead of collapsing to a single optimum, it keeps exploring multiple directions.
This diversity helps it generalize better and reduces the risk of getting stuck in local minima.
The efficiency numbers.
Across six tasks, GEPA outperforms GRPO by 6 percentage points on average and up to 19 points, while using up to 35x fewer rollouts. It also outperforms the leading prompt optimizer MIPROv2 by over 10 percentage points, including +12 points on AIME-2025.
In practical terms: GEPA's reflection-based approach means prompts can be optimized with just 20–100 examples, making it practical for production scenarios where labeled data is expensive or domain-specific.
GRPO-style RL methods, by contrast, have consistently required anywhere from 100,000 to 512,000 rollouts in practice.
It also generalizes beyond simple system prompts.
There's an MCP adapter that optimizes tool descriptions and system prompts for Model Context Protocol servers
There's also a DSPy adapter that evolves entire programs including signatures, modules, and control flow (achieving 93% on MATH vs. 67% with basic DSPy), and a generic RAG adapter that optimizes query reformulation, context synthesis, and document reranking. [source]
RLM ♥️ GEPA
These operate at different layers of the same stack.
1. GEPA focuses on what goes into the context i.e instructions, retrieval queries, agent scaffolding. Making each token more intentional and task-specific.
2. RLM focuses on what happens at runtime, letting the model decide what to load, when, and how, instead of consuming a fixed block of text.
Together, they suggest a shift in how context is handled.
The static part of the system is optimized ahead of time (GEPA), while the dynamic part is managed during execution through recursive decomposition (RLM).
Instead of relying on ever-larger context windows, the direction here is toward selectivity — models that are better at deciding what matters and when.
The broader implication is a move away from “add more context” toward systems that actively curate and manage it.
This shift toward better context management isn’t just theoretical. It directly shapes how agent systems need to be built.
Parting note
We’re building a personal agent at @quarqlabs where the harness sits at the center.
The goal is straightforward: an agent that works out of the box, without requiring users to assemble infrastructure around it. That means treating context as a first-class problem. We’ll be opening an early beta soon.
If you want to see how this approach performs in practice, including benchmark results, you can join the waitlist.
References
https://alexzhang13.github.io/blog/2025/rlm/
https://arxiv.org/abs/2507.19457
https://gepa-ai.github.io/gepa/
https://arxiv.org/pdf/2507.19457
https://openreview.net/forum?id=RQm2KQTM5r
https://decagon.ai/blog/optimizing-gepa-for-production
## 相关链接
- [Quarq](https://x.com/quarqlabs)
- [@quarqlabs](https://x.com/quarqlabs)
- [23K](https://x.com/quarqlabs/status/2048802002877251600/analytics)
- [here](https://x.com/quarqlabs/status/2047936241078059067?s=20)
- [[source]](https://github.com/gepa-ai/gepa)
- [@quarqlabs](https://x.com/@quarqlabs)
- [waitlist](https://quarq.io/#waitlist)
- [https://alexzhang13.github.io/blog/2025/rlm/](https://alexzhang13.github.io/blog/2025/rlm/)
- [https://arxiv.org/abs/2507.19457](https://arxiv.org/abs/2507.19457)
- [https://gepa-ai.github.io/gepa/](https://gepa-ai.github.io/gepa/)
- [https://arxiv.org/pdf/2507.19457](https://arxiv.org/pdf/2507.19457)
- [https://openreview.net/forum?id=RQm2KQTM5r](https://openreview.net/forum?id=RQm2KQTM5r)
- [https://decagon.ai/blog/optimizing-gepa-for-production](https://decagon.ai/blog/optimizing-gepa-for-production)
- [Upgrade to Premium](https://x.com/i/premium_sign_up)
- [12:30 AM · Apr 28, 2026](https://x.com/quarqlabs/status/2048802002877251600)
- [23.1K Views](https://x.com/quarqlabs/status/2048802002877251600/analytics)
---
*导出时间: 2026/4/28 21:14:07*