What Context Engineering Is
Context engineering is the set of strategies for curating and maintaining the optimal set of tokens in a model's context window during inference. That includes the system prompt, tool definitions, retrieved documents, message history, and everything else that shares the window. Andrej Karpathy, who popularized the term, described it as "the art of filling the context window with exactly the right information at the right time." Anthropic's engineering guidance frames it as the natural successor to prompt engineering once you move from single-turn prompts to multi-turn, long-horizon agents.
Here is the claim this guide defends: context engineering is not a new discipline that replaces what you already know. It is a new name for a working set of established patterns — retrieval, memory, compaction, routing, isolation — applied with one shared question: what deserves to be in the window right now, and what does not?
Prompt engineering did not die. It became the smallest member of a larger family. The exact phrasing of an instruction is now a second-order variable; the first-order variable is which tokens are present at all.
Why This Became the Job
Two production realities forced the shift.
Context rot. Every frontier model gets measurably worse as its context grows — well before the window is full. Instructions buried in the middle of a long context get ignored, retrieved passages compete with each other, and stale tool results crowd out the current task. Million-token windows did not fix this; they made it easier to create. The failure mode of most production agents is not a weak model. It is a polluted context.
Long-horizon work. A single prompt is stateless. An agent that works for minutes or hours accumulates message history, tool results, and intermediate artifacts. Without a deliberate policy for what to keep, summarize, externalize, or drop, the context fills with the history of the work instead of the state of the work — and quality degrades exactly when the task gets hard.
The response to both is the same: treat the context window as a budget, not a container. Every token must earn its place.
The Four Strategies
Practitioners have converged on four core strategies — select, write, compress, isolate. None of them is new. Each one is implemented by patterns already in this catalog.
Select: put the right tokens in
Selection is deciding what enters the window from a corpus that is far larger than the window. This is the retrieval family:
- Basic RAG is the canonical select move: retrieve relevant passages, inject them, generate from them.
- Semantic Indexing and Hybrid Retrieval determine how well selection works — matching by meaning, and combining lexical with semantic signals.
- Retrieval Refinement reranks and filters candidates so that only the passages that survive scrutiny spend budget.
- Agentic RAG and Deep Search make selection iterative: the model decides what to look for next based on what it has seen.
- Few-Shot Prompting is selection applied to examples — choosing which demonstrations, not which documents, occupy the window.
- Semantic Router selects at one level higher: which prompt, toolset, or downstream pipeline a request should reach, so each context stays narrow.
Write: persist context outside the window
Writing is moving information out of the window into durable storage so it can be selected back in later, instead of riding along in every request:
- Conversation Memory manages what the current session carries forward turn to turn.
- Long-Term Memory persists facts, preferences, and outcomes across sessions, retrieved on demand rather than replayed in full.
- Plan-and-Execute writes the plan itself as an external artifact — the agent consults a compact plan instead of re-deriving intent from a long transcript.
Compress: make what stays smaller
Compression keeps information present while shrinking its footprint:
- Summarization inside Conversation Memory — periodically compacting older turns into a summary — is the workhorse. The trade-off is real: compaction loses detail, so decide what must survive it (decisions, constraints, open questions) before you compress.
- Prompt Chaining compresses by construction: each step passes forward a distilled output, not its full working context.
- Prompt Caching is compression's economic twin. It does not shrink the context, but it makes a stable prefix nearly free — which changes what you can afford to keep. A well-engineered context has a long cached prefix and a short volatile tail.
Isolate: split context across boundaries
Isolation prevents one concern's tokens from polluting another's window:
- Multi-Agent Collaboration gives each subagent its own window. A researcher subagent can burn fifty thousand tokens reading files and return a two-hundred-token conclusion to the orchestrator.
- Code Execution offloads computation entirely: instead of reasoning over a thousand rows in context, the model writes code that processes them outside the window and returns the result.
- Tool Calling and MCP are just-in-time isolation — capabilities and data stay behind an interface and enter the context only when invoked, rather than being preloaded.
The Supporting Cast
Two more groups of patterns do not manage the window directly but decide whether the managed context actually works:
- Verification. Grounded Generation enforces that answers trace to the selected context; Self-Check and LLM-as-Judge catch the cases where the model ignored its context or leaned on stale memory.
- Economics. Model Router and Cascading match context cost to task difficulty — a compact, well-selected context is what makes a smaller model viable in the first place.
Setting a Context Budget
The discipline that ties the four strategies together is a budget. Before optimizing anything, measure what a typical request actually spends: system prompt, tool definitions, retrieved content, history, scratch. Then set targets per section and enforce them in code, not in hope.
A practical starting policy:
- Fixed prefix, aggressively cached. System instructions and tool definitions go first, change rarely, and sit under a cache boundary (Prompt Caching).
- Selection over inclusion. Nothing from a corpus enters the window without passing retrieval and reranking (Retrieval Refinement). "It might be relevant" is not a reason; it is the failure mode.
- Compact on a trigger, not a schedule. When history crosses a threshold, summarize the oldest span and keep decisions, constraints, and open questions verbatim (Conversation Memory).
- Externalize state. Plans, findings, and long artifacts live in files or memory stores and are referenced, not replayed (Long-Term Memory, Plan-and-Execute).
- Isolate the expensive work. Anything that requires reading widely gets a subagent or a code sandbox, and only conclusions come back (Multi-Agent Collaboration, Code Execution).
Then evaluate. Context engineering without measurement regresses into vibes: build a small evaluation set that includes long-context cases, and check that answers still trace to the tokens you selected (LLM-as-Judge).
Where to Go Next
If you are new to the catalog, the shortest useful path through it in context-engineering order is: Basic RAG for selection, Conversation Memory for writing and compression, Prompt Caching for the economics, and Multi-Agent Collaboration for isolation. Everything else in the catalog refines one of those four moves.