Context Engineering
Context engineering is the discipline of designing, assembling, and managing the entire context that an LLM receives — not just the user prompt, but the system prompt, retrieved documents, tool outputs, conversation history, and memory. It is the evolution of prompt engineering for the agentic era.
In 2024, prompt engineering meant crafting clever prompts. In 2026, context engineering means architecting the full information pipeline that feeds into an LLM. The difference is like the difference between writing a good SQL query (prompt engineering) and designing the entire database schema, indexing strategy, and caching layer (context engineering).
info
| Aspect | Prompt Engineering | Context Engineering |
|---|---|---|
| Scope | Single prompt text | Entire context window (system + RAG + history + tools) |
| Focus | Wording, formatting, examples | Information architecture, relevance, token budget |
| Timeframe | One-shot interactions | Multi-turn conversations, long-running agents |
| Key challenge | Getting the right output format | Getting the right information into limited context |
| Tools | Prompt templates, few-shot examples | RAG, compression, memory, context windows, pruning |
The context window is the LLM working memory — a finite resource measured in tokens. Every token you spend on irrelevant information is a token not available for reasoning. Modern models support 128K-1M token contexts, but performance does not scale linearly with context length.
Context Window Challenges
Lost in the Middle
Models attend strongly to the beginning and end of context but poorly to the middle. Information buried at position 50,000 of a 100,000-token context is effectively invisible.
Attention Degradation
As context grows, the model attention is spread thinner. A 128K context does not mean 128K tokens of useful attention — effective attention degrades well before the hard limit.
Cost Scaling
Every input token costs money. A 100K context costs 100x more than a 1K context per request. Context engineering directly impacts your API bill.
A production agent assembles context from multiple sources. The order and priority of these sources significantly impact output quality.
| 1 | // Context assembly pipeline for a production agent |
| 2 | interface ContextLayer { |
| 3 | name: string; |
| 4 | tokens: number; |
| 5 | priority: number; // 1 = highest (keep), 5 = lowest (prune first) |
| 6 | content: string; |
| 7 | } |
| 8 | |
| 9 | function assembleContext( |
| 10 | systemPrompt: string, |
| 11 | retrievedDocs: string[], |
| 12 | toolOutputs: string[], |
| 13 | conversationHistory: string[], |
| 14 | userMessage: string, |
| 15 | maxTokens: number = 128_000 |
| 16 | ): ContextLayer[] { |
| 17 | const layers: ContextLayer[] = [ |
| 18 | // Priority 1: System prompt — always included |
| 19 | { name: "system", tokens: countTokens(systemPrompt), priority: 1, content: systemPrompt }, |
| 20 | |
| 21 | // Priority 1: Current user message — always included |
| 22 | { name: "user", tokens: countTokens(userMessage), priority: 1, content: userMessage }, |
| 23 | |
| 24 | // Priority 2: Most relevant retrieved documents |
| 25 | ...retrievedDocs.map((doc, i) => ({ |
| 26 | name: `rag-${i}`, |
| 27 | tokens: countTokens(doc), |
| 28 | priority: 2, |
| 29 | content: doc, |
| 30 | })), |
| 31 | |
| 32 | // Priority 3: Recent tool outputs (last 3) |
| 33 | ...toolOutputs.slice(-3).map((output, i) => ({ |
| 34 | name: `tool-${i}`, |
| 35 | tokens: countTokens(output), |
| 36 | priority: 3, |
| 37 | content: output, |
| 38 | })), |
| 39 | |
| 40 | // Priority 4: Recent conversation history (compressed) |
| 41 | ...compressHistory(conversationHistory).map((msg, i) => ({ |
| 42 | name: `history-${i}`, |
| 43 | tokens: countTokens(msg), |
| 44 | priority: 4, |
| 45 | content: msg, |
| 46 | })), |
| 47 | ]; |
| 48 | |
| 49 | // Sort by priority, then trim to fit token budget |
| 50 | return layers |
| 51 | .sort((a, b) => a.priority - b.priority) |
| 52 | .reduce((acc, layer) => { |
| 53 | const totalTokens = acc.reduce((sum, l) => sum + l.tokens, 0); |
| 54 | if (totalTokens + layer.tokens <= maxTokens) { |
| 55 | acc.push(layer); |
| 56 | } |
| 57 | return acc; |
| 58 | }, [] as ContextLayer[]); |
| 59 | } |
When context exceeds the token budget, you need strategies to reduce size while preserving the most important information.
Summarization Compression
Use a cheaper model to compress older conversation turns into summaries. Keep the last 3-5 turns verbatim, summarize everything before that.
| 1 | async function compressHistory( |
| 2 | messages: Message[], |
| 3 | keepRecent: number = 5 |
| 4 | ): Promise<string[]> { |
| 5 | if (messages.length <= keepRecent) { |
| 6 | return messages.map(m => m.content); |
| 7 | } |
| 8 | |
| 9 | const recent = messages.slice(-keepRecent); |
| 10 | const old = messages.slice(0, -keepRecent); |
| 11 | |
| 12 | // Summarize older messages with a cheap model |
| 13 | const summary = await openai.chat.completions.create({ |
| 14 | model: "gpt-4o-mini", |
| 15 | messages: [{ |
| 16 | role: "user", |
| 17 | content: `Summarize this conversation history in 200 words, |
| 18 | preserving key decisions, code changes, and action items: |
| 19 | |
| 20 | ${old.map(m => `${m.role}: ${m.content}`).join("\n")}` |
| 21 | }], |
| 22 | }); |
| 23 | |
| 24 | return [ |
| 25 | `[Previous conversation summary]: ${summary.choices[0].message.content}`, |
| 26 | ...recent.map(m => `${m.role}: ${m.content}`), |
| 27 | ]; |
| 28 | } |
Relevance Filtering
Score each context item by relevance to the current query using embeddings. Drop items below a threshold.
| 1 | function filterByRelevance( |
| 2 | documents: Document[], |
| 3 | query: string, |
| 4 | threshold: number = 0.7 |
| 5 | ): Document[] { |
| 6 | const queryEmbedding = embed(query); |
| 7 | return documents |
| 8 | .map(doc => ({ |
| 9 | ...doc, |
| 10 | relevance: cosineSimilarity(queryEmbedding, embed(doc.content)), |
| 11 | })) |
| 12 | .filter(doc => doc.relevance >= threshold) |
| 13 | .sort((a, b) => b.relevance - a.relevance); |
| 14 | } |
Long-running agents need memory systems that persist across conversations. Three types of memory serve different purposes:
| Memory Type | Duration | Storage | Use Case |
|---|---|---|---|
| Short-term | Current conversation | Context window | Active task state, recent turns |
| Episodic | Across sessions | Vector DB | Past interactions, user preferences, project history |
| Semantic | Permanent | Knowledge graph / DB | Facts, relationships, domain knowledge |
| 1 | // Agent memory system combining all three types |
| 2 | class AgentMemory { |
| 3 | private shortTerm: Message[] = []; |
| 4 | private episodic: VectorStore; // Pinecone, Qdrant |
| 5 | private semantic: KnowledgeGraph; // Neo4j, TypeDB |
| 6 | |
| 7 | async recall(query: string, context: ConversationContext) { |
| 8 | // 1. Short-term: always available in context window |
| 9 | const recent = this.shortTerm.slice(-10); |
| 10 | |
| 11 | // 2. Episodic: search past interactions |
| 12 | const pastEpisodes = await this.episodic.search(query, { |
| 13 | filter: { userId: context.userId, projectId: context.projectId }, |
| 14 | limit: 5, |
| 15 | }); |
| 16 | |
| 17 | // 3. Semantic: retrieve relevant facts |
| 18 | const facts = await this.semantic.query( |
| 19 | `What do we know about ${context.topic}?` |
| 20 | ); |
| 21 | |
| 22 | return { recent, pastEpisodes, facts }; |
| 23 | } |
| 24 | |
| 25 | async store(episode: Episode) { |
| 26 | // Store in episodic memory for future recall |
| 27 | await this.episodic.upsert({ |
| 28 | id: episode.id, |
| 29 | vector: embed(episode.summary), |
| 30 | metadata: { |
| 31 | userId: episode.userId, |
| 32 | timestamp: episode.timestamp, |
| 33 | topics: episode.topics, |
| 34 | }, |
| 35 | }); |
| 36 | } |
| 37 | } |
Multi-agent systems introduce unique context challenges: how much context to share between agents, how to isolate sensitive context, and how to handle context handoffs.
Context Isolation
Each subagent gets its own context window. The parent agent passes only the necessary context for the subtask — never the full conversation history. This prevents context pollution and keeps each subagent focused.
Handoff Protocol
When a subagent completes, it returns a structured summary (not the full context). The parent merges this into its own context. This is like a function returning a value vs dumping its entire stack trace.
Shared Context Store
For agents that need shared knowledge (e.g., a codebase map), use an external store (vector DB, knowledge graph) rather than passing large contexts between agents. Each agent queries the store as needed.
Community
Get help on Slack, Discord or VIP
Stuck on a guide? Join the community and ask.