A Bigger Context Window Won't Save You

Part 2 · Beyond the Prompt series

When an AI agent keeps getting your codebase wrong, the answer is to decide, deliberately, what it gets to see. A smarter model or a longer context window rarely helps.

When an agent starts flailing on real code, the reflex is to give it more. Paste in more files. Upgrade to the model with the million-token window. Dump the whole repository in and let it sort things out. It feels like the obvious move, and it mostly backfires. An agent handed everything attends to nothing in particular, and the one detail that actually mattered gets buried under thousands that didn't.

The skill that fixes this is context engineering: managing what enters the model's window as the scarce, decisive resource it is. What follows is what that looks like in practice.

The window is a budget

A language model starts every request from nothing. Everything it knows about your task, your instructions, the relevant code, the conventions, the prior steps, has to fit inside one finite context window. That window is a budget, and it is far smaller than your codebase.

The counterintuitive part is that filling it doesn't help. Models attend unevenly to long inputs, and detail parked in the middle of a huge context tends to get lost. Past a point, more tokens buy you worse performance, because whatever you cared about is now surrounded by things you didn't. So the goal is to get the right context in and keep everything else out. A longer window raises the ceiling on what you can include. It does nothing to tell you what you should.

Persistent context: the things that are always true

Some context should never have to be rediscovered. Your architecture, your stack, the code conventions, the security non-negotiables, the modules a new feature should reuse: these are true on every task, and making the agent re-infer them each time is both wasteful and unreliable.

The move is to write them down once as durable, always-loaded ground truth, often called a constitution, and anchor the agent to it on every request. An agent that carries your conventions and your hard rules into each feature without being reminded behaves consistently across a whole project instead of improvising a new style every session. This is the cheapest, highest-leverage thing most teams aren't doing. A page of stable, authoritative project rules outperforms paragraphs of per-task instructions, because it governs everything rather than just the task in front of you.

Retrieval: finding the handful of files that matter

Out of thousands of files, only a handful matter for the change in front of you, and the model has no way to know which. Surfacing those few is a retrieval problem, and it is the heart of working in a real codebase.

The reinvention failure makes this concrete. You ask for a feature, and the agent writes its own date helper, its own validation, its own API client, blind to the three you already have. The existing code simply never entered the window, so as far as the agent could tell, it didn't exist. The fix is to point the agent at the exemplar to follow and name the modules to reuse, explicitly, so the right context is present at the moment it decides. Anchor the work to what already exists and the agent extends your system. Leave it guessing and you get a second, foreign way of doing the same thing.

Hygiene: keep the window clean as you go

Context engineering keeps going for as long as the task does. As an agent works, abandoned attempts, dead-end reasoning, and stale output pile up in the window, and that residue quietly degrades everything that comes after, because the model is now reasoning over its own discarded mistakes as if they were still relevant.

So you prune. You clear what's no longer load-bearing, and when a thread is genuinely exhausted, you start fresh rather than dragging its full history forward. A clean window beats a full one almost every time. Treat the context as something you maintain as you go, and quality stops sliding as a session runs long.

It's half the method

Context engineering is one half of getting real work out of an AI agent: control what it sees. The other half is the specification: control what it builds, by giving it a target it cannot misread. Apart, each helps. Together they give you an agent you can actually direct, because it is both well-informed and aimed at the right thing.