The Best Prompt in the World Can't Answer From Information That Got Truncated

Prompt engineering tells the model what to do. Context engineering determines whether the model has what it needs to do it. In production agents, that second problem covers most of the token surface and a large share of the failure surface.

The failure behind most agent postmortems

A perfect instruction cannot recover evidence that the model never saw.

That is the quiet failure behind many production agents. The prompt is clean. The model is capable. The task is reasonable. But the relevant memory was truncated, the tool output buried the signal, or the retrieval step filled the window with plausible noise. The system doesn't fail because the wording was weak. It fails because the information environment was.

Prompt engineering optimizes the instruction. Context engineering optimizes everything the instruction operates on: memory, history, retrieved documents, tool outputs, system state, and the rules that decide which of those tokens reach the model at all. This post is the case, with evidence, for why that second discipline has become the largest source of reliability issues in production agents.

June 2025: the vocabulary catches up

Sometimes a field renames itself in public. In mid-2025, Cognition's Walden Yan published a widely read essay on building agents, drawing a line under the era of prompt-first thinking. Prompt engineering, he argued, was the term for formatting your task well for an LLM chatbot. Context engineering was the next level: doing that work automatically, inside a dynamic system, and it was effectively the number-one job of engineers building AI agents.

Days later, Shopify CEO Tobi Lütke wrote that he preferred "context engineering" because it better describes the core skill: the art of providing all the context needed for the task to be plausibly solvable by the LLM. Andrej Karpathy gave it his "+1," pointing out that people associate prompts with the short task descriptions of day-to-day chat. At the same time, every industrial-strength LLM app lives or dies by the delicate art and science of filling the context window.

Karpathy also supplied the mental model that stuck: the LLM is the CPU, the context window is RAM, and context engineering is deciding what gets loaded into that limited working memory.

A year on, the naming debate matters less than the architecture. The research literature now treats context engineering as a broader discipline, with prompt design as one component; a 1,400-paper academic survey formalizes it as the systematic optimization of the information payloads LLMs receive. Prompting happens inside the window. Context engineering decides what fills it.

To be clear: prompt engineering still matters

This is not a takedown. A sloppy system prompt produces a sloppy agent, and the craft of clear instructions, good examples, and well-chosen structure is real. In healthy systems, both disciplines are present, and the system prompt is itself part of the context, so the two nest rather than compete.

The claim we're making concerns leverage and is specific to production and agentic systems. In a single-turn chat, the prompt and the context are nearly identical, so comparing the disciplines is meaningless. The moment a system runs across turns, tools, and memory, the ratio flips. The prompt is a few hundred tokens you write once. The context consists of tens of thousands of tokens assembled dynamically on every single call: conversation history, persistent memory, retrieved documents, and tool outputs, which are frequently the largest.

Impact scales with surface area. One of these things is a sentence. The other is the system.

Why context is where systems break

If we look at the deeper reasons, we see three pillars of evidence, pointing the same way.

First, context degrades models in measurable and universal ways. Chroma's "context rot" research tested 18 frontier models, including GPT-4.1, Claude 4, and Gemini 2.5, and found that every one of them becomes less reliable as input length increases, even on simple tasks, irrespective of the window length. The decline is continuous, not a fall-off, and it accelerates when relevant information sits among semantically similar distractors. This compounds with the well-documented lost-in-the-middle effect: models attend well to the start and end of a context and poorly to the middle. Anthropic's engineering guidance draws the practical conclusion that models have a finite attention budget, and that good context engineering means finding the smallest set of high-signal tokens that maximize the likelihood of the desired outcome. No amount of prompt wordsmithing can fix an attention problem caused by everything around the prompt.

Second, context failures account for a large share of what appears to be model failure. When an agent forgets the one thing it was supposed to remember, the postmortem usually blames the model. But teams shipping real agents keep finding the same thing underneath: the fact that got truncated, the tool output that buried the signal and the stale memory that contradicted the fresh one. Independent benchmark work points in the same direction; failure analyses of coding agents rank context loss and poor exploration among the dominant failure modes. Agents fail for other reasons, too, such as planning errors and tool misuse. Still, a large share of what gets diagnosed as "the model isn't smart enough" is an information environment problem wearing a model problem's clothes.

Third, the trend line favors context. As models improve, many teams find they need less brittle prompt phrasing and more reliable information assembly; newer models forgive imperfect wording in ways they don't forgive missing evidence. The marginal returns to clever phrasing may shrink. The need to decide what evidence enters the window does not.

The asymmetry that settles it

If you want the whole argument in one observation, it's this: the two disciplines fail in different ways.

Good context selection can often rescue an ordinary prompt. The model, handed the right facts and a clear enough task, can usually muddle through imperfect phrasing. But no prompt, however brilliant, can rescue a context window in which the needed fact was truncated or drowned out by noise. You cannot phrase your way to information that the model doesn't have.

Within a single call, one failure is recoverable. The other is terminal. That asymmetry is why the disciplines aren't equals.

It's also why TokenOps starts with context selection rather than prompt templates. The highest-leverage decision is not how to phrase the instruction. It is the evidence that the instruction operates on.

We measured it

At Nasiko, we didn't want to leave this at the level of argument, so our research team ran the controlled version of the experiment. In our paper on the PACMS method, Submodular Context Selection as a Pluggable Engine for LLM Agents (arXiv:2606.20047), the prompt was held constant: same instructions, same questions, same readers. The only variable was the context selection policy that determined which items from conversation turns, memory, and tool outputs made it into the window under a fixed token budget.

Answer accuracy varied by up to 26 points between the best and worst policies.

The difference between a coverage-based selection method and naive recency truncation was the difference between an agent that answered and one that guessed. And notably, retention alone didn't explain the gap: PACMS matched the standard MMR baseline on how much evidence it kept, yet beat it by 8 to 12 points on end-to-end answers. What mattered wasn't just keeping the right information, but assembling a prompt that the model could actually extract from.

That is context engineering's whole thesis, isolated in a lab: with the prompt frozen, context decisions alone made and unmade the system.

Closing out

Last time, we argued that larger context windows raise the ceiling, while TokenOps determines what earns that capacity. This is the same argument one level up. Prompting is necessary but insufficient, just as capacity is necessary but insufficient. The frontier in both cases is the same unowned seam: the assembly step, where the system decides, per call, what the model gets to see.

The field even has a map for it now. LangChain's widely cited taxonomy breaks context engineering into four strategies: write, select, compress, and isolate. Writing good instructions is well served by existing craft. Compression and isolation have growing tooling. Our view is that selection, deciding which of the pooled candidates deserve the budget, is the least owned and highest-leverage part of that map. That's where the PACMS method lives, and it's where TokenOps plants its flag: selecting, measuring, and governing what enters an agent's context window at runtime, in the open, above any single framework.

Prompt engineering made us better at talking to models. Context engineering makes systems worth talking to. If you're building agents, the 2025 vocabulary shift was pointing at something real, and the data since has only sharpened it: the window, not the wording, is where your reliability is decided.

The PACMS method, its evaluation harnesses, and a reference context-engine plugin are open source. If you want to see what context selection is doing to your own agents, run it against your sessions and watch the keep/drop decisions. The prompt you wrote is probably fine. What's around it usually isn't.