Context Engineering: What Actually Replaced Prompt Engineering

This post is for engineers who run an LLM agent past its first dozen tool calls and watch the bill, the latency, or the answers get worse. Context engineering is the name the field settled on for the fix: deciding, on every call, which tokens the model sees. Most writing on it is a list of principles. Instead, this post measures them. It replays one 60-call coding-agent session under eight context policies and reports what each one sent, what the prompt cache could reuse, and which facts the agent could still see at the end. Two results cut against common advice, and both are cheap to act on.

The numbers below came from this environment:

Measured 2026-11-05
Windows 11 Pro 25H2 (build 26200), AMD Ryzen 5 8600G (6C/12T), 16 GB RAM
Dart SDK 3.9.0 (stable)
Tokenizer: o200k_base vocabulary (SHA-256 446a9538...cfb1a2d), ported to Dart,
  checked against tiktoken 0.14.0 splits
Session: 60 model calls, tool results read from the Dart SDK 3.9.0 lib/ sources, seed 42

No model was called. The bench measures what goes into the context window, which is deterministic, and leaves what a model does with it to the published research cited below. Consequently, every number here can be reproduced exactly, and two consecutive runs produced byte-identical output.

What Is Context Engineering?

Context engineering is the practice of choosing, on every model call, exactly which tokens enter the context window: the system prompt, tool definitions, retrieved documents, conversation history and tool results. Prompt engineering tunes one instruction. In contrast, context engineering manages a budget that an agent refills on every turn of its loop.

The distinction matters because agents changed what a prompt is. A chat app sends a prompt the developer wrote plus a user message. An agent, however, sends a prompt that is mostly text the developer never saw, namely file contents, search results and test logs that its own tools returned. Anthropic’s engineering team put it this way in Effective context engineering for AI agents, published in September 2025: context is a finite resource, and the curation happens each time you decide what to pass to the model.

If the instruction-writing side is what you need, the site’s guide to prompt engineering best practices still applies. Prompt engineering did not die. Rather, it became one of five inputs that context engineering has to balance.

Why Did Prompt Engineering Stop Being Enough?

Prompt engineering stopped being enough because the prompt stopped being the largest thing in the request. In the session measured below, the system prompt was 292 tokens out of 182,088 at the final call. Nothing a developer rewrites in those 292 tokens changes the cost or the attention profile of the other 181,796.

Long contexts also degrade the model itself. Chroma’s Context Rot report from July 2025 tested 18 models and found that performance becomes less reliable as input length grows, even on trivial tasks such as repeating a word. Earlier, Lost in the Middle by Liu et al. showed models using information at the start and end of a long context better than information in the middle. Neither effect is measured here. Still, both say the same thing: a token you did not need to send is not neutral.

The Bench: One Agent Session, Eight Context Policies

The session is a coding agent fixing a double-charge bug in a checkout service. Over 60 calls it reads files, greps, runs tests and edits code. The bench builds its tool results from real Dart SDK source files, so the token counts reflect real code rather than filler text.

The session plants eight facts the agent must still know at the end. Six arrive as user messages, such as the deploy target and the ticket number for commits. The other two, F3 and F6, appear only inside tool output: a flaky-test note in a test log and a production feature flag in a config file. That split turns out to be the most important detail in the bench.

Fact	Arrives at call	Source	Content
F1	1	user	Deploy target is eu-west-2 staging only
F2	5	user	Do not touch migrations/
F3	9	tool	A named test is known flaky, ticket QA-2291
F4	13	user	Keep the OrderRepository public API unchanged
F5	18	user	Use the Money type, never a double
F6	23	tool	checkout_v2_enabled is false in production config
F7	28	user	Commit messages reference PAY-4417
F8	34	user	Ping the payments on-call, it is freeze week

How Tokens Were Counted

Token counts use o200k_base, the vocabulary behind OpenAI’s GPT-4o, GPT-5 and o-series models. The bench ports tiktoken’s byte-pair merge to Dart, verifies the vocabulary file’s SHA-256 on load, and checks the port against splits that tiktoken 0.14.0 produced for the earlier tokenizers compared post:

dart run verify_tokenizer.dart o200k_base.tiktoken

OK     5 (tiktoken 5)  ت|ج|ميع|␣الات|صالات
OK    20 (tiktoken 20)  ␣|␣const|␣response|␣=|␣await|␣fetch|(`|https|://|api|.example|.com|/v|1|/in|voices|/${|id|}`,|␣{
OK    30 (tiktoken 30)  9|f|8|b|2|c|14|-|6|d|3|e|-|4|a|71|-|9|c|05|-|2|be|7|f|1|a|8|d|340

Arabic, code and a UUID all split identically, piece for piece. Claude and Gemini use different tokenizers, so their absolute counts would differ. The ratios between policies, however, depend on which blocks are sent, not on the tokenizer, and those ratios are the findings.

What Each Call Measures

For each call the bench assembles the context a policy would send and then records four things. First, the total tokens sent. Second, the longest token prefix shared with the previous call, which is the most a provider’s prompt cache could reuse. Third, whether any earlier block changed, which counts as a cache break. Finally, after the last call, which of the eight facts are still present anywhere in the context.

// bench.dart (excerpt): the per-call measurement.
for (var call = 1; call <= calls; call++) {
  ctx = p.contextFor(call, session, enc);
  final ids = [for (final b in ctx) ...enc.encode(b.text)];
  var lcp = 0;
  if (previous != null) {
    final n = min(previous.length, ids.length);
    while (lcp < n && previous[lcp] == ids[lcp]) {
      lcp++;
    }
  }
  // Providers only cache prefixes of 1,024 tokens or more.
  final cached = lcp >= 1024 ? lcp : 0;
  sent += ids.length;
  shared += cached;
  if (previous != null && lcp < previous.length) breaks++; // an earlier block changed
  previous = ids;
}

The 1,024-token floor comes from OpenAI’s prompt caching guide, which also states that cache reuse requires the entire rendered prefix to match. Anthropic’s API works on the same prefix principle with explicit breakpoints, as covered in the site’s guide to Anthropic prompt caching. Treating the shared prefix as cached is therefore an upper bound: it assumes every call lands inside the cache’s lifetime.

The “weighted” column prices a cached token at 0.1 of an uncached one, which is the cache-read rate on most current OpenAI models and Anthropic’s standard rate. Some models discount further, to 0.05, and that widens every gap below. Cache write premiums are not included.

The Eight Policies

1. Append everything: system prompt, tools, then the full history. The baseline most agent loops start with.
2. Append, timestamp in system prompt: the same, with Current time: ... as the first line of the system prompt.
3. Append, tools re-ranked per call: the same, with the 12 tool definitions reordered by a per-call relevance score.
4. Sliding window: system prompt, tools and the last 16 messages.
5. Clear tool results at 100K, keep 3: once the context passes 100,000 tokens, old tool results become [tool result cleared], keeping the 3 most recent. These match the defaults of Anthropic’s clear_tool_uses_20250919 context editing strategy.
6. Clear tool results at 40K, keep 3: the same with an aggressive trigger.
7. Compaction at 100K: the history becomes one summary message holding the user’s constraints verbatim plus one line per step taken, followed by the last 6 messages.
8. Compaction at 40K: the same with the aggressive trigger.

The compaction summary is deterministic, not written by a model. It keeps what a well-designed notes file keeps, meaning the user’s instructions and a step log. It does not copy tool output, and that is a deliberate model of how such summaries are usually specified.

Context Engineering Results: Tokens, Cache Hits and Facts Kept

Here is the full run:

dart run bench.dart o200k_base.tiktoken "$DART_SDK/lib"

system prompt: 292 tokens; 12 tool definitions: 785 tokens

policy                                       per-call tokens and totals over 60 calls
append everything                            peak  182088  sent   4924396  cached  96.3%  weighted    656319  over200K 0  breaks  0  facts 8/8 
append, timestamp in system prompt           peak  182108  sent   4925596  cached   0.0%  weighted   4925596  over200K 0  breaks 59  facts 8/8 
append, tools re-ranked per call             peak  182088  sent   4924396  cached   0.0%  weighted   4924396  over200K 0  breaks 59  facts 8/8 
sliding window, last 16 messages             peak   51233  sent   1325777  cached   6.3%  weighted   1251016  over200K 0  breaks 52  facts 0/8 -F1 -F2 -F3 -F4 -F5 -F6 -F7 -F8
clear tool results, trigger 100K, keep 3     peak   96476  sent   3038034  cached  93.2%  weighted    490365  over200K 0  breaks  2  facts 6/8 -F3 -F6
clear tool results, trigger 40K, keep 3      peak   38439  sent   1384666  cached  84.5%  weighted    331965  over200K 0  breaks  6  facts 6/8 -F3 -F6
compaction at 100K, notes + last 6 messages  peak   95525  sent   3015525  cached  93.2%  weighted    486884  over200K 0  breaks  2  facts 6/8 -F3 -F6
compaction at 40K, notes + last 6 messages   peak   39705  sent   1368299  cached  84.3%  weighted    330313  over200K 0  breaks  6  facts 6/8 -F3 -F6

context size at calls 10, 20, 30, 40, 50, 60
append everything                              26609  56902  66137 107033 146855 182088
append, timestamp in system prompt             26629  56922  66157 107053 146875 182108
append, tools re-ranked per call               26609  56902  66137 107033 146855 182088
sliding window, last 16 messages               25664  28072   7480  34454  25223  35416
clear tool results, trigger 100K, keep 3       26609  56902  66137  32821  72643  23310
clear tool results, trigger 40K, keep 3        26609  29204  38439  32821  12677  23310
compaction at 100K, notes + last 6 messages    26609  56902  66137  31870  71692  21830
compaction at 40K, notes + last 6 messages     26609  28281  37516  31870  11422  21830

The same results as a comparison table, with the weighted cost relative to the append-everything baseline:

Policy	Peak tokens	Tokens sent	Cached	Weighted vs baseline	Facts kept
Append everything	182,088	4,924,396	96.3%	1.00x	8/8
Timestamp in system prompt	182,108	4,925,596	0.0%	7.50x	8/8
Tools re-ranked per call	182,088	4,924,396	0.0%	7.50x	8/8
Sliding window, 16 messages	51,233	1,325,777	6.3%	1.91x	0/8
Clear tool results at 100K	96,476	3,038,034	93.2%	0.75x	6/8
Clear tool results at 40K	38,439	1,384,666	84.5%	0.51x	6/8
Compaction at 100K	95,525	3,015,525	93.2%	0.74x	6/8
Compaction at 40K	39,705	1,368,299	84.3%	0.50x	6/8

Three findings stand out, and the first two contradict advice that appears in many agent tutorials.

One Timestamp Line Cost 7.5 Times More

Adding Current time: ... to the top of the system prompt changed 20 tokens per call. As a result, the cache reused nothing on any of the 59 follow-up calls, and the weighted cost rose from 656,319 to 4,925,596, a factor of 7.50. Re-ranking the tool definitions per call produced the same zero, because the first changed tool sits at token 292, below the 1,024-token caching floor.

Both patterns come from good intentions. A timestamp helps the model reason about dates, and relevance-ranked tools look like tidy context engineering. The fix is ordering, not removal. Put everything stable first and in a fixed order, then put anything that changes per call, such as the time, in the newest user message:

// bench.dart (excerpt): the stable prefix every policy shares.
List<Block> prefix({String? timestamp, int? reorderSeed}) {
  final ordered = [...tools];
  if (reorderSeed != null) ordered.shuffle(Random(reorderSeed)); // per-call relevance ranking
  return [
    (kind: 'system', text: timestamp == null ? systemPrompt : 'Current time: $timestamp\n$systemPrompt'),
    for (final t in ordered) (kind: 'tool_def', text: jsonEncode(t)),
  ];
}

With both parameters null, that prefix is 1,077 tokens and byte-identical on every call, which is what lets the baseline cache 96.3% of what it sends.

The Sliding Window Sent 73% Less and Cost 91% More

The sliding window is the most common first attempt at context engineering, and on raw tokens it looks like a clear win. It sent 1,325,777 tokens against 4,924,396, which is 73.1% fewer. However, its window start moves on almost every call, which changes the first message after the tool definitions. The cache broke on 52 of 59 follow-up calls, reuse fell to 6.3%, and the weighted cost came out at 1.91 times the baseline.

It also kept zero of eight facts. By call 60 even F8, which arrived at call 34, had scrolled out. A raw sliding window therefore manages to be more expensive and less informed at the same time, which is why the provider-side features clear and compact rather than truncate.

Clearing and Compaction Halved Cost but Lost the Tool-Sourced Facts

The two edit-based policies behaved almost identically. At a 40K trigger, both held the peak under 40,000 tokens, kept the cache warm on all but six calls, and cut the weighted cost roughly in half. At the 100K default trigger, they saved about a quarter.

Both also lost exactly F3 and F6, the two facts that only ever appeared in tool output. Clearing removed the test log and the config file. The compaction summary recorded “Read config/prod.yaml” as a step but not what the file said. Meanwhile, every fact the user typed survived, because both policies keep user and assistant messages or copy the constraints verbatim. The general lesson: a context policy keeps whatever it was designed to keep, and nobody designs for a fact they have not seen yet.

Where the Tokens in an Agent Context Actually Go

A breakdown of the final append-everything context shows why tool results dominate every decision above:

system         292    0.2%
tool_def       785    0.4%
user           115    0.1%
assistant     2091    1.1%
tool        178805   98.2%
grep       calls 11  tokens    4846  largest 913
read_file  calls 41  tokens  173559  largest 14878
run_tests  calls  6  tokens    7757  largest 1306
edit_file  calls  2  tokens      26  largest 13

Tool output made up 98.2% of the context. Across all 60 calls, read_file alone produced 173,559 of the 186,188 tool tokens. The single largest file read cost 14,878 tokens, which is more than fifty copies of the system prompt. In practice, the highest-leverage context engineering in an agent is not in the prompt at all. Rather, it sits in the tool layer: line limits on reads, match caps on searches, and truncation on logs. The token counting and budget management post covers enforcing those budgets in code.

Where the Facts Sat in a 182K Context

The bench also recorded where each fact started in the final append-everything context:

fact depth in the final append-everything context (182088 tokens)
F1 (user, call  1)  starts at token    1077  =   0.6% of the way in
F2 (user, call  5)  starts at token    3679  =   2.0% of the way in
F3 (tool, call  9)  starts at token   25305  =  13.9% of the way in
F4 (user, call 13)  starts at token   29907  =  16.4% of the way in
F5 (user, call 18)  starts at token   56220  =  30.9% of the way in
F6 (tool, call 23)  starts at token   59764  =  32.8% of the way in
F7 (user, call 28)  starts at token   64702  =  35.5% of the way in
F8 (user, call 34)  starts at token   74358  =  40.8% of the way in

Every constraint sits in the first 41% of the context, behind more than 100,000 tokens of later file reads. Whether a given model reliably uses an instruction at that depth was not measured here. However, it is exactly the position the long-context research flags as weakest. So “append everything” keeping 8/8 facts means the facts are present, not that they are used.

How to Engineer Context for an Agent

Based on these measurements, a practical order of work looks like this:

1. Freeze the prefix. Keep the system prompt and tool definitions byte-identical and in a fixed order across calls, and move timestamps and per-user data into the newest message.
2. Cap tool output at the source. Add line limits to file reads, match limits to searches and truncation to logs, because that is where 98% of the tokens come from.
3. Clear old tool results before anything else. Clearing kept every user instruction and cost about half of the baseline at a 40K trigger.
4. Clear in batches, not every call. A trigger plus a minimum amount to clear means the cache breaks a few times per session instead of on every call.
5. Give the agent somewhere to write down what it learns from tools. A notes file or memory tool is the only policy here that could have kept F3 and F6.
6. Compact only when clearing is not enough, and specify what the summary must preserve, including findings from tool output.

Step 5 is the one this bench could not measure, because whether a model writes the note is model behavior. Anthropic pairs context editing with its memory tool for this reason: as the context nears the clearing threshold, the model receives a warning to save important information before the results disappear.

Configuring Tool-Result Clearing on Anthropic’s API

Anthropic exposes the clearing policy as a request option behind the context-management-2025-06-27 beta header. This configuration comes from the context editing documentation. It sits close to the aggressive policy above, with a lower trigger and a minimum clear size:

"context_management": {
  "edits": [
    {
      "type": "clear_tool_uses_20250919",
      "trigger": { "type": "input_tokens", "value": 30000 },
      "keep": { "type": "tool_uses", "value": 3 },
      "clear_at_least": { "type": "input_tokens", "value": 5000 },
      "exclude_tools": ["web_search"]
    }
  ]
}

Two options deserve attention. clear_at_least skips a clearing pass that would free too little, which the documentation describes as deciding whether clearing is worth breaking the prompt cache. Meanwhile, exclude_tools protects a tool’s results from clearing altogether. In this bench, excluding run_tests and reads of config files would have kept F3 and F6 for a small token cost.

What Changed Between Prompt and Context Engineering

The two disciplines differ in what they control and how often they run:

	Prompt engineering	Context engineering
Unit of work	One instruction	Every call in a loop
Main input	Text the developer wrote	Tool output, history, retrieval
Share of an agent request	0.2% in this session	The other 99.8%
Failure mode	Vague or ambiguous wording	Bloat, broken cache, lost facts
Main levers	Wording, examples, structure	Ordering, truncation, clearing, memory
When it runs	At design time	At runtime, per call

Retrieval belongs on the context side too. A RAG pipeline is a context policy for documents, and the agentic RAG post covers the variant where the agent decides when to retrieve. Likewise, the loop that produces all these tool results is covered in building AI agents with tools, planning and execution.

A Real-World Scenario: The Support Agent Whose Bill Jumped

Consider a small team running an internal support agent that answers questions over a few dozen tools. Sessions run long, often 30 to 50 calls, and most of the context is ticket histories and log excerpts. One day someone adds the current date and the user’s name to the top of the system prompt, so answers can say “yesterday” correctly and greet people by name. The feature works, and nobody connects it to the token bill.

Over the next billing cycle, input costs climb several times over with no change in traffic. The usage dashboard shows the same number of input tokens as before, which makes the jump confusing. Cached input tokens, however, have dropped to near zero. That matches the timestamp row of this bench: identical tokens sent, a 7.5x weighted cost, and nothing in the response quality to give it away.

Fixing it takes an afternoon. The date and user name move into the first user message, the tool list gets sorted once by name, and cache reads return on the next deploy. The trade-off is that the team now has a rule to enforce in review, because any per-request value that drifts into the prefix silently brings the cost back.

When to Use Context Editing

• Agent sessions regularly exceed a few dozen tool calls or 30,000 tokens of context
• Tool results are large and mostly needed once, such as file reads, search results and logs
• The provider supports prompt caching and the bill is dominated by input tokens
• User instructions arrive in messages that the editing policy can leave untouched
• You can name the tools whose results must never be cleared

When NOT to Use Context Editing

• Short sessions that never approach the trigger, where editing only adds moving parts
• Tasks where an early tool result is the main subject, such as reviewing one large file in depth
• Agents without a notes file or memory tool, when tool-sourced findings must survive
• Single-shot chat or extraction calls, where there is no history to manage
• Pipelines where every call must be reproducible from the raw transcript for audit

Common Mistakes with Context Engineering

• Putting timestamps, user names or request IDs in the system prompt, which breaks the cache on every call
• Reordering or filtering tool definitions per call without measuring the cache cost
• Reaching for a sliding window first, which in this bench cost 1.91 times more and kept no facts
• Tuning the system prompt while uncapped file reads make up 98% of the context
• Writing compaction summaries that record which files were read but not what they said
• Treating “the fact is in the context” as “the model will use it” when 100,000 tokens of later output sit behind it
• Measuring raw tokens sent instead of cached and uncached tokens separately

What This Measurement Does Not Cover

No model ran in this bench. It measures what each policy puts in the window, not answer quality, so it cannot say whether a model follows an instruction at 40% depth or performs better after clearing. Likewise, the compaction summary is a deterministic stand-in for a model-written one, and a good model-written summary could keep F3 and F6.

The cache numbers are upper bounds. They assume every call lands within the cache lifetime, ignore cache write premiums, and use one tokenizer for every provider. Chat-template overhead tokens, parallel tool calls and output tokens are not counted either. Finally, this is one session shape. An agent with small tool results and long user conversations would shift the balance toward history and away from tool output. The bench reruns in about five seconds, so changing the session and re-measuring is cheap, and it will be re-run on the next major change to the caching rules of either provider.

Conclusion

Context engineering replaced prompt engineering as the main lever because the prompt became 0.2% of an agent’s request and tool output became 98%. In this measured session, a single timestamp line multiplied weighted input cost by 7.5, a sliding window cost 91% more while keeping no facts, and tool-result clearing at a 40K trigger halved cost while losing only the facts that came from tools.

Start with the cheapest change today: check your system prompt and tool list for anything that varies per call, move it into the newest message, and confirm cached input tokens rise in your provider’s usage data. Next, read the site’s guide to Anthropic prompt caching to place cache breakpoints deliberately rather than by accident.