Summarizing a Conversation Took 36 Seconds, So Claude Moved That Wait Out of the Request

Anthropic opened an on-demand compaction beta on the Messages API on September 14, 2026. The beta header is compact-2026-09-04.

Compaction replaces a long conversation with a single block of summary text. The more files an agent reads and tools it calls, the longer the transcript grows, and that whole transcript goes back into the model on every request. Sonar's instrumentation of a single 800-line PR that billed 152.8 million cache-read tokens put a number on that cost. Compaction shrinks the transcript to a summary so the next request carries less.

Server-side compaction already existed. What changed is who picks the moment

The Claude API has supported server-side compaction since a January 2026 beta (compact-2026-01-12). That mechanism is threshold compaction. When input tokens reach a configured value, the API builds a summary in the middle of handling your request, shrinks the context with it, and then writes the answer. The default threshold is 150,000 input tokens, with a 50,000 minimum.

The user's request is still in flight while that summary is being written. The open-source agent oh-my-pi recorded measurements while wiring threshold compaction into its own code: compacting one roughly 80,000-token conversation took 36 to 55 seconds depending on the thinking configuration. That is one project's measurement, not an official figure. When compaction runs inside the request, the user waits out those 36 to 55 seconds. The same notes show follow-up turns after compaction dropping to 3 input tokens through cache reuse.

The new beta splits the summary into its own request. Put a top-level compaction parameter on a request and the API summarizes every message in that request once, produces no answer, and returns a single summary block. The conversation keeps running on the original transcript, and you swap the block in when it arrives.

The summary request hands back no answer

The request is the conversation you were already sending, plus one parameter and the beta header.

curl https://api.anthropic.com/v1/messages \
  -H "x-api-key: $ANTHROPIC_API_KEY" \
  -H "anthropic-version: 2023-06-01" \
  -H "anthropic-beta: compact-2026-09-04" \
  -H "content-type: application/json" \
  -d '{
    "model": "claude-opus-5",
    "max_tokens": 4096,
    "messages": [ ... ],
    "compaction": {"type": "summarize"}
  }'

The response carries no answer text. content holds one compaction block and stop_reason is "compaction". The block contains the summary and a signature value. You have to send the block back exactly as received, signature included, or the API rejects it.

On later requests, put the block at the head of messages and delete the messages it summarized. Any old message left in front of the block returns a 400 with compaction_block_misplaced. Messages after the block pass through to the model untouched.

The API summarizes every message you send, so the way to keep recent turns is to leave them out of the compaction request entirely. They then follow the summary block. On models that preserve thinking, the thinking on those kept turns stays valid under two conditions: the kept turns came immediately after the summarized messages, and system plus any tools not marked defer_loading: true match the compaction request. An agent in the middle of a long job does not lose its reasoning at the moment you compact.

You can also replace the summarization prompt. instructions accepts up to 16,384 characters and substitutes for the default prompt wholesale. On Fable 5.1 and Mythos 5.1, threshold compaction dropped prior thinking from the summarizer's input whenever custom instructions were supplied. On-demand compaction reads prior thinking either way.

Your bill shows zero tokens

Summary calls are not free. Each one is a real model call using that request's model, system, tools, thinking configuration, and max_tokens. It is billed like any other request and counts against rate limits. The top-level token fields in the response are nonetheless 0, because no answer was generated. The real usage lands only in the compaction entry inside usage.iterations.

{
  "stop_reason": "compaction",
  "usage": {
    "input_tokens": 0,
    "output_tokens": 0,
    "iterations": [{ "type": "compaction", "input_tokens": 144, "output_tokens": 276 }]
  }
}

If your cost accounting sums only usage.input_tokens from responses, compaction calls land in the ledger as zero. A summary call feeds an entire long conversation in as input, so its per-call cost is not small. Fix the accounting to add up iterations before you turn this on, or you have no before-and-after comparison.

Nor does a summary always come back. The block is produced only when the summary call ends normally on text without calling a tool. Otherwise you get a 200 with empty content, and the call is still billed. stop_reason tells you which failure you hit. "max_tokens" means the summary was truncated, so raise max_tokens and retry. "model_context_window_exceeded" means you need shorter instructions or fewer messages. "tool_use" means the model called a tool instead of summarizing, which you fix by telling it not to in instructions. "refusal" means a safety policy fired, with the policy category in stop_details.

What it takes to turn on today

This is a developer feature. It is not a toggle in the Claude app or in Claude Code settings. It turns on when you add the header to code that calls the Claude API directly.

The harder constraint is not geographic, it is which endpoint you call. On-demand compaction does not work on Amazon Bedrock or Google Cloud, unlike threshold compaction, which is in beta on both. Teams across APAC commonly route Claude through Bedrock in ap-southeast-1 or through Vertex AI in asia-southeast1, often because a data-processing commitment under Singapore's PDPA or a customer contract pins where the traffic lands. Those deployments get nothing from this beta today. Moving to the first-party Claude API to use it means moving where the requests are processed, which is a compliance decision before it is an engineering one.

Other rejections are narrower. You cannot send compaction and context_management in the same request, and threshold compaction (compact_20260112) does not run on a request carrying a signed block. stop_sequences, structured output via output_config.format, and the any and tool forms of tool_choice are all refused on a compaction request. So is a state where the last assistant turn ended in a tool call with no result, so send the tool results first.

The failure that is hardest to notice is data loss. Images, documents, container_upload blocks, and fetched URLs inside the summarized range disappear the moment the block takes their place. If a later turn needs them, you resend the content or re-upload the file. The conversation also has to still fit inside the model's context window for the summary to run at all, so compact before you overflow it, not after.

If you already run your own summarization logic in an agent, take one long session and send the exact messages you would have summarized as a compaction request instead. Write the token counts from usage.iterations next to what your current summary call costs and the difference shows up immediately. Teams sending traffic to Amazon Bedrock or Google Cloud are not covered by this beta yet. Threshold compaction with pause_after_compaction is the option that remains there, and support for the two platforms would land in the Claude Platform release notes.