GPT-6 Astra's 99.9% Came From OpenAI's Own Harness; the Neutral One Says 62.7%

AI 요약

• The same GPT-6 Astra scored 62.7% and 99.9% on ARC-AGI-3.
• The difference was not the model but whether reasoning state carried into the next call.
• ARC Prize will now publish both harness conditions side by side.

ARC Prize published its ARC-AGI-3 measurement of GPT-6 Astra on September 3. Same model, same set of games, two scores: 62.7% and 99.9%. What changed was neither the model nor the puzzles, but the code wrapped around the model call.

ARC Prize calls that outer code the harness. It is the program that decides which tools the model can reach, what it keeps between calls, and what to throw away when a conversation runs long. If you have written an agent yourself, you already own one. The loop that sends a prompt, reads the response, and assembles the next request is a harness.

What the two harnesses actually did differently

ARC Prize ran the same model through two harnesses with different rules. The Standard harness is provider-neutral. It gives the model everything it needs to solve the game but leaves the model to decide what to carry forward, and the only thing it can carry is a note a human could read. It exists so OpenAI, Anthropic, and Google models can be compared under identical conditions.

The Provider Adapter harness lets each vendor use the features it built for its own models. For Astra, ARC Prize described it as preserving the opaque reasoning state (which they cannot see) between requests, and using compaction to manage longer conversations.

Put plainly: under Standard, the model has to write down what it was thinking each turn, and anything it does not write down is gone. Under Provider Adapter, internal reasoning survives to the next call without ever being turned into text. Why that gap reaches 36 percentage points depends on what kind of test ARC-AGI-3 is.

ARC-AGI-3 is a turn-based game that never tells you the rules. The agent moves, guesses what the goal is, builds its own model of how the environment behaves, and plans from there. Humans solve it 100% of the time. Across roughly 500 ordinary people ARC Prize recruited, one attempt at a game cost about $12.78 of human time. It is exactly the kind of task where losing your working hypothesis each turn means starting the guessing over.

Here are the results by reasoning effort. The Provider Adapter columns are on the right.

At maximum reasoning effort the gap is close to 36 percentage points. The sharper detail is that the Provider Adapter run scores 96.7% with reasoning turned off entirely, 34 points above the Standard harness at maximum reasoning.

Cost runs the other way too. Across the 167 game-reasoning pairs both harnesses solved, the Provider Adapter runs were about 3.66x faster by cumulative elapsed time and spent 49% fewer total tokens. The higher score was also the cheaper one, because a model that keeps its earlier reasoning does not have to regenerate it.

Why the gap is this large on Astra

Astra's reasoning architecture may be what widens it. This part rests on reporting rather than vendor documentation, so treat it as unconfirmed.

The Information reported that Astra uses recurrent depth, also called opaque recurrence: the same query passes through a neural loop several times, and less of the thinking ends up as human-readable text. OpenAI's system card does not name that architecture. It does publish an internal evaluation finding that Astra's stated reasoning is harder to monitor than GPT-5.6 Sol's, which is the point safety researchers raised in TechCrunch's September 2 report.

If the reporting is right, the Standard harness penalizes Astra specifically. The more reasoning a model leaves unwritten, the more it loses under a rule that keeps only what was written. For reference, other models on the same set under the Standard harness: Claude Opus 5 at 30.2%, GPT-5.6 Sol at 7.8%. The widely quoted 99.9% versus 7.8% sets two different conditions next to each other. Like for like, it is 62.7% versus 7.8%.

Turning the same thing on in your own agent loop

What the Provider Adapter did is not a private feature. Both vendors document it as an API option. This section is for developers building their own agent loop; if you use someone else's agent, the question is whether that tool already turns it on.

In OpenAI's Responses API, adding include=["reasoning.encrypted_content"] to a request returns encrypted reasoning tokens that you pass back verbatim on the next request. Your client holds the encrypted payload; OpenAI does not store it and decrypts it in memory only when it arrives in a request. If you would rather keep conversation state server-side, passing previous_response_id gives the model access to the earlier reasoning items on its own.

In Anthropic's Messages API, when extended thinking is on and tools are in play, you must return the thinking blocks in the final assistant turn complete and unmodified. Do not reorder them or edit their contents. The documentation's standing advice is to always pass every thinking block back.

One reason to keep expectations low: OpenAI's own measurement puts the gain from passing reasoning items back at roughly 3% on SWE-bench. The 36-point spread on ARC-AGI-3 is not what a normal coding task will give you. ARC-AGI-3 is only solvable if each turn's hypothesis survives into the next one, so it exposes the value of preserved state at an extreme. Editing one file does not.

It is still worth checking. Whether the gain is 3% or 36 points, the cost of turning it on is one request parameter, and it moves token usage down rather than up, so the bill does not rise.

The leaderboard splits in two

ARC Prize said it will publish Standard harness and Provider Adapter harness results side by side with condition labels from now on. One model, one score is over.

Why that change was needed showed up the same week in a round of quiet number revisions. Per TNW's writeup, within hours of the September 4 Astra launch announcement the hallucination rate moved from 4.2% to 2% and back to 4.2%. GPT-5.6 Sol's ExploitBench score doubled from 5.5% to 11.5%. The ARC-AGI-3 figure went from 98.6% in a pre-publication draft to 99.99% in the posted version. OpenAI told Fortune that most evaluations carry noise of a few percentage points depending on checkpoint and evaluation conditions. It did not explain the harness gap itself.

ARC Prize co-founder Mike Knoop drew a line under any AGI reading of the result. The ARC Prize post says the same: saturating the benchmark is not proof of achieving AGI, and ARC-AGI-3 has a tightly bounded scope and format with deterministic, closed-ended mechanics.

What you can use today

Item	Terms
ARC-AGI-3 Semi-Private set	Not public. You cannot reproduce these scores directly
ARC-AGI-3-Agents harness code	Open. Clone github.com/arcprize/ARC-AGI-3-Agents and run it free with an API key issued on the site
Providers the harness supports	Anthropic, OpenAI, Google, xAI, DeepSeek, Groq, OpenRouter, Fireworks. Bring your own key for each
OpenAI reasoning item passthrough	Every Responses API user. No separate plan, approval, or surcharge
Anthropic thinking block passthrough	Every Messages API user on a model that supports extended thinking
Regional availability	No geographic restriction on any of the three. ARC-AGI-3 key issuance and both vendor APIs are open worldwide, APAC included

If your team runs its own agent loop, start by checking whether the requests you send right now carry the previous turn's reasoning. On OpenAI, look for reasoning items in the response and whether you put them back in the next request. On Anthropic, check that you are not stripping thinking blocks. Pick one long task that calls tools several times, run it once with passthrough on and once with it off, and compare success and total tokens; that tells you what the gap is worth on your own work rather than on a benchmark. And when you read the next results table, add the question raised at Astra's launch: whose harness produced this number?