An AI Edited 0.016% of Another AI's Weights and Math Jumped 6x: It Got No Smarter

GPT-6 Astra edited the weights of the open model Qwen3.5-0.8B directly, with no training step, and its math score went up six times. The MATH-500 benchmark score moved from 7.9% to 48.1%. The edit touched 131,072 of roughly 800 million weights, or 0.016%. Four MIT CSAIL researchers published the work on November 2, and it reached the top of Hacker News with 1,003 points.

Weights are the numbers a model picks up during training. Until now those numbers were set by optimizers such as gradient descent, not read and rewritten by anyone. The researchers call having a general-purpose LLM read and write them directly weight-space programming.

Open the experiment logs the authors published, though, and the 6x figure means something narrower than it sounds. Astra did not teach Qwen math. It made answers that never finished finish.

What Astra did and did not do

There are eight experiments, scaling from tiny networks up to a real language model. Each one restricted Astra in some way, switching off tools or forbidding training, gradient computation, or model execution.

Impressive as that list is, the larger the target, the more Astra leaned on tools and on execution budget. Only targets up to about 171,000 weights were pasted into the prompt as literal numbers. The 1.48M-weight network and the 800M-parameter model were handled through Python tools.

The car experiment shows how Astra works. The PPO policy drove straight at the ramp and stalled there every time. Astra read the car's position and velocity and added a small circuit, a few units per layer, that reverses first. Once the car passes the red strip, that circuit switches off. This is behavior designed the way a person edits code, not behavior learned.

The 6x is answers that started finishing

Qwen3.5-0.8B had a problem that sat upstream of math. With thinking mode on, it repeated the same sentences until it hit the 32,768-token limit and produced no answer at all. That is how 93% of its MATH-500 responses ended. Qwen's own model card warns that this model falls into reasoning loops more readily than the other Qwen3.5 sizes.

Astra found one direction in the layer 15 and layer 19 MLPs that produced the repetition and subtracted it from 64 output columns in each layer. The edit in the logs is a single line.

W = W0 - 6 * outer(u, mask64 * (u @ W0))

The decisive numbers are not in the blog post but in the run's own final check: after Astra chose the edit, it compared before and after on 1,152 practice problems it had not used while selecting.

Accuracy on the problems it already finished did not move by a single point. Almost every new correct answer, 576 of 579, came from responses that used to be cut off. The model started finishing problems it could already solve. It did not buy the score by cutting answers short either: all 696 correct answers in the edited model closed their reasoning normally, at a median length of 9,172 tokens.

The benchmark table says the same thing. Scores jumped where truncation disappeared, and hard problems stayed hard.

All figures are percentages, and 10, 25, and 50 are the model-execution budgets Astra was given. MMLU-Pro, where truncation was rare to begin with at 28.9%, gained only 5.2 points. Gemma 3 270M truncates just 8.5% of its answers, and in the same experiment Astra could move its MATH-500 score only from 6.8% to at most 7.7%. The authors write the same thing: the edit surfaces capability that was already there and does not add capability that was not.

The side effect matters more in production. Average IFEval output fell from 25,296 tokens to 4,241, roughly one sixth. Inference time and cost drop with the wasted repetition.

What the logs say about the limits

48.1% comes from a single run. In the published results table, it is the only run evaluated to completion at the 50-call budget. Five things belong next to it when judging reproducibility.

• Each run chose a different edit. The 10-call run subtracted a direction from 30 matrices; the 25-call run switched off a single attention head. A bigger budget did not refine the same edit. IFEval at 25 calls (39.9%) came in below the 10-call run (49.0%).
• Some runs produced nothing. The 5-call run listed in the same table, and a 10-call run with a changed prompt, submitted no edit at all.
• 50 calls is not a small number. One call generates up to 512 problems, and a single generation took up to about 30 minutes. The 50-call run took 158 turns and 20.3 hours of wall-clock time. There are no gradients, but there is a great deal of measurement and trial and error.
• There is no control arm. Qwen's model card recommends adjusting sampling and detecting repetition in the stream to cut it off. What the score would be with sampling changes alone, and no weight edit, is not in the published results.
• No third party has reproduced it. This is a pre-peer-review blog post. One Hacker News commenter tried the same approach on a small transformer and got results far worse than trained weights (4.47 bpb against 2.53). Another reported beating AdamW on a small network. Both are individual attempts.

The authors say they are building a benchmark and a run harness, but that code is not in this repository. Publishing the full logs so every number can be traced is the right call, and the log docs even carry a correction noting that the 24.5% pre-edit GPQA figure in the post's chart is a median and the final value is 22.7%.

The same technique works on safety guardrails

The edit itself is not a new technique. Arditi et al. 2024, which the authors cite, showed that refusal behavior in 13 open models rides on a single direction in activation space and presented a rank-1 weight edit that erases that direction as a jailbreak method. What changed is who does the work. A general-purpose LLM, not a human researcher, handled diagnosis through verification.

This does not mean Astra removed any guardrail. No such attempt appears in these experiments. What remains is that 0.016% of the numbers in a file can change behavior substantially. Put that next to the recent pretraining data poisoning work and the Mistral Large 4 weight release, and downloaded weights have one more reason to be pinned by revision and hash the way code dependencies are.

Can you use this today

Anyone can apply the published edit, but rerunning the experiment requires Astra.

The edit file (edit.npz) is about 388KB. The script checks a per-tensor hash and rebuilds the exact model used in the evaluation, bit for bit.

python exp8_math/code/apply_edit.py model.safetensors exp8_math/runs/p50c/edit edited.safetensors

If your team runs a small open model in thinking mode, start by counting how many of last week's responses were cut off at the token limit. If that share is high and the model is Qwen3.5-0.8B, apply this edit and run 100 of your usual prompts before and after, comparing truncation rate and accuracy. Whether the same effect shows up in your own prompts, not in the benchmark, is what decides adoption.