TL;DR
Get everyday essentials delivered free with Prime
- Fast, free delivery on millions of items
- Prime Video, Amazon Music and more included
- Member-only deals all year
A November 2 blog post by four researchers affiliated with MIT CSAIL reports that GPT-6 Astra edited two layers of Qwen3.5-0.8B to reduce repetitive reasoning. Its MATH-500 score rose from 7.9% to 48.1%, but 33.1% of answers still reached the token limit, and the work has not been peer-reviewed.
Four researchers affiliated with MIT CSAIL report that OpenAI’s GPT-6 Astra edited the weights of the small open model Qwen3.5-0.8B, reducing its tendency to repeat reasoning and raising its reported score on MATH-500 from 7.9% to 48.1%. The November 2 blog post describes an experimental weight edit, not a peer-reviewed result; after the change, 33.1% of answers still hit the benchmark’s 32,768-token limit.
In the experiment, Astra searched the model’s weights for a cause of repetitive outputs, then changed two layers. The researchers say it identified one internal direction associated with looping and subtracted it from 64 output columns in each layer. The edit affected 131,072 weights, or 0.016% of the model’s reported 0.8 billion weights. The researchers published the edited model on Hugging Face, where they say only the two down-projection tensors in layers 15 and 19 differ from the original.
The method used feedback from practice problems and repeated model evaluations, but no gradient descent, backward passes or fitting to a loss. Published logs record 158 turns, 80 Python runs and 50 forward passes over 20.3 hours. Each forward pass could cover up to 512 problems. The researchers say Astra was not shown the final benchmark questions. They also report that runs with budgets of 10 and 25 passes scored 12.3% and 14.1% on MATH-500, respectively.
The post reports that the original model reached the token ceiling on 93.0% of its MATH-500 answers; after editing, that fell to 33.1%. The authors caution that reducing loops is not the same as making a model better at math. They describe the work as “very early results” and say the capabilities remain limited.
Astra changed model weights but didn’t prove smarter math
Researchers report that GPT-6 Astra edited two layers of Qwen3.5-0.8B to reduce repetitive reasoning. The MATH-500 score rose, but many answers still ran to the token limit.
A targeted weight edit
The researchers say Astra changed two layers to address looping in Qwen3.5-0.8B.
Astra used practice problems and repeated evaluations to search the model’s weights.
It subtracted a direction associated with loops from output columns in layers 15 and 19.
The reported MATH-500 score rose, while some answers still exhausted the token budget.
What changed in the run
What the result means
- Weight edits may help target a specific failure pattern.
- The published checkpoint can be evaluated by other researchers and operators.
- One model and one benchmark do not establish broader math ability.
- Effects on other tasks and model behaviors remain unclear.
Evidence is still early
The November 2, 2026 blog post reports an experiment that has not been peer-reviewed. The authors describe the capabilities as limited.
A Targeted Repair With Limits
The result points to a possible way to alter some behaviors in open-weight models without running conventional training. If the approach proves reliable, researchers and model operators could investigate specific failure patterns by editing weights directly. In this case, the reported score increase came alongside a major reduction in answers that exhausted the token budget, so the result is closely tied to fixing a known failure mode.
The experiment does not establish that Astra can broadly improve a model’s reasoning or intelligence. The benchmark gain is one result from one model and one task, and a substantial share of responses still ran to the limit. The authors also report that the same general effort produced no change greater than about one percentage point on Gemma 3 270M. Each edit changes the model itself, so a patched checkpoint should be treated as a distinct version whose behavior needs evaluation.
Products Worth Considering
Bose SoundLink Flex Bluetooth Speaker (2nd Gen), Waterproof, Dustproof, USB‑C, Up to 12 hr Battery, Black
Bose S1 Pro+ All-in-One PA Speaker
Apple Watch SE (2nd Gen) GPS 40mm with Starlight Aluminum Case and Sport Band (S/M)
As an affiliate, we earn on qualifying purchases.
How the Weight Search Worked
Qwen’s model card warns that Qwen3.5-0.8B is prone to thinking loops. In the researchers’ practice run, 85 of the first 96 answers reached the 32,768-token ceiling. They gave Astra access to a Python environment for inspecting and changing the model’s weights, plus practice questions from GSM8K and MATH. The environment ran on a CPU without internet access, and the logs specify a maximum of 300 code runs.
The report, Sparks of Weight-Space Programming: LLMs can natively read & write neural network weights, covers eight experiments, not all successful. The authors say Astra could not identify which of two fine-tuned Gemma models had been trained to flatter users. The Qwen math test is the experiment that produced the reported benchmark gain. The post cites earlier research on removing refusal behavior from models by deleting a direction in their weights, placing the Qwen edit within a broader line of work on targeted weight changes.
The authors are Rikiya Takehi, Jayden Teoh, Alex Zhang and Omar Khattab, listed with MIT CSAIL; Takehi and Teoh are marked as equal contributors. The post was published on Takehi’s site on November 2. It is not a peer-reviewed paper, and the source material says it had not been posted on arXiv. OpenAI is not listed as an author, and the post reports no statement from the company about the experiment.
““Fixing loops, however, is not the same as making a model better at math.””
— The researchers, in “Sparks of Weight-Space Programming”
How General the Fix Is
The post does not establish whether the same approach will work across other model sizes, architectures or tasks. The provided results show a large MATH-500 score change for Qwen3.5-0.8B, but the edited model still hit the token ceiling on 33.1% of answers. The source material does not provide independent replication or peer review, and it does not report a statement from OpenAI about Astra’s role.
It is also unclear from these results whether the math score gain persists across broader evaluations or whether the weight change affects other model behaviors. The researchers say they ran multiple experiments, including unsuccessful ones, but the reported Qwen result alone cannot resolve those questions.
Further Tests and Replication
The immediate next step is independent evaluation of the published edited checkpoint, including tests of its math performance, repetition rate and behavior on tasks beyond MATH-500. Replication on other open-weight models would help show whether the technique applies beyond this specific model and failure mode.
The researchers’ post presents the findings as early work and does not specify a publication or follow-up schedule. Whether the result will be submitted for peer review, expanded to more models or independently reproduced remains unknown.
Key Questions
What did GPT-6 Astra change?
The researchers say Astra subtracted an internal direction associated with repetitive outputs from 64 output columns in each of Qwen3.5-0.8B’s layers 15 and 19.How much did the reported MATH-500 score change?
The post reports a rise from 7.9% before the edit to 48.1% afterward. It also reports that 33.1% of edited-model answers still reached the 32,768-token limit.Was the model changed without any testing?
No. The method avoided gradient descent, but Astra used practice problems and repeated model evaluations to guide an iterative search. The logs report 50 forward passes in the run that produced the edit.Has the result been peer-reviewed?
The source post says the work is an early blog report, not a peer-reviewed paper, and had not been posted on arXiv.Source: Thorsten Meyer AI
