TL;DR
Get business pricing on tech for your team
- Business-only prices and quantity discounts
- Tax-exempt purchasing
- Multiple users, one account, clear invoices
A November 2, 2026 blog post by four researchers listed with MIT CSAIL reports that GPT-6 Astra edited two weight tensors in the open-weight Qwen3.5-0.8B model, raising its MATH-500 score from 7.9% to 48.1%. The result comes from an early, non-peer-reviewed experiment: loops fell but persisted, and the reported improvement required extensive model testing and feedback rather than a single automated edit.
GPT-6 Astra edited the weights of the open-weight Qwen3.5-0.8B model to reduce repetitive reasoning on math problems, according to a research post published November 2 by four researchers listed with MIT CSAIL. The authors report that its MATH-500 score rose from 7.9% to 48.1%, but the edit did not eliminate looping, and the result has not been peer-reviewed.
The experiment targeted a specific failure: Qwen3.5-0.8B frequently repeated itself until reaching the benchmark’s 32,768-token output limit. Before editing, 93.0% of its answers on MATH-500 hit that limit, according to the post. Afterward, the share fell to 33.1%. The researchers describe the work as a repair of a known weakness, not evidence that the model gained a general ability to solve math problems.
According to the published logs, GPT-6 Astra identified a direction associated with looping in the model’s multilayer perceptrons at layers 15 and 19. It subtracted that direction from 64 output columns in each layer, changing 131,072 weights, or about 0.016% of the model’s 0.8 billion weights. The researchers say only the two layers’ down-projection tensors differ in the released edited model; they also published hashes for those tensors.
The process did not use gradient descent, backward passes, or fitting against a loss function. But it was not a one-shot edit: the highest-budget run involved 158 turns, 80 Python runs and 50 forward passes, over 20.3 hours. A forward pass allowed Astra to inspect model outputs on batches of problems or texts. The benchmark questions used for final scoring were withheld from Astra, according to the post.
Inside an AI-to-AI repair GPT-6 Astra edits model weights
Researchers report a targeted edit that sharply reduced repetitive answers in Qwen3.5-0.8B. The early, non-peer-reviewed result shows a benchmark gain, but not a general math upgrade.
What changed
Astra targeted a direction linked to repetitive reasoning in the model’s multilayer perceptrons.
The logs identify looping directions in layers 15 and 19.
Astra subtracted the direction from 64 output columns in each layer.
The highest-budget run used 50 forward passes and extensive feedback.
A targeted repair, not a broad upgrade
The benchmark gain came with a clear limit: repetitive output fell, but did not disappear.
- MATH-500 score rose from 7.9% to 48.1%.
- Fewer answers reached the 32,768-token limit.
- One-third of answers still hit the limit.
- Effects on other tasks have not been established.
Evidence and practical limits
The result comes from one small model and one targeted experiment.
A different editing route
The researchers changed accessible weights without gradient descent or loss-based fitting.
Not independently validated
The post is not peer-reviewed, and its authors describe the findings as very early.
Test before deployment
Organizations would need to check side effects and performance on their own tasks.
A New Route to Editing Open Models
The experiment points to a possible way of changing an open model’s behavior without running a conventional training process. Instead of adjusting weights through gradient descent, an AI system can inspect outputs, propose a targeted change, and test the result. If this approach proves reliable, developers could use it to investigate or patch specific failure modes in models whose weights they can access.
The practical limits matter as much as the headline score. The strongest result required substantial testing, and one-third of answers still reached the output limit. The researchers also report that smaller search budgets had modest results: 10 and 25 forward passes produced MATH-500 scores of 12.3% and 14.1%. On Gemma 3 270M, no edit improved the score by more than about a point. The evidence therefore supports a promising, targeted intervention—not a general method for upgrading models.
Any successful weight patch also creates a modified model that needs its own evaluation. Changes intended to suppress repetition could affect other behaviors, and the reported benchmark score does not establish safety, reliability, or performance across different tasks. Organizations would need to test a patched model against their own use cases before deployment.
Products Worth Considering
Sony WH-1000XM6 Wireless Noise-Canceling Headphones, Black
Bose QuietComfort Headphones (2nd Gen), Wireless Over-Ear Noise-Cancelling Headphones, Eucalyptus Green
Jackery HomePower 3600 Plus Portable Power Station, 3584Wh, 3600W
As an affiliate, we earn on qualifying purchases.
An Early Weight-Space Experiment
The work appears in a post titled Sparks of Weight-Space Programming: LLMs can natively read & write neural network weights, published on Rikiya Takehi’s site on November 2. Its four authors are listed with MIT CSAIL; Takehi and Jayden Teoh are marked as equal contributors. The post describes eight experiments, including attempts to identify a model’s training task and repair a car-driving policy. One test failed: Astra could not determine which of two fine-tuned Gemma models had been trained to flatter users.
The authors call the results “very early”, and the post is not peer-reviewed or posted on arXiv, according to the supplied source material. The experiment focused on Qwen3.5-0.8B, whose model card already warns that it is prone to thinking loops. The researchers provided practice questions from GSM8K and MATH but reserved MATH-500 for final evaluation.
The post used GPT-6 Astra through the Codex command-line tool at its highest reasoning setting, with built-in tools disabled. A custom, CPU-only harness provided access to the model weights and a Python kernel, with no internet access and a limit of 300 code runs. OpenAI is not listed as an author, and the post gives no statement from the company about the work. The source reports that CNBC covered Astra’s rollout on September 3, 2026.
““Fixing loops, however, is not the same as making a model better at math.””
— The researchers, in their November 2 post
How General Is the Repair?
The reported result comes from one targeted experiment on one small model and has not been independently validated through peer review, based on the supplied material. It is not yet clear whether the same method can reliably improve other models, reduce different failure modes, or preserve performance across unrelated tasks. The post’s mixed results across its experiments counsel against treating the Qwen finding as a broadly established capability.
Important deployment questions also remain open. The source does not report a full evaluation of possible side effects from the weight change, nor does it establish whether the MATH-500 improvement holds under other testing conditions. The looping rate fell substantially but remained at 33.1%, so the behavior was reduced rather than removed. OpenAI’s role is also limited in the available account: the experiment used Astra, but OpenAI is not an author and has not provided a statement in the post.
Independent Tests and Broader Benchmarks
The immediate next step is independent replication of the published edit and evaluation logs, including checks of the released model tensors and performance beyond MATH-500. Further testing would need to determine whether the score gain persists across other math sets and whether the change affects the model’s performance on non-math tasks.
The post presents early evidence rather than a confirmed product feature or a broadly validated editing system. The researchers’ released logs and edited model provide material for other researchers to examine, but the supplied source does not announce a peer-reviewed paper, follow-up release, or official OpenAI response. Those developments, along with tests of how the edit behaves in practical use, remain pending.
Key Questions
What did GPT-6 Astra change?
According to the researchers, Astra altered 131,072 weights across two down-projection tensors in Qwen3.5-0.8B’s layers 15 and 19. The change was intended to reduce a direction associated with repetitive output.
Did the edit eliminate Qwen’s math loops?
No. The reported share of MATH-500 answers that hit the 32,768-token limit fell from 93.0% to 33.1%. The researchers say the change reduced looping; it did not make the behavior disappear.
Was the model trained?
The researchers say they did not use gradient descent, backward passes, or loss-based weight fitting. However, Astra used an iterative process with test feedback, including 50 forward passes in the highest-budget run, so “without training” does not mean without extensive experimentation.
Has the result been peer-reviewed?
Not according to the supplied source material. The post was published on November 2, 2026, and its authors describe the findings as very early. It is not identified there as a peer-reviewed paper or an arXiv preprint.
Source: Thorsten Meyer AI
