GPT-6 Astra Edits Qwen Weights: What the MIT Test Shows

The questions behind the headlines are: what did Astra actually do, is the 48.1% real, does it work on any model, can I get the edited model, and what would it cost me to try? We keep official facts, what users report and our take apart.

Official facts

Checked on 3 November 2026. The primary source is the authors' blog post by Rikiya Takehi, Jayden Teoh, Alex Zhang and Omar Khattab (MIT CSAIL affiliations). It is a blog post with released logs, not a peer-reviewed paper, and OpenAI is not an author. The post does not name the exact Astra snapshot or settings it used.

Item	Fact	Source
Published	2 November 2026	authors' post
Editor model	GPT-6 Astra (OpenAI), used "as it is", no fine-tuning	authors' post
Target model	Qwen3.5-0.8B (thinking on), plus Gemma 3 270M and small custom networks	authors' post
Edit size	131,072 weights, 0.016% of the model	authors' post
Rules	no gradient descent, no training loop; 10, 25 or 50 forward passes on practice problems	authors' post
Result	MATH-500 7.9% to 48.1%, GSM8K 13.8% to 67.1%, IFEval 21.0% to 62.3%	authors' post
Edited model	public on Hugging Face, Apache-2.0, about 1.75 GB	model page
Logs	every prompt, reply, code run and score released	GitHub

What is weight-space programming?

It is the authors' name for a general-purpose LLM reading and writing a network's weights directly, in natural language, instead of an optimizer such as Adam doing it. The LLM is given the weights (as text or in a Python session) and a plain-language goal, for example "make this model better at math", and returns new weights. The authors say it is not the same as asking an LLM to write training code: in these runs "Astra" was not allowed to write training loops.

Can GPT-6 Astra really read and edit weights?

The authors report eight experiments. In short, per their post:

• Reading. Given only 17,253 weights as text, Astra named the game each of four networks was trained on (Breakout, Pong, Pac-Man, Freeway). It told three tiny diffusion models apart (circle, square, triangle) and, from the middle layers alone of three 7.2M-parameter language models, named the training language (Python, romaji, English) and found the "copying" attention heads that the authors measured by running the models.
• Fixing. A PPO network could not get a car over a ramp in the Kinetix environment (0 of 256 episodes). With one reply and no tools, Astra edited 3.5% of the 171,664 weights and the car succeeded in all 1,024 test episodes.
• Writing from scratch. With every weight starting at zero and no data, Astra wrote networks for three MinAtar games. The post reports they beat PPO trained for 100 million steps in all three. Against 500 million steps of PPO, Breakout was far higher, Asterix close and Seaquest about 60%.
• Go. For a 9x9 Go network with 1,480,274 weights, too large for the context window, Astra used a Python sandbox. With 50 allowed forward passes it beat a PPO network trained on 20 million games.

These are small networks and single setups. The authors call the results "exciting but still limited".

What exactly did Astra change in Qwen3.5-0.8B?

Qwen's own model card says the 0.8B model runs in non-thinking mode by default and, in thinking mode with the recommended sampling settings, is "more prone to entering thinking loops compared to other Qwen3.5 models". In the authors' setup the base model hit the 32,768-token limit on 93% of MATH-500 answers, so most answers never finished.

Astra traced the looping to one direction written by the MLPs of layers 15 and 19 and subtracted it from 64 output columns in each layer (131,072 weights). No new knowledge was added.

Did the edit really make Qwen better at math?

By the authors' numbers, yes in their setup; the reason is narrower than "learned math". The edit mostly lets the model finish answers: the share of MATH-500 answers hitting the token limit fell from 93.0% to 33.1%. With 10 or 25 forward passes the loops barely went away.

Qwen3.5-0.8B	MATH-500	GSM8K	IFEval	Hit token limit
Base	7.9%	13.8%	21.0%	93.0%
Edited, 10 passes	12.3%	25.7%	49.0%	87.2%
Edited, 25 passes	14.1%	25.1%	39.9%	86.4%
Edited, 50 passes	48.1%	67.1%	62.3%	33.1%

The Hugging Face card of the edited model adds an independent re-run by the person who rebuilt the checkpoint: 51.0% on the full MATH-500 (500 problems, 4 samples each, 31.6% hitting the limit). That is the card's own statement; we did not repeat it.

Why is the Qwen baseline only 7.9%?

Because the test forces thinking mode with a loop-prone sampling setup and a 32,768-token cap, and the model rarely ends. A Hacker News commenter, pama, questioned whether the baseline reflects a setup issue and said (unverified) that a non-thinking Qwen variant scores 55.2% on MATH-500, above the edited 48.1%. We could not confirm those figures. What the Qwen card does confirm is the loop warning and that non-thinking is the default for this size. The honest reading: the edit repairs a failure of one configuration, and the 7.9% is not what most users would see.

Is this just abliteration?

Partly, by commenters' reading. Abliteration removes a "refusal direction" from a model's weights; the idea comes from Arditi et al., 2024. Here the removed direction causes looping instead. Aurornis wrote that a motivated human with the right tools could have done the same, and augment_me noted that Astra cannot insert new knowledge, only adjust existing routing. The authors present the new part as a general-purpose LLM doing the whole loop itself, from reading to editing, with a budget of forward passes to check its changes.

Where did Astra fail?

Two results in the same post go the other way:

• Gemma 3 270M, a model that rarely loops (8.5% hit the limit). Astra's edits stayed small and "none moved MATH-500 by more than about a point" (6.8% base; 7.0, 7.6, 7.7 and 7.0 after edits). The authors write that editing weights "could not add an ability the model lacked".
• Finding the sycophantic model from weights alone. In all four attempts Astra could not tell which of two Gemma copies was trained to agree with the user. It succeeded in all three pairs only after it was given attention patterns from texts it wrote itself.

Commenters picked up the same point: epihelix said it is "not nearly at the we-can-improve-everything stage", and BoorishBears noted that Astra did nothing with raw numeric weights once the work moved to Qwen; there it used a Python kernel and forward passes.

Can I download the edited Qwen model?

Yes. The authors published it as RikiyaT/Qwen3.5-0.8B-weight-space-programming on Hugging Face under Apache-2.0 (a Qwen3_5ForConditionalGeneration checkpoint, one 1.75 GB safetensors file plus evaluation files). A Hugging Face staff member, Ahsen Khaliq, is credited in the post for rebuilding it and for a live demo Space. We did not download or run it. At the time of writing the page showed 10 likes and 0 downloads, so there is little community feedback yet.

How do I stop Qwen3.5-0.8B from looping?

You do not need Astra for the common case. Qwen's card recommends tuning the sampling parameters for your use case and using streaming so you can detect and interrupt a runaway generation. It also lists non-thinking mode as the default for this model. The card's recommended thinking-mode text settings are temperature 1.0, top-p 0.95, top-k 20, presence penalty 1.5. The authors' test used temperature 0.6 with the same other values, which is one more reason to treat the baseline carefully. We did not test which setting loops least.

Can I check the results myself?

The authors released logs for all eight experiments on GitHub: inputs sent to Astra, its replies and reasoning summaries, the code it ran and the outputs, the networks, and the scoring. The repository licence is listed as "Other". We did not rerun anything. For Qwen, the edited checkpoint plus the base model are public, so anyone with a GPU can repeat MATH-500 with the settings in the post.

GPT-6 Astra: price and access

The editor in this study is a commercial model. From OpenAI's model page and pricing page, read 3 November 2026 (Astra was released on 3 September 2026 per the changelog):

Item	GPT-6 Astra
Model ID	gpt-6-astra
Context window	1,050,000 tokens (922,000 max input, 128,000 max output)
Knowledge cutoff	30 April 2026
Input / cached / output	$10 / $1 / $50 per 1M tokens
Over 272K input tokens	input and cache 2x, output 1.5x for the whole request
Batch and Flex	50% of standard; Fast mode 2x
Reasoning effort	low, medium, high, xhigh, max (no none)
Endpoints	Responses, Chat Completions, Batch

Astra does not accept custom temperature or top_p. Use the Responses API with the model ID above; the usage steps are in the OpenAI docs and we did not call it.

What one experiment could cost (our arithmetic, not OpenAI's). The car network in Experiment 4 has 171,664 weights. If each number takes 3 to 5 tokens as text (our assumption, not a published figure), one read is about 0.5 to 0.9 million tokens. Above 272K input tokens the long-context rate applies: $20 per 1M, so roughly $10 to $17 for the input of a single pass, before reasoning tokens billed at $50 per 1M output (75 per 1M in long context). The Qwen run avoided this by giving Astra a Python session instead of pasting 0.8 billion weights.

GPT-6 Astra vs Sol: which should I use?

OpenAI sells Astra as its most capable model at $10 / $50 and GPT-6.1 Sol at $2 / $10 for complex coding and professional work at a lower cost. The study used Astra only; nobody has published a Sol comparison on weight editing as far as we found. For ordinary API work, see our guide to GPT-6.1 Terra and the rest of the lineup.

What users report

From the Hacker News thread (2 to 3 November 2026, over 2,300 points and 250 comments at check). These are opinions; we verified none. Reddit blocks our fetch tools, so we did not read it.

• Astonishment. vessenes wrote it is "very, very surprising" and upsets some interpretability assumptions; jstummbillig called it obvious in hindsight and possibly very important.
• "Not as big as it sounds." l9o argued the Qwen result is not "taught math": the model already knew math and was looping, and the Gemma failure shows the limit. swiftcoder said the method is the same as human jailbreak or ablation work, and the surprise is that Astra does it at all.
• Doubts about the baseline. pama questioned the 7.9% baseline and added that, in their reading, the tuned model is worse than earlier Qwen checkpoints that did not loop (unverified).
• Qwen is known to loop. StilesCrisis said Qwen in reasoning mode "ends up going in circles" in general and that others have fine-tuned this away, but not "by hand".
• Self-improvement fears. lifty predicted labs will train models for recursive self-improvement; kristiandupont said they expected it once programming skill passed humans, and that this could be faster.

Is this recursive self-improvement?

Not yet, on this evidence. The authors list recursive self-improvement as a future direction, where a model could inspect and improve its own weights or a newer model's. The experiments edit other, much smaller models, and Astra needed forward passes and a harness to check its changes. Nothing in the post shows Astra editing itself or an OpenAI model. Commenters read more into it than the authors claim.

Our take

• Interpretability and safety people: the reading results (language of a model from middle layers, copying heads found without running anything) are the most interesting part and cheap to check against the released logs.
• People who run small local models: the practical lesson is that loop failures can be a one-direction problem. Try the sampling and streaming advice on Qwen's card before anything exotic.
• Anyone hoping to "upgrade" a model with a prompt: wait. The one clean upgrade came from removing a defect, and the authors' own Gemma test shows it does not add ability.
• Be careful with the 48.1%. It is a single-source number from a blog post, with a baseline that others question. The independent 51.0% re-run on the model card is encouraging but comes from the same release.
• Watch for: the benchmark and harness the authors say they are building, and any replication by an unrelated group.

What we did not test

• We did not use GPT-6 Astra, call the API or edit any weights.
• We did not download or run the edited Qwen model or repeat any benchmark.
• Numbers from the authors' post and the Hugging Face card are quoted, not reproduced.
• The commenter claims about other Qwen scores are unverified.
• We could not see the exact Astra snapshot or sampling settings the authors used, because the post does not state them.
• We did not read Reddit, X or the OpenAI announcement pages.

FAQ

What is weight-space programming?

It is the authors' term for a general-purpose LLM reading and writing a neural network's weights directly from a plain-language goal, without gradient descent or training code.

Did GPT-6 Astra edit Qwen's weights?

According to the authors' 2 November post, yes: Astra edited 131,072 weights (0.016%) of Qwen3.5-0.8B and MATH-500 went from 7.9% to 48.1% in their setup. We did not reproduce it.

Can GPT-6 Astra make any model better at math?

No evidence of that. On Gemma 3 270M, which rarely loops, none of Astra's edits moved MATH-500 by more than about a point. The Qwen gain came from fixing thinking loops.

Can I download the edited Qwen3.5-0.8B?

Yes. It is on Hugging Face as RikiyaT/Qwen3.5-0.8B-weight-space-programming under Apache-2.0, about 1.75 GB, together with its evaluation files. The authors' logs are on GitHub too. We did not download or test it.

How much does GPT-6 Astra cost?

$10 per million input tokens, $1 cached and $50 per million output, per OpenAI's pricing page. Prompts above 272K input tokens cost 2x input and 1.5x output for the whole request.

Is this abliteration for math?

Similar in spirit: both remove a direction from the weights. Commenters note that the loop-causing direction here is analogous to the refusal direction in abliterated models. The new part is a general LLM doing the search and edit.

Does this mean AI can improve itself now?

No. The post edits much smaller models and uses forward passes to check edits. Self-improvement is only listed as a future possibility.

Related guides

• GPT-6.1 Terra: API price, context and limits for the rest of OpenAI's GPT-6 lineup and its prices.
• Mistral Large 4 open weights: licence, price, hardware for another recent case of what "open weights" lets you do.
• Ollama models for coding: what fits in 8, 16 and 24 GB if you want to run small models like Qwen locally.

Sources

• Takehi, Teoh, Zhang, Khattab: Sparks of Weight-Space Programming (2 Nov 2026, read in full 3 Nov).
• Hugging Face: RikiyaT/Qwen3.5-0.8B-weight-space-programming (edited model card, re-run note).
• GitHub: RikiyaT/lm_understand-logs (experiment logs).
• Qwen: Qwen3.5-0.8B model card (thinking-loop note, sampling settings).
• OpenAI: GPT-6 Astra model page, pricing, API changelog (read 3 Nov 2026).
• Arditi et al.: Refusal in Language Models Is Mediated by a Single Direction (2024).
• Hacker News: Astra can read and write weights to improve Qwen (2 Nov 2026).
• Google autocomplete suggestions for "gpt-6 astra", "gpt-6 astra api" and "qwen3.5 0.8b" (3 Nov 2026).

Update log

• 3 November 2026: first version, from the authors' post, the Hugging Face cards, OpenAI's docs and the Hacker News thread. We will update it if the authors publish their benchmark or if an independent group replicates the Qwen result.

Last verified: November 2026 (3 November 2026). Research results and prices change; we will update this page and the log above.

Header image: Image: DevOpsCompass.