Language Models Can Rewrite Their Own Weights, Improving Without Retraining
Researchers have demonstrated a language model that can modify its own internal weights through its own reasoning, achieving self-improvement without traditional retraining or gradients. This breakthrough allows models to enhance accuracy on new tasks and even repair damaged copies of themselves.

By Clara Tan, Artificial Intelligence
3 min read
A new research paper from Jiajun Fan and a team of collaborators introduces a groundbreaking capability in artificial intelligence: language models that can rewrite their own weights. This self-rewriting mechanism allows a model to improve its performance and even repair itself without the need for traditional training, gradients, or external optimizers.
Traditionally, language models are trained once and then their weights are frozen. Any improvements typically require another training run, prompt engineering, or additional computation during inference. However, this new work demonstrates a third way: a model diagnosing its own failures and directly modifying the weights responsible for those failures.
The researchers used an open 27B parameter model, providing it with its own weights, graded answers, and the ability to run itself. The model was able to diagnose its own shortcomings and rewrite its internal parameters. This self-editing process led to a 6.6-point increase in accuracy on problems the model had never seen before.
Perhaps most remarkably, the model can also repair a damaged copy of itself. When given a corrupted version that could solve almost nothing, the model restored it to over 80% of the original accuracy on image tasks it had never encountered, all without ever reading the original weights and without any retraining.
The capability emerges with scale: in one model family, smaller models either made no edits or broke themselves, while only the largest model successfully improved and repaired itself. This suggests self-rewriting may be an emergent property of sufficiently large models.
Unlike test-time scaling, self-rewriting writes its gains into the weights at no cost per answer. Unlike training, it runs only forward passes, as serving does. The researchers argue the wall between training and inference is a limit of our tools, not a property of intelligence, and believe self-rewriting is a necessary step toward recursive self-improvement.
The paper is available on arXiv (2611.21885) and the project page is at jiajunfan.com/projects/self-rewriting.
Sources
- 1arXiv paperarxiv.org
- 2Project pagejiajunfan.com