🔍 Read the full analysis: Weight-Space Programming At Home: A Zero-Liability AI Editing Test on ThorstenMeyerAI.com
Get business pricing on office and shipping supplies
- Business-only prices and quantity discounts
- Tax-exempt purchasing
- Multiple users, one account, clear invoices
TL;DR
A home experiment used a directional weight edit to stop a local AI model from looping through answers. On the 10 published Lexassay tasks, the edited model completed all 10, up from 4 in the same comparison, but its legal score remained zero. The result suggests the edit improved completion, not legal knowledge, and is limited by a small test set and single runs.
A home test of Ornith 1.5 35B-A3B found that a targeted edit to the model’s weights helped it finish legal benchmark tasks, but did not improve its legal score. On the 10 published Lexassay tasks, it answered all 10 in the edited run, compared with four in the matched comparison, while scoring zero on the legal benchmark.
The experiment was inspired by MIT CSAIL researchers’ work on “weight-space programming,” but it did not reproduce their method. In the home test, a Claude Opus 5.5 session wrote code to run the local model, identify a direction associated with looping, and remove that direction at half strength from the output weights of 27 of its 40 layers. The model was not trained on new data, and no AI read its weights as text.
The report says the edit changed whether answers finished, not whether they were legally sound. The 10 edited answers contained 55 wrong or invented citations. On 50 checked criteria, the model met seven, compared with three before the edit; its overall legal score remained zero. The benchmark checks cited statutes and court decisions against primary sources.
The test’s author disclosed that Claude carried out the edit and Claude agents graded the answers. OpenAI’s GPT-6 Astra, running in Codex, reviewed the code, figures and claims and identified errors, according to the report. The author describes this as one experiment with an independent reviewer, not two separate experiments.
Infographic 01 · What we changed
One loop direction, weakened at half strength
Inspired by the MIT post, but a simpler, classic technique (directional ablation): code written by a Claude Opus 5.5 session measured one loop direction by running the model and projected it out of the weights. No training, and no AI reading weights as text.
The six steps
- Write test material that is not the benchmark: 24 legal development questions plus 8 general controls, 0 overlap with benchmark prompts
- Find the true loop onset at token level: from there, at least 90% of the next 1,500 positions repeat earlier 32-token sequences
- Run the model and measure one loop direction per layer: average internal signal inside the loop minus the reasoning just before it
- Weaken it in the weights:
W ← W − α·r·(rᵀW)on the MLP output matrices of all 256 experts and the shared expert - Bake an edited copy; the original files stay untouched and the edited weights stay internal
- Benchmark original and edit on the same local setup, graded blind by three Claude agents; figures cover the 10 published tasks
Quick test: share of pure copying
Restart 20 tokens before a known loop onset, 8 cases, 3,000 tokens. Lower is better.
The cost of the edit
- Full strength: 23 of 24 legal questions finished, but control answers broke: tokens dropped out of short formulas, and month counting went wrong
- Half strength: 23 of 24 finished, 9 of 9 checked control facts right, and one German control question was answered in Chinese once
- Before the edit: 8 of 24 legal development questions did not finish; all 8 controls did
One run per setting. The loop detector was revised three times during the experiment, so the development numbers come from a moving yardstick. The direction is a statistical description, not a single “loop neuron”.
Sources: our experiment records, 4–5 Nov 2026 (executing Claude Opus 5.5 session; review by GPT-6 Astra in Codex); Ornith-1.5-35B-A3B model card.
Powered by Thorsten Meyer AI
Completion Is Not Legal Accuracy
The result highlights an important distinction for people evaluating AI tools: a model can produce a complete response without producing a correct one. In this case, the edit appears to have addressed a failure to finish, while the benchmark found no reliable legal performance. A higher count of completed tasks would be misleading if presented on its own as evidence of improved expertise.
That distinction matters especially in legal work, where invented or incorrect citations can make an answer unusable. The test does not establish that weight edits are broadly effective, or that this edited model is suitable for legal use. It reports one model, one experiment and 10 published tasks; the author explicitly says the experiment is not legal advice.
Popular right now
LG 77-Inch OLED evo C6 4K Smart TV
Ninja DualZone Air Fryer 10 Qt Smart 6-in-1 Grey
Dyson V8 Cyclone Cordless Vacuum
As an affiliate, we earn on qualifying purchases.
From MIT’s Edit to a Home Test
The MIT CSAIL work described in the source material used a different approach: GPT-6 Astra examined another model’s weights directly and edited them by reasoning about those values. In the researchers’ reported Qwen example, the model stopped looping and its math score rose from 7.9% to 48.1%. The researchers cautioned that removing an obstacle might expose an existing ability, but would not add an ability the model lacked.
The home experiment tested a related idea through directional ablation, a technique that measures internal model activity and removes a selected direction. Ornith was chosen because earlier benchmark runs showed it looping: two answers did not arrive before a 62,000-token limit, and the report says much of the final output repeated itself. The other candidate models either failed without looping or could not be run under comparable conditions.
Lexassay covers questions on German, U.S. and European Union law, with no internet or external tools. It has 15 questions in total, but only 10 tasks are published; five remain hidden to prevent tuning against them. The updated figures in the report refer to those 10 published tasks, and the runs, answers, grades and citation checks are available on Lexassay.com.
“The model finishes is not the answer is right.”
— The home experiment’s author
Limits of the Published Test
The results are preliminary. The report says there was one run per setting, a small development set and a loop detector that changed during the experiment. Three graders came from the same model family, while the independent GPT-6 Astra review was a check of the work rather than a second experiment.
Comparisons across the report also require care. The unchanged model answered eight of 10 tasks in its published run but answered four of 10 under the other setup used for the matched comparison. The source says these two local setups are not directly comparable. It is not clear from the material whether the edited result will hold across repeated runs, other prompts, hidden tasks or different hardware and sampling settings.
The published runs and the experiment method are available through Lexassay, allowing readers to inspect the answers, grades and citation checks. The five hidden benchmark tasks remain unpublished, according to the report, to reduce the chance of models being tuned directly to the test.
The next useful evidence would be repeated runs under the same settings, a stable loop-detection method and evaluation on additional tasks, including the hidden set when it can be assessed without compromising it. Until then, the reported finding is narrow: the edit helped this model complete more published tasks, but the test found no improvement in its legal score.
Key Questions
What changed after the weight edit?
In the matched comparison on the 10 published Lexassay tasks, the edited model answered all 10, up from four. The report attributes the change to removing a direction associated with looping from selected model weights.
Did the edit improve the model’s legal accuracy?
The report says the legal score stayed at zero. The edited answers included 55 wrong or invented citations, and the model met seven of 50 checked criteria, up from three.
Was this the same method used by the MIT researchers?
No. The home test used code to measure the model’s internal signals and apply directional ablation. The MIT work described in the source involved an AI reasoning directly over another model’s weights.
Can the result be treated as proof that the technique works generally?
No. It was one experiment with one run per setting, using a small set of 10 published tasks. The report says the setups were not all directly comparable, and performance on repeated runs or hidden tasks remains unknown.
Source: ThorstenMeyerAI.com
