Where this started
One rule can serve many facts
I want to find out how far a small open model can be pushed toward frontier level by editing its weights directly and grafting in new parts, without a heavy training run. Training is the usual route. You push billions of questions and answers through a model and gradient descent works out the rules on its own. But we can also open a model up, read what it holds partway through a problem, and change it. So can some of those rules be written in by hand?
My working theory is that a lot of reasoning runs on rule sets, where one rule serves many facts. Carrying the one is the simplest case. Nobody memorizes every sum, because one carry rule covers all of them, at every digit place. In my notes I put it this way:
Maybe a rule from one fact would coincide with the rule for another.
Facts are expensive for a small model to store, and rules are cheap. The first multiplication circuit in this write-up is built for every problem from 2 × 10 up to 9 × 99, and it's 25,270 weights.
The same conversations turned up a second idea that pairs with the first: a learning ruleset for picking up something new, alongside logic rules for applying what you already know. This write-up is about the logic side.
What the model already has
The digits are in there
Before building anything, we looked at what SmolLM2 holds at the moment it starts writing an answer. A linear read of its hidden state at the answer position, fit by least squares, pulls out the digits of both numbers almost perfectly. For three-digit addition one layer wasn't enough, but joining layers 14, 20 and 21 reads every digit at 99% on fresh problems.
Three-digit addition. Share of fresh problems where a linear read gets all six digits of the two numbers right, reading one layer at a time, then layers 14, 20 and 21 joined. Each read is fit on 1,500 problems.
76.6% Best single layer, layer 21
11.8% Last layer, layer 29
97.9% Layers 14, 20 and 21 joined, and 99.2% when that read is fit on 3,000 problems
All 30 layers as a table
| Layer | All six digits right |
|---|---|
| 0 | 0.0% |
| 1 | 4.3% |
| 2 | 26.0% |
| 3 | 50.1% |
| 4 | 52.2% |
| 5 | 47.1% |
| 6 | 31.4% |
| 7 | 20.4% |
| 8 | 18.8% |
| 9 | 18.2% |
| 10 | 27.1% |
| 11 | 27.9% |
| 12 | 39.1% |
| 13 | 47.2% |
| 14 | 74.8% |
| 15 | 70.1% |
| 16 | 64.0% |
| 17 | 63.0% |
| 18 | 64.2% |
| 19 | 63.5% |
| 20 | 72.9% |
| 21 | 76.6% |
| 22 | 71.5% |
| 23 | 62.9% |
| 24 | 48.8% |
| 25 | 30.5% |
| 26 | 19.0% |
| 27 | 13.9% |
| 28 | 11.7% |
| 29 | 11.8% |
So the model has read the question. What it can't do reliably is combine the digits: the carry, the borrow, the times table. That combining step is the part we supply.
How a graft works
Four parts, none trained by gradient descent
- ReadA linear read for each digit of the two numbers, taken from the model's own hidden state.
- RulesA small lookup table for each digit of the answer. The units digit comes from the two units digits; the tens digit comes from the tens digits plus the carry, and so on up.
- WriteA push toward the right digit token at the model's last layer. The model still writes its own reply. The graft only tips which digit comes next.
- SwitchesTwo fitted switches decide when the graft is on. They are fit to fire only while the model writes the answer to this kind of problem, and to stay off on other math, word problems, code and ordinary text.
The reads and switches are fit by closed-form least squares, and the rule tables are written out by a program (by hand, for the first multiplication graft). The base model file is never touched. Each graft is added alongside it.
How it grew
From one graft to a pipeline
- Neuron graftThe first try added 24 new neurons to the last layer to supply the carry for two-digit addition. It held up on sealed problems, +19.7 points. It worked by nudging the model's own arithmetic, so the next step was to do the arithmetic outright.
- Hand-wired circuitFor multiplication we wrote the rule tables by hand and fed them the model's own digit reads. The first version scored 83.5% on sealed held-out problems, up from 21.3%. A second version with better switches scored 83.1%, and its worst change on other math was −0.75 points.
- Rules found by a programA rule-finding program got 340 solved problems as digits and answers. For each answer digit, it looked for the smallest set of inputs that decides it. It found the carry for addition, the borrow for subtraction and the carry chain for multiplication on its own. It also found that the tens digit is the units rule reused on the tens digits, plus a carry step: one rule serving several digits, which is the idea this started from. Its multiplication circuit scored 82.7% held-out, within half a point of the hand-wired one.
- Same pipeline, new skillsSubtraction and two-digit addition went through the same script, with settings copied from multiplication and no tuning. Three-digit addition took three fixes, each tested before its held-out set was opened: the read joined from three layers, a second switch, and a push strength set for each digit place.
Why it worked
Why the program could find the rules
The rules didn't come from the model. A separate program found them, outside the model, from 340 solved examples. The model's part is supplying the numbers, which the graft reads from its hidden state. Four things made it possible.
- Splitting problems into digitsAs whole numbers, 7 × 48 = 336 is one fact among 720 possible problems. Split into digits, a pattern shows up: the last digit of the answer depends only on the 7 and the 8. This was the most important choice we made, and it suits SmolLM2, which stores each digit as its own token.
- Asking for the simplest explanationFor each answer digit, the program searches for the smallest set of input digits that always decides it, and keeps a rule only if it holds on every example. A rule keyed on two digits has about 100 possible entries, and 340 examples fill nearly all of them, so it works on new problems that reuse the same pairs. Memorizing would mean keying on every input digit, with far more combinations than 340 examples can cover.
- Reusing what it already foundFor the next digit up, it tries the units rule again and checks whether one extra signal explains what's left over. That signal turned out to be the carry. Nobody told the program the word; it found the variable because the data needed one. Repeating the step for the hundreds digit gives the carry chain.
- Checking on unseen problemsThe rules were tested on problems they hadn't seen, then compiled into lookup tables inside the graft.
Gradient training has no built-in pull toward the simplest explanation, and with few examples it memorized. On multiplication, the same-size gradient graft scored 72.7% on its own training problems and 25.1% on the 140 dev problems. The circuit scored 94.7% on the problems it was built from and 88.7% on the same dev problems.
We set the digit framing, and the program finds rules within it. The next step toward skills beyond arithmetic is a program that also finds the right way to break a problem down.
Results
Five skills, on problems nobody tuned on
Each skill's held-out problems were set aside before any building, and nothing was tuned on them. Every score is two seeds and four runs per problem, with the same sampling settings for the model alone and with the graft. The multiplication set was opened once for the first hand-wired version, then once more for the final two.
Share of problems answered exactly right. 140 problems for multiplication, 500 for the rest.
- SmolLM2 alone
- With the graft
- Same graft, wiring scrambled
| Skill | Problems | SmolLM2 alone | With the graft | Gain, points (95% lower bound) | Scrambled wiring | Added weights |
|---|---|---|---|---|---|---|
| Multiplication, rules wired by hand7 × 48 | 140 | 21.3% | 83.1% | +61.8 (+56.1) | 2.4% | 25,2700.019% |
| Multiplication, rules found by a program7 × 48 | 140 | 21.3% | 82.7% | +61.3 (+55.5) | 6.3% | 32,5190.024% |
| Subtraction, two digits83 − 27 | 500 | 11.3% | 68.3% | +57.1 (+54.0) | 7.8% | 32,4460.024% |
| Addition, two digits38 + 47 | 500 | 49.7% | 86.5% | +36.8 (+33.8) | 16.7% | 32,4760.024% |
| Addition, three digits347 + 285 | 500 | 26.1% | 83.7% | +57.6 (+54.6) | 21.4% | 121,1980.090% |
- No original weight changed, and there was no gradient training. Each graft adds 25,270 to 121,198 weights, 0.019% to 0.090% of the model. The cap we set ourselves was 0.1%.
- The scrambled control is the same graft with its wiring shuffled. It scored at or below the model alone on every skill, 2.4% to 21.4%, so the extra weights do nothing without the right rules.
- The three two-digit grafts (multiplication, subtraction and addition) also run together in one model, 92,496 weights in all. Each kept its gain on dev problems. The worst change on other math was −0.25 points, and 200 of 200 unrelated prompts gave the same answer as before.
Gains are paired by problem, with a one-sided 95% bootstrap lower bound. The side-effect check covers eight other arithmetic and word-problem sets, plus GSM8K and code. Unrelated prompts are 200 IFEval prompts, answered greedily. The rules and predictions for every run were written down before its result.
Against gradient training
Three head-to-heads
Gradient training is the standard way to add a skill, so we ran it as the competitor. Each gradient graft was tuned over a grid of settings before its test.
140 problems, 2 seeds, 4 runs each. The first hand-wired circuit against two tuned gradient-trained grafts.
The circuit was ahead by 57.9 and 59.8 points.
140 dev problems, 2 seeds, 4 runs each. LoRA used the best setting from a grid search.
49.3 points ahead, with a 95% lower bound of +42.8, using under a quarter of the weights.
500 dev problems, 2 seeds, 4 runs each. The circuit's settings were copied from multiplication, untuned.
A tie with a gradient graft four times its size, and about 17 points ahead of the 14-neuron one.
The circuit's subtraction rules came from 340 problems. The gradient grafts trained on 1,500, the same 1,500 the circuit's digit reads were fit on.
Still in progress
Transplants from bigger models
We're also trying to move skills over from larger models. Qwen3.5-4B got 300 of 300 of these multiplication problems right in our check, so we tried translating a set of its neurons into SmolLM2. Separately, we mapped a correction signal from Qwen3.5-4B into Qwen3.5-0.8B on grade-school word problems. Neither has worked yet, and both are still open. So far the bigger model has been most useful as a guide to where the small one breaks.
What a graft is
A calculator fed by the model's own reading
The graft does the arithmetic itself, from numbers the model already holds. SmolLM2 doesn't learn to multiply. It works on the plain form of the question ("What is 347 + 285?"), and the switches keep it off on word problems on purpose. All of this is one model so far.
The method carries over to new skills. Rules are found by a program from 340 examples, compiled into tables, and placed with the same settings for each skill. Next is the learning side: getting SmolLM2 to use a fact it's handed in the prompt.