Teaching a twin without making it forget

At Tsuin.ai we’re building an AI twin that learns how you think: your logic, your reasons, the way you make decisions. Teaching it one thing turned out to be the easy part. Teaching it the second thing without wiping out the first is the hard part.

Researchers call this continual learning. For us it’s simply the problem standing between a demo and a twin that actually grows with you. Here’s what we measured, what broke, and what finally worked.

The loop that has to close

Every twin runs the same simple loop. You work. The twin guesses what you meant. You correct it with your real reasoning. We turn those corrections into a small training update.

Step four is where it gets tricky. Today’s update lands on a model that already carries yesterday’s. If the new lesson quietly erases the old one, the twin isn’t learning. It’s just changing.

The bad news: every gain had a price

We wanted two things from each update. Pick up the person’s voice. Keep the twin’s good judgement, like saying “I can’t do that” to an impossible task. We couldn’t get both.

The update that finally learned a voice refused only 4 impossible tasks out of 60. The base model refuses 27. I hoped there was a sweet spot in the middle. There isn’t: at the medium size, the judgement is already gone and the voice still hasn’t arrived.

We tried five different fixes. None kept both. One even looked like a clear win, until we checked the thing it wasn’t aiming at: tool use dropped from 23 out of 24 to 7. Since then, every fix gets tested on what it might break, not only on what it should repair.

What finally worked

The breakthrough was a little counter-intuitive. Instead of re-teaching the twin its good habits, we mixed in about 8% of the original model’s own answers while training. The best mix had no refusal examples in it at all. We weren’t teaching restraint back. We were stopping it from being erased.

We also learned that skills live at specific depths inside the model. Voice sits around layers 8 to 19. Training only layers 12 to 15 gave us 98.8% of the effect of training the whole thing, for a fraction of the cost.

Then came the economics question: when is training even worth it? The fast option is lookup, where the twin pulls up your closest past correction and copies the pattern.

Lookup is good from day one, but it never improves, because only one example fits in the model’s short memory. Training starts weak and then wins clearly. So our rule is simple: use lookup until a person has given 80 corrections, then train. On our test teacher, the trained twin scored 0.91, against 0.68 for lookup and 0.35 for the stock model, in just over an hour of GPU time.

Where we are now

Three twins are live today: two for developers, and one, new this week, for someone in marketing who doesn’t write code at all. None has reached 80 real corrections yet, so all three still run on lookup. The training path works, but so far it’s only been proven on a test teacher. Those 80 corrections are the one number between us and a real result.

The lesson that stung most wasn’t in any chart. One twin collected nothing for six weeks. A routine deploy had quietly replaced the code that captured corrections, and nothing alerted us. We only noticed because we asked why the count was stuck at 20. A learning system fails silently, because no new data looks exactly like a quiet week.

What’s still open:

• Nobody has given 80 corrections yet
• Our smallest GPU can’t fit the context we need (1,462 tokens against 1,024)
• We can predict the voice-versus-judgement trade, but we can’t yet explain why it happens