This is the fifth entry in the Digital Companion Build Journal. Midway through the project, the upstream open-source system contributed only an LLM relay. We had replaced its memory, voice, and interface. The largest operation followed: write the dialogue engine and make the system independent. This article records the brain’s design and the long fight against a self-repeating catchphrase after launch.
Why sessions were removed
Multi-session chat comes from task software: open a session for a task and archive it when finished. A companion is not a task. She represents one continuous relationship, and “new session” contradicts that meaning. The upstream session switch also had a persistent defect requiring refresh. Instead of repairing a concept that did not belong, the project removed it.
The replacement is one persistent timeline plus four memory layers:
- Recent window. Roughly thirty recent messages remain verbatim and preserve immediate continuity.
- Rolling summaries. Older material is compressed by a model and maintained separately for the two content contexts described in the third entry.
- Long-term memory pads. Distilled stable facts—habits, anniversaries, agreements—live in plain text that a person can edit directly.
- Full archive plus retrieval. Every sentence is stored permanently. Each user turn triggers lightweight relevance search based on character-bigram overlap, inserting the most relevant old dialogue as a memory. Mention something from last month and she can continue it.
Migration was accepted only when the original 22 content-isolation tests passed unchanged.
The catchphrase war
Several days after launch, the most instructive failure in the project appeared.
Symptom. One short speech habit began closing nearly every reply. Counting showed the same phrase in eleven of her last twenty messages. Worse, the rolling summary now said that “she often uses ××”—a temporary mannerism had been consolidated into long-term personality.
The pathology required three layers:
- Origin. One entirely normal generation used the phrase naturally in its local context.
- Self-reinforcement. Once inside the recent window, it became an example. A language model imitates its own visible history, making the second occurrence more likely than the first and creating a snowball. Non-reasoning models without a self-review step are especially sensitive.
- Consolidation. The summariser truthfully recorded that she often used the phrase. It then appeared in the long-term context read by every generation. The window would eventually flush; the summary would not. The loop was complete.
The first counterattack failed. Adding “do not repeat the same phrase” to the system prompt had little effect. The conclusion deserves emphasis: instructions lose to examples. A dozen demonstrations already in context outweigh one abstract rule.
The second counterattack changed the mechanism in four parts:
- Summary surgery. Remove the consolidated catchphrase manually while preserving every factual memory.
- Summariser vaccine. Permanently instruct summary folding to preserve facts and emotions but never record specific catchphrases or sentence forms, cutting the future consolidation route.
- Generation guard. Keep anti-repetition instructions. They work in a clean window even though they cannot defeat a polluted one.
- Context cut. Add an action that folds every currently unsummarised message and then moves the live-context boundary to the present. Earlier text remains in the archive and remains retrievable, but no longer appears as direct dialogue examples. Facts survive; the voice infection dies. The action became a console button.
Repetition stopped immediately after the cut. The underlying tendency cannot be eliminated—it follows from autoregressive generation—but it is now manageable.
Fast and slow models
One final observation shaped the architecture: LLM work divides naturally by sensitivity to latency.
| Task | Latency sensitivity | Model role |
|---|---|---|
| Conversation reply | Extreme: the user is waiting for her to speak | Fast non-reasoning model |
| Summary folding / context cut | None: background work with no waiting user | Strong reasoning model |
Background compression runs only once every few dozen turns. The stronger model costs less than one yuan per month at that frequency and yields better memory, which improves every future reply. Conversation stays on the fast model because, once the planned streaming synthesis pipeline is ready, first-token speed will directly determine the delay before she begins speaking.
Reusable principles
- Match the concept to the setting: tasks use sessions; relationships use timelines.
- Treat style disease with data mechanisms—cleaning, cutting, and immunisation—not instructions alone.
- The summariser is memory’s gatekeeper. Every trait it records becomes a self-fulfilling prediction, so constrain it to facts.
- Assign models by latency sensitivity. Strong reasoning can work where the user never has to wait for it.