Alias ArchiveArchive in progress
Archive in progress

A Homegrown Brain: Four Memory Layers, Fast and Slow Models, and a Catchphrase War

Digital Companion Build Journal05 / SERIES

From borrowing an open-source dialogue engine to a fully independent brain: one relationship timeline, four memory layers, a self-reinforcing style infection, and latency-based model roles.

Filed
Field
Other
Edition
EN / Reading copy

This is the fifth entry in the Digital Companion Build Journal. Midway through the project, the upstream open-source system contributed only an LLM relay. We had replaced its memory, voice, and interface. The largest operation followed: write the dialogue engine and make the system independent. This article records the brain’s design and the long fight against a self-repeating catchphrase after launch.

Why sessions were removed

Multi-session chat comes from task software: open a session for a task and archive it when finished. A companion is not a task. She represents one continuous relationship, and “new session” contradicts that meaning. The upstream session switch also had a persistent defect requiring refresh. Instead of repairing a concept that did not belong, the project removed it.

The replacement is one persistent timeline plus four memory layers:

Four-layer memory diagram: roughly thirty recent verbatim messages enter context; older material is folded into separate rolling summaries; distilled facts live in editable long-term notes; and a full archive supports per-turn retrieval. A context cut keeps earlier text in the archive and summary but removes it from the live window
Every reply assembles context from four layers. The context cut becomes the decisive tool in the catchphrase war.
  1. Recent window. Roughly thirty recent messages remain verbatim and preserve immediate continuity.
  2. Rolling summaries. Older material is compressed by a model and maintained separately for the two content contexts described in the third entry.
  3. Long-term memory pads. Distilled stable facts—habits, anniversaries, agreements—live in plain text that a person can edit directly.
  4. Full archive plus retrieval. Every sentence is stored permanently. Each user turn triggers lightweight relevance search based on character-bigram overlap, inserting the most relevant old dialogue as a memory. Mention something from last month and she can continue it.

Migration was accepted only when the original 22 content-isolation tests passed unchanged.

The catchphrase war

Several days after launch, the most instructive failure in the project appeared.

Symptom. One short speech habit began closing nearly every reply. Counting showed the same phrase in eleven of her last twenty messages. Worse, the rolling summary now said that “she often uses ××”—a temporary mannerism had been consolidated into long-term personality.

The pathology required three layers:

  1. Origin. One entirely normal generation used the phrase naturally in its local context.
  2. Self-reinforcement. Once inside the recent window, it became an example. A language model imitates its own visible history, making the second occurrence more likely than the first and creating a snowball. Non-reasoning models without a self-review step are especially sensitive.
  3. Consolidation. The summariser truthfully recorded that she often used the phrase. It then appeared in the long-term context read by every generation. The window would eventually flush; the summary would not. The loop was complete.

The first counterattack failed. Adding “do not repeat the same phrase” to the system prompt had little effect. The conclusion deserves emphasis: instructions lose to examples. A dozen demonstrations already in context outweigh one abstract rule.

The second counterattack changed the mechanism in four parts:

  • Summary surgery. Remove the consolidated catchphrase manually while preserving every factual memory.
  • Summariser vaccine. Permanently instruct summary folding to preserve facts and emotions but never record specific catchphrases or sentence forms, cutting the future consolidation route.
  • Generation guard. Keep anti-repetition instructions. They work in a clean window even though they cannot defeat a polluted one.
  • Context cut. Add an action that folds every currently unsummarised message and then moves the live-context boundary to the present. Earlier text remains in the archive and remains retrievable, but no longer appears as direct dialogue examples. Facts survive; the voice infection dies. The action became a console button.

Repetition stopped immediately after the cut. The underlying tendency cannot be eliminated—it follows from autoregressive generation—but it is now manageable.

Fast and slow models

One final observation shaped the architecture: LLM work divides naturally by sensitivity to latency.

TaskLatency sensitivityModel role
Conversation replyExtreme: the user is waiting for her to speakFast non-reasoning model
Summary folding / context cutNone: background work with no waiting userStrong reasoning model

Background compression runs only once every few dozen turns. The stronger model costs less than one yuan per month at that frequency and yields better memory, which improves every future reply. Conversation stays on the fast model because, once the planned streaming synthesis pipeline is ready, first-token speed will directly determine the delay before she begins speaking.

Reusable principles

  • Match the concept to the setting: tasks use sessions; relationships use timelines.
  • Treat style disease with data mechanisms—cleaning, cutting, and immunisation—not instructions alone.
  • The summariser is memory’s gatekeeper. Every trait it records becomes a self-fulfilling prediction, so constrain it to facts.
  • Assign models by latency sensitivity. Strong reasoning can work where the user never has to wait for it.

Related reading

03 / LINKS
C01

Building a Digital Companion: A Journal and Reading Map

Two weeks after starting from an open-source desktop companion, the result was a wholly self-owned system. This is both the chronological account and the index to five technical records.

Read post ↗
C02

Valves and Redaction: Content Isolation for a Two-Context Companion

How an AI companion isolates two content contexts structurally: a fail-closed valve, segment-level memory masking, constant bridge text, and three isolation failures.

Read post ↗
C03

A Selfie System That Chooses Its Own Outfit: Controlled Randomness and Image–Caption Consistency

Designing an AI companion that sends selfies in tune with a conversation: a layered outfit engine, explicit image–caption constraints, restrained proactive sharing, and two image-to-image API failures that returned success while ignoring the reference.

Read post ↗