Aria's memory, Part 3: Keeping it tidy, and in your hands

After Part 1 and Part 2, Aria can decide what’s worth remembering and find it again when it matters. Run that for a few months and a different problem appears. The memory store slowly turns into a junk drawer: three slightly different versions of the same fact, things that stopped being true, and the occasional misunderstanding that keeps coming back.

This last memory post is about maintenance, the unglamorous part of RAG, and about the rule from the introduction that I care about most: nothing hidden. Anything Aria remembers about you, you can see, fix or delete.

Consolidation: merging what’s really one fact

Deduplication (Part 1) catches near-identical facts as they arrive. It can’t catch facts that grew apart over time:

• “The user works as a developer.”
• “The user writes C# at work.”
• “The user’s job is in software.”

Each one was different enough to be stored on its own. Together they’re one fact said three ways, and they waste three of the twelve memory slots in a prompt.

So about ten minutes after new memories arrive, a consolidation job runs (it can also be started on request through the API). It works entirely from vectors that already exist:

1. For each active memory, Qdrant finds its nearest neighbours among your other memories, reusing the stored vectors rather than embedding anything again.
2. Memories with a similarity of 0.82 or more are grouped.
3. The analysis model gets each group and decides whether it really is one fact. If it is, it writes the merged version: “The user is a software developer who writes C# at work.”
4. The merged memory replaces the group and keeps every source of the originals, so its provenance is complete.
5. The originals aren’t deleted. They’re archived, marked as merged into the new one, and stay visible.

Two guard rails keep the model honest. A merged memory that comes back longer than all the originals combined is rejected, because a merge that adds words is inventing details. And pinned memories are never consolidated: if you pinned it, it stays exactly as you left it.

Forgetting on purpose

Some memories should fade. Part 1 covered the ones with an expiry date (“the user is preparing for a job interview” lasts 1 to 30 days). Recency in the ranking (Part 2) handles the rest gently: a memory nobody has used or confirmed in a year still exists, but it has to be a somewhat better match to come up.

Deleting is different, because deleting has to mean deleted:

• The row in PostgreSQL is removed immediately. From that moment the memory can’t be retrieved, because every search is checked against PostgreSQL (Part 2).
• The vector in Qdrant is removed by a background job, retried until it succeeds. If Qdrant was down at the time, the memory is still gone; only its vector lingers.
• The reconciler sweeps for leftovers every 60 seconds, removing any vector whose memory no longer exists.

That order matters. If deletion waited for Qdrant, a down index would mean a memory you deleted could still turn up. With PostgreSQL first, the worst case is an orphaned vector that nothing can ever return.

The same reconciler repairs the opposite case: memories that were saved while the embedding model was unavailable are indexed as soon as it’s back. Between them, the database and the index drift apart for at most a minute, and in both directions the database wins.

The Memories page

All of this would be worrying if it happened out of sight. So everything Aria remembers is on the Memories page, and it’s not a read-only view.

For each memory you can see:

• The memory itself, its type and its category.
• Why it matters, when Aria knows (“Why: …”).
• Where it came from, as links to the messages it was learned from.
• How it was learned: said by you, inferred, or a pattern noticed over time. Inferred memories are marked as held tentatively.
• Its emotional significance.
• Its history: every change, from what to what, and why, or “Unchanged since it was learned.”
• Its state: pinned, held until the relationship is closer, merged, archived, or waiting to be indexed.

And you can act on it:

• Search your memories in your own words, and filter by category or pinned only.
• Add a memory yourself. Useful for facts you’d rather state than wait for Aria to pick up.
• Edit one that’s wrong. The edit is saved, and the memory is re-embedded from the new text.
• Pin one so it’s always included in the profile, whatever you’re talking about.
• Delete one, with the guarantees above.

There’s also a global switch (MEMORY_ENABLED). With memory turned off, nothing is extracted, retrieved or added to prompts.

Life: the things that change

Some of what Aria learns doesn’t fit a list of facts. Your life moves: you start a new project, finish another, have a birthday, have a rough month. With life tracking turned on, Aria keeps three things on a separate Life tab, noticed from conversations as they happen:

• Focus: what you’re working on or dealing with right now, and what you’ve moved on from.
• Milestones: dated events like a new job, a move or a wedding, with their anniversaries so they can be remembered a year later.
• The mood journal: how you’ve been feeling lately, and why, kept for a set number of days. It’s switched on separately, because a log of your moods is about as personal as data gets.

Like memories, everything there can be corrected or removed, and the whole mood journal can be cleared in one go. When it’s included in a prompt, it’s labelled as data with a reminder to use it naturally: “do not recite it.” There’s nothing quite like a friend reading your own diary back to you to ruin a conversation.

What I learned

• Maintenance is part of the RAG design, not an afterthought. Without consolidation and reconciliation, retrieval quality slowly decays even if the retrieval itself never changes.
• Delete from the source of truth first. If deletion depends on the index being up, “delete” quietly becomes “delete eventually”.
• Transparency keeps the system honest. Being able to see why a memory exists, where it came from and how it changed makes bad extractions easy to spot and fix, and an AI that remembers you is a lot easier to trust when you can check what it remembers.

That’s memory. Next up, the other half of what makes Aria feel continuous: Conversations, Part 1, where one ongoing conversation streams in real time and keeps going even when you close the tab.