Mindcache Tree-First Memory Reset

Date: 2026-04-11 Status: Published note Scope: Runtime architecture reset

Over the last few weeks, Mindcache's runtime slowly taught us where its original design was too ambitious and where it was simply wrong.

The early instinct was graph-first:

import a source
extract structured nodes
connect them immediately
grow a memory graph over time
derive a topic tree as a faster view on top

That was elegant on paper, but it kept breaking in practice.

Small snippets became too fragmented. Large documents became too heavy. Relation-building grew more complex as the node count grew. Search over a growing graph started to carry the exact scalability burden we were trying to escape.

This update is the point where we stopped treating those symptoms as implementation bugs and started treating them as architectural feedback.

The result is a reset:

the long-term structure is now tree-first
the durable atom ontology is now fact / event / plan
raw inputs are treated as source material, not durable knowledge units
daily digest is responsible for reconciliation and consolidation
graph inspection remains available, but it is now a secondary atom-layer/debug surface

What Changed

The new runtime separates the system into three clearer layers.

1. Source / Material Layer

Inputs are first preserved as source material:

snippet
resource
writeup

This layer exists to preserve provenance and to retain the original inputs that later consolidation can revisit.

This means external reading material is no longer treated as if it were already memory.

A resource is something the system may later learn from, but on import it is just:

a preserved source
a short summary
keywords
suggested topics
and, optionally, an event like "saved this resource today"

2. Atom Memory Layer

The durable atom ontology is now intentionally small:

fact
event
plan

This is the core change.

We removed entity, summary, and document from the runtime's primary memory ontology because they were adding more conceptual and maintenance cost than retrieval value.

plan is also no longer treated as a thing that can sit forever in a "completed" state. If a planned thing happens, it becomes an event. If it does not happen, it remains a plan with a missed or canceled status.

3. Topic Tree Layer

The topic tree is no longer "just a convenient view."

It is now the primary retrieval structure.

This changes the role of search completely:

retrieval starts from topics
topics route us downward
atoms provide detail

The old graph mental model assumed the system should precompute lots of pairwise relations between nodes. The new model assumes that most useful relatedness is better expressed as:

shared topic membership
parent/child topic hierarchy
daily and temporal organization

That is much closer to how a human memory system feels in practice.

Why We Made This Change

The two biggest runtime problems were pointing in the same direction.

Small snippets were too granular

When a lot of small notes are imported over time, a graph-first system starts to overfit on local pairwise relations:

too many candidate links
too much traversal
too much ambiguity about which edges matter

Even if the relations were correct, search became too dependent on leaf-level navigation.

Large documents were too coarse

When a large document was recalled, too much context came back at once. We were paying a high token cost to answer questions that often depended on only a small part of a source.

This combination is exactly what pushed us away from "everything should become a graph node with relations" and toward a more layered design:

source material stays source material
atoms remain small and durable
topics become the main navigation structure

The Role of Daily Digest

One of the clearest conceptual shifts in this reset is the role of digest.

We no longer want import-time to behave like full consolidation.

Import should do only what it can do well:

preserve source material
classify the input
extract atom nodes

Digest is where memory is allowed to reorganize.

Daily digest now has two jobs:

1. Reconcile atoms through a daily writeup

The system drafts a daily writeup from the day's inputs and extracted atoms, then lets the model refine the atom set by:

keeping
adding
modifying
deleting

The writeup is not a second parallel memory layer. It is an editing surface for the atom layer.

2. Maintain the topic tree

After reconciliation, digest updates the topic tree.

This is where topic structure is allowed to grow, merge, split, and re-route retrieval over time.

Cold Start vs Consolidated Memory

Another major insight from this round is that the runtime should not behave the same way at every scale.

Cold start

When the atom count is still small, the tree is not stable enough to be trusted as a strong search index.

So during cold start:

search effectively degenerates to the full atom set
digest can rebuild the whole topic tree
large-scale topic reshaping is allowed

This is the "childhood" phase of the memory system.

Post-cold-start

Once the atom count passes a threshold, we stop rebuilding the whole tree every time.

Instead:

tree search returns the top candidate paths
overlapping paths are deduplicated
isolated atoms are attached locally
parent topics can be revised or split when needed

That makes the system much more scalable.

Topic Tree Search

This reset also changed how we think about retrieval performance.

We no longer want search complexity to scale with the number of leaf atoms.

The current retrieval direction is:

lexical-first
topic-tree-first
global beam of size k
cumulative path scoring
threshold pruning

The important detail is that we do not keep a per-layer top-k.

We keep a global top-k set of active paths, which keeps the search budget bounded much more tightly.

This is effectively a beam-style search over the topic tree rather than a search over a dense memory graph.

Influence: mempalace and llm-wiki

Two public references helped sharpen this reset.

`mempalace`

mempalace is interesting because it takes hierarchical organization very seriously. It keeps raw information around, but navigates through a structured memory palace instead of assuming that everything should be flattened into a fully connected graph.

That reinforced a key idea for us:

the main job of a memory structure is not to maximize pairwise connectivity; it is to make retrieval navigable.

Reference:

[mempalace](https://github.com/milla-jovovich/mempalace)

Karpathy's `llm-wiki`

Karpathy's llm-wiki note is valuable for a different reason. It argues for a compiled intermediate knowledge layer between raw source material and final answers, instead of treating retrieval as repeated ad hoc RAG over raw files.

That maps well to our current direction:

raw source inputs are preserved
atom memory nodes provide durable units
daily writeups and topic summaries act like compiled knowledge artifacts

Reference:

[llm-wiki](https://gist.github.com/karpathy/442a6bf555914893e9891c11519de94f)

The New Principle

If I had to compress this entire reset into one sentence, it would be:

Mindcache should be tree-first for retrieval, atom-first for durable memory, and source-preserving for future re-grounding.

That means:

sources are preserved
atoms are durable
topics organize retrieval
digest consolidates
the graph is secondary

This is a much smaller and more disciplined architecture than where we started, but it is also much more believable.

What Is Still Not Solved

This reset made the runtime cleaner, but not complete.

The hardest open problems now are:

improving cold-start topic grouping quality
making topic lexical surfaces better for tree search
tuning local topic revision and splitting
deciding how and when source recall should re-ground older topic structure

These are still active design questions.

But the runtime is now operating under a more honest model:

memory is not born as a graph.

It is first recorded, then refined, then organized.

And only some parts of it ever deserve to become stable structure.