ai forecasts in the bay have been extremely heavy; the other day, i met a 17yr old from iran who raised $XXXk to build virtual memory stores (CDNs for memory recall). my aspirated brain only caught up last week and i took some time off to build a memory layer agent for claude code because of how garrulous it was becoming. and because memory intrigued me, i dug around and the word “forget” stuck with me. we first say the agent remembers you; then a user asks us to make it forget, and everyone nods like the two words should be obvious. but they aren’t. forget presupposes remember, and remember presupposes a single store that once held the thing. and the more i built, the more i could see the crude imitations of whatever the real memory systems are doing today.
we can filter down memory retrieval to two substrates, and they are not mutually exclusive; (a) retrieval: rows, vectors, a graph, text so anything you can point at with an address, and (b) weights: the parameters, where whatever the model swallowed in training is smeared across billions of numbers with no address and no row. and usually, forgetting ends up meaning the opposite thing on each side. in retrieval, DELETE is instant and total, which isn’t forgetting but erasure. in weights there is nothing to point at to erase. so you get two failures that look identical from a distance and are actually antonymous. any deployed model is stateless between calls; the weights don’t update when you chat, and there is no fine-tune per user. so if claude code seems to remember you across sessions, that memory physically cannot be in the weights, because nothing put it there. it’s its context. so my solution attempts to solve “forgetting” mimics how our brains recall.
whenever memory feels like the ominous guy, the obvious next move everyone reaches for, is infinite memory: store everything, drop nothing, let retrieval sort it out later. i think that’s backwards. i think we can agree that storage is the cheapest compute in 2026. the scarcity (and the beauty of this problem) lies in findability, and infinite memory is just an infinite haystack so every note you keep is one more thing the right answer has to deal with. so “store everything” doesn’t solve retrieval. rather, it only adds to the intricacies of navigation. and once you see it as navigation, you will almost never agree with “the agent forgot” being the model’s fault. the real issue is that similarity is a weak navigator. embeddings build on top of resemblance, and the memory you need is often not similar to the query at all. if i want to get my sister a gift, i need my agent to recall some history that says “sister went vegan/hates chocolate/wants a bowtie” which shares almost vocabulary with the question.
a flat vector store could be the bottleneck here, and also maybe why i looked into a zettelkasten. this is interestingly the second part of our problem statement. one would surmise that graphs and links are navigation infrastructure and rightly so, because similarity search only ever measures distance to the query itself. it reaches whatever resembles your question and stops but it can’t hop from there to the note that resembles the answer. that second hop is what a graph stores as an edge. but even then, to route to the right memory you first have to infer how deliberate the query’s intent is (which we know as semantic search), and infer what is relevant even when it doesn’t appear to be. that’s inference for you. and maybe the frontier version of this is to stop hard-coding the route at all and let the model write its own traversal over the store, pick which links to walk, even write the query itself (now that LLMs have the bash tool). in short, we need to rely more and more on non-determinism (an idea i do not totally converge with in my other blog).
so i first tried decay across my memories, but demoted memory would sink so far that reranking never reconsidered it. after some hit and miss, i finally settled on decay on disuse. so finally, my agent now ranks “my first kiss last year” higher than “i ate chips last month”. eventually, navigation boils down to how optimally you split the labor between inference and recalling because for LLMs, forgetting is never the opposite of memory. a more honest goal is to forget well enough that what’s left can still be found.
i’m expecting at least 2 PRs out of this blog.