Field notes · AI memory

Why AI models are so limited.

How modern AI systems remember and forget.

It helps to separate the technology into two distinct layers. Each one forgets in its own way.

1

The core AI model

The architecture itself. What it can hold in mind at one time.

Part 1 ↓
 2

The application layer

The software wrapper, like a browser or desktop app. How it fakes a longer memory.

Part 2 ↓
1the core model

Context windows: the core model's short-term mind.

The context window is the most text a model can process in a single interaction. It is measured in tokens; one token is about three quarters of a word.

The analogy

It works like RAM

The context window is working memory. Every time you hit Enter, the entire conversation (your words, its replies, the full history) is bundled up and sent again. Between prompts, the model remembers nothing.

The mechanism

Self-attention

Frontier models are built on an architecture called the Transformer. It relies on self-attention, which maps the mathematical relationship between every token in the context window and every other token.1

The limits

Huge windows. Steep limits.

Frontier models now advertise windows of 128,000 tokens, and some reach one or two million. Size doesn't solve two problems.

01

Computational cost

Standard self-attention scales quadratically: O(N²). Double the window and you quadruple the compute. Latency rises. Infrastructure costs climb.

A window of N tokens needs N squared comparisons. Doubling the window to 2N needs four times as many. N tokens 1× compute → 2N tokens · 4× compute
Every token is compared with every other token.
02

The middle goes soft

A model may accept 1,000,000 tokens, but that doesn't mean it reads them all with equal clarity. Studies show models retrieve information well from the very beginning or the very end of a long prompt. Accuracy drops significantly when the crucial information is buried in the middle.2

Recall is high for information at the start and end of the prompt and dips in the middle. lost here start middle of the prompt end recall
The shape of the effect, drawn for illustration.

Over the limit

What happens when you go past it?

The model can't absorb any more, so the system has to drop data. The failures are specific.

The cliff

Hard truncation

The oldest messages are deleted from what gets sent to the model.

The drift

Loss of early instructions

The rules you set at the start of the conversation get pushed out. Formatting slips. The persona shifts. The objective fades.

The invention

Contradictions and hallucinations

Early facts fall away. The model contradicts itself. It fills in the blanks by making things up.

firstThe format.
thenThe voice.
thenThe goal.
2the application layer

How browser and desktop apps fake a memory.

Context windows are expensive and finite. So consumer apps, like the browser and desktop versions, use workarounds that create the illusion of persistent memory. They keep track of you in two main ways.

Workaround 1

Rolling windows: summarize and delete.

When a chat gets too long, the app quietly asks a smaller, faster model to summarize the earliest part. Then it deletes the original messages and puts the summary at the top.

Failure mode

A game of telephone

Granular details go. Nuance goes. Code snippets go. Names go. The model ends up reacting to the cliff notes, not the complete record.

Workaround 2

Vector databases and RAG.

When an app says it remembers you across separate chats (your job, your birth year, your preferences), it uses retrieval-augmented generation, or RAG. It draws on a small database it keeps about you.

  1. You state a fact: “I work as a bartender.”
  2. The app decides it's a lasting preference and pulls the fact out.
  3. It saves the fact as a mathematical embedding in a vector database on a server.
  4. Tomorrow, in a new chat, the app searches that database for relevant facts.
  5. It silently adds them to the hidden instructions of your new message.
A few megabytes.

These memory stores don't hold millions of words per user. They hold a few hundred explicit memory slots: a few megabytes of text, at most.

How RAG memory fails

Retrieval dilution

The wrong memory wins

Conflicting memories compete. The system pulls the wrong background and carries on from the wrong information.

Stale facts

Out of sync with reality

Life changes. The old memory isn't deleted or overwritten. The AI stays anchored to outdated data, and its answers are confident but wrong.

Fragmentation

Facts without the story

Memories arrive as isolated facts, not a continuous narrative. The AI knows what you prefer, but not why, or how that preference came about.

So what is memory here?

Memory is a scattershot of what you have told the model. It is never the whole story.

Your marketing is the whole story: years of customers, campaigns and results. That's why I built AtmaOS. See how it works →

Need a team that remembers?

Let's talk!Find a time here

Notes

  1. Ashish Vaswani et al., “Attention Is All You Need,” in Advances in Neural Information Processing Systems 30 (2017). arxiv.org/abs/1706.03762 ↩
  2. Nelson F. Liu et al., “Lost in the Middle: How Language Models Use Long Contexts,” Transactions of the Association for Computational Linguistics 12 (2024): 157–73. arxiv.org/abs/2307.03172 ↩