Suppose a customer tells an assistant that invoices should go to one email address. Months later the customer changes the billing contact. The assistant still remembers the first instruction and confidently uses it.
There is no obvious technical failure. The memory store works, retrieval works, the model uses the information it was given. The problem is that the product treated yesterday's context as today's truth.
Once an AI application remembers things between sessions, it becomes responsible for deciding which facts deserve to survive, who can change them and when they should stop influencing decisions.
Conversation history is the wrong starting point
The easiest implementation is to save every interaction and search it later. It is also an easy way to accumulate contradictions, temporary requests, sensitive details and guesses the model once made about the user.
A transcript answers the question “what was said?” Memory needs to answer another one: “what information may reasonably influence the next task?”
These are different responsibilities. The fact that something appeared in a conversation is not permission to turn it into a durable customer attribute. Nor does it make the information accurate.
Short-term context can help complete the current request. Long-term memory should have a much higher bar for being written.
Write less, with a reason
I would start with a few explicit classes of memory rather than an unrestricted collection of messages: preferences the user has chosen to save, facts supplied through a trusted system, and task-specific information with a defined lifetime.
Each stored item should carry context. Where did it come from? Was it said by the user, loaded from a business system or inferred by the model? When was it last confirmed? Who owns it? When does it expire?
That provenance matters when facts disagree. A current customer record should not silently lose to an old conversational preference. An inference such as “this user probably prefers…” should not be presented later as something the user explicitly requested.
The model can help suggest what might be useful to remember. The application still needs a rule for what is actually committed to long-term storage.
Retrieval is another product decision
Even correct memories should not be inserted into every prompt. A remembered dietary preference has little reason to appear in a technical support conversation. Unrelated context increases cost and can distort the answer.
Retrieval needs a purpose: which memories are relevant to this request, how recently were they checked and is the user entitled to use them in the current context?
That last question becomes important when an assistant serves a whole company. Something learned in a private meeting should not become available in a different team's workflow just because the embedding search returned a similar phrase.
Memory and the model's context window are different problems. A longer context window does not decide what should be stored, who can read it or which fact is current.
Correction and deletion must work beyond the interface
Users eventually change their minds. The company also changes its prices, processes, people and policies. A memory system without correction is a mechanism for making outdated information more persistent.
A sensible product should let people see relevant saved information, correct it and remove it where appropriate. Deletion has to reach the underlying records and any derived stores that still use them, subject to legitimate retention requirements. Hiding an item from a settings page is not the same as stopping it from influencing future answers.
There is a less obvious design problem here: what happens when a deleted fact was used to produce another summary? If the system stores derived memories, their relationship to the source needs to be traceable.
This work is less exciting than an impressive memory demo. It is also the work that makes memory trustworthy over time.
Test the damage memory can do
Memory evaluation should not stop at “the assistant recalled a useful fact.” I would also test whether it ignores irrelevant memories, notices contradictions, respects access boundaries and stops using something that was corrected or removed.
Where personal or company-sensitive information is involved, data retention and access controls should be agreed before enabling the feature. The wider question of data movement is covered in AI Data Residency and Privacy.
For some products, there is no persuasive case for long-term memory at all. A document extraction service might work perfectly well with the context supplied for each request. Adding a personal history would create an obligation without creating much value.
Useful memory is selective. The aim is not to prove that an AI can remember everything. It is to make sure the few things it remembers continue to deserve the user's trust.