Agent Memory Patterns: What to Persist, What to Forget

Stuffing the entire transcript into the next model call is not memory design. It is how you pay for tokens you do not need and how bad conclusions become permanent personality.

This spoke belongs to the Agentic Systems Operating Manual. It pairs with RAG That Does Not Lie: retrieval is for documents; memory is for run state, preferences, and promoted facts.

AI agent memory design: four stores

Separate these even if they share infrastructure:

1. Ephemeral run context

Prompt assembly for this run. Dies when the run ends. Includes the job contract, current artifact, and the last evaluator failures. Does not include every failed thought from last Tuesday.

2. Working scratch

Intermediate files: outlines, tool raw dumps, temporary extractions. Readable by the worker during the run. Default TTL: end of run or 24 hours. Never customer-facing. Never silently promoted.

3. Durable facts

Things you would defend in a meeting: customer preferred language, account IDs, approved SOP pointers, “do not contact before 10am.” Owned fields with schema, write auth, and audit. Humans or strict promotion rules write here — not every model utterance.

4. Run history / traces

Ops gold: states, tool calls, costs, scores. Used for debugging and offline learning. Not re-injected wholesale into prompts. Summaries may be derived; raw traces stay in the observability store.

If your architecture has one blob called memory, split it before you scale.

Short-term vs long-term agent memory

	Short-term	Long-term
Lifespan	Run or session	Weeks to permanent
Contents	Job context, scratch, open questions	Preferences, identifiers, approved facts
Write authority	Worker (scratch); system (context)	Controlled promotion / human
Risk if wrong	Contained to one job	Repeated wrongness across jobs
Typical store	Redis, workflow static data, temp objects	DB rows, CRM fields, config service

Short-term should be generous enough to finish the job and aggressive about deletion. Long-term should be stingy.

What to persist

• Stable identifiers (customer_id, ticket_id, tenant_id)
• Explicit user preferences with timestamp and source
• Pointers to authoritative docs (not a paraphrase of the whole doc)
• Budgets and policy version IDs used for the run
• Evaluator criterion versions for audit

What to forget

• Chain-of-thought and speculative plans
• Raw tool payloads that contain secrets or PII beyond need
• Failed hypotheses that never passed the evaluator
• Entire chat transcripts as a default “memory layer”
• Injected instructions found inside untrusted documents

Forgetting is a feature. It limits contamination.

Promotion rules (how scratch becomes fact)

Nothing moves from scratch to durable facts without a rule such as:

• Human approved the write, or
• Evaluator passed a specific “preference extraction” criterion and the field is in an allowlist, or
• A nightly job reconciles structured outputs against CRM with validation

“The model said they like blue” is not a preference write. “User clicked Save preference: language=es” is.

Memory vs RAG vs fine-tuning

• Memory — instance-specific state and preferences.
• RAG — organization documents under a retrieval contract.
• Fine-tuning / style adapters — behavioral priors, not a substitute for either.

Most “our agent needs better memory” tickets are actually “our agent needs CRM fields and a retrieval contract.” Fix the boring stores first.

Multi-agent memory

Do not share a mutable mind across agents. Share a handoff package and read-only access to durable facts. If agent A pollutes a shared scratchpad, agent B inherits the pollution. See Multi-Agent Handoffs.

Privacy and retention

Memory design is a privacy design. Define retention per store. Encrypt at rest where you store PII. Redact traces. Give customers a deletion path that actually clears durable facts and vectors derived from their data. “We will remember you” is not a charming product line when the data is wrong or sensitive.

Testing memory

• Injection tests: can untrusted content write durable facts? It must not.
• Contamination tests: does a failed run’s conclusion appear in the next run’s context? It must not unless promoted.
• Budget tests: does memory assembly blow the token budget? Cap and summarize with schema.

Pilot guidance

For a $1,500 · 5-day Spurlock Studios pilot we usually ship ephemeral + scratch + one or two durable fields. Fancy long-term “agent brains” wait until the job clears evaluation. You keep what we build.

/agentic · /contact?intent=agentic-pilot

Anti-patterns

Infinite context as strategy. Models still miss; costs do not.

Vectorizing every Slack message as memory. That is an ungoverned corpus, not memory.

Silent preference writes. No audit, no schema, no owner.

Cross-tenant leakage. Shared caches without tenant keys. Instant trust-ender.

Session memory versus customer memory

Session memory helps a multi-turn operator UI: what file you uploaded, which ticket is active. Customer memory is durable and dangerous. Do not conflate them in one key-value soup.

Session memory may live in the workflow’s static data or a short-TTL store. Customer memory belongs in systems of record (CRM fields, account settings) with the same permissions a human would need to edit those fields. If the agent can write a preference a human analyst cannot see in the CRM UI, you have created a shadow database. Shadow databases always diverge.

Summarization as a controlled transform

When context grows, summarize with a schema: {open_questions[], decisions[], artifacts[]}. Discard prose. Run the summary through mechanical validation. Treat summarization failures as escalate triggers — a bad summary is how long-term wrongness enters through the side door.

Never summarize away evaluator failures. Those must remain verbatim until resolved.

Cross-run learning

There is a legitimate pattern: cluster escalate reasons weekly and feed human-approved lessons into SOPs or evaluator criteria — not into a mystery vector memory. Learning that cannot be reviewed is how agents acquire superstitions.

Spurlock Studios keeps pilot memory minimal on purpose. The $1,500 week proves the job; memory architecture expands in builds once promotion rules have owners. /agentic · operating manual.

Concrete schemas for durable facts

{
  "customer_id": "cus_9",
  "pref_language": "es",
  "source": "user_settings_form",
  "updated_at": "2026-05-01T12:00:00Z",
  "updated_by": "agent:prefs_v2|human:u_33"
}

Require source and updated_by. Ban free-text “memory blobs” as the only store. Free text is where unverifiable claims hide.

Forgetting schedules

Store	Default TTL
Scratch	end of run
Session	7 days idle
Durable prefs	until user/company deletes
Traces	30–180 days per policy

Put TTLs in config. Review annually with counsel for regulated industries.

Debugging “it remembered wrong”

Check promotion logs first, then session assembly, then RAG contamination mistaken for memory. Most “memory bugs” are retrieval or prompt-assembly bugs. Keep the stores separate so diagnosis is possible.

Pilot keeps this thin on purpose — /agentic.

Short-term vs long-term agent memory in one diagram (textual)

trigger → assemble short-term context (contract + scratch refs + allowed durable facts) → act/evaluate loop → maybe promote → persist traces → drop scratch

Promotion is the only arrow into long-term facts. Everything else dies or stays in ops storage.

Multi-tenant memory isolation tests

Attempt to assemble context for tenant A with an ID from tenant B. The assembler must hard-fail. Add this test before any fancy memory feature. Cross-tenant memory is an extinction-level trust event.

Operator-visible memory

Let operators see and edit durable facts the agent can use. Invisible memory trains conspiracy theories about “what the AI knows.” Visible fields in CRM/settings keep humans in charge.

AI agent memory design stays boring on purpose. Expand after the $1,500 pilot proves the job: /agentic · manual.

Worked example: support agent memory

Short-term: current ticket id, last evaluator failures, draft summary URI.

Durable: customer language preference, VIP flag, do-not-contact window — all CRM fields.

Forbidden: past model guesses about mood; raw prior ticket transcripts dumped into every prompt; unverified “customer promised to renew.”

Promotion: VIP flag only via human or billing system, never via ticket text saying “I am VIP.”

Memory in evaluations

Add golden cases where a wrong durable fact exists in the DB. The agent should not invent fixes; it should use the fact or escalate if contradictory evidence arrives. Memory is data; criteria still rule.

Migration off transcript-as-memory

If you already stuffed chats into a store, freeze writes, export, extract structured prefs with human review, then delete raw blobs from the prompt path. Painful once beats chronic contamination.

Short-term vs long-term agent memory stays a policy conversation as much as a storage one. Spurlock Studios will keep pilots minimal — /agentic — and expand memory in builds when owners exist. See the operating manual.

Closing note on forgetting

The adult move in AI agent memory design is deleting scratch by default and promoting almost nothing. Short-term vs long-term agent memory only stays clean when promotion has owners, schemas, and audits. If a fact is important, put it in the CRM where humans already look. Then let the agent read it — not invent a parallel brain. Prove the job first on a $1,500 pilot: /contact?intent=agentic-pilot.

One more operating rule

Operators should be able to open the CRM and see every durable fact an agent can read. If they cannot, you built a shadow brain — delete it or surface it. That single rule prevents most memory mysticism.

Write retention timers into config reviews the same day you ship a new store. Forgotten TTLs are how scratch becomes accidental long-term memory.

FAQ

What is AI agent memory design?

It is the policy and storage layout for what an agent may remember across steps and runs: which stores exist, who can write, what TTLs apply, and how facts get promoted. It is not merely a larger context window.

What is the difference between short-term and long-term agent memory?

Short-term covers the current job and scratch and should die quickly. Long-term covers approved facts and preferences with strict write rules. Mixing them is how errors become permanent.

Should agents remember every conversation?

No. Persist structured outcomes and preferences. Keep full transcripts in ops storage if you need them for audit, not as default prompt fuel.

How do you prevent bad memories?

Evaluator-gated promotion, allowlisted fields, human approval for sensitive writes, TTLs on scratch, and tests that failed conclusions do not auto-promote.

Does Spurlock Studios build memory layers in the pilot?

Only as needed for the one job — usually minimal durable fields. Deeper memory systems land in full builds after the pilot proves value. See /agentic.

How does memory connect to the operating manual?

Memory is one layer alongside evaluators, sandboxes, state machines, and RAG contracts. The manual shows the full stack order.