Domain 3 of 3 · Chapter 2 of 3

Design and implement agent memory stores

Unlock the complete study guide + 25 practice exams totaling 1,500 questions.

Bundled into the existing Building AI Applications with Azure Cosmos DB premium course — no separate purchase.

14-day refund guarantee on one-time purchases and your first subscription charge.

Included in this chapter:

  • Why an agent needs a durable memory store
  • Short-term and long-term memory
  • Choosing the memory partition key
  • The recommended item model: one document per turn
  • Alternative item models: per response and per thread
  • Agent Framework history providers
  • Local history versus service-managed conversations
  • Persisting history after each model call (Python)
  • Persisting and restoring the whole session
  • Bounding the model's context without losing history
  • Four query patterns for recalling memories
  • Semantic cache: reusing completions
  • Maintaining a semantic cache
  • Recognizing agent memory scenarios in exam questions

The three memory item models

AspectOne document per turnOne response per itemOne thread per item
What one item holdsOne complete exchange: prompt and reply, or tool call and resultOne user, agent or tool messageEvery turn of the thread in an array, plus an optional summary
Status in Microsoft guidanceRecommended; the most common modelAlternativeAlternative; typically not recommended
Embedding granularityOne vector per exchange, optionally one per messageOne vector per messageOne coarse vector, usually over a thread summary
Loading the latest turnsFilter by threadId, order by turnIndex or timestampFilter by threadId, order by turnIndex or timestampOne read of the whole thread, then client-side sorting
Expiry (TTL) granularityPer exchangePer messageWhole thread only
Write cost as the thread growsA small new item per exchangeA small new item per messageEach append updates a growing item and can incur a higher request unit (RU) charge
Fits bestAgents that recall whole exchanges; semantic cachingPer-utterance search and per-message analyticsShort, bounded threads loaded in one read

Decision tree

Need the latest turnsin conversation order?YesNoMost-recent queryORDER BY timestampDoes matching thequery's words matter?NoYesSemantic searchVectorDistanceMust every resultcontain those words?YesNoKeyword searchFullTextContainsHybrid searchReciprocal RankFusion (RRF)Any pattern: scope WHEREto a thread, user or tenant

Cheat sheet

  • Agent memory must be persisted outside process memory to survive a restart
  • A thread-based memory key fits conversation-local reads
  • Tenant and thread hierarchy fits multitenant conversation memory
  • Per-item GUID memory keys favor write-heavy logging over conversation replay
  • Short-term working memory and long-term knowledge need different retention policies
  • Restart-safe agent persistence includes the complete session state
  • A custom history provider loads context and stores new messages separately
  • A shared history provider must keep conversation-specific keys in session state
  • A session should use one authoritative conversation-history model
  • Tool-calling runs can need history persistence before the overall run finishes
  • Serialized sessions must be restored with compatible agent and provider configuration
  • Resume shared application conversations through an owner-bound server-side mapping
  • One document per complete exchange is the recommended general chat-memory model
  • One response per item suits per-utterance retrieval and analytics
  • One thread per item is suitable only for deliberately bounded conversations
  • Conversation memory should preserve who produced each message
  • Summary memory can consolidate older exchanges without retaining every detail in active context
  • Recent conversation context is retrieved by thread and explicit sequence
  • Persisted chat history affects an answer only when it is supplied as model context
  • The active model context can be smaller than the durable conversation history
  • Durable working memory can include intermediate task and tool state
  • Audit history can be persisted without adding it all to future prompts
  • Semantic memory retrieves relevant earlier information beyond the recent window
  • Long-term user knowledge must be recalled at a scope broader than one thread
  • A memory embedding should represent the unit the agent needs to recall
  • Semantic memory can require freshness without switching to a purely chronological search
  • Memory retrieval can require an explicit mention rather than semantic closeness
  • A semantic cache reuses a prior completion for a sufficiently similar request
  • Semantic cache keys must include enough conversation context to disambiguate follow-ups
  • A semantic cache acceptance threshold balances hit rate against relevance
  • Semantic cache entries can expire independently of durable conversation history
  • A freshness predicate can exclude old cache entries while retaining them for analysis
  • Cache-hit metadata can support selective retention of frequently reused answers

Unlock with Premium — includes all practice exams and the complete study guide.

References

  1. https://learn.microsoft.com/en-us/azure/cosmos-db/gen-ai/semantic-cache
  2. https://learn.microsoft.com/en-us/azure/cosmos-db/gen-ai/agentic-memories
  3. https://learn.microsoft.com/en-us/azure/cosmos-db/gen-ai/integrations
  4. https://learn.microsoft.com/en-us/agent-framework/concepts/agents/conversations/storage
  5. https://learn.microsoft.com/en-us/azure/cosmos-db/time-to-live
  6. https://learn.microsoft.com/en-us/azure/cosmos-db/partitioning-overview
  7. https://learn.microsoft.com/en-us/azure/cosmos-db/hierarchical-partition-keys
  8. https://learn.microsoft.com/en-us/azure/cosmos-db/vector-search
  9. https://learn.microsoft.com/en-us/azure/cosmos-db/request-units
  10. https://learn.microsoft.com/en-us/dotnet/api/microsoft.agents.ai.cosmoschathistoryprovider?view=agent-framework-dotnet-latest
  11. https://learn.microsoft.com/en-us/agent-framework/concepts/agents/conversations/compaction
  12. https://learn.microsoft.com/en-us/azure/cosmos-db/gen-ai/hybrid-search
  13. https://learn.microsoft.com/en-us/cosmos-db/query/fulltextcontains
  14. https://learn.microsoft.com/en-us/azure/cosmos-db/how-to-query-container
  15. https://learn.microsoft.com/en-us/azure/cosmos-db/partial-document-update
  16. https://learn.microsoft.com/en-us/credentials/certifications/resources/study-guides/dp-420