Design and implement agent memory stores
Unlock the complete study guide + 25 practice exams totaling 1,500 questions.
Bundled into the existing Building AI Applications with Azure Cosmos DB premium course — no separate purchase.
14-day refund guarantee on one-time purchases and your first subscription charge.
Included in this chapter:
- Why an agent needs a durable memory store
- Short-term and long-term memory
- Choosing the memory partition key
- The recommended item model: one document per turn
- Alternative item models: per response and per thread
- Agent Framework history providers
- Local history versus service-managed conversations
- Persisting history after each model call (Python)
- Persisting and restoring the whole session
- Bounding the model's context without losing history
- Four query patterns for recalling memories
- Semantic cache: reusing completions
- Maintaining a semantic cache
- Recognizing agent memory scenarios in exam questions
The three memory item models
| Aspect | One document per turn | One response per item | One thread per item |
|---|---|---|---|
| What one item holds | One complete exchange: prompt and reply, or tool call and result | One user, agent or tool message | Every turn of the thread in an array, plus an optional summary |
| Status in Microsoft guidance | Recommended; the most common model | Alternative | Alternative; typically not recommended |
| Embedding granularity | One vector per exchange, optionally one per message | One vector per message | One coarse vector, usually over a thread summary |
| Loading the latest turns | Filter by threadId, order by turnIndex or timestamp | Filter by threadId, order by turnIndex or timestamp | One read of the whole thread, then client-side sorting |
| Expiry (TTL) granularity | Per exchange | Per message | Whole thread only |
| Write cost as the thread grows | A small new item per exchange | A small new item per message | Each append updates a growing item and can incur a higher request unit (RU) charge |
| Fits best | Agents that recall whole exchanges; semantic caching | Per-utterance search and per-message analytics | Short, bounded threads loaded in one read |
Decision tree
Cheat sheet
Unlock with Premium — includes all practice exams and the complete study guide.
References
- https://learn.microsoft.com/en-us/azure/cosmos-db/gen-ai/semantic-cache
- https://learn.microsoft.com/en-us/azure/cosmos-db/gen-ai/agentic-memories
- https://learn.microsoft.com/en-us/azure/cosmos-db/gen-ai/integrations
- https://learn.microsoft.com/en-us/agent-framework/concepts/agents/conversations/storage
- https://learn.microsoft.com/en-us/azure/cosmos-db/time-to-live
- https://learn.microsoft.com/en-us/azure/cosmos-db/partitioning-overview
- https://learn.microsoft.com/en-us/azure/cosmos-db/hierarchical-partition-keys
- https://learn.microsoft.com/en-us/azure/cosmos-db/vector-search
- https://learn.microsoft.com/en-us/azure/cosmos-db/request-units
- https://learn.microsoft.com/en-us/dotnet/api/microsoft.agents.ai.cosmoschathistoryprovider?view=agent-framework-dotnet-latest
- https://learn.microsoft.com/en-us/agent-framework/concepts/agents/conversations/compaction
- https://learn.microsoft.com/en-us/azure/cosmos-db/gen-ai/hybrid-search
- https://learn.microsoft.com/en-us/cosmos-db/query/fulltextcontains
- https://learn.microsoft.com/en-us/azure/cosmos-db/how-to-query-container
- https://learn.microsoft.com/en-us/azure/cosmos-db/partial-document-update
- https://learn.microsoft.com/en-us/credentials/certifications/resources/study-guides/dp-420