Implement a Cosmos DB change feed processor

A retrieval pipeline stores a document chunk and its embedding in the same Azure Cosmos DB item. Someone edits the chunk. Nothing in the container knows the embedding is now stale, and no query will tell you which items changed since the last time you looked, because a query answers a question about the container's current state and not about its history. The change feed is the piece that closes that gap: the change feed in Azure Cosmos DB is a persistent record of changes to a container in the order the changes occur^[1], and it outputs the sorted list of documents that were changed in the order in which they were modified.

The two pages before this one in the domain covered the two ways of reading a container's current state: addressing a known item and searching for matching ones, and what those reads cost and how consistent they are. This page owns the third shape. Addressing means you already know the identity of one item. Searching means you know a predicate and the engine finds the matches. Following means you know neither, and instead you ask the container to hand you its writes in order, forward from a position you remember. Nothing about that changes the scope rungs the domain runs on, and you will see them again below: the partition key still decides how much of the container a consumer covers, because the feed is served per partition key range, which is the slice of the container the feed is delivered in, one range per physical partition.

Before any of the mechanics, settle which path is open to your language, because it decides how much of this page is code you write and how much is machinery you configure. The change feed processor is a library, and it exists only in two SDKs. Microsoft's supported-SDK table marks it available in .NET V3 and Java and unavailable in Node.js and Python, and states the consequence outright: for Python and Node.js, use the change feed pull model ... because the change feed processor library is only available for .NET and Java^[2]. AI-200 is a Python exam, so read every "the processor does this for you" sentence below as a description of machinery a Python developer either writes by hand (the pull model) or rents from the platform (an Azure Functions trigger). The mechanics are still worth knowing exactly, because they are what the other two paths are measured against, and because the Functions trigger is the processor wearing a different hat.

The pull model is the Python-native path. It is one method on a container, and you drive the loop.

Listing 1: reading the change feed from Python with the pull model

# LatestVersion is the default mode, so `mode` does not have to be passed.
response_iterator = container.query_items_change_feed(start_time="Beginning")
for doc in response_iterator:
    handle(doc)          # your own processing
# ...
# Persist this yourself if you want to resume later.
continuation_token = container.client_connection.last_response_headers["etag"]

The start_time argument is where reading begins, handle(doc) stands in for whatever your application does with a change, and the # ... marks the polling and error handling a real service would add around this. The one line that matters most is the last one: with the pull model, the continuation token, if specified, takes precedence over the start time and start from beginning values^[3], and storing it is your job. That single responsibility is the difference the rest of this page keeps circling back to, because the change feed processor's whole reason to exist is to store that position for you, in a container, safely, across many machines.

One more thing the pull model gives you that the processor does not, and it matters for multitenant AI workloads: Microsoft's own comparison lists processing changes from only a single partition key as supported by the pull model and not supported by the change feed processor. If a tenant is a partition key value and you want that tenant's changes alone, the pull model with partition_key= is the only library path to it.

The decision tree in the overview above walks the choice between the three paths from the top. The rest of this page explains what sits behind them, in the order you need it: which mode decides what the feed contains, the four components and the loop they run, what a batch promises about size and order, where progress is stored, what caps scale-out, where reading starts, and how to tell that a consumer has fallen behind. The last four are where working developers lose days: a handler that is not idempotent breaks on the first redelivery, an under-provisioned lease container stalls everything behind it, a machine added past the lease count does nothing at all, and a start setting changed after the first run does nothing either.

Which operations reach a consumer is not a property of the consumer, it is a property of the mode the feed is read in, and the default mode is lossy on purpose. Azure Cosmos DB offers two change feed modes^[4], and each application picks exactly one: an individual change feed application can only be configured to read the change feed in one mode, although different applications may read the same container in different modes at the same time.

Latest version mode is the default, and Microsoft describes it as a persistent record of changes made to items from creates and updates. Three consequences are worth memorising because each one is a plausible exam distractor. You get the latest version of each item, so if an item is created and then updated before you read the feed, only the updated version appears and the intermediate one is gone. Deletes are not logged at all, and once an item is deleted it is no longer available in the feed. And there is no fixed data retention period, so changes can be synchronised from any point in time, including the beginning of the container. Older SDK surfaces call this same mode Incremental; the .NET pull-model guidance notes that both Incremental and LatestVersion refer to latest version mode of the change feed, and applications that use either mode see the same behaviour^[3].

All versions and deletes mode is the one that captures deletions. It records create, update and delete operations, it includes the intermediate changes that happened between two of your reads, and each change arrives with metadata naming the operation type, including whether a delete was caused by a TTL expiry. It is not free, and the price is paid at the account level rather than in code. Reading it requires continuous backups configured for your Azure Cosmos DB account^[4]; it is supported for Azure Cosmos DB for NoSQL accounts only; accounts that have ever merged a partition are not supported; and you can only read changes that occurred within the continuous backup retention window, with an attempt to read outside it returning an error. It also removes options: in this mode you cannot start from the beginning of the container or from a past timestamp, only from "now" or from a lease or continuation token you already hold.

When the deletes matter but the account-level dependency does not fit, Microsoft documents the workaround inside the latest-version-mode feature list itself, and it is the pattern the exam expects you to recognise. You capture deletes by setting a soft-delete flag within your items instead of deleting them directly: add an attribute such as deleted with the value true, and set a Time to Live^[5] on the item. The change feed captures the flag as an ordinary update, and the item is automatically removed when the TTL expires. The trade the docs attach to it is real: with a finite expiry you have to process the changes within a shorter time interval than the TTL expiration period, or the item vanishes before your consumer reaches it.

A second gap in latest version mode catches people out for the same reason. You cannot filter the feed for a specific type of operation, and Microsoft's suggested alternative is the same trick in miniature: add a soft marker on the item for updates and filter on that marker when you process items in the change feed. There is no "only inserts" subscription to configure.

The takeaway is a single question to ask before writing any consumer: does this workload need to see deletions, or every intermediate version? If yes, all versions and deletes mode with continuous backups is the supported answer. If no, stay on latest version mode, keep the freedom to replay from the beginning of the container, and reach for soft-delete plus TTL when a logical deletion still has to reach the consumer.

The change feed processor is small enough to hold in your head, and the whole page follows from its shape. Microsoft's own decomposition is that the change feed processor has four main components^[2], and each one answers a different question.

The monitored container is where the data lives; any inserts and updates to it are reflected in its change feed. The lease container acts as state storage and coordinates the processing of the change feed across multiple workers, which is the subject of the next section. The compute instance hosts the processor, and the docs are deliberately vague about what it physically is: a virtual machine, a Kubernetes pod, an Azure App Service instance, or a physical machine, each carrying a unique instance name. The prose also calls a running one a host instance, and the two words mean the same thing here. The delegate is the code that defines what you, the developer, want to do with each batch of changes that the change feed processor reads.

Those four run one loop, and Microsoft numbers its steps. Read the change feed. If there are no changes, sleep for a predefined amount of time, which is configurable through the poll interval, and go back to reading. If there are changes, send them to the delegate. And when the delegate finishes processing the changes successfully, update the lease store with the latest processed point in time, then go back to reading. The figure below traces those four steps and adds the branch the numbered list does not draw: what happens when the delegate throws.

Step four is where the guarantee comes from, and it is worth stating as a rule before the elaboration. The checkpoint is written after your code succeeds, never before. If your delegate implementation has an unhandled exception, the thread that processes that particular batch stops and a new thread is eventually created; the new thread checks the latest point in time that the lease store saved for that range and restarts from there, effectively sending the same batch of changes to the delegate. Microsoft says in as many words that this behaviour is the reason the change feed processor has an "at least once" guarantee^[2].

Two practical corollaries follow, and both are exam material. First, a delegate must be idempotent, because it will eventually see the same change twice: a handler that appends a row or increments a counter double-counts on the first crash, where one that upserts by item id does not. Second, a delegate that never succeeds never checkpoints, so it retries the same batch forever. The documented remedy is not to swallow the exception but to write the unprocessable documents, upon exception, to an errored-message queue, so you keep track of unprocessed changes while still being able to continue to process future changes. The store does not matter; another Azure Cosmos DB container is fine.

There is exactly one documented hole in the retry, and it runs the other way. If the failure happens on the first-ever delegate execution, the lease store has no previous saved state to be used on the retry, so the retry falls back to the initial starting configuration, which might or might not include the last batch. A brand-new deployment is therefore the one moment where at-least-once is not yet in force.

One related warning belongs here rather than in a traps list, because it looks like good practice and is not. If your delegate starts asynchronous work and returns before that work finishes, the processor may checkpoint the lease before all asynchronous operations complete, which can lead to missed events during recovery. Returning from the delegate is a promise that the batch is done; keep it.

Hold on to one sentence from this section and it should be the fourth step. Where the checkpoint is written decides what the processor can promise, what your handler has to tolerate, and, in the next sections, how far the whole thing scales.

Once the loop is clear, two questions decide whether a handler you write is correct: how much arrives at a time, and in what order. Both have precise answers, and both have an edge that trips people.

Changes arrive as a batch, and the batch has a configurable ceiling. In the .NET processor the builder exposes WithMaxItems, which sets the maximum number of items to be returned in the enumeration operation^[6]; the Java builder has the same idea, and the Azure Functions trigger surfaces it as MaxItemsPerInvocation. The edge is that this ceiling is not absolute. Microsoft attaches the same caveat wherever the setting appears: if operations in the monitored container are performed through stored procedures, transaction scope is preserved when reading items from the change feed, so the number of items received could be higher than the specified value^[7] in order that the items changed by the same transaction are returned as part of one atomic batch.

That is worth reconciling against a rule the sibling page on the SDK teaches, because the two look contradictory and are not. For a query, max_item_count is a hard upper bound the engine never exceeds and a page may come back short. For the change feed, the maximum can be exceeded, and only ever upward, to keep a transaction's writes together. Two different engines, two different promises, one similar-looking knob: the query pagination rule lives on the querying page, and this one applies to the feed.

Order is guaranteed, but only inside one partition key. Microsoft's statement is exact: change feed items come in the order of their modification time, and this sort order is guaranteed per partition key, and there's no guaranteed order across the partition key values^[1]. That is the same scope idea the domain has used throughout, applied to time instead of cost. If two items share a partition key value, your handler sees their changes in the order they were written. If they do not, there is no promise at all about their relative order, and code that reconstructs a global sequence from the feed is building on nothing.

One exception sits inside the guarantee. Items written in the scope of a transactional batch, a stored procedure, or a bulk mode request share the same modification time, and changes within that scope may be delivered in any order, even within one partition key. The ordering promise is about distinct modification times, not about ties.

The practical shape of a correct handler follows from these two rules together: treat each batch as a set rather than a sequence unless every item in it shares a partition key value, make every write idempotent, and never assume the batch you received is exactly the size you configured.

Ask where the processor keeps its memory and every other behaviour on this page falls out of the answer. The lease container acts as state storage and coordinates the processing of the change feed across multiple workers^[2]. It is an ordinary Azure Cosmos DB container that you point the processor at, and it can live in the same account as the monitored container or in a separate account.

What it stores is one document per partition key range, which is the unit the change feed is served in. A container's partition key values are distributed in ranges, each range representing a physical partition, and a range's progress is maintained separately from other ranges in the lease container through a lease document. A lease records how far that range has been processed, expressed as a continuation, and which instance currently owns it. The combination of the leases represents the current state of the change feed processor. The figure below shows that mapping: three ranges on the monitored side, three lease documents on the lease side, and nothing else holding the position.

Three operational facts about that container are easy to skip and expensive to get wrong.

It has a partition key of its own, and it is not a free choice. Microsoft states the requirement in the Azure Functions guidance, where the same lease mechanism is used: partitioned lease containers are required to have a /id partition key definition^[8]. Create it with anything else and the processor cannot use it.

It costs request units, and they are charged to it, not absorbed by the monitored container. Operations on the lease container for updating and maintaining state consume request units, and the higher the number of instances that use the same lease container, the higher the potential consumption. Microsoft is blunt about the failure mode: make sure the lease container isn't experiencing throttling, because throttling adds delays in receiving change feed events and can even completely end processing^[2]. It also states the corresponding cost model plainly, that the only cost for the change feed is the lease container's provisioned throughput and request units for each request^[1]. If you are sizing throughput, the request unit page in this domain applies to the lease container exactly as it does to any other.

You do not always create it by hand. With the Azure Functions trigger the container name defaults to leases, and setting CreateLeaseContainerIfNotExists to true makes the platform create it for you, although the default for that property is false. There is a documented catch when you authenticate with Microsoft Entra ID rather than a key: with Entra identities, creating containers is not an allowed operation and your Function won't be able to start^[7] if you leave that property on. The .NET library also offers WithInMemoryLeaseContainer, which keeps lease state in memory instead of a container and, as its own reference notes, restricts the scaling capability to just the instance running the current processor. Neither of those is a Python option, and neither changes the rule for a real deployment: durable, shared progress needs a real container.

One last consequence, and it is the reason identity-based access to the change feed needs planning. The processor writes to the lease container constantly, so the two containers need different permissions. On the monitored container the identity needs readMetadata and readChangeFeed; on the lease container it needs item read, create, replace, delete and query. These are Azure Cosmos DB native data-plane actions, granted by a data-plane role definition, not by Azure role-based access control (Azure RBAC) over the resource. That distinction is the same one the SDK page makes for ordinary reads, and it bites harder here because a control-plane owner with no data-plane role sees a processor that starts and then silently fails to persist a single lease.

Treat the lease container as a first-class part of the deployment rather than a bookkeeping detail: it has a partition key you cannot choose, a throughput bill of its own, a permission set distinct from the monitored container's, and, as the next section shows, it is the thing that decides how many machines are worth running.

Scaling the processor is not a matter of starting more copies and hoping. Copies only cooperate when they form what Microsoft calls a deployment unit, and that takes three conditions holding at the same time: all instances should have the same lease container configuration, all instances should have the same value for processorName, and each instance needs to have a different instance name^[2]. Miss the first two and you have not scaled out at all; you have built a second, independent deployment unit that reads the entire feed again in parallel with the first. Miss the third and instances collide over identity instead of dividing work.

With all three satisfied, the behaviour is automatic. The processor distributes all the leases that are in the lease container across all running instances of that deployment unit and parallelises compute using an equal-distribution algorithm. The number of instances can grow and shrink, and the processor dynamically adjusts the load by redistributing it. It also follows the data: if the container's throughput or storage grows and Azure Cosmos DB adds physical partitions, the processor transparently increases the leases and distributes the new ones among existing instances.

That leaves one hard ceiling, and it is the single most useful number on this page even though it is not a number. A lease is owned by one instance at any time, so the number of instances shouldn't be greater than the number of leases. Leases come from partition key ranges, and ranges come from physical partitions, so your parallelism is bounded by how the data is partitioned, never by how many machines you are willing to pay for. The figure below draws that: three leases, two working instances, and a third instance with nothing to own. Scaling out past the lease count buys idle processes. When throughput is the problem and the leases are already spread thin, the lever is the partitioning of the monitored container, not the instance count.

The mirror image of the deployment-unit rule is sharing. You can share a lease container across multiple deployment units, and in that arrangement each unit either listens to a different monitored container or uses a different value for processorName, so each maintains independent state in the same container. Review the request unit consumption when you do, because every unit's lease traffic lands on the same throughput.

The Azure Functions trigger inherits the same arithmetic through the same lease documents, and its configuration exposes the same choice under different names. Microsoft states the rule directly: if you configure multiple functions to use an Azure Cosmos DB trigger for the same collection, each function should use a dedicated lease collection or specify a different LeaseCollectionPrefix for each function. Otherwise, only one of the functions is triggered^[7]. Two Python functions pointed at one container and one leases container with no prefix is not two consumers; it is one consumer and one function that never fires.

Listing 2: a Python function bound to the change feed, with its lease settings

import azure.functions as func

app = func.FunctionApp()

@app.function_name(name="CosmosDBTrigger")
@app.cosmos_db_trigger(arg_name="documents",
                       database_name="%COSMOS_DATABASE_NAME%",
                       container_name="%COSMOS_CONTAINER_NAME%",
                       connection="COSMOS_CONNECTION",
                       lease_container_name="leases",
                       create_lease_container_if_not_exists="true")
def cosmos_trigger(documents: func.DocumentList) -> str:
    for doc in documents:
        handle(doc)     # your own processing
        # ...

The documents parameter is the batch, so this function body is the delegate under another name, and lease_container_name is the same lease container the library would have used. The # ... stands in for the error handling a production handler needs. Everything this page says about idempotence applies unchanged, because the platform is running the same processor. What you give up is the fine control: the per-partition-key read that opened this page as the pull model's exclusive ability is not among the trigger's settings either. What you gain is that Microsoft's stated point of the trigger is exactly this, using the change feed processor's scaling and reliable event detection functionality without the need to maintain any worker infrastructure^[8]. For a Python team that is usually the right trade, and the Functions hosting page covers what running it costs.

A new consumer has to decide how much history it wants, and the setting that decides it is a first-run setting. Getting this wrong is a classic exam trap because the code looks right and simply does nothing.

The default is now. When a change feed processor starts for the first time it initialises the lease container and begins its life cycle, and any changes that happened in the monitored container before the change feed processor is initialised for the first time aren't detected^[2]. A processor started against a container with a million existing items therefore sees nothing until the next write.

Two overrides exist for latest version mode. You can start from a specific date and time, which the .NET builder expresses as WithStartTime and Java as a start time on the processor options. You can start from the beginning of the container's lifetime, which .NET spells as WithStartTime(DateTime.MinValue.ToUniversalTime()) and Java as setStartFromBeginning(true). The Azure Functions trigger exposes the second one as the StartFromBeginning property, and the Python pull model expresses both as the start_time argument shown in Listing 1. Microsoft notes that the precision of a start time is approximately five seconds, so it is a coarse cursor rather than an exact one.

The rule that matters more than either override is what happens on the second run. The lease wins. Microsoft states it for the library as a plain constraint, that these customisation options work only to set up the starting point in time of the change feed processor and, after the lease container is initialised for the first time, changing these options has no effect^[2]. The Functions trigger reference says the same thing about its own property in operational terms: reading from the beginning only works the first time the trigger starts, because in subsequent runs the checkpoints are already stored, and setting this option to true when there are leases already created has no effect^[7]. If you truly need to replay from the beginning of a container that a consumer has already processed, you change the state, not the setting: a fresh lease container, or a different processor name so the deployment unit starts with leases of its own.

The pull model expresses the same precedence through a different object. A continuation token, if specified, takes precedence over the start time and start-from-beginning values, which is the same rule with the stored position named explicitly instead of hidden in a lease document.

Mode narrows the choice further. Customising the starting point is only available for latest version change feed mode; in all versions and deletes mode you must start from the time the processor is started, or resume from a prior lease state that falls within the account's continuous backup retention period. So the full precedence, in the order to remember it: an existing lease or continuation token first, then the configured start setting, then the default of now, with the two overrides available only in latest version mode.

A consumer that is running is not the same as a consumer that is keeping up, and the difference is invisible from the outside. Your deployment processes changes at a rate set by its CPU, memory and network, and if this rate is slower than the rate at which your changes happen in your Azure Cosmos DB container, your processor starts to lag behind^[9]. The change feed estimator is the supported way to see that.

What it measures is a direct consequence of where progress is stored, which is why it belongs at the end of this page rather than the start. The estimator measures the difference between the last processed item, defined by the state of the lease container, and the latest change in the container. Because the lease already holds the position, the estimator needs no cooperation from your delegate: it reads the same leases the processor writes. It comes in two shapes. As a push model it hands a delegate a number representing how many changes are pending to be read by the processor, on an interval you choose. On demand it returns more: the estimated lag per lease, and the instance owning and processing each lease, which is how you tell a globally slow deployment from one bad instance.

Two constraints keep the estimator honest. It is configured with the same lease container and the same name as the processor it measures, so it is scoped to one deployment unit and not to the container in general. And the number is an estimate in the strict sense: for both change feed modes, the estimate provided isn't guaranteed to be an exact count of outstanding changes to process^[9]. Treat a rising trend as the signal, not an individual value.

Running it is not free and Microsoft recommends where to put it. Each estimation consumes request units from both the monitored and the lease containers, so a frequency of one minute between measurements is a good starting point, and lowering the interval raises the request units consumed. The estimator does not need to be deployed as part of your processor or even in the same project; a single estimator instance can track the progress of all the leases and instances in a deployment. Deploying it separately is the recommendation, and it also means a wedged processor does not take its own monitoring down with it.

Like the processor itself, the estimator is documented for .NET and Java. A Python consumer built on the pull model has no equivalent library call, so the same signal has to be produced another way, for example by emitting the age of the last processed change from the handler. That gap is a genuine reason to prefer the Azure Functions trigger for a Python workload that needs to be operated rather than merely run.

When the lag does climb, there are only two real remedies and this page has already named both. Add instances, up to the lease count and no further. Or fix the throughput that is throttling the read, on the monitored container or on the lease container, since throttling on either adds delay and throttling on the lease container can stop processing altogether.

The questions on this subtopic cluster around a handful of distinctions, and each one has a sentence in the documentation behind it.

A scenario that says Python and asks you to "implement the change feed processor" is testing the SDK table. The processor library is .NET and Java only. The Python answers are the pull model, query_items_change_feed, or the Azure Functions trigger for Azure Cosmos DB. A stem that hands you a Python service and asks for the least operational work is usually pointing at the Functions trigger, because it is the processor hosted for you.

Any stem containing "deleted" is a mode question. Latest version mode does not log deletes and the item leaves the feed entirely. The two correct responses are all versions and deletes mode, which costs continuous backups, an account limited to the API for NoSQL, and a retention window, or the soft-delete flag with a TTL, which keeps the default mode and turns the deletion into an update.

Any stem where a handler runs twice is testing at-least-once. The checkpoint is written after the delegate succeeds, so a crash mid-batch means the batch is re-delivered. "Make the handler idempotent" is the answer; "switch to exactly-once" is not an option that exists.

A stem where adding instances changes nothing is testing the lease ceiling. One lease per partition key range, one owner per lease, so instances past the lease count sit idle. The distractor to reject is anything that raises the instance count further; the real levers are the container's partitioning and its throughput.

A stem where a second consumer sees no changes is testing the deployment unit. For the library, instances of one unit share a lease container and a processorName and differ only by instance name; two units that must both see every change need different processor names or different lease containers. For Azure Functions the same rule reads as a dedicated lease container or a distinct lease container prefix per function, and the documented symptom of getting it wrong is that only one of the functions is triggered.

A stem where StartFromBeginning appears to be ignored is testing lease precedence. The start setting applies only when the lease container is initialised for the first time; afterwards the stored checkpoint wins. Replaying history means new leases or a new processor name, never a changed flag.

A stem about "how far behind are we" is the change feed estimator. It compares the lease's position against the latest change, is configured with the processor's lease container and name, returns per-lease lag and the owning instance on demand, and is explicitly not an exact count.

Two details are worth carrying in for the harder questions. The lease container is partitioned by /id, and it consumes its own request units, so it is a throughput line item and a throttling risk of its own. And ordering is guaranteed per partition key only, with no promise across partition key values and no promise within a single transactional batch, so a stem that needs a global sequence is describing a design that the change feed cannot supply on its own.