Retrieval and grounding pipelines with Azure AI Search

Two pipelines, one index

Grounding starts with a picture of two pipelines, not a single call. On one side an ingestion pipeline reads your source content, breaks it into pieces small enough to search, turns each piece into a vector, and writes the result into an Azure AI Search index^[1]. On the other side a query runs against that index at question time, pulls back the handful of pieces most relevant to the user, and passes them to a language model as grounding for its answer. Retrieval-augmented generation (RAG) is the name for that second half feeding the model from the first half's index.

Three terms carry the rest of the page, so pin them down now. An indexer is the crawler-and-loader that pulls from a data source and drives ingestion. A skillset is the ordered set of enrichment steps the indexer applies to each document, such as chunking, embedding, and language enrichment. An index is the schema-defined store the results land in and the only thing a query ever touches. A chunk is one searchable piece of a document, and an embedding, or vector, is the numeric representation of a chunk's meaning.

What this page owns, and what the sibling page owns

The sibling page on document content extraction owns turning one file into a clean structured or Markdown representation with Content Understanding: cracking a PDF, running optical character recognition (OCR) on it, reading its layout, and pulling named fields out. This page owns everything about the searchable index built from that content and the query that grounds an answer: index design, chunking for retrieval, integrated vectorization, multimodal ingestion, the query modes, security trimming, wiring retrieval into agents, and proving it works. If the question is 'what did this document say', that is the sibling page; if it is 'which chunks should ground this answer', that is here.

Figure 1 traces one flow through both pipelines. Read it as two lanes that share one box: the ingestion lane on top writes the index, the query lane below reads it. Nothing in the query lane can retrieve a field the ingestion lane never wrote, which is why so much of this page is really about decisions made before any question is ever asked.

Figure 1: the ingestion pipeline writes the index; the query pipeline reads it.

The index schema is a contract you set once

A field's attributes, not its contents, decide what a grounding query can do with it. Two fields can hold the exact same text and behave completely differently, because capability in Azure AI Search is declared per field at design time and then largely frozen.

Five attributes^[2] map to five things a query can do. A searchable field takes part in full-text BM25 matching. A filterable field can appear in a $filter, which is the single mechanism behind permission trimming, metadata scoping, and a fixed filter on an agent's search tool. A sortable field can be named in orderby. A facetable field can drive facets. A retrievable field can be returned to the caller to be quoted or cited. Exactly one field is the key, and it must be of type Edm.String. Vector fields, of type Collection(Edm.Single), are searchable but can never be filterable, sortable, or facetable.

Why the design is nearly permanent

New fields can be added to a live index at any time, but an existing field's data type and most of its attributes are locked for the life of the index. Turning on filtering or sorting after data is loaded is therefore not a toggle: it means adding a new field, or dropping and rebuilding the index and re-indexing every document. Figure 2 maps each attribute to the capability it unlocks, so the design conversation happens before ingestion, not after.

The trap here is subtle and common. retrievable only controls whether a value comes back to the caller; it says nothing about filtering or sorting. A field whose value you can plainly see in results will still reject a $filter or an orderby until it is redefined as filterable or sortable, and existing documents will not pick up that change without a full rebuild. So the takeaway is a sequencing rule: enumerate every query, filter, and sort you will need, set the attributes to match, then load the data.

Figure 2: each index field attribute and the one query capability it unlocks.

Chunking content to fit the embedder

Chunking exists first for a hard mechanical reason, and only second as a relevance tuning knob. Embedding models impose a fixed input ceiling: text-embedding-3-small accepts 8,191 tokens^[3], roughly 6,000 words, and any content past that limit is truncated rather than embedded, silently dropping data. Even below the limit, one vector over a document that covers several subtopics represents none of them well, so splitting still helps recall.

Two ways to split, and when each fits

The Text Split skill^[4] cuts on character or token counts. Its textSplitMode chooses between pages, the default, which produces chunks of several sentences bounded by maximumPageLength, and sentences, which emits one sentence per chunk with boundaries decided by defaultLanguageCode. Page mode adds pageOverlapLength, which must be less than half the maximum page length, and maximumPagesToTake, which defaults to 0 meaning take every chunk. Choosing sentences mode for a long PDF explodes the chunk count into fragments too small to carry an answerable thought, so pages is the grounding default.

The Azure Content Understanding skill splits differently: it performs semantic chunking with Markdown output and produces units that preserve meaning across page boundaries and keep cross-page tables intact, combining extraction and chunking in one skill. This is semantic chunking, an ingestion-time splitting step; do not confuse it with the semantic ranker that reorders query results, covered later. They share only the word. On a layout-heavy corpus of contracts or reports, a character-based splitter slices tables and clauses in half so that no single retrieved chunk holds a complete answer; semantic chunking is the fix. Figure 3 walks that choice.

Sizing the chunks

For fixed-size chunking Microsoft's documented starting point^[3] is about 512 tokens, roughly 2,000 characters, with about 25 percent overlap, roughly 128 tokens or 500 characters, then tuning by content type. Overlap preserves continuity across chunk boundaries. Pushing it toward 50 percent on the theory that more context is always better backfires: it duplicates content, inflates index size, and, if the overlap is large relative to the actual content length, can collapse to no usable overlap at all. Start near 2,000 and 500, then measure.

Figure 3: choosing between semantic chunking and the Text Split skill's two modes.

Skills enrich content while the indexer runs

Every enrichment in ingestion is a skill, and each skill's output reaches the index only if you wire it there. A skill runs inside the skillset, over each document, during the indexer run, and what it emits lives only in memory as a node in the enriched-document tree for the duration of that run. Nothing persists on its own.

Cracking: two skills, different depth

The first skill usually cracks the document into page text and inline images. The Document Extraction skill^[5] is the lightweight cracker: it returns image location metadata for PDFs only, with no table extraction and no built-in chunking. The Azure Content Understanding skill is the heavier one: it extracts text location metadata, preserves tables including those spanning pages, produces semantic units that cross page boundaries, and works across PDF, DOCX, XLSX, and PPTX. Microsoft directs new skillsets to the Content Understanding skill and keeps the older Document Layout skill supported only for existing pipelines. The deep analyzer behaviour of Content Understanding is the sibling page's subject; here it is simply the cracking step that feeds chunking. Pick Document Extraction for cheap image-and-text cracking, Content Understanding when tables and cross-page units matter.

Built-in language skills need an output field mapping

Entity Recognition, Key Phrase Extraction, Language Detection, PII Detection, Sentiment, and Text Translation^[6] are billable built-in language skills that run pretrained models over each document so you can later filter, facet, scope, or redact grounding data on their output. Because that output lives only in the enriched-document tree, you persist it by adding an outputFieldMappings^[7] entry whose sourceFieldName is the /document/... path of the skill output and whose targetFieldName is a top-level simple field or collection in the index. The predictable mistake is reaching for fieldMappings instead: fieldMappings maps verbatim source fields to index fields and can never address a skill output. A skill configured correctly but never mapped enriches nothing, and the index looks as though the skill never ran. Figure 4 draws both paths side by side. One more note worth keeping straight: the Foundry resource attached to these skills is attached for billing only, and Azure AI Search runs them on its own internal resources.

The custom extension point

When no built-in skill fits, Microsoft.Skills.Custom.WebApiSkill^[8] calls your own endpoint. The uri must use HTTPS, and the indexer sends up to batchSize records per call, default 1000, as a top-level values array whose elements each carry a unique recordId and a data object matching the skill's declared inputs. Your service must reply with the same recordId values, a data object matching the declared outputs, and errors and warnings properties that are required but may be null. A non-JSON response, a missing recordId, or a duplicate one means that record is silently not enriched. Set authResourceId or authIdentity so the search service's managed identity authenticates instead of embedding a function key in the uri. It is distinct from the Azure Machine Learning skill, which targets a model deployed on an AML online endpoint rather than an arbitrary Web API. Across all three skill kinds one rule holds: a skill enriches the index only when its output is wired there, so match the skill to the job and never skip the mapping.

Figure 4: fieldMappings copies source fields; outputFieldMappings persists skill output.

Custom Web API skill request and response envelope

The indexer POSTs a batch to your HTTPS endpoint and expects the same envelope back. Only the field names inside data are yours; the values, recordId, data, errors, and warnings keys are fixed.

Request the indexer sends

{
  "values": [
    { "recordId": "0", "data": { "text": "Contoso Brew 300 stopped heating." } },
    { "recordId": "1", "data": { "text": "Refund issued on 14 March." } }
  ]
}

Response your service must return

{
  "values": [
    { "recordId": "0", "data": { "category": "fault" }, "errors": null, "warnings": null },
    { "recordId": "1", "data": { "category": "resolution" }, "errors": null, "warnings": null }
  ]
}

Every recordId in the request must reappear in the response. Any record you drop, or any recordId you return that was not asked for, is discarded rather than enriched. errors and warnings are required keys even when there is nothing to report, which is why both are present as null here.

Integrated vectorization: embeddings inside the pipeline

Integrated vectorization means the indexer generates embeddings for you, so no code of yours ever calls an embedding model. It depends on three pieces working together^[9]: an indexer that pulls from a supported data source and drives the pipeline, a skillset that combines a chunking strategy with an embedding skill such as the AzureOpenAIEmbedding skill, and a search index with vector fields to receive the results. Remove any one of them and you are back to generating and pushing vectors from your own application. Defining vector fields on the index alone does nothing, because nothing in the pipeline is calling a model.

The vectorizer has to match the embedder

Query-time text-to-vector conversion comes from a vectorizer declared in the index schema, assigned to a vector profile, which is in turn assigned to the vector field. The vectorizer must use the same embedding model^[9] that encoded the content: the AzureOpenAIEmbedding skill pairs with the Azure OpenAI vectorizer, the Azure Machine Learning skill with the Foundry model-catalog vectorizer, and so on. Mismatched models put the query vector and the document vectors in different spaces and relevance collapses. Upgrading the indexing model without re-embedding the corpus and updating the vectorizer is the quiet way to wreck a working index.

Index projections keep chunk and document grain

Optional index projections^[10] let one indexer run populate a granular chunk index alongside a document-level index, both from the same source document. Figure 5 shows the fork. The chat application matches on the fine-grained secondary index for precision, then returns the richer parent document from the primary index for the title, date, and summary fields that make a complete answer possible. Flattening everything into one chunk index throws those document-level fields away.

Throttling is handled for you, so schedule the runs

Azure AI Search has internal, non-configurable retry policies for the throttling errors an AzureOpenAIEmbedding^[11] deployment raises when it exhausts its tokens-per-minute allowance. Because the batching and retry behaviour is not exposed for tuning, the documented move is to put the indexer on a schedule so any call dropped despite the retries is picked up on the next run. Tokens-per-minute limits apply per model per subscription, so sharing one embedding deployment between ingestion and the query workload makes both throttle.

Figure 5: one indexer run fills a secondary chunk index and the primary document index.

Grounding on images and other modalities

Two different image goals need two different ingestion skills, and picking the wrong one is a design dead end rather than a tuning problem. The question that decides it: will the user search with text about an image, or query with an image itself?

Verbalization: describe the image as text

Image verbalization calls a language model once per extracted image at ingestion time through the GenAI Prompt skill^[12], storing a concise description such as 'five-step HR access workflow that begins with manager approval' next to the surrounding document text. Because the picture is now expressed in language, the pipeline can explain relationships inside a diagram and hand the model a caption it can cite verbatim, at the cost of one model call per image. Its limit is exact: the GenAI Prompt skill supports text-to-vector hybrid queries but not image-to-vector queries. An index built this way cannot accept an uploaded photo as the query, no matter that it already 'understands' images.

Multimodal embeddings: query with an image

A 'find things that look like this' experience needs a multimodal embedding model^[12] and its matching vectorizer, built with the Azure Machine Learning skill or the Azure AI Vision multimodal embeddings skill. These convert an image to a vector at query time and need no model call during indexing, but they carry no explanation of why two images are related and give the model no ready-made text to cite. Figure 6 lays the two paths side by side. Verbalization is one flavour of image handling and multimodal embeddings is the other; a pipeline can run both, but each answers only its own kind of question.

Where the pixels live

Either way, a multimodal pipeline stores the images it pulls out of source documents in a knowledge store, and the index keeps each image's location so the application can render the original figure beside the cited text. Returning image bytes from the search index itself is the antipattern: the index holds a pointer, the knowledge store holds the picture. That separation is what lets a grounded answer show both a textual citation and the diagram snippet it came from.

Figure 6: verbalization for text-about-an-image, multimodal embeddings for image-as-query.

Choosing a retrieval mode: keyword, vector, hybrid

A hybrid query is one request that carries both a full-text search string and one or more vectorQueries. The two run in parallel, BM25 for the text and HNSW or exhaustive KNN for the vectors^[13], and a Reciprocal Rank Fusion (RRF) algorithm^[14] merges the two result sets into one ranked list. This matters because it is one call, not two: you do not run separate searches and blend the scores in application code. Figure 7 traces that single request through the parallel branches into the fused result.

Hybrid is the documented default for grounding for a concrete reason. Keyword matching catches product codes, jargon, dates, and names that pure vector similarity misses, while vectors catch paraphrase and meaning that keyword search misses. Adding vectors to an index does not make keyword search redundant; the two cover each other's blind spots.

The vector knobs: recall versus latency

A vector query defaults to approximate nearest-neighbour traversal of the HNSW graph^[15]. Setting exhaustive to true forces an exhaustive KNN scan that maximizes recall at the cost of latency. Its real job is measurement, not production: you run it to establish a ground-truth baseline for how much recall the approximate path is losing, not as an always-on 'quality' setting across a live index. The oversampling value is the companion knob. It widens the candidate set retrieved before rescoring, which recovers recall lost when vectors are stored compressed, or quantized. So the mental model is a pair of dials for the same quantity, recall: exhaustive measures the ceiling, oversampling buys back what compression gave away.

Figure 7: a hybrid query's parallel BM25 and vector branches fused by RRF.

A hybrid query request with a vector query

This is the whole search request body for a hybrid query: a search string for BM25 and a vectorQueries entry for the vector side, in one call. The service fuses them with RRF; your application does no score blending.

{
  "search": "brew 300 not heating",
  "vectorQueries": [
    {
      "kind": "text",
      "text": "the kettle stopped warming up",
      "fields": "contentVector",
      "k": 50,
      "exhaustive": false
    }
  ],
  "select": "title,content,sourceUrl",
  "top": 5
}

The kind: text vector query lets the index's vectorizer embed the query string for you, which only works when the vectorizer matches the indexing model. k is the number of nearest neighbours the vector side returns into the fusion, set to 50 here so a downstream reranker has enough candidates; top then bounds how many fused results come back. Leaving exhaustive at false keeps the fast approximate path; flip it to true only to measure recall.

Shaping the ranking, and the orderby trap

Retrieval decides which documents come back; ranking decides their order, and three levers shape it. They stack in a pipeline, and one clause switches the whole pipeline off. Figure 8 draws the stages and the bypass.

Semantic reranking

Semantic ranking is opt-in per query: the request must set queryType to semantic^[16] and reference a semanticConfiguration defined in the index, after which results carry a separate @search.rerankerScore alongside the ordinary @search.score. Because the reranker works only from the first-stage candidates, Microsoft advises setting the vector query k to 50 so it has enough input. The trap is believing relevance improved because a semantic configuration exists on the index, when no query ever set queryType to semantic and the ranker never fired.

Scoring profiles

A scoring profile^[17] is a named object in the index schema that boosts or suppresses the ranking of documents a query already matched. It is built from text weights over searchable fields plus optional functions: freshness over an Edm.DateTimeOffset field, magnitude over a numeric range, distance from a reference point, and tag for overlap with a caller-supplied list. Those functions can only be applied to filterable fields. A query uses one by naming it in scoringProfile, or through the index's defaultScoringProfile; an index can hold up to 100 profiles but only one applies to any given query, so multiple boosts must be combined as several functions inside a single profile. Because a profile only re-weights what the query already found, you can add, modify, or delete one with no index rebuild. It is the documented lever for 'prefer the newest revision' or 'boost this customer's documents', and it applies to nonvector fields only.

The orderby trap

An explicit orderby clause discards relevance ranking^[13] outright, including RRF and the reranker order, and returns rows in the sorted order instead. Sorting grounding results by a recency field to make answers 'fresher' silently destroys the ranking the model depends on. When you need recency to influence grounding, express it as a freshness function in a scoring profile, which re-weights without replacing the ranking, rather than as an orderby.

Figure 8: the ranking stages, and the orderby clause that bypasses them.

Grounding security: trimming in the query

Permission-scoped grounding is enforced by permission metadata in the index applied as a filter during query execution, not by prompt instructions and not by redacting the answer afterwards. Azure AI Search stores permission metadata alongside each indexed document^[18] and excludes non-matching documents inside the query pipeline, before results are ever returned. Figure 9 shows the trim happening at the filter stage, upstream of the model.

The reason this placement is non-negotiable: if you instead tell the model in its system prompt to ignore documents the user may not see, or redact the generated answer, the restricted content has already entered the prompt by then, and the model is not an authorization boundary. Trimming has to precede generation.

Two patterns, not to be inverted

The first is the security-filter pattern^[18], which is API-agnostic. You index a string field holding the user or group identities allowed to see each document; your application obtains the caller's identity at query time and passes it as a filter expression; results whose field does not match the caller are trimmed. It is plain string comparison that your application drives, so the field must be filterable.

The second is the native ACL and RBAC permission-filter pattern^[19], currently in preview. Permission filters are enabled on the index, and the caller's Microsoft Entra token is attached to the query with the x-ms-query-source-authorization header, after which the service compares its user, group, and scope claims to the stored metadata. Two details catch people: with permission filters the client app still needs the Search Index Data Reader role on the index in addition to the per-user token, and a permission change at the source only takes effect after the metadata is resynchronized to the index. The rule to carry: security filters are string comparison you drive; permission filters are Entra authentication the service recognizes. Do not describe one as the other.

Figure 9: document-level trimming happens at the query filter, before the model.

Wiring retrieval into agents and knowledge bases

Retrieval reaches an agent in one of two shapes, and they differ in who composes the answer. Figure 10 contrasts them.

The Azure AI Search tool

Configuring the Azure AI Search tool^[20] means supplying a project_connection_id and an index_name. The optional query_type accepts simple, vector, semantic, vector_simple_hybrid, or vector_semantic_hybrid and defaults to vector_semantic_hybrid; top_k defaults to 5; and any filter you set applies to every query the agent issues against that index, rather than being negotiated per question by the model. One tool instance can target exactly one index. To attach several indexes you add one tool per index or move to a knowledge base.

For grounded answers to carry usable citations, the index needs at least one retrievable text field holding the content plus a retrievable field containing the source URL, optionally a title, so url_citation annotations can link back. When responses come back with no citations at all, the usual cause is agent instructions that never asked for them. And setting query_type to semantic on an index that has vector fields but no semanticConfiguration is a misconfiguration, because the semantic path needs that configuration to exist.

The knowledge base

A knowledge base^[21] adds a planning layer and an outputMode. The default extractive behaviour returns merged grounding content that your application passes to its own model; setting outputMode to answer synthesis makes the knowledge base compose the answer itself with an assigned Azure OpenAI model, shaped by answerInstructions. The enum value for the extractive mode is spelled differently across doc versions, so key on the behaviour, returning chunks, rather than the exact string. A separate lever, retrievalInstructions, tells the planner which knowledge source to prefer for which kind of question. Confusing the two is the common error: retrievalInstructions steers source selection during planning, answerInstructions only shapes the wording of a synthesized answer.

Figure 10: the single-index search tool versus a knowledge base's output modes.

Proving retrieval quality and monitoring

You cannot tune what you do not measure, and the single most important habit here is measuring the retriever separately from the answer. Groundedness scores the generated answer against the context it was given and stays high even when the retriever fetched the wrong documents, so it can never diagnose a weak retriever. Figure 11 walks the choice of evaluator.

Pick the evaluator by whether you have labels

The Document Retrieval evaluator^[22] takes retrieval_ground_truth, a set of per-document query_relevance_label values, plus the retrieved_documents, and computes search metrics: ndcg@3, xdcg@3, fidelity, top1_relevance, top3_max_relevance, holes, and holes_ratio. It needs no model deployment because nothing is judged by a language model. When no labels exist, the Retrieval evaluator is the alternative: it uses an LLM judge over the query and context and scores 1 to 5. Labels buy you objective search metrics; their absence falls back to a model's opinion.

Sweep the parameters, and watch the holes

Microsoft documents parameter sweeping^[22] as the way to tune RAG retrieval: generate retrieval results for several search algorithms, top_k values, and chunk sizes, score each run with the retrieval metrics, and keep the settings that maximize quality. The holes metric guards the exercise, because a high hole count means the labeled set has gaps and the other numbers cannot be trusted. Comparing two configurations on end-to-end answer quality alone is the mistake it replaces, because that mixes retrieval regressions with generation noise.

Monitoring the live pipeline

Once the pipeline ships, the same separation applies to operations. Ingestion quality, search index health, and relevance performance are monitored as distinct signals: a healthy index that ingests cleanly can still return poor relevance, and each is watched on its own rather than rolled into a single 'is search working' number.

Figure 11: labels pick the Document Retrieval evaluator; groundedness rates the answer, not retrieval.

Azure AI Search tool query_type values

Query type	Keyword (BM25)	Vector	Semantic rerank	Best used for
simple	Yes	No	No	Exact-token lookups and structured filters
vector	No	Yes	No	Meaning-based recall over embeddings
semantic	Yes	No	Yes	Keyword recall reordered by the deep reranker
vector_simple_hybrid	Yes	Yes	No	Fused keyword and vector recall, no rerank
vector_semantic_hybrid	Yes	Yes	Yes	The default; fused recall then reranked for grounding

Decision tree

Cheat sheet

Sharp facts the exam loves — give these one last read before exam day.

Cheat sheet

Sharp facts the exam loves — scan these before test day.

A hybrid query is one request carrying both search and vectorQueries, and Reciprocal Rank Fusion merges the two result sets

Hybrid search issues a single request that specifies a full-text search string and one or more vectorQueries; the two run in parallel using BM25 for text and HNSW or exhaustive KNN for vectors, and a Reciprocal Rank Fusion (RRF) algorithm merges them into one ranked result set. Hybrid is the documented default choice because keyword matching handles product codes, jargon, dates, and names that pure vector similarity misses.

Trap Assuming you must run two separate calls and blend the scores in application code, or that adding vectors to an index makes keyword search redundant.

5 questions test this

Semantic ranking only runs when the query sets queryType to semantic and names a semanticConfiguration

Semantic ranking is opt-in per query: the request must set queryType to semantic and reference a semanticConfiguration defined in the index, after which results carry a separate @search.rerankerScore alongside the ordinary @search.score. Because the reranker works from the first-stage candidates, Microsoft advises setting the vector query k to 50 so the ranker has enough input to work with.

Trap Believing relevance improved because a semantic configuration exists on the index, when no query actually requests queryType=semantic and the ranker never fires.

5 questions test this

An explicit orderby clause discards relevance ranking, including RRF and reranker order

Explicit sort orders override relevance-ranked results, so a hybrid or semantic query that also specifies orderby returns rows in the sorted order rather than by fused relevance. Grounding queries that need the most relevant chunks must omit sorting and instead shape ranking with filters, scoring profiles, or the semantic ranker.

Trap Sorting grounding results by a recency field to make answers 'fresher', which silently destroys the ranking the LLM depends on.

6 questions test this

exhaustive switches a vector query from approximate HNSW to full KNN, and oversampling compensates for quantization

A vector query defaults to approximate nearest-neighbour traversal of the HNSW graph; setting exhaustive to true forces an exhaustive KNN scan that maximizes recall at the cost of latency, which is normally used to establish a ground-truth baseline. The oversampling value widens the candidate set retrieved before rescoring, which recovers recall lost to compressed vector storage.

Trap Turning on exhaustive search across a production index to 'improve quality', instead of using it only to measure how much recall the approximate path is losing.

Permission-scoped grounding is enforced by permission metadata in the index applied as a filter during query execution, not by prompt instructions or post-generation redaction

Azure AI Search enforces document-level access control by storing permission metadata alongside each indexed document and excluding non-matching documents inside the query pipeline, before results are returned. Two patterns exist: the API-agnostic security-filter pattern, where you index a string field holding user or group identities, your application obtains the caller's identity at query time and passes it as a filter expression, and results that do not match the string are trimmed; and the native ACL/RBAC permission-filter pattern (preview), where permission filters are enabled on the index and the caller's Microsoft Entra token is attached to the query with the x-ms-query-source-authorization header so the service compares its user, group, and scope claims to the stored metadata. Either way the trimming happens before the retrieved chunks ever reach the model.

Trap Instructing the model in the system prompt to ignore documents the user is not entitled to see, or redacting the generated answer afterwards — by then the restricted content has already entered the prompt, and the model is not an authorization boundary. Do not invert the two patterns either: security filters are plain string comparison that your application drives, while permission filters are recognized as Microsoft Entra authentication; with permission filters the client app still needs Search Index Data Reader on the index in addition to the per-user token, and permission changes at the source only take effect after the metadata is resynchronized to the index.

6 questions test this

A field's attributes, not its content, decide what a grounding query can do with it, and most attribute changes are not an in-place edit

In an Azure AI Search index only searchable fields take part in full-text BM25 matching, only filterable fields can appear in a $filter (the mechanism behind permission trimming, metadata scoping, and a fixed filter on an agent's search tool), only sortable fields can be named in orderby, only facetable fields drive facets, and only retrievable fields can be returned to be quoted or cited; exactly one field must be the key and it must be of type Edm.String, while vector fields of type Collection(Edm.Single) are searchable but cannot be filterable, sortable, or facetable. New fields can be added to a live index at any time, but an existing field's data type and most of its attributes are locked in for the lifetime of the index, so turning on filtering or sorting after the fact means adding a new field or dropping and rebuilding the index and re-indexing every document.

Trap Assuming a field can be filtered or sorted because its value is visible in results. retrievable only controls whether the value comes back to the caller; a security-trimming or metadata filter on that field fails until the field is redefined as filterable, which existing documents will not pick up without a rebuild and full re-index.

6 questions test this

A scoring profile boosts or suppresses the ranking of documents a query already matched, and only one profile applies to any given query

A scoring profile is a named object defined in the index schema that boosts or suppresses the ranking of matching documents; it is built from text weights over searchable fields plus optional functions - freshness over an Edm.DateTimeOffset field, magnitude over a numeric range, distance from a reference point, and tag for overlap with a caller-supplied tag list - and functions can only be applied to fields attributed as filterable. A query uses one by naming it in the scoringProfile parameter, with scoringParameters supplying the per-request reference point or tag list, or through the index's defaultScoringProfile; an index can hold up to 100 profiles but you can only specify one profile at a time in any given query, and profiles work in keyword, vector, hybrid, and semantically reranked queries yet apply only to nonvector fields. Because a profile only adjusts the score of documents the query already matched, you can add, modify, or delete one with no index rebuild and no effect on indexed documents, which makes it the documented lever for 'prefer the newest revision' or 'boost this customer's documents' in a grounding pipeline.

Trap Reaching for orderby or a $filter to express the same preference: sorting replaces relevance ranking outright and a filter deletes non-matching documents from the result set, whereas a scoring profile re-weights only what the query already found. Two further near-misses: naming two profiles in one request to combine boosts, when only one profile applies per query so the criteria must be combined as multiple functions inside a single profile; and expecting a profile to lift a purely vector match, when profiles apply only to nonvector fields and a function over a field that isn't filterable produces no boost at all.

5 questions test this

The Text Split skill's textSplitMode chooses between multi-sentence pages and one-sentence chunks

textSplitMode accepts pages (the default, producing chunks of several sentences bounded by maximumPageLength) or sentences (one sentence per chunk, with sentence boundaries decided by defaultLanguageCode). Page mode adds pageOverlapLength, which must be less than half the maximum page length, and maximumPagesToTake, which defaults to 0 meaning take every chunk.

Trap Choosing sentences mode for a long PDF corpus, which explodes the chunk count into fragments too small to carry answerable context.

12 questions test this

Microsoft's documented starting point is roughly 2,000 characters per chunk with about 500 characters of overlap

For fixed-size chunking Azure AI Search recommends starting at a chunk of about 512 tokens (roughly 2,000 characters) with about 25 percent overlap (roughly 128 tokens, or 500 characters), then tuning by content type. Overlap preserves continuity across chunk boundaries, but an overlap value set too large relative to the actual content length can result in no usable overlap at all.

Trap Pushing overlap toward 50 percent on the theory that more context is always better, which duplicates content, inflates index size, and can break the overlap entirely.

9 questions test this

The Azure Content Understanding skill does semantic chunking with Markdown output that can span page boundaries

The Text Split skill cuts on character or token counts and cannot cross a document's structural seams intelligently, while the Azure Content Understanding skill performs semantic chunking with Markdown output and produces units that preserve meaning across page boundaries and keep cross-page tables intact. Content Understanding therefore combines extraction and chunking in one skill instead of requiring a separate splitter.

Trap Keeping a character-based splitter on layout-heavy contracts and reports, which slices tables and clauses in half so no retrieved chunk contains a complete answer.

7 questions test this

Chunking exists first to stay under the embedding model's input token ceiling

Embedding models impose a hard input limit — text-embedding-3-small accepts 8,191 tokens, roughly 6,000 words — and content past the limit is truncated rather than embedded, silently losing data. Chunking is also worthwhile below the limit when one document covers several subtopics, because a single vector over mixed content represents none of them well.

Trap Treating chunking as purely a relevance-tuning knob and skipping it for documents that 'fit', without checking the model's token ceiling.

Integrated vectorization needs an indexer, a skillset with a chunking skill plus an embedding skill, and an index that receives the vectors

Indexing-time vectorization depends on three pieces working together: an indexer that pulls from a supported data source and drives the pipeline, a skillset combining a chunking strategy with an embedding skill such as the AzureOpenAIEmbedding skill, and a search index to receive the chunked, vectorized content. Removing any one of them means embeddings must be generated and pushed by your own code.

Trap Assuming that defining vector fields on the index is enough, when nothing in the pipeline actually calls an embedding model.

4 questions test this

The vectorizer declared in the index must use the same embedding model that encoded the content

Query-time text-to-vector conversion comes from a vectorizer defined in the index schema, assigned to a vector profile which is in turn assigned to the vector field; the vectorizer must match the embedding model used during indexing (AzureOpenAIEmbedding skill pairs with the Azure OpenAI vectorizer, the AML skill with the Foundry model catalog vectorizer, and so on). Mismatched models put query and document vectors in different spaces and relevance collapses.

Trap Upgrading the indexing embedding model to a newer version without re-embedding the corpus or updating the vectorizer, and blaming the drop in quality on chunk size.

5 questions test this

Index projections write chunk-grain rows to a secondary index while the parent document stays in the primary index

Optional index projections let one indexer run populate a granular chunk index alongside a document-level index, both from the same source document. The chat application matches on the fine-grained secondary index and then returns the richer parent document from the primary index, which is the documented pattern for question-answering and chat-style apps over long PDFs.

Trap Flattening everything into one chunk index, which loses the document-level title, date, and summary fields that make a complete answer possible.

3 questions test this

Indexer batching and retry on embedding throttling are built in and non-configurable, so run the indexer on a schedule

Azure AI Search has internal, non-configurable retry policies for throttling errors raised when an Azure OpenAI embedding deployment exhausts its tokens-per-minute allowance, and Microsoft recommends putting the indexer on a schedule so calls dropped despite those retries are picked up on the next run. Token-per-minute limits apply per model per subscription, so sharing one embedding deployment between the ingestion and query workloads makes both throttle.

Trap Tuning an indexer batch size to dodge throttling, when the batching and retry behavior is not exposed for configuration at all.

The Custom Web API skill is the skillset extension point when no built-in skill fits, and it imposes a fixed values/recordId batch contract on your endpoint

Microsoft.Skills.Custom.WebApiSkill calls your own endpoint from inside the skillset; the uri must use the HTTPS scheme, and the indexer sends up to batchSize records per call (default 1000) as a top-level values array whose elements each carry a unique recordId and a data object matching the skill's declared inputs. Your service must reply with the same recordIds, a data object matching the declared outputs, and errors and warnings properties that are required but may be null - a non-JSON response, a missing recordId, or a duplicate one means that record is not enriched. Set authResourceId or authIdentity so the search service's managed identity authenticates instead of embedding a function key in the uri.

Trap Assuming your enrichment endpoint can take and return whatever JSON shape it likes, or that a plain http:// endpoint is acceptable. The envelope is fixed and HTTPS-only, and any response record whose recordId was not in the request is discarded. It is also distinct from the Azure Machine Learning skill, which targets a model deployed in an AML online endpoint rather than an arbitrary Web API.

4 questions test this

Built-in language skills enrich grounding content during indexing, but their output reaches the index only through an output field mapping

Entity Recognition, Key Phrase Extraction, Language Detection, PII Detection (which can also mask the detected entities), Sentiment, and Text Translation are billable built-in skills that run pretrained Foundry Tools language models over each document inside the skillset, producing values you can then filter, facet, scope, or redact grounding data on. Everything a skill emits lives only in memory, as a node in the enriched-document tree, for the duration of the indexer run: to persist it you must add an outputFieldMappings entry to the indexer whose sourceFieldName is the /document/... path of the skill output and whose targetFieldName is a top-level simple field or collection in the index. A skill that is configured correctly but whose output is never mapped enriches nothing, and the index looks as though the skill never ran.

Trap Reaching for fieldMappings instead of outputFieldMappings. fieldMappings maps verbatim source fields to index fields and can never address a skill output; only outputFieldMappings maps in-memory enrichments, and its target must be a top-level simple field or collection, not a path into a complex type. A second near-miss is assuming the Foundry resource attached to the skillset does the processing - for these skills it is attached for billing only, and Azure AI Search executes them on internal resources.

3 questions test this

The Document Extraction skill is the lightweight cracker; the Azure Content Understanding skill is the one that keeps tables, positions, and cross-page units

Both skills crack a document into page text and inline images, but only the Azure Content Understanding skill extracts text location metadata, preserves tables including those spanning pages, produces semantic units that cross page boundaries, and works across PDF, DOCX, XLSX, and PPTX; the Document Extraction skill returns image location metadata for PDFs only and has no table extraction or built-in chunking. Microsoft directs new skillsets to the Content Understanding skill and keeps the older Document Layout skill supported only for existing pipelines.

Trap Reaching for the Document Layout skill on a new build, or picking Document Extraction for a table-heavy corpus because it is cheaper per document.

8 questions test this

The GenAI Prompt skill turns each extracted image into a natural-language description that is indexed and embedded as text

Image verbalization calls an LLM once per extracted image at ingestion time through the GenAI Prompt skill, storing a concise description such as "five-step HR access workflow that begins with manager approval" next to the surrounding document text. Because the picture is now expressed in language, the pipeline can explain relationships inside a diagram and hand an LLM a caption it can cite verbatim, at the cost of one model call per image.

Trap Expecting verbalization to also support image-as-query lookups; the GenAI Prompt skill supports text-to-vector hybrid queries but not image-to-vector queries.

9 questions test this

Querying with an image as input requires a multimodal embedding model and its matching vectorizer, not verbalization

Only multimodal embedding models expose vectorizers that convert an image into a vector at query time, so a "find things that look like this" experience must be built with the AML skill or the Azure Vision multimodal embeddings skill plus the equivalent vectorizer. Direct multimodal embeddings need no LLM at indexing time but carry no explanation of why two images are related and give the LLM no ready-made text to cite.

Trap Assuming an index built with the GenAI Prompt skill can accept an uploaded photo as the query, because it already 'understands' images.

8 questions test this

Extracted images live in a knowledge store, with their location recorded in the index for retrieval at answer time

A multimodal pipeline stores the images it pulls out of source documents in a knowledge store, and the index keeps each image's location so the application can render the original figure next to the cited text. That is what lets a RAG answer show both a textual citation and the diagram snippet it came from.

Trap Trying to return the image bytes from the search index itself rather than storing them and indexing a pointer.

The Azure AI Search agent tool defaults to vector_semantic_hybrid and accepts five query_type values

Configuring the tool means supplying project_connection_id and index_name; the optional query_type accepts simple, vector, semantic, vector_simple_hybrid, or vector_semantic_hybrid and defaults to vector_semantic_hybrid, top_k defaults to 5, and any filter you set applies to every query the agent issues against that index.

Trap Setting query_type to semantic on an index that has vector fields but no semantic configuration, or expecting filter to be negotiated per question by the model.

5 questions test this

The tool can target exactly one index, and citations need retrievable text plus a source URL field

One Azure AI Search tool instance can only target a single index, and for grounded answers to carry usable citations the index needs at least one retrievable text field holding the content plus a retrievable field containing the source URL (optionally a title) so url_citation annotations can link back. When responses come back with no citations at all, the usual cause is agent instructions that never ask for them.

Trap Attaching several indexes to one Azure AI Search tool definition rather than adding one tool per index or moving to a knowledge base.

3 questions test this

A knowledge base's outputMode decides whether the retrieve call returns grounding chunks or a synthesized answer

Setting outputMode to answerSynthesis makes the knowledge base compose an answer with the assigned Azure OpenAI model, shaped by answerInstructions, while the default extractive behaviour returns merged grounding content that your application passes to its own model. retrievalInstructions is the separate lever that tells the planner which knowledge source to prefer for which kind of question.

Trap Confusing retrievalInstructions with answerInstructions; the first steers source selection during planning, the second only shapes the synthesized answer's wording.

7 questions test this

The document_retrieval evaluator needs human relevance labels and returns search metrics, not an LLM judgment

The Document Retrieval evaluator takes retrieval_ground_truth (per-document query_relevance_label values) plus retrieved_documents and computes ndcg@3, xdcg@3, fidelity, top1_relevance, top3_max_relevance, holes, and holes_ratio; it needs no model deployment because nothing is judged by an LLM. The Retrieval evaluator is the alternative when no labels exist: it uses an LLM judge on query and context and scores 1 to 5.

Trap Reaching for Groundedness to diagnose bad retrieval — groundedness scores the generated answer against the context it was given, and stays high even when the retriever fetched the wrong documents.

4 questions test this

A parameter sweep replays the same labeled query set across retrieval settings and picks the highest-scoring configuration

Microsoft documents parameter sweeping as the way to tune RAG retrieval: generate retrieval results for several search algorithms, top-k values, and chunk sizes, then score each run with the retrieval metrics and keep the settings that maximize quality. The holes metric guards the exercise, because a high hole count means the labeled set has gaps and the other numbers cannot be trusted.

Trap Comparing two retrieval configurations on end-to-end answer quality alone, which mixes retrieval regressions with generation noise.

Two pipelines, one index

What this page owns, and what the sibling page owns

The index schema is a contract you set once

Why the design is nearly permanent

Chunking content to fit the embedder

Two ways to split, and when each fits

Sizing the chunks

Skills enrich content while the indexer runs

Cracking: two skills, different depth

Built-in language skills need an output field mapping

The custom extension point

Integrated vectorization: embeddings inside the pipeline

The vectorizer has to match the embedder

Index projections keep chunk and document grain

Throttling is handled for you, so schedule the runs

Grounding on images and other modalities

Verbalization: describe the image as text

Multimodal embeddings: query with an image

Where the pixels live

Choosing a retrieval mode: keyword, vector, hybrid

The vector knobs: recall versus latency

Shaping the ranking, and the orderby trap

Semantic reranking

Scoring profiles

The orderby trap

Grounding security: trimming in the query

Two patterns, not to be inverted

Wiring retrieval into agents and knowledge bases

The Azure AI Search tool

The knowledge base

Proving retrieval quality and monitoring

Pick the evaluator by whether you have labels

Sweep the parameters, and watch the holes

Monitoring the live pipeline

Azure AI Search tool query_type values

Decision tree

Cheat sheet

References