AI-103 Cheat Sheet

Model router is deployed like any other Foundry model; at request time it analyzes the prompt and routes it to one of its supported underlying models, so you get a single deployment and chat surface instead of managing many. The routing mode controls the trade-off: Balanced (default) picks the cheapest model within a narrow quality band of the best model for that prompt, Cost widens that band for more savings, and Quality always picks the highest-rated model regardless of cost.

Trap Treating model router as a gateway or load balancer across resources. It is a trained routing model inside one deployment, not Azure API Management traffic distribution, and its effective context window is capped by the smallest model in its subset.

The model leaderboard in the Foundry portal compares curated catalog models across quality (reasoning, knowledge, math, coding), safety, performance (latency and throughput), and estimated cost, with scenario-specific leaderboards and a side-by-side view for up to three models. Leaderboards are the base-model-selection stage of the evaluation lifecycle, ahead of pre-production evaluation.

Trap Treating leaderboard rank as proof of fitness for your workload. Benchmarks run on public datasets, so Microsoft directs you to re-evaluate shortlisted models against your own data with the evaluation SDK.

A reasoning model generates intermediate reasoning tokens and summarizes them before returning the first response token, so even streaming requests can stall until reasoning finishes and client-side timeouts must be set far higher than for a non-reasoning chat model. You control how much of that work happens with the reasoning effort parameter, not with sampling parameters.

Trap Raising temperature or top_p to make a chat model 'think harder'. Sampling parameters only change token-selection randomness; they add no reasoning pass and no reasoning tokens.

A vision-capable chat model is the right choice when the ask is open-ended interpretation, captioning, or question answering over an image or document. When the output must be named business fields in structured JSON with a numeric per-field confidence you can threshold for human review, the requirement points at a schema-driven Foundry Tool analyzer instead.

Trap Assuming a multimodal model's own certainty can gate a review queue. Chat completions return no calibrated per-field confidence score, so there is nothing to compare against a 0.80 threshold.

Small language models are compact generative models, typically from under 1 billion to around 14 billion parameters — Microsoft's Phi family is the canonical example — versus large language models with hundreds of billions. Compared with an LLM they need far fewer resources (a single GPU or even CPU), give lower latency and higher throughput per device, and can run on-premises or on-device through Foundry Local, while trading away some breadth of general knowledge. That makes an SLM the right selection for narrow, well-specified, high-volume work such as classification, intent routing, field extraction, or short summarization, and a frontier LLM the right selection when the task needs broad world knowledge or long multi-step reasoning.

Trap Defaulting to the largest frontier model because 'quality matters' when the requirement is thousands of cheap, fast, single-purpose calls — on a focused task a small model can match a larger one at a fraction of the latency and cost. The inverse trap is treating an SLM as simply a cheaper LLM: it is the breadth of general knowledge and long-horizon reasoning that is traded away, not just price, so an open-ended research or planning agent still needs a large model.

The Foundry model catalog carries code-specialized reasoning models - the Codex family such as gpt-5-codex, gpt-5.1-codex, and gpt-5.1-codex-mini - tuned for agentic software work: reading and refactoring files across a repository, writing tests, and opening pull requests from a terminal, VS Code, or a GitHub Actions runner against your Foundry project's deployments. Selecting one of them is a deployment decision about which model authors the code. The Code Interpreter tool sits on a different axis: it gives whatever model you already deployed a sandbox in which to execute Python it has written, for data analysis, computation, and charts over attached files.

Trap Answering a 'raise the quality of generated code' or 'refactor across the repository' requirement by attaching Code Interpreter to a general chat model. Code Interpreter only executes model-written Python in a sandbox against attached files; it does not change which model authors the code and it is not a repository-editing coding agent.

Supervised fine-tuning trains on labeled prompt/completion (or conversational) examples and is the starting technique for task specialization, output format, and instruction following, working best when a problem has finite correct solutions; direct preference optimization requires preferred and non-preferred responses supplied as pairs in the training set and aligns the model to subjective qualities such as tone, style, and content preference without fitting a separate reward model; reinforcement fine-tuning replaces example outputs with a grader that rewards the model incrementally, so it fits complex reasoning problems that have many possible solution paths but checkable answers. Foundry also documents stacking SFT first and DPO afterwards, and whichever technique you use the result is a separate custom model that must be deployed to its own endpoint before any client can call it, so a production deployment adds an ongoing hourly hosting charge to the one-time training cost and the usual per-token inference.

Trap Reaching for DPO because the requirement says 'better answers'. DPO consumes a preferred/non-preferred pair per example; a set of known-good outputs is SFT data, and a task whose correctness a scoring function can judge is RFT territory. Choosing the technique before checking which of those three datasets the team can actually produce is what makes fine-tuning projects stall.

Azure Translator in Foundry Tools is neural machine translation across 100+ languages through a single managed call, producing consistent output run to run at a low per-character cost, which suits high-volume plain copy. An LLM translation flow is the right pick only when tone, domain context, or nuance must be carried, because it costs more per token, adds latency, and can render the same source differently between runs.

Trap Fine-tuning a Custom Translator model per language pair when the source text has no domain terminology. Customization exists to teach in-domain vocabulary and style, so with generic copy it adds per-pair training and maintenance for no quality gain.

The Foundry guardrail system applies at model and agent intervention points, so it can only see prompts, tool traffic, and completions flowing through a Foundry deployment. Content that must be moderated before it ever reaches a model, such as a user upload being stored or routed to a human, requires a direct call to the Azure AI Content Safety analyze APIs, which return per-category severity you compare against your own threshold.

Trap Expecting a model deployment's guardrail to screen an attachment the application never sends to the model. If the payload does not traverse an intervention point, no control fires.

Content Understanding, Speech, Translator, Document Intelligence, and Content Safety are Foundry Tools available as part of the Microsoft Foundry resource rather than as separate standalone accounts you must each provision, connect, and secure. Consolidating them behind one account means a single endpoint, one set of role assignments, and one keyless authentication path for the whole tool surface.

Trap Provisioning a separate Cognitive Services account per capability and then wiring per-service keys, which duplicates the identity and networking work the Foundry resource already centralizes.

The Web Search and Grounding with Bing Search tools bring real-time public web results with URL citations into an agent's answer, which fits questions about current public information. Grounding with Bing Search retrieves from the web at large rather than specific web domains - to narrow results to domains you choose, use Grounding with Bing Custom Search or Web Search's custom_search_configuration allow/block lists. All of these reach only public, Bing-indexed content, so an internal catalog, policy library, or ticket history must be grounded through Azure AI Search or another knowledge source you own.

Trap Selecting Grounding with Bing to answer from an internal product catalog. It cannot reach private content at all, and its results carry no tenant permission model.

In agentic retrieval an application calls a knowledge base with a retrieve action; an LLM decomposes the request, using conversation history, into focused subqueries that execute simultaneously against the knowledge sources, are each semantically reranked, and are merged into unified grounding data with optional references and an activity log. Classic RAG sends one query to the index and leaves planning and the LLM handoff to your code.

Trap Choosing 'iterative retrieval' as if it were a third selectable pattern. Iterative search is an internal follow-up pass inside agentic retrieval at medium reasoning effort, and it is sequential, so it adds latency rather than parallelism.

Reasoning effort is configured on the knowledge base and defaults to low. At low and medium effort the knowledge base sends the query and conversation history to an LLM to generate subqueries; at minimal effort that planning step is skipped entirely and queries go straight to the knowledge sources. Lowering effort therefore cuts LLM token spend and latency at the cost of decomposition quality.

Trap Assuming minimal effort still uses conversation history. With planning bypassed there is no query rewriting or history-aware decomposition, so layered follow-up questions degrade.

Knowledge sources define the content an agentic retrieval pipeline draws on. An indexed knowledge source is backed by a search index you host on the Azure AI Search service, so content must be ingested first. A remote knowledge source retrieves content from an external platform at query time, which avoids an ingestion pipeline but makes freshness and permissions the source system's responsibility.

Trap Assuming every knowledge source needs its own indexer. Remote sources are queried live and never populate a local index.

Retrieval-augmented generation injects retrieved passages at inference time, so refreshed content is reflected as soon as the index updates with no retraining. Fine-tuning adapts behavior, style, task performance, and output format rather than storing new facts, and prompt engineering is where per-conversation instructions such as language or support tier belong. A single scenario often needs all three together.

Trap Fine-tuning on a daily-changing catalog. Weights cannot be updated per transaction, so the model fabricates stale detail while incurring repeated training and hosting cost.

The pull model attaches an indexer to a supported data source object and automates crawling, change detection and skillset execution on a schedule that can run as often as every five minutes, so index freshness is bounded by that run interval. The push model calls the Index Documents REST API or an SDK client to submit per-document upload, merge, mergeOrUpload and delete actions from your own code, and Microsoft documents no restriction on data source type and no restriction on execution frequency, which makes push the only option when content originates outside the supported sources or when the index must stay in sync faster than any schedule. The price is that skillsets attach to indexers and don't run independently, so AI enrichment and integrated vectorization are unavailable on the push path and your code must produce the chunks, the vectors and the deletes.

Trap Answering a 'source is an on-premises ERP and the index must reflect price changes within seconds' requirement with 'shorten the indexer schedule': five minutes is the floor for a recurring indexer run, so the freshness requirement still fails no matter how the source is staged. The mirror-image trap is choosing push for a Blob-hosted PDF corpus that needs OCR and vectorization, which throws away the only pipeline that can run a skillset.

Built-in tools such as Web search, Code Interpreter, File Search, Azure AI Search, and Azure Functions are preconfigured capabilities the service executes for you after basic configuration. Custom tools are the Model Context Protocol tool, the OpenAPI 3.0/3.1 tool, and Agent-to-Agent endpoints, which you host or describe yourself. Function calling sits apart: the agent proposes the call and your application executes it and returns the result.

Trap Confusing Function calling with the Azure Functions tool. With Function calling your own process runs the code and returns the value; the Azure Functions tool has the service invoke a deployed Azure Function.

The Model Context Protocol tool connects an agent to tools hosted on an MCP server endpoint, which is the documented fit when the tools are shared across multiple agents or owned by a different team. The OpenAPI tool connects the agent to an external HTTP API described by an OpenAPI 3.0 or 3.1 specification and supports anonymous, API key, and managed identity authentication defined inside the tool definition.

Trap Modeling an API key as a per-operation header parameter in the OpenAPI spec. The connection's key is injected only when the spec declares it as an apiKey security scheme referenced from a security section.

A toolbox is a curated bundle of tools configured once and exposed as a single MCP-compatible endpoint, so any MCP-capable runtime can consume it instead of every agent definition attaching each tool. The toolbox handles credential injection, token refresh, and policy enforcement centrally with Microsoft Entra ID and OAuth, and supports versions that agents on the consumer endpoint pick up when you promote a new default.

Trap Assuming toolbox promotion requires redeploying every consuming agent. Agents bound to the consumer endpoint receive the promoted default version without code changes.

The File Search tool augments an agent with knowledge from files developers or end users upload, backed by vector stores the agent creates. The Azure AI Search tool instead grounds the agent on an existing Azure AI Search index, which is what you use when the corpus already has its own ingestion pipeline, enrichment skills, semantic configuration, and permission model.

Trap Choosing File Search when the corpus must stay under your own index governance. File Search vector stores are created by the agent, so index schema, enrichment, and refresh are not yours to control.

The SharePoint tool grounds an agent on documents in a connected SharePoint site or folder using the Microsoft 365 Copilot retrieval stack, so you do not export content, build a semantic index, or manage refresh logic; the Microsoft Fabric data agent tool answers questions over governed structured data by calling a published Fabric data agent over its lakehouse, warehouse, KQL database, or Power BI semantic model sources. Both use identity passthrough (On-Behalf-Of): the retrieval or query runs as the signed-in user, so each end user must already hold read access to the SharePoint site or to the Fabric data agent and its underlying sources, app-only/service-principal authentication is unsupported, and the data source and Foundry project must be in the same tenant.

Trap Reaching for File Search or the Azure AI Search tool and building your own index over content that already lives in SharePoint or Fabric — that duplicates the corpus, adds a refresh pipeline, and drops the source's per-user permissions unless you rebuild them. The opposite trap is assuming a headless or app-only agent can use these tools, or that granting the agent's managed identity access is sufficient: with On-Behalf-Of the call fails unless the individual end user has access, which also rules these tools out for unattended batch jobs with no signed-in user.

Conversations store items rather than only chat messages, capturing user and assistant messages, tool call items, tool output items, and output items so the next turn can reuse that context within the session. Memory in Foundry Agent Service is a separate managed long-term store that extracts, consolidates, and later retrieves durable knowledge so an agent stays continuous across sessions, devices, and workflows.

Trap Reusing the same agent session ID to replay prior turns. With the Responses protocol, continuity comes from previous_response_id or a conversation ID; the session ID alone does not resend earlier messages to the model.

User profile memory holds durable preferences and personal context such as language or accessibility needs and should be retrieved near the start of a conversation to establish personalization. Chat summary memory holds distilled summaries of prior topics and is retrieved per turn for continuity. Procedural memory holds reusable how-to routines inferred from past interactions and is retrieved when the user asks for a recurring workflow.

Trap Expecting memory to be raw transcript storage. The service extracts and consolidates with an LLM, merging duplicates and resolving conflicting facts, so what is retrieved is distilled rather than verbatim.

Attaching the memory search tool to a prompt agent lets it read from and write to the memory store during conversations, which is the recommended default. The low-level Memory Store APIs are for advanced cases needing direct control of individual memory records, store-level default time to live, and explicit remember-or-forget behavior, but they require you to set scope explicitly on every request.

Trap Expecting the low-level APIs to infer scope from the caller. Automatic scope resolution is supported only through the memory search tool with scope bound to the user ID.

The Foundry SDK (Python package azure-ai-projects 2.x) is a thin client over all Foundry project APIs reached through a single project endpoint of the form https://.services.ai.azure.com/api/projects/. You construct AIProjectClient with that endpoint and a credential such as DefaultAzureCredential, so one configuration value covers models, agents, tools, and evaluations.

Trap Configuring the app with the Azure OpenAI resource endpoint (https://.openai.azure.com/openai/v1) instead - that surface serves model inference only and cannot reach Foundry agents, project connections, or evaluations.

AIProjectClient exposes Foundry-native work that has no OpenAI equivalent - reading project properties, listing and resolving connections, and enabling tracing. Calling project_client.get_openai_client() returns an OpenAI-compatible client used for anything modeled on OpenAI shapes: responses, conversations, vector stores, evaluations, and fine-tuning. Most production apps instantiate both.

Trap Expecting the project client itself to expose response or conversation methods, for example project.responses.create(...), instead of going through the OpenAI-compatible client it hands out.

The project endpoint used by the Foundry SDK does not currently route embedding calls. An application that generates its own vectors must send embedding requests to the Azure OpenAI /openai/v1 endpoint with the OpenAI SDK, even while the rest of the application talks to the project endpoint.

Trap Assuming the 'single endpoint' promise is absolute and pointing an embedding call at the project endpoint, which fails rather than transparently forwarding.

A Foundry project connection stores an external resource's target URI plus its authentication material - custom keys, OAuth app registration, project managed identity, or agent identity. Tool definitions reference the connection by project_connection_id rather than carrying the secret, so credentials never appear in agent definitions, prompts, or application configuration.

Trap Shipping the downstream API key in an environment variable or in the tool's header block, which defeats rotation and removes per-principal attribution.

Anthropic Claude deployments in Foundry are called through https://.services.ai.azure.com/anthropic using the Anthropic Messages API and the Anthropic SDK. They are not served by the OpenAI-compatible client, so an application that mixes model families needs a second client path.

Trap Assuming every catalog model is reachable through the same OpenAI-compatible responses call.

openai.responses.create(...) returns output_text for the final assistant text and an output list of typed items such as message, function_call, file_search_call, and web_search_call. Inspecting tool behavior therefore means iterating response.output and switching on item.type, not parsing the answer text.

Trap Trying to infer which tool ran from the wording of the final message instead of reading the output items.

The Azure OpenAI /openai/v1 surface offers the full OpenAI API shape with the lowest latency and best client-library compatibility. Foundry agents, Foundry-exclusive platform tools, and evaluations are only available through the Responses API served on the project endpoint, so a chat app that later needs agent tooling must move off the direct model route.

Trap Choosing the direct Azure OpenAI endpoint for a build that also needs file search, memory, or MCP tools.

Because the service persists response history, the next call can pass previous_response_id and inherit the prior turn's context without the client resending any messages. This is the lightest multi-turn pattern and needs no conversation object.

Trap Rebuilding and resending the whole message array on every turn when a response id already carries the context.

With store set to false the service does not persist the response, so previous_response_id has nothing to reference and the application must pass earlier output items back as input on the following request. This is the pattern for zero-data-retention environments and for teams that require full control of conversation state.

Trap Treating store=false as a logging switch and then finding follow-up turns have lost all context.

Inference calls pass the deployment name created in the Foundry project rather than a catalog model identifier. Repointing a deployment at a newer model version, or creating a same-named deployment of a different model, changes behavior without touching application code.

Trap Hard-coding a catalog model string that does not match any deployment name in the target project.

Setting the response text format to json_schema with strict set to true makes the model conform to the supplied schema. The supported subset requires every property to be listed in required, objects to declare additionalProperties as false, and forbids anyOf at the schema root.

Trap Writing a natural schema with genuinely optional fields; strict mode rejects it because all properties must be required (model optionality by allowing null instead).

JSON mode constrains the model to emit syntactically valid JSON and nothing more. Field names, nesting, types, and presence of required keys are unenforced, so a downstream parser can still break on a renamed or missing property. Only structured outputs enforces the contract itself.

Trap Selecting JSON mode for a schema-driven extraction pipeline and adding validation retries instead of enforcing the schema at generation time.

When a function tool is selected, the model emits a function_call item carrying name, JSON arguments, and a call_id. Your application executes the code and posts a function call output item keyed to that call_id; only then does the model produce its final message. Multiple function_call items can be emitted in one response and each needs its own output.

Trap Expecting Agent Service to run your Python function - only service-side tools such as Azure Functions, OpenAPI, and MCP tools are executed by the platform.

Setting tool_choice to required makes the model invoke one of its tools before answering, which is how you guarantee that a retrieval or delegation step actually ran and that citations exist. The default auto lets the model answer from parametric knowledge and skip the tool entirely.

Trap Diagnosing 'no citations returned' as a retrieval failure when the model simply chose not to call the tool.

Adding a file to a vector store triggers managed parsing, chunking, embedding, and indexing. Defaults are 800-token chunks with 400-token overlap, text-embedding-3-large at 256 dimensions, and a maximum of 20 chunks placed in context per query.

Trap Standing up a separate chunking and embedding pipeline before uploading, duplicating work the tool already performs.

Under basic agent setup, uploaded files sit in Microsoft-managed storage and vector stores are built on a Microsoft-managed search resource. Under standard agent setup, the same tool writes files to your connected Azure Blob Storage account and creates vector stores in your connected Azure AI Search resource. The application code is identical; only data residency and control differ.

Trap Assuming that bringing your own storage and search requires rewriting the tool configuration or the retrieval code.

Choose file search when the corpus arrives as user or application uploads and you want the service to handle ingestion end to end. Choose the Azure AI Search tool when an enriched index already exists and is refreshed by your own indexer or skillset pipeline, so the agent consumes it rather than re-ingesting the content.

Trap Re-uploading documents that are already indexed, which creates a second copy that drifts out of sync with the source index.

Ground volatile or high-cardinality facts with retrieval so answers track the source of truth; encode durable voice, formatting, and behavioral conventions with fine-tuning so they hold without prompt bloat; and carry per-session variables such as language or entitlement tier in the prompt. A requirement set that names all three needs all three.

Trap Fine-tuning on a catalog that changes daily, which bakes stale facts into weights and still needs retraining every refresh.

A vector store holds up to 10,000 files with a 512 MB per-file ceiling. At most one vector store can be attached to an agent and one to a conversation. Vector stores created through conversation helpers carry a default expiration of seven days after last activity, and runs on that conversation fail once it lapses.

Trap Attaching several vector stores to one agent to widen coverage instead of consolidating files into a single store.

A Foundry workflow is a declarative, versioned orchestration over agents and business logic. Human in the loop pauses to ask the user a question or collect an approval, Sequential passes each agent's result to the next in a fixed order, and Group chat hands control between agents dynamically based on context. Every save produces a new immutable workflow version.

Trap Reaching for Group chat when the process is a fixed pipeline; dynamic handoff adds nondeterminism to a sequence that must be auditable.

Workflow node types are Agent (invoke an agent), Logic (if/else, go to, for each), Data transformation (set or parse a variable), and Basic chat (send a message or ask a question). Power Fx formulas compute values inside these nodes and must prefix a variable with System. for built-in context or Local. for workflow variables.

Trap Referencing a variable without its scope prefix, which raises a 'Name isn't valid' formula error rather than silently resolving.

The visual workflow designer accepts prompt agents from the project but does not support hosted agents as nodes. When a containerized hosted agent must coordinate other agents or run a multistep process, that orchestration belongs in its own code using Microsoft Agent Framework workflows.

Trap Planning a portal workflow around a hosted agent and discovering only prompt agents can be assigned to agent nodes.

Agent Framework provides sequential, concurrent, handoff, and group-chat orchestrations in Python and .NET, authored as code or declarative YAML. Foundry is moving workflow authoring to this code-first model, and the result runs on the platform when packaged and deployed as a hosted agent.

Trap Assuming a code-first orchestration needs its own hosting; the hosted agent runtime supplies the endpoint, scaling, and identity.

The Groundedness evaluator uses a judge model you deploy and reports a 1 to 5 score against the retrieved context. Groundedness Pro (preview) calls the Azure AI Content Safety service instead, returns a binary pass or fail, and requires no judge-model deployment - so it costs less to stand up but cannot rank near-miss answers.

Trap Selecting Groundedness Pro when the team needs a graded threshold for routing borderline answers to review.

Retrieval scores how effectively the search step surfaced and ranked useful context, Groundedness scores whether the answer is actually supported by that context, and Relevance scores whether the answer addresses the user's query at all. A fabricated citation is a Groundedness signal; 'the right document never came back' is a Retrieval signal.

Trap Tuning the prompt when Retrieval scores are the ones failing, which cannot recover context that was never returned.

evaluation_level is set to turn (individual responses, the default) or conversation (the whole multi-turn interaction). Each evaluator declares its levels in supported_evaluation_levels, and every evaluator in a run must support the run's level, so you cannot mix incompatible levels. Customer Satisfaction, Task Completion, Coherence and Groundedness support both levels, whereas the tool-call and safety evaluators are turn-only and cannot be added to a conversation-level run.

Trap Assuming an evaluator's name implies its scope. Customer Satisfaction and Task Completion sound conversation-only but support both levels; the tool-call and safety evaluators are the turn-only ones that will break a conversation-level run.

F1 Score, BLEU, GLEU, ROUGE, and METEOR are deterministic overlap metrics computed against ground-truth references, so they are unusable without labeled answers. Coherence, Fluency, Relevance, and Similarity are AI-assisted evaluators that call a judge model and can score outputs with no reference text.

Trap Planning a BLEU or ROUGE gate for open-ended generation where no reference answer exists.

Image generation calls against a GPT-image-series deployment return the picture as base64 data in the response's b64_json field, and the response_format parameter is not supported for these models. The application must decode those bytes and persist the file itself; there is no URL variant to fetch later.

Trap Writing client code that reads a url field from data[0], which was the older DALL-E response shape and yields a KeyError against a GPT-image deployment.

The dall-e-3 model was retired on 4 March 2026 and is no longer available for new deployments, with existing deployments non-functional. A new Foundry image workload must be built on the GPT-image family (gpt-image-1, gpt-image-1-mini, gpt-image-1.5, gpt-image-2).

Trap Picking DALL-E 3 because it is remembered as the Azure image model, or because of its automatic prompt-rewriting behaviour.

Setting background to transparent produces real transparency only when output_format is png; requesting transparency with a JPEG output silently gives you an opaque background. The output_compression value (0-100) likewise applies only to JPEG and WEBP output and is ignored for PNG.

Trap Combining background: transparent with output_format: jpeg to keep asset sizes small, then wondering why product cut-outs have a white box behind them.

Setting stream to true together with partial_images (1-3) makes the GPT-image models emit progressively refined previews before the final render, which shortens perceived wait in interactive UIs. The quality setting (low, medium, high) is the other latency lever, trading render time against fidelity.

Trap Assuming partial images reduce total generation cost rather than only the time before the first visible frame.

The mask supplied to the image edits operation must be a PNG with exactly the same dimensions as the input image, and its fully transparent pixels (alpha of zero) define the area the model is allowed to change. Every opaque pixel in the mask is preserved untouched in the result.

Trap Painting the region to be edited in solid white or black, which inverts the intended edit area, or supplying a mask that has been resized away from the source dimensions.

Sending only an image and a prompt to the edits operation is a valid prompt-driven modification, but with no mask the model may re-render the entire frame. Only a mask constrains the change, so an unmasked call cannot guarantee that the untouched parts of an approved product shot survive.

Trap Believing that naming one object in the edit prompt is enough to freeze the rest of the composition.

The input_fidelity parameter on an edit request controls how much effort the model spends matching the style and features — especially facial features — of the input images. It is not supported by gpt-image-1-mini, so that model cannot be used where likeness preservation is a requirement.

Trap Choosing gpt-image-1-mini to cut cost on a headshot-retouch workflow that depends on faithful face preservation.

Creating a picture from a prompt uses the image generations endpoint with a JSON body, whereas editing uses the separate image edits endpoint with a multipart form that carries the source image file and, optionally, the mask file. Editing is available only on GPT-image-series deployments.

Trap Trying to pass an existing image into the generations endpoint as a JSON string field.

Creating a video returns a Video object immediately with a status of queued, which then moves through in_progress to completed or failed. The application polls the video by its id and, only once the status is completed, retrieves the finished MP4 from the video content endpoint.

Trap Writing synchronous code that expects the create call to return playable bytes, which instead yields a queued job with progress at 0.

The seconds parameter accepts 4, 8, or 12 and defaults to 4, and size is either portrait 720x1280 (the default) or landscape 1280x720. Requesting a width and height combination the model does not support fails the job with a 400 dimension error rather than snapping to the nearest supported value.

Trap Passing an arbitrary duration such as 30 seconds or a bespoke aspect ratio and expecting the service to round or letterbox it.

Sora 2 supports audio generation in its output videos, and that audio and dialogue are produced natively during the render. There is no operation that lays a music bed or narration track over an already-finished clip, so an audio change means a new generation.

Trap Looking for an 'add soundtrack' API on a completed video instead of re-prompting or remixing.

Independently of any content filter you configure, the Sora 2 API enforces restrictions: it rejects copyrighted characters and copyrighted music, refuses to generate real people including public figures, currently rejects input images containing human faces, and limits output to content suitable for audiences under 18.

Trap Diagnosing a failed promotional-video job as a quota or content-filter-threshold problem when the reference photo simply contains a person's face.

Video generation jobs remain available for roughly a day after creation; once that window passes the job must be re-created to produce the video again. A production pipeline therefore downloads the MP4 and stores it in its own blob storage as soon as the job completes.

Trap Treating the service-side video id as durable long-term storage and linking end users straight to it.

Calling the remix operation with the id of a previously completed generation plus an updated prompt makes Sora 2 maintain the original video's framework, scene transitions, and visual layout while applying only the requested change. The new Video object records its origin in remixed_from_video_id.

Trap Re-submitting the original prompt with an edit appended, which starts a fresh render and drifts the camera move, lighting, and staging every run.

The input_reference parameter accepts one still image (JPEG, PNG, or WEBP) that serves as the visual anchor for the opening frame of a new generation. The source image resolution has to match the requested output size exactly — 720x1280 or 1280x720 — or the request fails.

Trap Reaching for input_reference to adjust an already-approved clip, when it seeds a new render rather than editing an existing one.

Microsoft's guidance is to limit every remix to a single, narrowly described modification, because precise edits retain the greatest fidelity to the source material. Bundling several changes into one remix prompt degrades fidelity and raises the likelihood of visual defects.

Trap Batching a reviewer's full change list into one remix prompt to save a round trip, then getting a clip that no longer matches the approved scene.

The remix operation accepts exactly one source video id and produces exactly one new video. Sora 2 offers no stitching, trimming, or timeline-assembly operation, so joining several generated clips into a continuous piece is an external editing step.

Trap Expecting a multi-clip concatenation call to exist because the model can generate several variants of the same prompt.

The tool requires a gpt-image-1 deployment and a compatible orchestrator model deployment (for example gpt-4o or a gpt-4.1 variant) in the same Foundry project. Every Responses call must also carry the x-ms-oai-image-generation-deployment header naming the image deployment, or the tool call fails.

Trap Deploying only the image model, or naming the orchestrator model in the header, which routes the request to the wrong deployment.

When the tool executes, the response output contains an item whose type is image_generation_call and whose result field holds base64-encoded image data to decode and save. A response containing only a text message item means the request never routed to image generation.

Trap Parsing the assistant's prose reply for an image link and concluding the tool is broken when the picture is actually in a sibling output item.

The agent tool accepts an optional input_image_mask, supplied either as a base64 image_url or as a file_id, which lets the agent edit a specific region of an existing picture mid-conversation. Editing therefore does not require dropping out of the agent and calling the raw image edits endpoint.

Trap Assuming the agent tool is generation-only and building a separate service just to handle user-requested touch-ups.

Compared with calling the image API directly, the Agent Service tool offers two documented advantages: it can stream partial image outputs during generation to improve perceived latency, and it accepts image file ids as inputs in addition to raw image bytes.

Trap Choosing the tool for a batch, non-conversational render pipeline where neither streaming nor file-id inputs buys anything.

Setting response_format to {"type": "json_schema", "json_schema": {..., "strict": true}} forces a Foundry model to emit output that conforms to the JSON Schema you supply, which is what makes generative entity, topic, and field extraction machine-consumable. The older JSON mode (response_format of type json_object) guarantees only that the reply parses as valid JSON; it cannot guarantee the reply carries the fields, names, or types your extraction pipeline expects.

Trap Choosing JSON mode (json_object) for a schema-driven extraction job: it eliminates parse errors but still lets the model rename, omit, or invent fields.

In strict mode every property must appear in the object's required array and every object must set additionalProperties: false. There is no genuinely optional field: you emulate one by giving the property a union type that includes null (for example "type": ["string", "null"]) while still listing it in required. Output key ordering follows the order of the schema you send.

Trap Leaving a property out of required to make it optional — the schema is rejected rather than treated as an optional field.

The supported subset covers string, number, boolean, integer, object, array, enum, and anyOf (the root object cannot be anyOf), plus $defs and recursive references. Validation keywords are not honored: minLength, maxLength, pattern, and format on strings, minimum, maximum, and multipleOf on numbers, and minItems, maxItems, and uniqueItems on arrays. A schema may declare at most 100 object properties across five levels of nesting.

Trap Expecting the schema to enforce value ranges, regex patterns, or array bounds; those checks must run in your own code after parsing.

Structured outputs also apply to tool definitions: setting strict: true on a function forces the generated arguments to match the parameter schema exactly. Structured outputs are not supported together with parallel function calling, so parallel_tool_calls must be set to false whenever strict tool schemas are in use.

Trap Leaving parallel tool calls enabled and assuming the strict argument schema still holds for every emitted call.

Microsoft documents structured outputs as not supported with Azure OpenAI On Your Data (bring-your-own-data) scenarios, with the Assistants and Foundry Agent Service surfaces, and with the gpt-4o-audio-preview and gpt-4o-mini-audio-preview models. An extraction step that must return a guaranteed schema therefore calls chat completions directly rather than routing through an agent run.

Trap Designing an agent tool that relies on strict schema adherence from the agent run itself instead of from a direct model call.

Prebuilt named entity recognition returns spans typed against a preset category and subcategory list (person, location, organization, quantity, and so on) with an offset, length, and confidence score. Entity linking is a separate task that resolves an ambiguous mention to a single knowledge-base entry and returns its reference URL; named entity recognition alone never returns a knowledge-base identifier.

Trap Reaching for named entity recognition when the requirement is to decide whether 'Mars' means the planet or the company — that disambiguation is entity linking.

Key phrase extraction surfaces the main talking points of a document as a flat list of phrases with no entity category, no character offset, and no relevance ranking. Whenever the downstream consumer needs typed, positioned values — populating form fields, driving redaction, or filtering a search facet — prebuilt entity recognition or a schema-driven generative extraction is required instead.

Trap Selecting key phrase extraction for a pipeline that must know both the entity type and where in the document each value appeared.

Custom named entity recognition adds domain-specific entity types beyond the preset list, but only through an authoring project with human-labeled documents, a training run, and an explicit deployment that exposes a runtime prediction endpoint. Prebuilt entity recognition requires none of that — you call the analyze-text runtime immediately with the entity recognition task kind.

Trap Assuming a new domain entity type can be added by configuration or prompt alone, with no labeled data and no deployment step.

A synchronous analyze-text call runs a single task over the submitted documents, while the asynchronous job endpoint accepts several tasks — for example entity recognition, key phrase extraction, and PII detection — over the same document collection in one submission. Batching this way avoids re-sending the corpus once per feature.

Trap Issuing one synchronous call per feature over the same documents and paying the ingestion cost repeatedly.

Prebuilt Language tasks accept a per-document language code, and when none is supplied the analysis defaults to English. Supported entity categories also differ by input language, so a non-English document analyzed without its language code returns degraded results with no error, and categories not enabled by default for that language must be requested explicitly.

Trap Treating missing recall on multilingual text as a model-quality issue rather than an unset language code.

Extractive summarization selects the highest-ranked original sentences from the source and returns each with a rank score plus its start position and length, so every word in the summary is verbatim and traceable to a location in the input. Abstractive summarization writes new, coherent sentences that appear nowhere in the source and returns a contextual input range rather than sentence offsets.

Trap Choosing abstractive summarization when the requirement is that no wording may be invented and each summary line must map back to a source offset.

Conversation summarization accepts structured, speaker-tagged conversational input and is requested per aspect: issue and resolution for contact-center calls, recap for a single-paragraph summary, and chapterTitle together with narrative to segment a long conversation and summarize each segment. Text summarization accepts only a plain text block and has no aspect concept at all.

Trap Flattening a call transcript into plain text for text summarization and still expecting separated issue and resolution output.

Microsoft has published a retirement date for Summarization in Azure Language (31 March 2029) and directs both existing workloads and all new projects to Microsoft Foundry models for natural-language understanding. For a green-field Foundry build the task-specific summarization API is a migration path for legacy code, not the recommended default.

Trap Standardizing a brand-new Foundry solution on the prebuilt Language summarization task because it is the 'purpose-built' service.

Compliance and other domain summarization work carries output requirements the prebuilt task cannot express: mandatory sections, a fixed tone, named clause types to call out, and citations back to the source. A Foundry model steered by a system prompt plus a strict JSON schema delivers that control, while the prebuilt summarization API exposes only summary-length style controls over content it chooses itself.

Trap Assuming a task-optimized summarizer can be configured to always surface a specific clause type; it has no such steering surface.

Summarization is processed as an asynchronous job whose output is available for retrieval for 24 hours from ingestion, after which the results are purged and cannot be fetched again. A production pipeline must persist the returned summaries itself rather than treating the job identifier as durable storage.

Trap Storing only the job identifier and planning to re-read the result days later.

The sentiment task returns confidence scores between 0 and 1 for positive, neutral, and negative for the document as a whole and for each sentence inside it, then assigns the label with the highest score at each level. Because scoring is per sentence as well as per document, a long review whose document label is neutral can still contain individually negative sentences that a document-only reading would miss.

Trap Routing escalations on the document label alone and losing the negative sentences buried inside long feedback.

Opinion mining is an option on the sentiment task, not a separate service, and it performs aspect-based sentiment analysis: it links each expressed sentiment to the concrete target it is about — 'the room', 'the staff', 'battery life' — and returns target and assessment pairs. Base sentiment analysis returns only an overall label per sentence with no attribution to a product or service attribute.

Trap Expecting per-attribute sentiment from the sentiment task without enabling opinion mining.

The Content Safety text model rates Hate, Sexual, Violence, and SelfHarm on the full 0-7 severity scale and can return either that full scale or a trimmed one in which each adjacent pair of levels collapses to a single value (0, 2, 4, 6). Text and multimodal image-with-text scoring offer the eight-level output; classification is multi-label, so one passage can be flagged under more than one category at once.

Trap Building text thresholds on the assumption only four buckets exist, which cannot express the distinction between a level 4 and a level 5 finding.

PII detection evaluates unstructured text for predefined personal and health information categories and returns two things in one response: the entity list with category, offset, length, and confidence, plus a redactedText string in which each detected span is already masked. Called synchronously the feature is stateless — no input is stored in the resource — so a pipeline can log entity metadata and forward only the redacted copy downstream.

Trap Running generic entity recognition and hand-assembling the masked string, which discards the category-aware spans PII detection already produced.

Redaction behavior is chosen with a redaction policy kind: characterMask (the default) replaces the span with a repeated character and preserves the original length and offsets, entityMask substitutes a typed placeholder such as [PERSON_1], noMask returns the response without a redactedText field at all, and syntheticReplacement swaps in realistic but fictitious values. Only length-preserving masking keeps downstream character offsets valid.

Trap Selecting entityMask in a pipeline whose later stages index by character offset into the original document.

The Language sentiment analysis feature assigns the labels positive, neutral, and negative with confidence scores at both document and sentence level, plus a derived mixed label at document level when a document contains both positive and negative sentences, and opinion mining only adds the target an opinion is attached to; none of these labels has any vocabulary for tone or emotion classes such as frustration, urgency, politeness, sarcasm, or escalation risk. Detecting those means defining the taxonomy yourself - a chat model prompted to classify into a constrained enum with a structured output schema, an evaluator or LLM-judge rubric applied to conversations, or a Content Understanding audio analyzer with custom generative fields when the tone is carried by spoken delivery rather than wording.

Trap Equating a strongly negative sentiment score with a specific tone: 'negative' cannot distinguish an angry customer from a disappointed one, or flag an urgent-but-polite escalation, because it is one axis with three labels. Enabling opinion mining does not add emotion labels either — it is aspect-based sentiment returning target and assessment pairs on the same three-label scale — and neither does lowering a confidence threshold.

A trained Custom Translator system is invoked by passing its category ID in the category query parameter of the translate call. allowFallback defaults to true, letting the request quietly fall back to the general system when no custom system exists for that language pair; setting allowFallback=false makes the request return HTTP 400 instead of a non-custom translation, which is how you prove every returned string used the trained terminology.

Trap Leaving allowFallback at its default in a regulated workflow and assuming every translation applied the custom glossary.

The dynamic dictionary supplies a known rendering for a single span inline as <mstrans:dictionary translation="...">phrase. It is case-sensitive, requires the from parameter because autodetection is not allowed with it, and is documented as safe only for proper nouns such as person and product names. Systematic terminology and style control belongs in Custom Translator, which learns those choices from in-context training data.

Trap Injecting dynamic-dictionary markup for an entire glossary instead of training a custom system, and losing source-language autodetection in the process.

By default (profanityAction=NoAction) Translator carries profanity from source to target. Deleted removes the profane words from the output with no replacement, while Marked replaces them — with asterisks by default, or wrapped in tags when profanityMarker=Tag, which leaves the workflow able to detect and post-process the occurrence.

Trap Choosing Deleted in a moderation workflow that still needs to know profanity was present in the source.

Setting textType=html tells Translator the payload is well-formed markup so tags are preserved instead of being translated as prose, and any element carrying class="notranslate" is returned in its source language. That combination is how boilerplate, code samples, brand names, and legal identifiers are excluded from a translated page.

Trap Submitting HTML with the default plain text type, which translates the markup itself and corrupts the document.

Azure Translator document translation has two processes: asynchronous batch, which handles multiple and large documents through an Azure Blob Storage account with separate source and target containers authorized by SAS token or managed identity, while you poll job and per-document status; and synchronous single-file, which accepts one document, needs no storage account, and returns the translated file directly in the response. Both preserve the original layout and formatting and can apply a custom translation model and a glossary, but only the batch path accepts PDFs, where OCR extracts and translates the text of a scanned PDF while retaining the original layout - the synchronous path is limited to .txt, .tsv, .csv, .html, .mhtml, .docx, .pptx, .xlsx, .msg, and .xlf. The text translation API is a different surface entirely - it takes and returns strings, so it cannot round-trip a .docx or .pptx.

Trap Extracting text from each file and looping it through the text /translate endpoint, which discards exactly the layout and formatting document translation exists to preserve. The other inversion is picking the synchronous path for a whole library: synchronous accepts only a single document per request, and it is the asynchronous batch mode - not the synchronous one - that requires the blob source and target containers.

Hybrid search issues a single request that specifies a full-text search string and one or more vectorQueries; the two run in parallel using BM25 for text and HNSW or exhaustive KNN for vectors, and a Reciprocal Rank Fusion (RRF) algorithm merges them into one ranked result set. Hybrid is the documented default choice because keyword matching handles product codes, jargon, dates, and names that pure vector similarity misses.

Trap Assuming you must run two separate calls and blend the scores in application code, or that adding vectors to an index makes keyword search redundant.

Semantic ranking is opt-in per query: the request must set queryType to semantic and reference a semanticConfiguration defined in the index, after which results carry a separate @search.rerankerScore alongside the ordinary @search.score. Because the reranker works from the first-stage candidates, Microsoft advises setting the vector query k to 50 so the ranker has enough input to work with.

Trap Believing relevance improved because a semantic configuration exists on the index, when no query actually requests queryType=semantic and the ranker never fires.

Explicit sort orders override relevance-ranked results, so a hybrid or semantic query that also specifies orderby returns rows in the sorted order rather than by fused relevance. Grounding queries that need the most relevant chunks must omit sorting and instead shape ranking with filters, scoring profiles, or the semantic ranker.

Trap Sorting grounding results by a recency field to make answers 'fresher', which silently destroys the ranking the LLM depends on.

A vector query defaults to approximate nearest-neighbour traversal of the HNSW graph; setting exhaustive to true forces an exhaustive KNN scan that maximizes recall at the cost of latency, which is normally used to establish a ground-truth baseline. The oversampling value widens the candidate set retrieved before rescoring, which recovers recall lost to compressed vector storage.

Trap Turning on exhaustive search across a production index to 'improve quality', instead of using it only to measure how much recall the approximate path is losing.

Azure AI Search enforces document-level access control by storing permission metadata alongside each indexed document and excluding non-matching documents inside the query pipeline, before results are returned. Two patterns exist: the API-agnostic security-filter pattern, where you index a string field holding user or group identities, your application obtains the caller's identity at query time and passes it as a filter expression, and results that do not match the string are trimmed; and the native ACL/RBAC permission-filter pattern (preview), where permission filters are enabled on the index and the caller's Microsoft Entra token is attached to the query with the x-ms-query-source-authorization header so the service compares its user, group, and scope claims to the stored metadata. Either way the trimming happens before the retrieved chunks ever reach the model.

Trap Instructing the model in the system prompt to ignore documents the user is not entitled to see, or redacting the generated answer afterwards — by then the restricted content has already entered the prompt, and the model is not an authorization boundary. Do not invert the two patterns either: security filters are plain string comparison that your application drives, while permission filters are recognized as Microsoft Entra authentication; with permission filters the client app still needs Search Index Data Reader on the index in addition to the per-user token, and permission changes at the source only take effect after the metadata is resynchronized to the index.

In an Azure AI Search index only searchable fields take part in full-text BM25 matching, only filterable fields can appear in a $filter (the mechanism behind permission trimming, metadata scoping, and a fixed filter on an agent's search tool), only sortable fields can be named in orderby, only facetable fields drive facets, and only retrievable fields can be returned to be quoted or cited; exactly one field must be the key and it must be of type Edm.String, while vector fields of type Collection(Edm.Single) are searchable but cannot be filterable, sortable, or facetable. New fields can be added to a live index at any time, but an existing field's data type and most of its attributes are locked in for the lifetime of the index, so turning on filtering or sorting after the fact means adding a new field or dropping and rebuilding the index and re-indexing every document.

Trap Assuming a field can be filtered or sorted because its value is visible in results. retrievable only controls whether the value comes back to the caller; a security-trimming or metadata filter on that field fails until the field is redefined as filterable, which existing documents will not pick up without a rebuild and full re-index.

A scoring profile is a named object defined in the index schema that boosts or suppresses the ranking of matching documents; it is built from text weights over searchable fields plus optional functions - freshness over an Edm.DateTimeOffset field, magnitude over a numeric range, distance from a reference point, and tag for overlap with a caller-supplied tag list - and functions can only be applied to fields attributed as filterable. A query uses one by naming it in the scoringProfile parameter, with scoringParameters supplying the per-request reference point or tag list, or through the index's defaultScoringProfile; an index can hold up to 100 profiles but you can only specify one profile at a time in any given query, and profiles work in keyword, vector, hybrid, and semantically reranked queries yet apply only to nonvector fields. Because a profile only adjusts the score of documents the query already matched, you can add, modify, or delete one with no index rebuild and no effect on indexed documents, which makes it the documented lever for 'prefer the newest revision' or 'boost this customer's documents' in a grounding pipeline.

Trap Reaching for orderby or a $filter to express the same preference: sorting replaces relevance ranking outright and a filter deletes non-matching documents from the result set, whereas a scoring profile re-weights only what the query already found. Two further near-misses: naming two profiles in one request to combine boosts, when only one profile applies per query so the criteria must be combined as multiple functions inside a single profile; and expecting a profile to lift a purely vector match, when profiles apply only to nonvector fields and a function over a field that isn't filterable produces no boost at all.

textSplitMode accepts pages (the default, producing chunks of several sentences bounded by maximumPageLength) or sentences (one sentence per chunk, with sentence boundaries decided by defaultLanguageCode). Page mode adds pageOverlapLength, which must be less than half the maximum page length, and maximumPagesToTake, which defaults to 0 meaning take every chunk.

Trap Choosing sentences mode for a long PDF corpus, which explodes the chunk count into fragments too small to carry answerable context.

For fixed-size chunking Azure AI Search recommends starting at a chunk of about 512 tokens (roughly 2,000 characters) with about 25 percent overlap (roughly 128 tokens, or 500 characters), then tuning by content type. Overlap preserves continuity across chunk boundaries, but an overlap value set too large relative to the actual content length can result in no usable overlap at all.

Trap Pushing overlap toward 50 percent on the theory that more context is always better, which duplicates content, inflates index size, and can break the overlap entirely.

The Text Split skill cuts on character or token counts and cannot cross a document's structural seams intelligently, while the Azure Content Understanding skill performs semantic chunking with Markdown output and produces units that preserve meaning across page boundaries and keep cross-page tables intact. Content Understanding therefore combines extraction and chunking in one skill instead of requiring a separate splitter.

Trap Keeping a character-based splitter on layout-heavy contracts and reports, which slices tables and clauses in half so no retrieved chunk contains a complete answer.

Embedding models impose a hard input limit — text-embedding-3-small accepts 8,191 tokens, roughly 6,000 words — and content past the limit is truncated rather than embedded, silently losing data. Chunking is also worthwhile below the limit when one document covers several subtopics, because a single vector over mixed content represents none of them well.

Trap Treating chunking as purely a relevance-tuning knob and skipping it for documents that 'fit', without checking the model's token ceiling.

Indexing-time vectorization depends on three pieces working together: an indexer that pulls from a supported data source and drives the pipeline, a skillset combining a chunking strategy with an embedding skill such as the AzureOpenAIEmbedding skill, and a search index to receive the chunked, vectorized content. Removing any one of them means embeddings must be generated and pushed by your own code.

Trap Assuming that defining vector fields on the index is enough, when nothing in the pipeline actually calls an embedding model.

Query-time text-to-vector conversion comes from a vectorizer defined in the index schema, assigned to a vector profile which is in turn assigned to the vector field; the vectorizer must match the embedding model used during indexing (AzureOpenAIEmbedding skill pairs with the Azure OpenAI vectorizer, the AML skill with the Foundry model catalog vectorizer, and so on). Mismatched models put query and document vectors in different spaces and relevance collapses.

Trap Upgrading the indexing embedding model to a newer version without re-embedding the corpus or updating the vectorizer, and blaming the drop in quality on chunk size.

Optional index projections let one indexer run populate a granular chunk index alongside a document-level index, both from the same source document. The chat application matches on the fine-grained secondary index and then returns the richer parent document from the primary index, which is the documented pattern for question-answering and chat-style apps over long PDFs.

Trap Flattening everything into one chunk index, which loses the document-level title, date, and summary fields that make a complete answer possible.

Azure AI Search has internal, non-configurable retry policies for throttling errors raised when an Azure OpenAI embedding deployment exhausts its tokens-per-minute allowance, and Microsoft recommends putting the indexer on a schedule so calls dropped despite those retries are picked up on the next run. Token-per-minute limits apply per model per subscription, so sharing one embedding deployment between the ingestion and query workloads makes both throttle.

Trap Tuning an indexer batch size to dodge throttling, when the batching and retry behavior is not exposed for configuration at all.

Microsoft.Skills.Custom.WebApiSkill calls your own endpoint from inside the skillset; the uri must use the HTTPS scheme, and the indexer sends up to batchSize records per call (default 1000) as a top-level values array whose elements each carry a unique recordId and a data object matching the skill's declared inputs. Your service must reply with the same recordIds, a data object matching the declared outputs, and errors and warnings properties that are required but may be null - a non-JSON response, a missing recordId, or a duplicate one means that record is not enriched. Set authResourceId or authIdentity so the search service's managed identity authenticates instead of embedding a function key in the uri.

Trap Assuming your enrichment endpoint can take and return whatever JSON shape it likes, or that a plain http:// endpoint is acceptable. The envelope is fixed and HTTPS-only, and any response record whose recordId was not in the request is discarded. It is also distinct from the Azure Machine Learning skill, which targets a model deployed in an AML online endpoint rather than an arbitrary Web API.

Entity Recognition, Key Phrase Extraction, Language Detection, PII Detection (which can also mask the detected entities), Sentiment, and Text Translation are billable built-in skills that run pretrained Foundry Tools language models over each document inside the skillset, producing values you can then filter, facet, scope, or redact grounding data on. Everything a skill emits lives only in memory, as a node in the enriched-document tree, for the duration of the indexer run: to persist it you must add an outputFieldMappings entry to the indexer whose sourceFieldName is the /document/... path of the skill output and whose targetFieldName is a top-level simple field or collection in the index. A skill that is configured correctly but whose output is never mapped enriches nothing, and the index looks as though the skill never ran.

Trap Reaching for fieldMappings instead of outputFieldMappings. fieldMappings maps verbatim source fields to index fields and can never address a skill output; only outputFieldMappings maps in-memory enrichments, and its target must be a top-level simple field or collection, not a path into a complex type. A second near-miss is assuming the Foundry resource attached to the skillset does the processing - for these skills it is attached for billing only, and Azure AI Search executes them on internal resources.

Both skills crack a document into page text and inline images, but only the Azure Content Understanding skill extracts text location metadata, preserves tables including those spanning pages, produces semantic units that cross page boundaries, and works across PDF, DOCX, XLSX, and PPTX; the Document Extraction skill returns image location metadata for PDFs only and has no table extraction or built-in chunking. Microsoft directs new skillsets to the Content Understanding skill and keeps the older Document Layout skill supported only for existing pipelines.

Trap Reaching for the Document Layout skill on a new build, or picking Document Extraction for a table-heavy corpus because it is cheaper per document.

Image verbalization calls an LLM once per extracted image at ingestion time through the GenAI Prompt skill, storing a concise description such as "five-step HR access workflow that begins with manager approval" next to the surrounding document text. Because the picture is now expressed in language, the pipeline can explain relationships inside a diagram and hand an LLM a caption it can cite verbatim, at the cost of one model call per image.

Trap Expecting verbalization to also support image-as-query lookups; the GenAI Prompt skill supports text-to-vector hybrid queries but not image-to-vector queries.

Only multimodal embedding models expose vectorizers that convert an image into a vector at query time, so a "find things that look like this" experience must be built with the AML skill or the Azure Vision multimodal embeddings skill plus the equivalent vectorizer. Direct multimodal embeddings need no LLM at indexing time but carry no explanation of why two images are related and give the LLM no ready-made text to cite.

Trap Assuming an index built with the GenAI Prompt skill can accept an uploaded photo as the query, because it already 'understands' images.

A multimodal pipeline stores the images it pulls out of source documents in a knowledge store, and the index keeps each image's location so the application can render the original figure next to the cited text. That is what lets a RAG answer show both a textual citation and the diagram snippet it came from.

Trap Trying to return the image bytes from the search index itself rather than storing them and indexing a pointer.

Configuring the tool means supplying project_connection_id and index_name; the optional query_type accepts simple, vector, semantic, vector_simple_hybrid, or vector_semantic_hybrid and defaults to vector_semantic_hybrid, top_k defaults to 5, and any filter you set applies to every query the agent issues against that index.

Trap Setting query_type to semantic on an index that has vector fields but no semantic configuration, or expecting filter to be negotiated per question by the model.

One Azure AI Search tool instance can only target a single index, and for grounded answers to carry usable citations the index needs at least one retrievable text field holding the content plus a retrievable field containing the source URL (optionally a title) so url_citation annotations can link back. When responses come back with no citations at all, the usual cause is agent instructions that never ask for them.

Trap Attaching several indexes to one Azure AI Search tool definition rather than adding one tool per index or moving to a knowledge base.

Setting outputMode to answerSynthesis makes the knowledge base compose an answer with the assigned Azure OpenAI model, shaped by answerInstructions, while the default extractive behaviour returns merged grounding content that your application passes to its own model. retrievalInstructions is the separate lever that tells the planner which knowledge source to prefer for which kind of question.

Trap Confusing retrievalInstructions with answerInstructions; the first steers source selection during planning, the second only shapes the synthesized answer's wording.

The Document Retrieval evaluator takes retrieval_ground_truth (per-document query_relevance_label values) plus retrieved_documents and computes ndcg@3, xdcg@3, fidelity, top1_relevance, top3_max_relevance, holes, and holes_ratio; it needs no model deployment because nothing is judged by an LLM. The Retrieval evaluator is the alternative when no labels exist: it uses an LLM judge on query and context and scores 1 to 5.

Trap Reaching for Groundedness to diagnose bad retrieval — groundedness scores the generated answer against the context it was given, and stays high even when the retriever fetched the wrong documents.

Microsoft documents parameter sweeping as the way to tune RAG retrieval: generate retrieval results for several search algorithms, top-k values, and chunk sizes, then score each run with the retrieval metrics and keep the settings that maximize quality. The holes metric guards the exercise, because a high hole count means the labeled set has gaps and the other numbers cannot be trusted.

Trap Comparing two retrieval configurations on end-to-end answer quality alone, which mixes retrieval regressions with generation noise.