AI-300 Cheat Sheet

The Azure Machine Learning workspace is the top-level resource and the centralized place where teams collaborate and where assets (datasets, models, components, environments, and jobs) are created and stored. Because every other Azure ML task (registering data or models, running jobs, creating endpoints) happens inside a workspace, a shared workspace must be provisioned first.

Trap A Microsoft Foundry project is for building generative AI apps and agents, not the top-level container for classic Azure ML training assets, so it does not replace the workspace.

If you do not supply them, creating a workspace automatically provisions the associated resources it depends on: an Azure Storage account and an Azure Key Vault, plus Application Insights for endpoint monitoring. An Azure Container Registry (ACR) is created lazily, only the first time you build a custom Docker image.

Trap ACR is not always created up front; a workspace can exist without an ACR until a custom environment image is built.

A hub workspace groups multiple project workspaces under shared security settings, connections, and compute; it is the same resource type as a Microsoft Foundry hub, so it can be used from both Azure ML studio and Foundry. A workspace cannot be moved to a different subscription or tenant.

A datastore is a reference to an existing Azure storage service; it does not create or copy the storage. It securely keeps the connection information and credentials in the workspace so scripts connect by a friendly datastore name instead of embedding storage connection strings and secrets in code.

Trap A datastore does not hold the data itself; it stores a pointer plus credentials to the underlying Blob, ADLS, or File storage.

Datastores connect to Azure Blob Storage, Azure Data Lake Storage Gen2, Azure Files, and OneLake (Microsoft Fabric). Each is authenticated with an account key, a SAS token, or a service principal, or created credential-less for identity-based (Microsoft Entra) access. Data Lake Storage Gen1 datastores retired in February 2024.

Trap A credential-less (identity-based) datastore uses the caller's Entra identity at access time rather than a stored account key or SAS token.

Every workspace is created with built-in default datastores; workspaceblobstore is the default that Azure ML uses when you upload data or write job outputs (path form azureml://datastores/workspaceblobstore/paths/...). Create additional datastores with az ml datastore create --file .

Trap Uploaded data and job outputs land in workspaceblobstore by default; you do not have to create a datastore first to start working.

An Azure ML compute cluster (AmlCompute) autoscales between its minimum and maximum node counts, provisioning dedicated nodes per submitted job (so workloads stay isolated) and de-allocating back to the minimum when idle. Setting the minimum node count to 0 means you pay nothing between runs.

Trap A fixed-size cluster (minimum nodes greater than 0) keeps billing while idle; only autoscale-to-zero satisfies both per-job isolation and zero idle cost.

A compute cluster can be set to the low-priority (Spot) VM tier to run training and batch workloads on Azure surplus capacity at a steep discount. Compute instances, Azure Container Instances, and AKS inference do not offer a low-priority tier.

Trap Low-priority/Spot discounting applies to training and batch compute clusters, not to AKS or ACI real-time inference.

A compute instance is a single-node managed dev workstation for one user (with idle shutdown); a compute cluster (AmlCompute) is multi-node autoscaling training and batch compute; managed online endpoints run on Microsoft-managed compute; attached Kubernetes (AKS or Arc) serves high-scale production inference; serverless compute is Microsoft-managed with no cluster to create.

Trap A compute instance is for interactive development, not distributed training or production serving.

Serverless compute is a fully managed, on-demand compute that Azure Machine Learning creates, scales, and tears down for you, so there is no cluster to create or manage. You run a command, sweep, or AutoML job on it simply by omitting the compute parameter (for a pipeline job, set default_compute to "serverless"), and if you also leave out resources it defaults the instance count and picks an instance_type using your quota, cost, performance, and disk size, so it selects an appropriate VM size from your quota to avoid insufficient-quota failures.

Trap Reaching for a compute cluster, Azure Container Instances, AKS, or local compute for a 'run training jobs without provisioning or managing infrastructure' scenario; omitting the compute target to use serverless is the no-management answer, and it consumes the same Azure ML compute quota with no min/max node cluster to size.

Azure ML provides built-in roles: AzureML Data Scientist can perform all workspace actions except creating/deleting compute or modifying the workspace itself; AzureML Compute Operator can create, manage, delete, and access compute; plus Reader, Contributor, and Owner. Combine AzureML Data Scientist and AzureML Compute Operator to let a user run experiments and self-serve compute.

Trap Granting Owner or Contributor over-privileges a data scientist; AzureML Data Scientist is the least-privilege built-in role for running experiments.

The workspace uses a managed identity (system-assigned by default, or user-assigned) to reach its dependent resources such as storage, key vault, and ACR. For workspaces created after 2024-11-19 that identity is granted the Azure AI Administrator role on the resource group, which is narrower (least-privilege) than the older Contributor default.

Trap Prefer scoped Azure RBAC roles and the workspace managed identity over embedding credentials or granting broad Contributor access.

Azure Machine Learning uses MLflow as its native tracking framework. A job run on Azure ML compute automatically has its MLflow tracking URI set to the workspace, so parameters, metrics, and artifacts logged with mlflow.log_param / mlflow.log_metric are stored in the workspace and shown on the job's Metrics tab.

Trap Assuming you must stand up and configure a separate MLflow tracking server; the Azure ML workspace already IS the tracking backend.

Calling mlflow.autolog() before training enables automatic logging of parameters, metrics, and the trained model artifact for supported frameworks (scikit-learn, PyTorch, TensorFlow, XGBoost) without writing explicit log calls for each value.

Trap Believing you must hand-log every metric; autolog captures the standard framework metrics automatically.

Code running outside Azure ML managed compute (for example on a local machine) does not inherit the workspace tracking URI. You point MLflow at the workspace with mlflow.set_tracking_uri() so those runs are also recorded in the workspace.

Automated ML explores algorithms and hyperparameters in parallel for classification, regression, time-series forecasting, computer vision, and NLP tasks. You set the task type and a primary_metric (for example accuracy, AUC_weighted, or normalized_root_mean_squared_error) that AutoML optimizes to pick the best model.

Trap Thinking AutoML only handles classification; it also covers regression, forecasting, vision, and NLP.

In an AutoML job, featurization (scaling and normalization, missing-value handling, text-to-numeric encoding) runs automatically by default, and those steps become part of the produced model so they are reapplied at inference. You can set featurization to custom to override the defaults.

Trap Assuming you must manually engineer and encode all features first; AutoML featurizes automatically unless you customize it.

An AutoML job ends when it meets the exit criteria you configure, such as experiment_timeout_minutes, max_trials (iterations), or a metric score threshold. Voting and stacking ensemble models are enabled by default and usually appear as the final trials.

Trap Expecting AutoML to run indefinitely; it halts at the timeout, trial cap, or score threshold you set.

Interactive experimentation and notebooks run on an Azure Machine Learning compute instance, a managed single-node cloud workstation (personal to one user) with Jupyter, JupyterLab, and VS Code. You start and stop it, and you can configure idle shutdown to avoid paying while it sits unused.

Trap Trying to open notebooks directly on a compute cluster; clusters run submitted jobs, not the interactive notebook environment.

A compute instance is single-node and stays running while started, for interactive authoring; a compute cluster (AmlCompute) is multi-node, autoscales dedicated nodes per submitted job, and can scale to 0 nodes when idle. Use the instance for notebooks and the cluster for training jobs and sweeps.

Trap Picking a cluster for interactive dev, or an instance for a distributed multi-node training job.

A compute instance is a single dedicated VM that keeps billing compute hours the whole time it is running and does not scale to zero like a compute cluster, so its main cost control is enabling idle shutdown (set idle_time_before_shutdown_minutes; the inactivity window must be at least 15 minutes and at most 3 days) and attaching start/stop schedules (up to four per instance). It only counts as inactive when there are no running kernels, terminals, jobs, VS Code connections, or custom apps.

Trap Expecting a compute instance to auto-scale to zero when idle like a compute cluster; it keeps billing until idle shutdown or a schedule stops it.

A sweep job automates hyperparameter tuning by exploring a defined search space with a sampling algorithm: random, grid, or bayesian sampling. You also set a primary_metric plus its goal (maximize or minimize) that the sweep optimizes across trials.

Trap Choosing grid sampling for a large or continuous search space, where random or bayesian sampling is far more efficient.

To cut cost, a sweep job can apply an early-termination policy that cancels underperforming trials early: bandit (slack_factor or slack_amount off the best run), median_stopping, or truncation_selection. Policies use evaluation_interval and delay_evaluation to control when they start acting.

Trap Confusing bandit's slack (distance from the best run) with truncation selection's fixed percentage of worst trials.

Bayesian sampling chooses each new set of hyperparameter values from the results of prior trials and does NOT support early-termination policies (set the policy to None). Random and grid sampling do support early termination.

Trap Pairing bayesian sampling with a bandit or median-stopping policy; only random and grid sampling allow early termination.

You run a training script as a command job that specifies the command (for example python train.py), the code folder, an environment (a curated or custom Docker/conda image), and a compute target. Submit it with az ml job create -f job.yml.

Trap Confusing environment with compute; a command job needs both an environment (dependencies) and a compute target (hardware).

A command job's inputs mount or download data assets and URIs into the script, and its outputs write results back to workspace-managed storage. You reference them in the command with ${{inputs.}} and ${{outputs.}} placeholders instead of hardcoding storage paths.

Trap Hardcoding blob/datastore paths in the script rather than using typed inputs/outputs, which breaks portability and lineage.

To train large or deep-learning models across multiple nodes, a command job sets a distribution with type pytorch, tensorflow, or mpi (for example Horovod), together with instance_count (number of nodes) and process_count_per_instance (processes or GPUs per node).

Trap Setting instance_count greater than 1 without a distribution config; the workers then run independently instead of coordinating.

process_count_per_instance is typically set to the number of GPUs on each node, and scaling out with a larger instance_count requires a compute cluster whose VM SKU provides those GPUs.

Set distribution.type (or use the typed PyTorchDistribution, TensorFlowDistribution, or MpiDistribution object) to match the framework the script uses: PyTorch for torch.distributed/DistributedDataParallel scripts, TensorFlow for tf.distribute (using worker_count, plus parameter_server_count for the parameter-server strategy), and mpi for MPI or Horovod. DeepSpeed is not a separate distribution type; you launch it through either the PyTorch or the MPI distribution.

Trap Selecting mpi for a plain PyTorch DistributedDataParallel script, which needs the PyTorch type, or treating DeepSpeed as its own distribution.type instead of launching it through the PyTorch or MPI distribution.

A training pipeline is a pipeline job made of components, where each component is a self-contained step with typed inputs/outputs, an environment, and code. Components are registered, versioned assets that can be reused across pipelines and shared, and each step can run on its own compute.

Trap Treating a standalone command job as reusable; the component is the registerable, shareable unit that pipeline steps consume.

Azure ML can reuse a prior step's cached output when that component's code and inputs are unchanged, skipping recomputation to save time and cost on subsequent pipeline runs.

Because each run logs metrics to the workspace via MLflow, you compare model performance across jobs by selecting multiple runs in an experiment and viewing their logged metrics side by side in the studio (or querying them with the MLflow API) to identify the best model.

Trap Comparing on training loss instead of the agreed primary/validation metric used to select the production candidate.

AutoML and sweep jobs already rank trials by the configured primary_metric and surface the single best trial, so choosing a production candidate means taking that best trial's model rather than manually re-ranking every run.

Trap Re-picking a model by a different metric than the primary_metric the job optimized, which can select a worse model.

Microsoft Foundry offers two project types. A Foundry project is the default, generally available choice and the only one that exposes the Foundry API and Foundry Agent Service; a hub-based project is required only when you need Azure Machine Learning compatibility, managed compute, or open-source model hosting and fine-tuning.

Trap Choosing a hub-based project 'to use prompt flow' — prompt flow is on a retirement path (support ends 2027-04-20), so it is not a reason to pick a hub project for new work.

A Microsoft Foundry resource is the top-level Azure resource you create first; projects are child scopes inside it that group models, agents, connections, and deployments. The Azure Machine Learning workspace remains the top-level resource for classic ML assets, not for generative AI work.

Trap Treating the Azure Machine Learning workspace as the container for Foundry generative-AI projects.

A Foundry connection is a named, project-scoped object that stores the endpoint and credentials for an external dependency such as Azure Storage, Azure AI Search, or Application Insights, so components and agents reuse the connection instead of embedding secrets in each definition.

Trap Hard-coding a resource key into each agent or prompt rather than referencing one shared connection.

A Foundry connection can authenticate to its target resource with a Microsoft Entra identity (the resource's managed identity) rather than an API key, keeping secrets out of the connection object and letting Azure RBAC govern who can use it.

A Foundry resource can use a system-assigned managed identity (created and deleted with the resource, one-to-one) or a user-assigned managed identity (a standalone Azure resource you can attach to several resources and manage independently) to access dependencies such as Storage or Key Vault without stored credentials.

Trap Using an account key or SAS token where a managed identity is required for keyless, credential-free access.

For production Foundry access, Microsoft Entra ID token authentication with a managed identity is preferred over API keys; you can disable local (key) authentication so every data-plane call is authorized through Entra ID and RBAC rather than a shared secret.

Trap Assuming API keys are as safe as Entra ID — keys cannot be scoped by RBAC and are harder to rotate or revoke per user.

The built-in Foundry User role (formerly Azure AI User) grants the least-privilege access needed to work inside a project and call its deployed models; assign it to team members who build and test rather than granting a broad Owner or Contributor role.

Trap Granting Owner or Contributor for everyday build-and-test work, over-provisioning access beyond what Foundry User already allows.

The Foundry Project Manager role (formerly Azure AI Project Manager) is the elevated developer role: it is the minimum role that can publish agents (Foundry Account Owner cannot publish) and it can conditionally assign the Foundry User role to others. Microsoft pages currently disagree on whether it can also create projects or manage model deployments - the canonical rbac-foundry matrix (2026-07) marks both as NOT granted - so questions must not anchor on those claims; Foundry Account Owner and Foundry Owner cover full resource management and model deployment.

Trap Assuming Project Manager can create projects or deploy models; the canonical RBAC matrix denies both, and its distinctive documented power is agent publishing plus conditional Foundry User assignment.

A Foundry managed virtual network supports two isolation modes: Allow internet outbound (agents may reach public endpoints) and Allow only approved outbound (outbound is limited to approved private endpoints and FQDN rules). Only Allow only approved outbound enables automatic data-exfiltration protection, and adding an outbound FQDN rule automatically provisions an Azure Firewall.

Trap Believing Allow internet outbound provides data-exfiltration protection — only Allow only approved outbound does.

To keep traffic off the public internet, create a private endpoint for the Foundry resource and set public network access to Disabled, so the resource is reachable only over a private IP inside your virtual network. Managed private endpoints created by the managed VNet are fully Microsoft-managed and do not expose a customer-visible network interface (NIC).

Trap Using a service endpoint instead of a private endpoint, or leaving public network access enabled after adding the private endpoint.

Define Foundry resources, projects, and model deployments as Bicep templates and deploy them with the Azure CLI (for example az deployment group create) so environments are versioned, reviewable, and reproducible. A managed-network Foundry resource currently has no portal creation UI and must be deployed via Bicep or Terraform, or with az rest / Azure CLI.

Trap Relying on portal click-ops, which is not source-controlled or repeatable across dev/test/prod environments.

The Azure AI Evaluation SDK evaluate() API takes a test dataset as a file path, not as an in-memory list or a plain JSON array. JSON Lines (JSONL, one JSON object per line) is the standard format used throughout the documentation and tooling, and the current API reference also accepts CSV; each record supplies the fields the chosen evaluators need.

Trap Assuming evaluate() ingests an in-memory Python list or a plain JSON array file; it requires a dataset file path (JSONL standard).

An evaluation record can contain up to four elements: query (the input sent to the app), response (the app output), context (the grounding/source documents), and ground_truth (a human reference answer). Which are required depends on the evaluator; groundedness and relevance need context, while similarity and F1 need ground_truth.

Trap Omitting context for a groundedness run or ground_truth for a similarity run, so the evaluator scores against empty input.

When dataset column names differ from the keywords an evaluator expects, you map them with column_mapping inside evaluator_config, using ${data.} to reference a dataset field and ${outputs.} to reference a value produced by a target; a default mapping applies to all evaluators. Comprehensive evaluation is achieved by mapping every required field for each evaluator.

Trap Assuming columns must literally be named query/response; mismatched names without a column_mapping feed the evaluator empty values.

The azure.ai.evaluation.simulator.Simulator class produces non-adversarial synthetic query/response conversations from a text blob or search index, giving consistent, repeatable evaluation inputs when no production data exists. It is the tool for building a controlled dataset to compare prompt variants without live user traffic.

Trap Reaching for content filters or a blocklist to create evaluation inputs; those moderate content, they do not generate a controlled test dataset.

AdversarialSimulator, driven by an AdversarialScenario enum, generates adversarial (red-team) datasets using a service-hosted model with safety behaviors turned off; it requires a Foundry project and runs only in East US 2, France Central, UK South, and Sweden Central. DirectAttackSimulator produces user-prompt (UPIA) jailbreak data and IndirectAttackSimulator produces cross-domain (XPIA) jailbreak data.

Trap Expecting adversarial simulation to run in any region; the hosted safety service is limited to four regions and needs azure_ai_project.

Groundedness, relevance, coherence, and fluency are the AI-assisted quality evaluators. Groundedness and relevance are RAG-oriented (groundedness checks the response against supplied context; relevance checks the response against the query), while coherence and fluency are general-purpose language-quality metrics — coherence scores how well the response hangs together as an answer to the query (requires query + response), whereas only fluency judges the response text alone.

Trap Selecting groundedness when no context/grounding documents are provided; without context there is nothing to measure groundedness against.

AI-assisted quality evaluators use an LLM-as-a-judge, so you must supply a GPT model deployment in model_config (AzureOpenAIModelConfiguration). They score on a 1-to-5 Likert scale where higher is better, apply a default pass threshold of 3, and return a reason for the score.

Trap Configuring a model_config for the risk and safety evaluators; only the AI-assisted quality/RAG evaluators use a GPT judge, safety evaluators do not.

GroundednessProEvaluator (preview) detects ungrounded or hallucinated claims using the hosted Foundry evaluation service, so it takes azure_ai_project rather than a GPT model_config and returns a pass/fail label with a reason instead of a 1-to-5 score.

The textual-similarity evaluators (F1Score, BleuScore, GleuScore, RougeScore, MeteorScore) are built-in, generally available metrics that score a response purely by math-based token or n-gram overlap against a supplied ground_truth, so they take only ground_truth plus response and require no GPT model deployment or LLM-as-a-judge (each returns a 0-1 float against a default 0.5 threshold, except RougeScore, which returns separate precision, recall, and F1 scores). The selection rule is deterministic: when a fixed reference answer exists and exact overlap is the criterion (machine translation, fixed-answer retrieval, NLP benchmarking) choose a textual-similarity evaluator; when there is no ground truth to compare against, choose the AI-assisted quality metrics (groundedness, relevance, coherence, fluency) instead.

Trap Reaching for an AI-assisted quality evaluator, or supplying a model_config, when a fixed ground_truth answer already exists — the textual-similarity evaluators need no LLM-judge deployment; conversely, they cannot run at all without a ground_truth.

The risk and safety evaluators (ViolenceEvaluator, SexualEvaluator, SelfHarmEvaluator, HateUnfairnessEvaluator) are executed by the hosted Microsoft Foundry evaluation service, so they are instantiated with your Foundry project information (azure_ai_project) and a credential, not with a GPT model_config or a deployment_name.

Trap Passing a GPT model_config to a safety evaluator; harm detection is done by Microsoft-hosted safety models reached through your Foundry project.

Content safety evaluators return a 0-to-7 severity score bucketed as Very low (0-1), Low (2-3), Medium (4-5), and High (6-7). Given a default threshold of 3, a score at or below the threshold passes and a higher score fails, with a reason explaining the level.

Trap Reading the safety scale like a quality score; here a HIGHER number means MORE harmful, the opposite of the 1-to-5 quality metrics.

Over a dataset the service reports a defect rate, the percentage of responses whose severity exceeds the threshold, as the aggregate safety signal. ContentSafetyEvaluator is a composite evaluator that runs violence, sexual, self-harm, and hate/unfairness together for a single combined output.

Trap Reporting a single worst-case score instead of the defect rate, which is the intended dataset-level harmful-content metric.

The hosted safety evaluation service (and adversarial simulation) is available only in select regions, so the Foundry project must be created in a supported region such as East US 2, France Central, UK South, or Sweden Central for these evaluators to run.

Beyond the four content-harm evaluators, the built-in risk and safety set includes IndirectAttack (XPIA / cross-domain prompt injection), which returns a boolean pass/fail verdict where fail means the response fell for a jailbreak instruction injected into retrieved context or a source document, and ProtectedMaterial, which flags text under copyright (song lyrics, recipes, articles) using the Azure AI Content Safety Protected Material for Text service. Both output pass/fail rather than the 0-to-7 severity score used for hate/unfairness, sexual, violence, and self-harm.

Trap Assuming every risk and safety evaluator emits a 0-to-7 severity score, or that IndirectAttack inspects the user's direct prompt; it is a boolean verdict and targets injections hidden in retrieved or grounding content, not the immediate user turn.

The AI Red Teaming Agent (preview) is Microsoft Foundry's automated adversarial-scanning tool, built on the open-source PyRIT framework, that probes a model, application, or agent endpoint with simulated attacks (applying PyRIT attack strategies such as Base64, Flip, or Crescendo across easy, moderate, and difficult complexity), scores each attack-response pair with the hosted Risk and Safety Evaluators, and reduces the run to an Attack Success Rate (ASR), the percentage of attacks that successfully elicit an undesirable response. It emits a risk-category scorecard broken down by harm category and attack complexity to drive a pre-deployment go/no-go decision, and is distinct from the AdversarialSimulator, which only generates a red-team dataset, and from the hosted per-response safety evaluators, which score a single response at a time rather than run an end-to-end attack campaign.

Trap Confusing the AI Red Teaming Agent with the AdversarialSimulator (which only builds the adversarial dataset) or with the hosted content-safety evaluators (which score the harm severity of one response); the AI Red Teaming Agent runs the whole scan, scoring attack-response pairs into the ASR scorecard used for the deployment go/no-go.

The evaluate() API runs multiple built-in and custom evaluators over one dataset in a single pass, returning aggregate metrics plus per-row scores, and can log results to a Foundry project (surfaced via result.studio_url). Composite evaluators such as QAEvaluator bundle several quality metrics at once.

Trap Running each evaluator by hand row by row; evaluate() batches all evaluators and produces the aggregated report.

When built-in metrics do not fit, you author custom evaluators either as a code-based Python callable/class (for example a deterministic score like answer length) or as a prompt-based Prompty file that instructs a judge model, then pass them to evaluate() alongside the built-in evaluators.

Trap Assuming only built-in metrics exist; domain-specific criteria are handled by code-based or Prompty custom evaluators.

Cloud (remote) evaluation runs let you integrate evaluation into CI/CD pipelines such as GitHub Actions as automated quality gates, failing a build or blocking a release when quality or safety scores fall below thresholds, which operationalizes evaluation rather than leaving it a manual step.

Trap Treating evaluation as a one-off local check instead of a scheduled or CI/CD-gated automated workflow.

The Azure AI Evaluation SDK is the successor to the retired Evaluate-with-prompt-flow workflow; new automated evaluation should be built on azure-ai-evaluation plus Foundry evaluations, not on prompt flow, whose feature development ended on 2026-04-20 and support ends 2027-04-20.

Trap Selecting prompt flow to author or run evaluation workflows; it is on the retirement path and the Azure AI Evaluation SDK is its replacement.

Agentic evaluators judge agent behavior beyond final text: IntentResolutionEvaluator checks whether the agent correctly understood the user's intent, ToolCallAccuracyEvaluator checks whether the right tools were called with correct parameters, and TaskAdherenceEvaluator checks whether the agent followed its instructions and stayed on task.

Trap Evaluating an agent with only groundedness/relevance; agent quality also depends on intent resolution, tool-call accuracy, and task adherence.

Agent evaluators read the agent message schema, including tool_calls, so evaluating an agent means capturing its tool-invocation trace as input, not only the final assistant message. This is why agent evaluation is tightly coupled to tracing.

Trap Feeding only query/response to an agent evaluator; without the tool_calls it cannot assess tool-call accuracy.

In Azure AI Search integrated vectorization, a skillset runs the Text Split skill to break documents into chunks and then an embedding skill (for example the Azure OpenAI Embedding skill) vectorizes each chunk. textSplitMode is set to "pages" (default fixed-size chunks) or "sentences", and maximumPageLength plus pageOverlapLength control chunk size and overlap. Chunking is required because embedding models cap input length (text-embedding-3 models accept at most 8,191 tokens per call).

Trap Trying to embed whole documents directly: inputs over the model's 8,191-token limit are rejected, so documents must be chunked first.

Smaller chunks return more precise matches but can lose surrounding context, while larger chunks preserve context but dilute relevance and cost more to embed. Microsoft recommends starting near a 512-token chunk size with about 25 percent overlap (roughly 128 tokens); pageOverlapLength repeats text at chunk boundaries so a relevant passage is not split across two chunks.

Trap Setting overlap to 0 to avoid duplicate text: zero overlap can cut a relevant passage in half across two chunks and lower recall.

A vector field's algorithm sets the retrieval strategy: HNSW performs fast approximate nearest-neighbor search (tunable with m, efConstruction, and efSearch) and scales to large indexes, while exhaustive KNN scans every vector for exact nearest neighbors, which is more accurate but slower and costlier. An HNSW field can be forced to run exact search on a per-query basis with "exhaustive": true.

Trap Choosing exhaustive KNN for a large production index because it is 'exact': it does not scale; HNSW approximate search is the default for large indexes.

A vector query's k (k-nearest-neighbors) controls how many documents the retrieval step returns; increasing k raises recall by giving the reranker or LLM more candidates, at the cost of latency and noise. For hybrid queries that feed semantic ranking, Microsoft recommends setting k to 50 so the reranker receives a full candidate set.

Trap Confusing k with the embedding dimension count: k is the number of returned neighbors, unrelated to the 1,536/3,072 vector dimensions.

The vector field's similarity metric must match how the embedding model was trained: cosine is the default and correct choice for Azure OpenAI text-embedding models, with dotProduct and euclidean as the alternatives. On pure single-vector queries, a per-query similarity threshold (preview) can additionally drop low-scoring matches so weakly related chunks never reach the generator; hybrid/RRF queries are not eligible because their fused score ranges are too small and volatile for a cutoff.

Trap Switching the metric to euclidean to 'improve' OpenAI embedding relevance: OpenAI embeddings are tuned for cosine, so a mismatched metric degrades results.

A pure vector query always returns its k nearest neighbors even when they are barely similar, because the nearest-neighbor search just finds the closest vectors regardless of score. To drop weak, off-topic matches, set a per-query threshold (kind vectorSimilarity, in preview) that excludes any result scoring below the minimum even if fewer than k results remain; this is a precision lever distinct from top-k and from the similarity metric. Apply it only to single-vector queries, not to hybrid/RRF queries, whose fused rank ranges are too small and volatile for a score cutoff.

Trap Expecting a smaller k or a different similarity metric to filter out irrelevant chunks; k always returns k results, so only a similarity threshold removes weak matches, and the threshold does not apply to hybrid/RRF queries.

text-embedding-3-large outputs 3,072-dimensional vectors by default and text-embedding-3-small outputs 1,536; both accept up to 8,191 input tokens. A larger, higher-dimensional model captures more semantic nuance to improve domain accuracy, but it increases vector storage and query cost.

Trap Believing text-embedding-ada-002 produces 3,072 dimensions: ada-002 is fixed at 1,536, and only the v3 models reach 3,072.

text-embedding-3 models support a dimensions parameter that truncates the output vector (for example from 3,072 down to 1,536 or 256) using Matryoshka Representation Learning, trading a small accuracy loss for lower storage and faster search. The same dimensions value must be used at indexing time and query time or the vectors are not comparable.

Trap Reducing dimensions only on the query side: index and query embeddings must share the same dimension count or the similarity comparison breaks.

Azure OpenAI text-embedding-3 models cannot be fine-tuned; fine-tuning applies to chat/completion models only. To raise domain-specific retrieval accuracy you select the right embedding model and dimension count, improve chunking, or add hybrid and semantic ranking, rather than training the embedding model itself.

Trap Fine-tuning text-embedding-3-large on domain data: Azure OpenAI offers no embedding-model fine-tuning; model and dimension selection plus retrieval tuning are the levers.

Hybrid search runs a vector (HNSW) query and a BM25 keyword query in parallel over one index and merges them with Reciprocal Rank Fusion (RRF), which combines results by rank position rather than by raw score because BM25 and cosine scores are on incompatible scales. Hybrid retrieval usually beats either method alone, especially for queries that mix exact keywords with semantic intent.

Trap Thinking RRF averages the BM25 and cosine scores: it fuses by rank position, and the raw scores are never added or averaged.

Setting queryType=semantic adds an L2 reranking pass that rescores the top 50 RRF/BM25 results with Microsoft's language models and returns @search.rerankerScore from 4.0 (highly relevant) down to 0.0 (irrelevant). It reranks only the existing top 50 candidates and never re-queries the whole corpus, so first-stage recall still depends on the vector/keyword query.

Trap Expecting semantic ranker to surface documents the first-stage query missed: it only reorders the top 50 already retrieved, so raise recall in the retrieval query, not the reranker.

Beyond L2 reranking, semantic ranker returns verbatim semantic captions (usually under 200 words) and optional semantic answers extracted from indexed text, and with query rewrite enabled it expands a query into up to 10 semantically similar variants that each run and are rescored. Captions and answers are always verbatim from the index; no generative model composes new text.

Trap Assuming semantic answers are generated by an LLM: they are extracted verbatim from your index, not synthesized.

The Azure AI Evaluation SDK and Foundry evaluations provide built-in RAG evaluators that each return a 1-to-5 score: Retrieval rates how well the retrieved chunks rank the relevant context, Groundedness rates whether the response is supported by that context without fabrication, and Relevance rates whether the response addresses the user's query. Combine them, often with content-safety evaluators, for a full quality picture.

Trap Treating Groundedness and Relevance as the same metric: Groundedness checks support by the retrieved context (hallucination), while Relevance checks that the answer addresses the question.

To improve a RAG system, run A/B tests that change one variable at a time (chunk size, overlap, k, embedding model or dimensions, hybrid on/off, semantic ranker on/off) and score each variant on the same evaluation dataset, then keep the configuration with the higher aggregate evaluator scores. A consistent, controlled test set, synthetic if production data is unavailable, is what makes the comparison valid.

Trap Comparing two configurations on different query sets: only a shared, fixed evaluation dataset yields a valid A/B comparison.

The evaluators show where to fix a RAG system: a low Retrieval score points to indexing (chunking, embedding model or dimensions, k, or hybrid search), while a high Retrieval score with low Groundedness or Relevance points to the generation step (prompt or model), because the right context was retrieved but the answer did not use it. Fix retrieval first when Retrieval is the weak metric.

Trap Rewriting the prompt when Retrieval is low: if the right chunks were never retrieved, no prompt change can ground the answer, so fix retrieval first.