Identify AI model components and configurations

Feed a model the fragment "I heard a dog" and it continues with "bark", because "heard" and "dog" are the strongest clues about what comes next. If you have ever typed a question into a chat assistant and read the reply, you have already met that behavior from the outside; this page is the machinery underneath it, and the Python listings later ask nothing of you beyond reading a dictionary literal. The other pages in this domain ask which responsible-AI principle governs a solution and which workload family a scenario belongs to; this one stays inside the model. Work through it and you can take a requirement and settle the three decisions an exam question sets beside it: which model class, which access path, and which setting.

Tokens are the units a model actually reads

A model does not work in words. A tokenizer first breaks the text into tokens: whole words, sub-words such as the "un" in "unbelievable" and "unlikely", punctuation, and other commonly used sequences of characters. Each distinct token gets a unique integer identifier, and a current model's vocabulary runs to hundreds of thousands of them, built from large volumes of training text (tokenization in large language models^[1]). Every limit and every per-token charge later on this page is counted in tokens, never in words, so a 400-word prompt and a 400-token prompt are different quantities.

Embeddings give tokens a position in a meaning space

Each token starts as a vector, an array of numeric values. A transformer model converts those starting vectors into ones that carry meaning through two blocks: an encoder block that creates the embeddings by applying a technique called attention, and a decoder block that uses those embeddings to determine the next most probable token in a sequence. Attention looks at each token in the context of the tokens around it and weights those neighbors by how much they influence it, which is why "heard" and "dog" count for more than "I" or "a" when the model considers what follows. The result is an embedding, a vector whose direction encodes meaning: tokens used in similar contexts point in similar directions, and that closeness is measured as cosine similarity. Keep that definition close, because embeddings return later as a model class you can call directly.

Generation is a loop, not a lookup

The decoder predicts one token, that token is appended to the sequence, and the process repeats with the longer sequence as its context until the decoder predicts that the sequence has ended. Nothing in that loop retrieves a stored answer keyed to your prompt, which is the most common misreading of these models: the response is assembled one token at a time out of probabilities, so the same prompt can produce different wording on different runs. The figure below traces a single pass around that loop.

Training happened once; a request only applies it

The probabilities come from parameters fixed during training, when the model was exposed to large volumes of text and iteratively learned which tokens occur together and in similar contexts. At request time, which is called inference, the model applies those learned patterns and changes nothing about itself. A deployed model therefore does not retrain on user prompts, and it does not check a generated claim against a factual database as part of answering, which is why a wrong answer is a grounding problem or a model-choice problem rather than a settings problem.

The short version: tokens in, one probable token out, appended, repeat. Everything else on this page is either which model runs that loop or how much room and randomness it gets.

This section covers the shape of the request itself, because a chat model does not accept one unstructured string. It accepts an ordered transcript of messages, each labeled with a role, and returns a model-generated message; that format is what supports multi-turn conversation and it also works well for tasks that are not conversations at all (work with chat completion models^[2]).

Three roles, three jobs

• System. The system message sits at the beginning of the array and provides the initial instructions to the model: a brief description of the assistant, personality traits, rules you want followed, and data the model needs, such as relevant questions from an FAQ. It is optional, but Microsoft's guidance is to include at least a basic one for the best results. On reasoning models a developer message is functionally the same as a system message, and you should not use both in one request (reasoning models^[3]).
• User. The request itself. End the transcript with a user message, because that is what signals it is the assistant's turn to respond.
• Assistant. What the model said, or what you want a good answer to look like. Seeding assistant messages alongside user messages is how few-shot examples are supplied.

Application-wide rules belong in the system message, not the assistant role. It is an easy slip to make, because assistant content does influence later turns, but an instruction parked there reads as one more example turn rather than as a standing constraint on behavior.

Listing 1: a transcript carrying one seeded example exchange

The listing below puts the three roles just described into a single transcript. It is complete as shown; nothing has been cut. The role key is the structure the model reads, and the content key holds the text for that turn.

messages = [
    # Standing behavior and constraints go here, not in an assistant turn.
    {"role": "system", "content": "Assistant helps users answer tax-related questions."},
    # A user/assistant pair used as a few-shot example of the answer style.
    {"role": "user", "content": "When do I need to file my taxes by?"},
    {"role": "assistant", "content": "Check the current individual filing deadline on the IRS website."},
    # The transcript ends on a user turn, which is what prompts a response.
    {"role": "user", "content": "How can I check the status of my tax refund?"},
]

The model keeps no memory between calls

Every request sends the running transcript again. Because the model has no memory, an application that stops resending earlier turns loses the context of the previous questions and answers, so conversation state is the caller's job rather than the model's. The same message format also covers non-chat work: an entity-extraction prompt is a system message describing the JSON shape to return plus a user message holding the raw text.

Two things to take from this: the roles are a contract about what kind of content each message is, and the transcript you resend on every turn is the thing that spends the budget described next.

A model's context window is the token budget for a single request, and the messages you send and the tokens the model generates both spend it. That one sentence settles most exam questions in this area, so start there rather than with the arithmetic.

What draws on the budget

Three kinds of token draw on the same window:

Token kind	Where it comes from	Visible to the user
Input tokens	The transcript you send, including the system message and every earlier turn	Yes
Output tokens	The assistant message the model generates	Yes
Reasoning tokens	Hidden intermediate work on reasoning models, reported under completion_tokens_details	No

Reasoning tokens are the ones that surprise people. They are not returned as part of the message content, but they are used by the model to help produce the final answer, and a higher reasoning_effort setting generally produces more of them (reasoning models^[3]). A request can therefore consume far more generated tokens than the visible answer suggests.

Because everything shares one window, the published input maximum and output maximum for a model are not two independent allowances you can each spend in full. Filling the input side leaves less headroom for generation, and the operative rule from Microsoft's chat-completions guidance is that the combined count of your messages plus the requested output tokens must stay within the model's limit (managing conversations^[2]). Per-model limits are published on the models reference and change as models are added, so treat the specific numbers as a lookup rather than something to memorize. The figure below puts a short prompt beside a long one inside the same window.

What happens at the boundary is described more than one way

Microsoft's documentation does not give a single answer here, so do not learn one. The chat-completions guidance states that a request whose combined count exceeds the model's limit fails, and the same page offers the Responses API as a way to have truncation and management of the conversation history handled for you. Its own sample instead trims the transcript client-side, deleting the oldest non-system messages until the request fits, and notes that this gradually degrades quality as the model loses the earlier parts of the conversation. All three are documented; what they agree on is that the budget is something an application manages deliberately, not something that manages itself by default.

Practical consequences

• Track the token count of a growing conversation rather than assuming it stays small.
• Preserve the system message when trimming, since it carries the standing instructions.
• Alternatively, cap a conversation at a number of turns and start a fresh one, which restores the full budget instead of degrading it.
• Expect a response that stops because it hit a length ceiling to be reported as such, distinct from a response that stopped because the model finished.

The conclusion for the exam: one window, several things drawing on it, and no automatic rescue you can name with confidence.

You have a requirement in hand, say ranking support articles by meaning rather than by keyword, and a catalog of models in front of you. The catalog will not narrow itself, so work through two questions in this order before you look at any model name: what must the solution hand back, and what will the prompt hand in. Output settles the model class; what kind of content the prompt hands in, its modality, settles whether one kind of input is enough. The workloads page runs a similar test one level up, reading a scenario for the workload family it belongs to; the question here is the narrower one of which model inside that family returns the required shape.

The model classes you are expected to tell apart

The comparison table at the top of this page sets out the four classes AI-901 leans on hardest. Microsoft's catalog guidance divides the same space more finely, grouping models by the task they perform, including the following (explore the model catalog^[4]); read it as a zoom-in on those four rather than a rival list. The catalog labels a few classes differently, notably chat completion for chat models and image analysis for multimodal models, and this page keeps one name per class throughout. The list is representative rather than exhaustive, and the catalog gains new categories over time.

Class	Returns	Reach for it when
Chat model	A generated assistant message	The answer is newly written natural language
Reasoning model	A generated answer, with the model breaking the problem into steps	Mathematics, coding, science, strategy, and similar multi-step problems
Embedding model	A numerical representation of the input	Semantic search, recommendations, and retrieval by meaning rather than exact keywords
Image-generation model	A newly created image	Marketing material, illustrations, or design mockups from a description
Video-generation model	Newly created video content	Video from a text description
Multimodal model	Natural language about supplied images	Prompts that include images to be interpreted
Text-to-speech model	Synthesized speech	Reading text aloud
Speech-to-text model	A transcription	Turning spoken audio into text

Size is a second axis, not a replacement for the first. Large language models suit deep reasoning, complex content generation, and extensive context understanding while demanding more computational resources; small language models handle common natural language tasks more cheaply and can run on lower-end hardware or edge devices. Pick the class first, then the size.

Input modality is a separate check

A model that returns text is not automatically a model that accepts images. When a prompt combines modalities, the requirement is a multimodal model whose model card lists every input type you intend to supply, and the fact that you want a text answer back tells you nothing about what the model will accept going in. This is the single most reliable trap in this objective, so make the model card the deciding evidence rather than the output format.

The evidence surfaces in the Foundry portal

Three surfaces answer three different questions, and mixing them up costs marks:

1. Catalog filters narrow the field. The catalog supports keyword search plus filters on collection, capabilities such as reasoning, tool calling, and multimodal processing, source, inference tasks, fine-tuning methods, and industry. The product documentation lists a partly different set, adding region, deployment options, deployment SKU, lifecycle stage, and supported features (model catalog capabilities^[5]). The two lists overlap without matching, so learn the kinds of filter rather than one canonical list.
2. The model card confirms a single candidate. It shows the provider, capabilities, benchmark metrics, responsible AI considerations, and deployment options, organized in the portal as quick facts plus Details, Deployments, Benchmarks, and License tabs. Supported data types live on the Details tab, which is where the multimodal check above is settled.
3. The model leaderboard (preview) and Compare models rank and contrast candidates. The leaderboard sorts on quality, safety, estimated cost, and throughput; trade-off charts plot two of those metrics against each other; leaderboards by scenario rank models for a capability such as reasoning, coding, or question answering. Compare models opens up to three models side by side across performance benchmarks, model details including context window, supported endpoints, and feature support (compare models using the model leaderboard^[6]).

Evaluators are a different tool for a different job. They score the outputs of a solution you have built; they do not tell you which candidate model costs less or supports vision. When a question asks how to compare candidate models on capability and cost, the leaderboard and Compare models are the answer.

Read benchmarks with their limits attached

Benchmark data is not available for all models in the catalog, and a model without a Benchmarks tab simply has no published results yet. Scores are normalized indexes: higher is better for quality and safety, while for cost and throughput a lower estimated cost and a higher throughput are generally preferred. Public benchmarks use standardized datasets and might not reflect performance on your own data, which is what the separate evaluation workflow exists for.

One number worth not memorizing: Microsoft's own pages currently give different sizes for the catalog, one saying over 1,900 models and another over 10,000. The size is not the testable fact; the selection method is.

So the sequence is output class, then input modality, then evidence from the card and the leaderboard. Nothing about that order changes with the model of the month.

You have settled on a model, your client code is open, and there is still no address to send a request to. Selecting a model in the catalog does not make it callable, and adding it to a collection does not create an endpoint. Something has to hand your application that address, and Foundry offers more than one way to get one. Read the options and their selection criteria first; the mechanics of each follow.

• Instant access (preview) lets you call a supported model by name with no deployment at all. It fits getting started, prototyping, and trying a model the day it is released.
• A deployment creates a named inference target that you then address by its deployment name. It fits everything that needs reserved capacity, data residency, per-model content filtering, version pinning, quota partitioning, or a fine-tuned model.
• Managed compute (preview) runs model weights on dedicated managed GPU capacity. It fits open-source, partner, industry, and custom-weight models that are not served from a Microsoft-hosted endpoint.

Instant access, and what it gives up

With instant access you pass a supported model name where you would otherwise pass a deployment name, using the same API, SDK, and client (instant access to models^[7]). It draws on a per-model global quota pool that is separate from the regional quota standard deployments use, and by default it resolves to the latest evergreen version unless you append a version suffix to pin one. During preview it is limited to projects in West US 3, fine-tuned models are not supported, and guardrails, which are the safety controls applied around a model, along with custom responsible-AI policies and content filters, cannot be configured per model. If an existing deployment is named the same as a model, the deployment wins and instant access for that name is unavailable in that project.

Deployments, and the type you pick with them

Creating a deployment means choosing a deployment type, and that choice determines where your data is processed, how you pay, and the performance characteristics you get (deployment types^[8]). The categories are standard, which is pay-per-token, and provisioned, which is reserved capacity bought as provisioned throughput units; within each you choose global, data zone, or single-region processing.

Deployment type	Data processing	Billing
Global Standard	Any Azure region	Pay-per-token
Data Zone Standard	Within the specified data zone (US, EU, or APAC)	Pay-per-token
Standard	The deployment region only	Pay-per-token
Global Provisioned	Any Azure region	Reserved provisioned throughput units
Regional Provisioned	The deployment region only	Reserved provisioned throughput units
Global Batch and Data Zone Batch	Any region, or within the data zone	Discounted, with a 24-hour target turnaround
Developer	Any Azure region	Pay-per-token, fine-tuned model evaluation only

The pattern behind the table is worth more than the rows: the first word of the type names the data-processing scope, and the second names the billing model. The table shows the common combinations rather than every published SKU, and because the naming is generative, a type you have not met before, such as Data Zone Provisioned, decodes the same way. Data stored at rest stays in the designated Azure geography for every type; it is inferencing data whose processing location varies. Not every model supports every type, and a Developer deployment has a fixed 24-hour lifetime with no service-level agreement.

Two names in that table collide with names used elsewhere, and a question can turn on which one is meant. Standard with a capital S is a single deployment type, the single-region pay-per-token one, whereas standard deployment in a Foundry resource is the broader access path under which those deployment types are offered. The Developer deployment type likewise has nothing to do with the developer message role described earlier.

Among the paths that use a deployment, standard deployment in a Foundry resource, the primary resource type for new Foundry projects, is documented as the preferred and most capable option, supporting the widest range of deployment types, regional, data zone, or global processing, built-in and customizable content filtering, keyless authentication with Microsoft Entra ID, private networking, and provisioned throughput (deployment overview^[9]). Reaching for dedicated virtual machines because dedicated sounds more capable inverts the actual guidance.

Managed compute, and what it costs

Managed compute (preview) is a managed GPU platform-as-a-service that hosts open-source and custom-weight models on dedicated capacity, with no virtual machines, clusters, or serving runtimes for you to own. Billing is hourly per accelerator SKU rather than per token, and auto-scale with an idle timeout lets a deployment scale to zero so billing stops. Quota is granted per accelerator SKU per region through the Foundry quota process and is separate from Azure virtual machine (VM) quota, so existing VM quota cannot be applied to it. In public preview it is available for global deployment only, and content filtering is not available through it; you use the Azure AI Content Safety APIs instead. Example collections that require it include Hugging Face models, NVIDIA inference microservices, industry models, and custom models. The figure below groups the paths by who holds the capacity.

Two shapes, several names

Microsoft's pages currently label and count these paths differently. The deployment overview describes standard deployment in Foundry resources alongside managed compute; the model catalog overview describes managed compute alongside serverless deployments, using that second name for the Microsoft-hosted, API-based path where you are billed for inputs and outputs, typically in tokens. Because the naming and the totals do not line up across pages, learn the two hosting-and-billing shapes, a framing used on this page rather than Microsoft's own vocabulary, and treat any specific count as unsafe: either Microsoft hosts the model and you pay for what you send and receive, or model weights run on capacity dedicated to you and you pay for the time it is up.

Your prototype answers well but rambles, and asking it the same question twice gives two different answers. Both complaints are settings. Request parameters change how the model samples and how long it may answer; they do not change what it knows, and holding those two clauses together explains every right answer in this part of the objective.

The Foundry model playground is where you meet them first: it lets you configure parameters such as temperature, top_p, and max_tokens, inject system prompts, and enable tools, then export the equivalent code (Foundry playgrounds^[10]). What you tune there is what you send in a request.

The three settings and what each one moves

Setting	What it changes	Reach for it when	What it will not do
Temperature	How random token selection is; lower is more focused and consistent, higher is more varied	Answers wander between runs, or you want more variety	Change how long a response may be
Top P	Caps the pool at the most probable tokens whose combined probability reaches a chosen mass (nucleus sampling), another way to steer the same randomness	You would rather bound the probability mass than set a temperature	Compound usefully with Temperature; adjust one or the other
Max Completion Tokens	An upper bound on the generated-token budget for the response	A response must fit a length, or generated-token cost must be bounded	Guarantee the model uses the full amount

Temperature and Top P steer the same behavior by different means, so the normal practice is to adjust one of them in a request rather than both. Neither one is a length control; that is a separate lever with a separate name, which is exactly the pair most often swapped in exam options.

Max Completion Tokens is a ceiling, including the invisible part

The portal label is Max Completion Tokens; in code the parameter name depends on the surface. Reasoning models take max_completion_tokens on the Chat Completions API and max_output_tokens on the Responses API (reasoning models^[3]). Whichever name applies, the value bounds generated tokens, and on applicable models that budget covers the hidden reasoning tokens described earlier as well as the visible answer. Set it too low on a reasoning model and the reply can be truncated before any visible text appears.

A response carries a finish reason, and length means the output was incomplete because of that parameter or the model's own token limit, as distinct from stop, which means the model finished (finish reasons^[2]). That is a request that ran and returned a shortened answer, a different outcome from the over-budget request in the context-window section, where the combined count is rejected before anything is generated. Microsoft's guidance is to set the value high enough for the expected response so the model does not stop before reaching the end of its message.

Listing 2: a request that sets both kinds of control

This is the same transcript idea as Listing 1, now sent through a client with the two settings attached. Only the call is shown; the client construction and the credential setup are omitted at the marked point.

# ... client construction omitted ...
response = client.chat.completions.create(
    model="YOUR-DEPLOYMENT-NAME",   # the deployment name from the access path above
    messages=conversation,          # the role-labeled transcript from Listing 1
    temperature=0.7,                # sampling: lower is more consistent
    max_completion_tokens=250       # ceiling on generated tokens, not a target
)

What these settings are not for

Parameters do not add knowledge, and they do not moderate content. A wrong-but-fluent answer is addressed by grounding the model on your own sources or choosing a different model. Harmful content is addressed by content filtering and guardrails attached to a deployment, which is also why instant access, where those are not configurable per model, is positioned for prototyping rather than for a workload with specific filtering requirements. Where data is processed is settled by the deployment type, not by anything in the request body.

The compact form: temperature and Top P for how it chooses, Max Completion Tokens for how far it goes, and neither for what it knows or what it is allowed to say.

Questions on this objective rarely ask what a term means. They describe a situation and ask which model, which access path, or which setting fits it, and the wrong options are usually adjacent facts that are true about something else. This section covers the recurring stems and the reason each distractor fails.

Stem cues and the answer they point to

The stem says	Correct choice	The option that looks right
Compare passages by meaning, cluster them, or retrieve by similarity	An embedding model	An image-generation model, because it also produces a representation
Compose a new natural-language answer to a user question	A chat completion model	Sentiment analysis or another text-analysis feature
The prompt includes an image to interpret	A multimodal model whose card lists image input	Any text-output model, on the assumption that output modality settles input
Compare candidate models on capability and estimated cost	The model leaderboard (preview) and Compare models	The evaluator catalog, which scores solution outputs
Make a catalog model callable	Create a deployment, or use instant access (preview)	Add the model to a project collection
Host open-source or custom model weights	Managed compute (preview)	Standard deployment, which does not host arbitrary weights
Keep a response within a length or generated-token cost limit	Max Completion Tokens	Temperature, which changes randomness
Make answers more consistent between runs	Lower Temperature, or use Top P	Raise Max Completion Tokens

Recurring misconceptions worth naming

• The model looks up a stored answer. It generates one token at a time from probabilities; there is no keyed retrieval of a matching response.
• The deployment retrains on user prompts. Training produced the parameters once; inference applies them and changes nothing.
• Input and output limits are additive. They share one context window, so spending the input allowance in full leaves less room for the answer, not the full output allowance.
• Application-wide rules belong in an assistant message. They belong in the system message; assistant content reads as example output.
• An embedding is the text handed back to the user. It is a numerical vector; the natural-language reply is a completion.
• Managed compute is the safer default because dedicated hardware means more features. Standard deployment in a Foundry resource is documented as the preferred and most capable path.
• A serverless deployment installs weights on your own virtual machines and bills by core-hour. That description belongs to managed compute; the Microsoft-hosted path bills for inputs and outputs, typically in tokens.

Where the objective stops

The published skills outline for this objective covers how generative AI models work, identifying an appropriate model based on capabilities, and identifying appropriate deployment options and configuration parameters (AI-901 study guide^[11]). Advanced data science, model training, and MLOps are named as out of scope for this exam and belong to the associate-level certifications, so a question that appears to require designing a training run is almost certainly testing something else, usually the selection or configuration decision sitting next to it.

A final caution on counts. Where two official pages disagree on a number, such as how many deployment options exist or how many models the catalog holds, an exam answer keyed to the number is a guess. Answer from the shapes: what the model returns, who hosts it, how it is billed, and which setting moves which behavior.