Build Generative Applications with Microsoft Foundry

A team ships a chat feature against the Azure OpenAI endpoint, then picks up a requirement to answer from the company's uploaded PDFs. The model does not change. The code does, because file search is a Foundry platform tool, meaning a capability the model can invoke while it is producing a response, and platform tools are only reachable through the Responses API in the project endpoint^[1], never through the direct model route. The Responses API is the request shape that carries those tools, and the third section takes it apart. One requirement, arriving late, moved the application onto a different endpoint. That is why this page walks its material in the order a build actually commits to it.

A platform tool is not the same thing as a Foundry Tool, despite the near-identical name: Foundry Tools are the prebuilt services such as Vision, Speech, and Content Safety, which have their own SDKs and their own tool-specific endpoints^[1]. Everything on this page means the first sense.

Microsoft Foundry (formerly Azure AI Foundry, and Azure AI Studio before that) is the platform all of this runs on, and this is the first of three pages covering it for generative and agentic work. It starts from a single project endpoint and the two clients built from it, taking nothing else about Foundry as familiar, and ends with an application that constrains its own output, grounds its answers, coordinates several steps, and is graded by evaluators that name which stage failed. This page owns the application: the endpoint it connects to, the call it makes, the shape it demands back, the tools and retrieval attached to that call, the way several steps are coordinated, and the evaluators that grade the result. The page on building Foundry agents owns the agent object itself, meaning how a prompt agent or hosted agent is defined, versioned, and equipped with memory and knowledge tools. The page on optimizing and operationalizing owns what happens after the application works: prompt structure, sampling parameters, tracing, caching, and model routing.

The seven decisions, in build order

These seven names are used consistently for the rest of the page, and each has its own section below. The figure below lays them out as one spine.

• Connect: which endpoint the application is configured with, and which client objects it builds from it.
• Call: which of the three request routes carries a given call, and which deployment name goes in the model argument.
• Output contract: whether the answer comes back as free text, as merely-parseable JSON, or as JSON bound to a schema you supply.
• Tools: which capabilities the model may invoke, and whether your process or the platform executes them.
• Grounding: where the facts in the answer come from, and which retrieval tool fetches them.
• Orchestration: how several agents or steps are sequenced when one call is not enough.
• Evaluation: which measurements tell you whether the result is good, and which failure each one isolates.

The order is not arbitrary. Each decision constrains the ones under it: the endpoint decides which call surfaces exist, the call surface decides which tools exist, and the tools decide which evaluators have anything to score. Read top to bottom the first time.

Configuration for a Foundry application is one value. The Foundry SDK is a thin client over all Foundry project APIs reached through a single project endpoint^[1] of the form https://<resource-name>.services.ai.azure.com/api/projects/<project-name>, where <resource-name> is the Foundry resource (or its custom subdomain, if the organization uses one) and <project-name> is the project inside it. Models, agents, tools, and evaluations all sit behind that one string.

From that endpoint the SDK gives you two client objects, because Foundry and OpenAI have different API shapes. The project client covers Foundry-native operations that have no OpenAI equivalent: reading project properties, listing and resolving connections, and enabling tracing. The OpenAI-compatible client, returned by get_openai_client(), covers everything modeled on OpenAI concepts: responses, conversations, files, vector stores (the indexed stores the file search tool queries, covered in the grounding section), evaluations, and fine-tuning. Most production applications hold both, and the figure below shows the split. A predictable early mistake is to reach for project.responses.create(...), which does not exist; responses live on the OpenAI-compatible client.

Building both clients (Python)

The AI-103 audience profile expects experience developing apps by using Python^[2], so this page's listings are Python, and this one is the shape every later listing assumes: the project client is constructed once from the endpoint plus a credential, and the OpenAI-compatible client is derived from it rather than configured separately. The Python package is azure-ai-projects 2.x, installed with pip install "azure-ai-projects>=2.0.0".

from azure.identity import DefaultAzureCredential
from azure.ai.projects import AIProjectClient

project = AIProjectClient(
    endpoint="https://<resource-name>.services.ai.azure.com/api/projects/<project-name>",
    credential=DefaultAzureCredential(),   # Entra ID token; no key in configuration
)

openai = project.get_openai_client()       # OpenAI-shaped surface: responses, vector stores, evaluations
response = openai.responses.create(
    model="gpt-5.2",                       # a DEPLOYMENT name in this project, not a catalog model id
    input="What is the size of France in square miles?",
)
print(response.output_text)
# ... error handling and client cleanup omitted

DefaultAzureCredential is the Azure Identity credential chain: it tries environment variables, a managed identity, and the signed-in developer tooling in turn, so the same code runs locally and in production without a key. The model argument is discussed in the next section; it names a deployment, not a catalog model.

Connections keep credentials out of the application

A project connection is a named pointer from the project to an external resource. Connections let you authenticate to Microsoft and other resources within your Foundry projects^[3] and are required for scenarios such as building standard agents or building with agent knowledge tools; different connection types support different authentication methods, and a Custom key connection stores keys alongside related properties such as targets and versions. For a tool, the connection holds the endpoint, the authentication selection, and any credentials, and the supported methods are key-based, Microsoft Entra with the agent identity, Microsoft Entra with the project managed identity, OAuth identity passthrough, and unauthenticated access^[4]. Tool definitions then reference the connection by its project_connection_id instead of carrying the secret themselves, which is why an Azure AI Search tool definition^[5] contains a connection resource ID and an index name but no key. Putting the downstream key in an environment variable or a tool header block works, and it also removes rotation and per-project scoping from the platform's hands, so the connection is the documented path.

Three things to carry forward from this section: one endpoint string configures the application, two clients come out of it and they are not interchangeable, and anything the application reaches outside Foundry arrives through a named connection rather than through a secret in your configuration.

The endpoint an application is configured with decides what it can ever reach, so this is the decision with the longest shadow. Foundry exposes three request routes for model and agent traffic, drawn in the figure below, and a project that mixes model families uses more than one of them. Foundry Tools such as Vision and Speech sit outside this set on their own tool-specific endpoints.

The project endpoint carries the Responses API and is the only route to Foundry agents, Foundry-exclusive platform tools, and Foundry evaluations. The Azure OpenAI route, https://<resource-name>.openai.azure.com/openai/v1, offers the full OpenAI API surface with the best latency and maximum OpenAI compatibility^[1], and it is where Chat Completions lives, but it does not provide access to agents, evaluations, or Foundry-exclusive tools. Anthropic Claude deployments answer on a third route, https://<resource-name>.services.ai.azure.com/anthropic, using the Anthropic Messages API and the Anthropic SDK rather than the OpenAI-compatible client. Two consequences follow directly. A chat application that later needs file search or a Model Context Protocol (MCP) server has to move off the direct Azure OpenAI route, and an application that adds a Claude deployment needs a second client path rather than a different model string.

Embeddings are the documented exception

The single-endpoint promise has one hole worth memorizing: the project endpoint used by the Foundry SDK doesn't currently route embedding requests^[1]. An application that generates its own vectors sends those calls to the Azure OpenAI /openai/v1 endpoint^[6] with the OpenAI SDK while the rest of it keeps talking to the project endpoint. The request is not transparently forwarded, so pointing an embedding call at the project endpoint fails rather than quietly working. Note that this only matters when the application builds its own vectors; the file search tool embeds on your behalf, which the grounding section covers.

The model argument names a deployment

On every route, the model argument carries the name of a deployment created in the project, not a catalog model identifier such as gpt-5.2. The two often look identical because deployments are conventionally named after the model they serve, and that convention is not a requirement. The practical consequence is that swapping models is a deployment-level change^[7]: repointing a deployment at a newer version, or creating a same-named deployment backed by a different model, changes behavior without any code edit. The matching failure is a hard-coded catalog string that matches no deployment in the target project, which is why the same code can work in development and fail in production.

The takeaway for the exam is a two-part test. Ask first which route a scenario needs, because that is what the Foundry SDK, OpenAI SDK, and Anthropic SDK choices come down to, and ask second whether a stated model name is a deployment or a catalog entry.

Multi-turn behavior on the Responses API is a storage decision before it is a code decision, because the service keeps response history by default. The store parameter defaults to true^[8], so a response is persisted and can be retrieved later. Three patterns follow from that default, and the figure below sets them side by side.

The lightest pattern passes previous_response_id on the next call. Because the prior turn is already stored, the service carries its context forward and the client sends only the new input, with no message array to rebuild and no conversation object to create. Rebuilding and resending the whole history when a response id already carries it is the classic wasted-token pattern here.

The second pattern creates a conversation object with conversations.create() and passes its id on each response. Items are appended to it automatically as responses generate, which gives several calls one shared thread of items rather than a chain of ids. Conversations are the state object Foundry agents are built around, so the Foundry agents page covers their lifecycle in depth; for a plain application, treat a conversation as a named container that a series of responses can share.

The third pattern turns persistence off. With store set to false the service does not persist the response, so previous_response_id has nothing to reference and the application must pass the earlier output items back as input on the next request. This is the pattern for zero-data-retention environments and for teams that want full control of conversation state, and its failure mode is quiet: a team sets store=false thinking of it as a logging switch, and follow-up turns arrive with no context at all. The two statements sit together cleanly once you read store as the thing that decides whether there is a server-side copy to point at, rather than as an audit setting.

A response also returns more than text. Alongside output_text, which holds the final assistant message, the call returns an output list of typed items such as message, function_call, function_call_output, code_interpreter_call, and reasoning. Inspecting what a turn actually did therefore means iterating response.output and switching on item.type, not reading the wording of the answer. That typed array is what the next two sections operate on.

Ask what the caller of your application does with the answer. If a person reads it, plain text is the contract. If a parser reads it, the question becomes how much of the shape the model is actually obliged to honor, and there are two very different answers. The figure below stacks the three options by how much they guarantee.

JSON mode^[9] constrains the model to emit syntactically valid JSON and stops there. Field names, types, nesting, and the presence of required keys are unenforced, so a downstream parser can still break on a renamed or missing property. Structured outputs is the stronger contract: it makes a model follow a JSON Schema definition that you provide as part of your inference API call^[10], and the docs draw the same distinction, describing JSON mode as the older feature that guaranteed valid JSON but could not ensure strict adherence to the supplied schema. Reaching for JSON mode in a schema-driven extraction pipeline and then adding validation retries is solving at parse time a problem the generation call can enforce.

The supported schema subset

Structured outputs binds generation to your schema only when strict is set to true, and strict mode accepts a documented subset of JSON Schema rather than the whole specification. The rules that decide most exam items are these:

Definitions via $defs and recursive schemas using # are supported, so a nested structure does not force you out of strict mode. Key ordering in the output follows the order of the schema you send.

A strict schema on a request

The listing below is the response-format block from a Chat Completions request; the Responses API expresses the same thing through the response text format. It shows the four constraints from the table working together on one small object, and the prose above the table names each key it uses.

{
  "type": "json_schema",
  "json_schema": {
    "name": "CalendarEventResponse",
    "strict": true,
    "schema": {
      "type": "object",
      "properties": {
        "name":  { "type": "string" },
        "date":  { "type": "string" },
        "venue": { "type": ["string", "null"] }
      },
      "required": ["name", "date", "venue"],
      "additionalProperties": false
    }
  }
}

The venue property is the optional-field pattern: it is listed in required like every other property, and its optionality is expressed by allowing null in its type. Note two documented gaps before planning around this feature. Structured outputs are not supported with parallel function calls, so a request that uses both sets parallel_tool_calls to false, and the docs list Foundry Agent Service among the scenarios structured outputs are not currently supported with.

So the decision reduces to one question about the consumer and one about the schema: a parser downstream means structured outputs rather than JSON mode, and a schema that survives strict mode is one where every property is required, every object closes with additionalProperties: false, and optionality is spelled as a union with null.

Two things called tools behave in opposite ways, and the difference is simply who runs the code. Service-side tools such as Azure Functions, OpenAPI tools, and MCP tools are invoked by the platform. A plain function tool is not: the model can only ask for it, and your own process has to answer. Expecting Foundry Agent Service to execute a Python function you registered is the misread this section exists to remove.

When the model selects a client-side function tool, it emits a function_call item into the response's output array carrying three things: the function name, its arguments as a JSON string, and a call_id. The turn then suspends. Your application parses the arguments, runs the code, and posts a function call output item keyed to that same call_id; only then does the model produce its final message. A single response can carry several function_call items, as the docs show with a single tool called for three different locations in one turn^[11], and each one needs its own output keyed to its own call_id. The figure below traces the whole loop, including the suspension in the middle that makes it two API calls rather than one.

Handling the suspended turn (Python)

This listing continues the openai client built in the connect section and fills in steps 4 and 5 of the figure: it walks the typed output array, executes each requested call, and sends the outputs back keyed by call_id.

tools = [{
    "type": "function",
    "name": "get_current_time",
    "description": "Get the current time in a given location",   # <= 1,024 characters
    "parameters": {
        "type": "object",
        "properties": {"location": {"type": "string"}},
        "required": ["location"],
    },
}]

first = openai.responses.create(model="gpt-5.2", input=user_text, tools=tools)

follow_up = []
for item in first.output:                       # the typed output array, not the answer text
    if item.type == "function_call":
        result = dispatch(item.name, item.arguments)   # your code, in your process
        follow_up.append({
            "type": "function_call_output",
            "call_id": item.call_id,            # must match the call_id the model emitted
            "output": result,
        })

second = openai.responses.create(
    model="gpt-5.2",
    previous_response_id=first.id,              # the prior turn is already stored
    input=follow_up,
)
print(second.output_text)
# ... argument validation and error paths omitted

The call_id is the join key of the whole exchange, and dispatch here stands in for your own router from a function name to a Python callable. Because the model fills the arguments from the tool name, description, and parameter schema alone, wrong-tool and bad-argument failures are fixed in the tool definition rather than in the prompt, and the docs cap tool and function descriptions at 1,024 characters.

Forcing or suppressing a tool call

tool_choice decides whether a tool has to be used on this turn. The default auto lets the model decide on its own whether to call a function and which one, which means it may answer from what it already knows and never touch the tool. Setting tool_choice to required makes it invoke one of its tools, which is how you guarantee that a retrieval or delegation step actually ran and that citations exist. You can also name one tool explicitly to force that specific function, or set none to force a user-facing message. This is worth holding onto for troubleshooting: an answer that comes back with no citations is often not a broken retriever but a model that simply chose not to call the tool, and the file search troubleshooting guidance recommends exactly this fix.

Grounding is where the facts in an answer come from, and inside a Foundry application there are two managed paths to it. The choice turns on one question: does the corpus arrive as uploads from your application, or does an index for it already exist? The figure below draws both paths converging on the same model context.

When the application owns the documents

The file search tool takes documents your application uploads and handles the whole ingestion pipeline itself, the managed ingestion step in the figure below. Adding a file to a vector store automatically parses, chunks, embeds, and stores^[12] it in a vector database supporting both keyword and semantic search, and at query time the tool rewrites the query, breaks complex queries into parallel searches, runs hybrid search, and reranks the results. Standing up a separate chunking and embedding pipeline in front of an upload duplicates work the tool already performs.

The defaults are documented, which makes them fair game for a question:

The limits around a vector store matter just as much. Each store holds up to 10,000 files, the maximum file size is 512 MB, and each file should contain no more than 5,000,000 tokens. At most one vector store attaches to an agent and at most one to a conversation, so widening coverage means consolidating files into a single store rather than attaching several. Vector stores created through conversation helpers carry a default expiration policy of seven days after they were last active, and when one expires the runs on that conversation fail; the documented fix is to recreate the store with the same files and reattach it.

Where the files and vector stores physically live depends on the environment rather than on the code. Under basic agent setup, uploaded files sit in Microsoft-managed storage and vector stores are built on a Microsoft-managed search resource. Under standard agent setup, the same tool stores files in your connected Azure Blob Storage account and creates vector stores in your connected Azure AI Search resource. The documentation is explicit that the code is identical for both setups and the only variation is where your files and vector stores are stored, so a residency requirement is answered by the environment choice, not by a rewrite.

When an index already exists

The Azure AI Search tool retrieves documents from an index you already maintain^[5] so the model can answer with inline citations. It is the right choice when an enriched index is already refreshed by your own indexer or skillset pipeline, because re-uploading that content into file search creates a second copy that drifts away from the source. The tool takes a project_connection_id and an index_name as required parameters, and optionally top_k, which defaults to 5, and query_type, which defaults to vector_semantic_hybrid and also accepts simple, vector, semantic, and vector_simple_hybrid. Two constraints shape designs around it: the tool can only target one index, and the search resource and the Foundry agent must be in the same tenant.

So the two paths differ in who owns ingestion, not in what the model finally sees. File search means the service ingests for you and you accept its chunking defaults; the Azure AI Search tool means you own ingestion and the agent consumes the result. The retrieval and grounding pipelines page covers building and enriching that index itself.

You get a requirement list asking for answers that track a product catalog updated nightly, always written in the company's support-desk voice, and rendered in the caller's language. It is tempting to read that as one problem with three candidate solutions. It is three problems, each with one right lever, and the figure below keys them to the question each one answers: how often does this thing change?

Retrieval-augmented generation (RAG) is the lever for facts that change faster than a model can be retrained, or that come in too many distinct values to fit in any prompt (high-cardinality facts, such as a per-customer price list). Microsoft's own guidance is to use RAG when you need answers grounded in private or frequently changing data, and to use fine-tuning when you need to change model behavior, style, or task performance rather than add fresh knowledge^[13]. The application fetches the current source of truth at query time and puts it in the model's context, so the answer tracks the data rather than the weights. Fine-tuning is the lever for conventions that do not change: a house voice, a fixed output shape, a domain vocabulary. Encoding those in weights means they hold without spending prompt tokens on them at every call. The prompt itself carries what varies per session, such as the caller's language, entitlement tier, or the current date.

The fastest way to get this wrong is to fine-tune on volatile data. A catalog that changes daily, baked into weights, produces answers that were correct at training time and needs a retraining cycle at every refresh, which is exactly the cost RAG exists to avoid. The mirror-image mistake is prompting your way to a durable convention: pasting a two-page style guide into every request works and it also pays for that style guide in tokens on every call, forever.

Because the three answer different questions, a requirement set that names all three needs all three, and the levers combine without conflict. In practice a Foundry application that grounds on file search, calls a fine-tuned deployment, and passes session variables in its system instructions is using one of each, and the evaluation section is where you find out which of the three is actually failing.

One call with several tools handles most applications. Coordination becomes its own decision when a process has stages that different agents own, such as a triage agent handing a refund case to a policy agent, or when a human has to approve that refund before the order moves on. Foundry offers two ways to express that, and which one you pick is now partly a calendar question.

A Foundry workflow is a declarative, predefined sequence of actions that orchestrates agents and business logic in a visual builder. Three templates cover the three coordination shapes, drawn in the figure below: Human in the loop asks the user a question and awaits input to proceed, Sequential passes the result from one agent to the next in a defined order, and Group chat dynamically passes control between agents based on context or rules^[14]. In the Human in the loop shape the run pauses at the question and resumes only once the user answers. Reaching for Group chat when the process is a fixed pipeline adds nondeterminism to a sequence that has to be auditable; Sequential is the template for a process whose order is part of the requirement.

Inside a workflow, the common node types are Agent to invoke an agent, Logic for if/else, go to, or for each, Data transformation to set a variable or parse a value, and Basic chat to send a message or ask a question. Power Fx formulas compute values inside those nodes, and every variable reference needs a scope prefix: System. for system variables and Local. for local ones. Omitting the prefix does not silently resolve; it raises a Name isn't valid formula error, which the troubleshooting table resolves by adding the prefix. Two operational details are easy to miss. Foundry does not save workflows automatically, and each save creates a new, unchangeable version, visible under the Version dropdown.

Two limits that redirect the design

Hosted agents aren't supported in the workflow designer. A hosted agent is your own container image run by Foundry, as opposed to a prompt agent, which is declarative configuration the platform runs with no container of yours; the Foundry agents page covers both types in full. When a hosted agent has to coordinate other agents or run a multistep process, the guidance is to use Microsoft Agent Framework workflows from inside that agent's code. Planning a portal workflow around a hosted agent and only then discovering that agent nodes accept prompt agents is a costly ordering mistake.

The second limit is a date. Microsoft is retiring workflows on December 1, 2026, and directs new work to Microsoft Agent Framework. The capability itself is not disappearing: the same orchestration patterns are expressible in code or declarative YAML, the exported workflow YAML is the portable artifact to carry across, and after that date Foundry continues to run YAML-based workflow definitions when they are deployed as a hosted agent. The documented alternatives are Agent Framework for a code-first runtime, Azure Logic Apps if a fully visual low-code designer is the reason you wanted workflows, and a direct agent-to-agent (A2A) call for a lightweight hand-off between two agents. Agent Framework is available in Python and .NET, and its foundry package depends on the Foundry SDK to access Foundry models, tools, and project configuration^[1], so a code-first orchestration still runs on the platform's hosted-agent runtime rather than needing hosting of its own.

For the exam, hold three things: which template matches which coordination shape, the two scope prefixes Power Fx requires, and the fact that the designer accepts prompt agents while hosted agents orchestrate in code.

A retrieval-augmented application that answers badly has at least three possible culprits, and tuning the wrong one wastes a sprint. Evaluators exist to tell them apart, and the useful mental model is a map rather than a leaderboard (an analogy for how to read them, not Microsoft terminology): each evaluator watches one junction of the pipeline. The figure below pins each one to the stage it observes. Note that the first evaluator shares its name with a pipeline stage; capitalized Retrieval in this section always means the evaluator, and retrieval step always means the search that runs before generation.

Retrieval measures how effectively the system retrieves relevant information^[15], Groundedness measures how grounded the response is in the retrieved context, and Relevance measures how relevant the response is with respect to the query. Reading those as three flavors of the same score is the trap. A fabricated citation is a Groundedness signal, because the answer went beyond what the context supported. An answer that misses the obviously right document is a Retrieval signal, and no amount of prompt work recovers context that never came back. An answer that is faithful to its context and still does not address what the user asked is a Relevance signal. Microsoft's own recommended combination for a RAG application is Retrieval plus Groundedness plus Relevance plus Content Safety, which is the same map read as a checklist.

Two Groundedness evaluators, two different answers

Groundedness returns a score from 1 to 5 using a model-based judgment, so it needs a judge model, meaning a model deployment of your own whose job is to score another model's output rather than to serve users, and it can rank a near-miss answer against a threshold. Groundedness Pro, which is in preview, uses the Azure AI Content Safety service instead, returns a binary pass or fail, and requires no model deployment. That makes Pro cheaper to stand up and unable to grade degrees, so a team that wants to route borderline answers to human review needs the graded evaluator, not the binary one.

Judges versus overlap metrics

The textual similarity family splits along whether it needs a reference answer. F1 Score, BLEU, GLEU, ROUGE, and METEOR all measure overlaps in n-grams or tokens between the response and a ground truth, so they are unusable when no labeled answer exists. Planning a BLEU or ROUGE gate for open-ended generation is therefore a dead end. Similarity, along with the general-purpose Coherence and Fluency evaluators and the RAG evaluators above, is AI-assisted: a judge model scores the output, so no reference text is required but a judge deployment is. Document Retrieval and Response Completeness sit on the ground-truth side of the same line.

Turn level or conversation level

One run-level setting quietly rejects otherwise sensible evaluator combinations. Each evaluator declares which levels it supports in supported_evaluation_levels, and an evaluation run carries an evaluation_level that is turn (individual agent responses) by default or conversation (entire multi-turn conversations, the same multi-turn exchange the state section called a conversation). All evaluators in a run must support the specified level, and you can't mix evaluators with incompatible levels in the same run. Name recognition is not a reliable guide to which is which, so check the catalog rather than assuming that an evaluator about satisfaction must be conversation level.

The scope line for this page ends here: choosing and reading evaluators for an application is part of building it, while calibrating a judge against human labels, writing custom weighted rubrics, and evaluating production traffic continuously belong to the optimize and operationalize page, and the agent-specific evaluators such as Intent Resolution and Tool Call Accuracy belong to the Foundry agents page.