Domain 1 of 5 · Chapter 2 of 4

Setting Up AI Solutions in Foundry

Unlock the complete study guide + 1,040 practice questions across 16 full exams.

Bundled into the existing AI-103 premium course — no separate purchase.

14-day money-back guarantee — no questions asked.

Included in this chapter:

  • Provisioning order and the capability host
  • Standard agent setup and your own state stores
  • Choosing a model deployment type
  • Version upgrade policy and model retirement
  • Choosing the agent runtime: prompt or hosted
  • Agent versions and structured inputs
  • Role assignments the setup depends on
  • Network isolation for agent traffic
  • Infrastructure as code and the release pipeline
  • Holding the footprint in its desired state
  • Designing agents to survive a dependency outage

The main Foundry deployment options, by decision axis

Decision axisStandard (pay-per-token)Provisioned (PTU)BatchManaged computeInstant access (preview)
Billing basisInput and output tokens consumedReserved provisioned throughput unitsTokens at 50 percent less than Global StandardHourly per accelerator SKUTokens against a global quota pool
Capacity you must holdToken quota only, no compute quotaPTUs, sized per model and version pairA separate enqueued-token quotaAccelerator quota per SKU and region, separate from VM quotaWhatever global quota Global Standard deployments leave
Latency behaviourBest effort; more variable at high sustained volumeLower and more consistent24-hour target turnaround, no real-time SLAFixed by the accelerators you reservedBest effort
Inference processing locationGlobal, data zone, or the deployment regionGlobal, data zone, or the deployment regionGlobal or data zoneGlobal onlyAny Azure region
Documented fitVariable, bursty online trafficConsistent high volume needing predictable throughputLarge asynchronous jobs that are not time sensitiveOpen-source and custom-weight modelsGetting started, prototyping, trying a new model

Decision tree

Model supports standard deployment rather than managed compute only No Managed compute global processing only Yes Governed production endpoint quota, filters, residency guarantee No Instant access (preview) no deployment created Yes Real-time responsiveness required a caller waits on the answer No Batch 50% cost, 24-hour target Yes Steady high volume, latency target consistent, not bursty Yes Provisioned (PTU) reserved capacity, billed idle No Standard (pay-per-token) best effort, bursty traffic Then pick the processing scope: Global, Data Zone, or Regional it sets where inference runs; data at rest stays in the designated geography

Cheat sheet

  • Foundry defines three assignable scopes: the resource, the project, and an individual agent
  • Connections are created at account level for shared dependencies and at project level for the resources that back a project's agents
  • Capability hosts enable Agent Service on the account and project, and they cannot be updated after they are set
  • Basic agent setup uses Microsoft-managed multitenant storage; standard setup pins agent state into your own single-tenant Cosmos DB, Storage, and AI Search
  • Pay-per-token standard deployments need no compute quota and bill on tokens; managed compute reserves accelerators and bills on uptime
  • Data at rest stays in the designated geography for every deployment type; the deployment type decides only where inferencing data is processed
  • Provisioned deployments buy reserved capacity in PTUs for lower and more consistent latency; standard deployments are best-effort
  • Batch deployment types trade real-time responsiveness for roughly half the cost and a separate enqueued-token quota
  • Instant access lets you call supported models by name with no deployment created at all
  • A standard deployment's version upgrade policy decides whether it follows the new default, waits for expiry, or stops serving at retirement
  • Which deployment options exist is a property of the chosen model, and partner or community models add an Azure Marketplace purchase step
  • Prompt agents are declarative definitions the service runs; hosted agents are your own container image on Microsoft-managed infrastructure
  • Agents are identified by name plus version, and creating a version is what provisions a hosted agent's runtime
  • Structured inputs override fixed tool configuration per request without creating a new agent version
  • Publishing an agent requires at least Foundry Project Manager at the Foundry resource scope
  • Standard agent setup provisions several Cosmos DB containers with per-container throughput requirements, and undersizing fails capability host provisioning
  • Standard setup requires account-level control-plane roles plus granular data-plane roles on each dependency for the project managed identity
  • Outbound isolation injects the agent client into a delegated subnet, and that outbound configuration cannot be changed after deployment
  • Private endpoints to the bring-your-own Search, Storage, and Cosmos DB dependencies are not created for you
  • Govern AI workloads with one inherited baseline policy assignment plus narrower scoped assignments and exemptions, never duplicated policy sets or a loosened global control
  • Retry with backoff only covers transient faults; surviving a dependency outage needs a cached last-known-good copy plus a circuit breaker
  • azd pipeline config wires the repository to Azure and GitHub Actions signs in with OpenID Connect rather than a stored secret
  • Foundry evaluation runs as an automated quality gate inside the pipeline and fails the job when scores fall below a threshold
  • Bicep and ARM templates provision the whole Foundry footprint, including existing resource IDs for bring-your-own dependencies
  • Multi-agent releases validate version compatibility and roll out progressively with rollback triggered by quality-regression metrics
  • IaC sets desired state at deploy time; a deployIfNotExists policy plus a remediation task is what re-fixes resources that drifted afterwards

Unlock with Premium — includes all practice exams and the complete study guide.

References

  1. Set up standard agent resources for Foundry Agent Service
  2. Role-based access control for Microsoft Foundry
  3. Capability hosts for Foundry Agent Service
  4. Set up private networking for Foundry Agent Service
  5. Use your own resources in the Foundry Agent Service
  6. Deployment overview for Microsoft Foundry Models
  7. Understanding deployment types in Microsoft Foundry Models
  8. Foundry Models from partners and community
  9. Model versioning in Microsoft Foundry Models
  10. Hosted agents in Foundry Agent Service
  11. Customize agent behavior at runtime with structured inputs
  12. Explore Azure Developer CLI support for CI/CD pipelines
  13. How to run an evaluation in GitHub Action
  14. Observability in generative AI
  15. Implement CI/CD pipelines for multi-agent systems with GitHub Actions
  16. Understand scope in Azure Policy
  17. Details of the policy exemption structure
  18. Azure Policy definitions effect basics
  19. Remediate non-compliant resources with Azure Policy
  20. Retry pattern Well-Architected
  21. Cache-Aside pattern Well-Architected
  22. Circuit Breaker pattern Well-Architected