AB-620 Cheat Sheet

Power Platform connectors wrap APIs as typed operations and are the preferred low-code integration when the target service and required action are already available. A connector can be used as an agent tool, inside a topic, or as an action in an agent flow.

Agent flows follow rule-based paths and can combine connector actions, conditions, loops, and data operations. Choose a flow when an enterprise operation requires orchestration across several steps rather than one tool call.

Trap Adding every connector action separately as agent-level tools, which leaves the generative orchestrator to reconstruct a fixed business transaction at runtime.

A REST API tool exposes selected endpoints and methods to the agent as named capabilities. It is a better reusable integration boundary than embedding a one-off HTTP call in every conversational branch.

A Model Context Protocol connection makes an MCP server's tools and resources available through one agent integration. Select it when the enterprise system already exposes MCP and the server should govern its capability catalog.

Knowledge sources ground generated answers, while tools and flow actions retrieve operational data or change external systems. An integration that creates, updates, or executes a business transaction belongs behind a tool or flow rather than a knowledge source.

Integration planning must review throughput, authentication, connector, flow, downstream API, and data-quality constraints before implementation. Model peak per-minute and per-hour traffic across the full path instead of relying only on aggregate volume.

For user-authenticated knowledge and tools, the agent uses the signed-in user's identity so results remain limited to content and operations that user can access. This is the correct pattern for permission-trimmed SharePoint, Dataverse, or connector data.

Maker-provided credentials make the tool operate with the maker's connection rather than each user's connection. Choose this only when the business resource is shared and authorization doesn't need to vary per caller.

Trap Using maker-provided credentials to avoid sign-in for a repository whose source permissions must be enforced per employee.

An agent configured for no authentication can't run tools with user credentials. Plan a supported authenticated channel and authentication mode before choosing integrations that require delegated user access.

Technical readiness includes confirming that required APIs exist and establishing their authentication modes, permissions, network reachability, and connector support. Validate these unknowns before committing the conversational design to an integration path.

Authenticate with Microsoft automatically uses Microsoft Entra ID for Teams, Power Apps, and Microsoft 365 Copilot and is the default for new agents. It doesn't provide the manual configuration needed to authenticate users on arbitrary non-Microsoft channels.

For an internal website or another channel outside the native Microsoft identity set, select Authenticate manually with Microsoft Entra ID. Turn on Require users to sign in when authentication must occur before the conversation begins.

Trap Selecting Authenticate with Microsoft because the users have Entra accounts, even though the deployment channel is a custom website.

Manual authentication can use Generic OAuth 2 for any identity provider that complies with OAuth 2. Use a Microsoft Entra ID service provider when access must be restricted specifically to users in the organization's Entra tenant.

No authentication allows anyone with access to the agent link to interact with it and disables user-credential tools. Use it only when anonymous access is intended and no capability depends on caller identity.

Authentication configuration changes take effect only after the agent is published. Include publishing and channel retesting in every identity-setting change plan.

To defer sign-in until a topic needs it, select Authenticate manually, clear Require users to sign in, and add an Authenticate node to that custom topic. The node creates success and failure paths; place logic that uses User.AccessToken under the success path and handle failed or canceled sign-in under the failure path.

Trap Skipping the Authenticate node and relying on the first use of User.AccessToken; that use can prompt sign-in, but it doesn't add the template's failure-handling path.

User.DisplayName and User.Id can be empty even when user authentication is configured, depending on the user's identity-provider configuration. Build topic logic to handle blank values and test it with a real user configured in the identity provider rather than treating either variable as guaranteed identity data.

Trap Use /debug set bot.UserID "" in the Test bot pane to simulate an arbitrary signed-in identity.

Manual Entra ID and Generic OAuth 2 authentication expose User.AccessToken for delegated calls. Never display, log, or place this token in user-visible message content.

Create a single-tenant app registration and add the Copilot Studio Web redirect URI, then grant delegated openid and profile permissions plus only the resource scopes the agent requires. In Settings > Security > Authentication, select Authenticate manually and prefer Microsoft Entra ID V2 with federated credentials, enter the app client ID, and register the issuer and value that Copilot Studio provides as a federated credential; use the client-secret provider only when federated credentials aren't available.

To prevent makers from publishing agents without authentication, block the Chat without Microsoft Entra ID authentication in Copilot Studio connector in a Power Platform data policy. In an affected environment, makers must use Authenticate with Microsoft or an allowed manual Microsoft Entra ID option; for a noncompliant agent, the violation makes the Publish button unavailable.

Trap Classifying the connector as Business, rather than blocking it, is sufficient to require authentication.

Connector and other authenticated tools default to end-user credentials. The user might need to create or authorize the corresponding service connection before the tool can run.

Choose end-user credentials when the target system must enforce each caller's permissions, and maker-provided credentials for a deliberately shared service identity. Agent authentication and tool authentication are related but separately configured controls.

Copilot Studio requires a supported authenticated channel before configuring a connector tool to use maker-provided credentials. Publishing and testing must verify the connection in the intended channel.

Trap Enabling web channel secured access alone and assuming that it signs the end user into the agent.

Connector single sign-on isn't supported when an agent uses custom Active Directory authentication and is deployed to Teams. In that combination, users authenticate to connector connections separately.

Copilot Studio can publish to Teams, Microsoft 365 Copilot, SharePoint, websites, mobile or custom apps, messaging platforms, and Azure Bot Service channels. Channel choice must account for where users work, how they authenticate, and which response features the host renders.

The prebuilt demo website is intended for teammates and stakeholders while the agent is being built. Deploy a live website integration for customer production use rather than sharing the demo URL publicly.

Trap Customizing the demo site's welcome message and treating the resulting URL as the production web deployment.

Channel support differs for Markdown, suggested actions, surveys, cards, welcome messages, and citations. Validate critical content on every target host instead of assuming the test panel's rendering is portable.

A mobile or custom native app connects through the custom application pattern and requires developer integration. It is distinct from embedding the agent in a live website.

Users can't engage with an agent until it has been published, and additional channels are configured after at least one publication. Publish again after content or configuration changes so connected channels receive the update.

Publishing applies the agent's latest version across all associated channels; separate content publication isn't performed per channel. Channel-specific connection and store-approval steps can still differ.

Publish and install the agent for yourself first, then validate the published experience before making it available to others or submitting it for organization approval. This separates technical publication from broad Teams app distribution.

Trap Submitting the agent to the organization app store as the first test of the published build.

The latest published content normally becomes available when a new session starts so active conversations aren't disrupted. Persistent channels can require Start over or a propagation delay before users see the new version.

Web channel security requires clients to present a Direct Line secret or a token generated from that secret. Requests without the configured security measure are rejected after the setting propagates.

Browser and mobile clients must receive a conversation token rather than the Direct Line secret. Keep the secret server-side because exposing it in client code compromises the agent channel.

Trap Hard-coding Secret 1 in the website's JavaScript because the page is hosted on an internal domain.

Require secured access controls which client applications can connect to the agent, while the agent authentication setting establishes who the conversational user is. Use both when a custom web host and its individual employees must each be authenticated.

Copilot Studio provides two simultaneously valid secrets so clients can move to the alternate secret before the old one is regenerated. Regenerating a secret disconnects profiles using that secret or a token issued from it.

A Direct Line token is valid for only one conversation and has an expires_in lifetime. For a conversation that continues, refresh the token before expiration to obtain a new token for that same conversation; an expired token can't be refreshed.

Responsible AI planning requires transparency that users are interacting with an agent using generative AI. A Conversation Start message can provide this disclosure before generated questions or responses appear.

Accountability requires clear roles for monitoring behavior, approving sensitive uses, reviewing incidents, and updating mitigations. These responsibilities must include legal, privacy, risk, and content owners where applicable.

Design generated answers to retain source citations and monitor actions through activity records and audit logs. Traceability lets reviewers connect an output or action to the information and execution path that produced it.

Trap Relying on a disclaimer alone while removing citations and retaining no execution records.

Transparency includes explaining the agent's intended capability, AI use, and appropriate escalation or fallback behavior. This helps users understand when to verify information or seek human assistance.

For scenarios where unsupported answers create material risk, restrict retrieval to approved sources and design a not-found or escalation path. Prompt wording alone isn't an access control or grounding guarantee.

Responsible AI is an ongoing process: use representative scenarios, user feedback, monitoring, and recurring review to detect disparities and harmful outputs. Update data, instructions, filters, or escalation behavior when evidence reveals a problem.

Use an approval or explicit confirmation before an agent commits a sensitive transaction. Human review should occur before the external side effect, not merely as a notification after completion.

An escalation path should preserve relevant conversational context and clearly tell users when automation can't safely complete their request. Copilot Studio's Escalate system topic can route this behavior to a configured handoff product.

Establish feedback mechanisms for inaccuracies and review usage, outcomes, transcripts, and quality signals after release. Treat go-live as the start of iterative monitoring rather than the end of validation.

Trap Completing a one-time prelaunch review and assuming platform content filters remove the need for production monitoring.

Data policies classify and control connectors so organizational data isn't unintentionally combined with disallowed services. Copilot Studio agents are subject to data policy enforcement; a prior exemption isn't a valid design assumption.

Block connectors, channels, and settings that the project doesn't require, then allow only the approved runtime path. This reduces the capabilities available to makers and agents beyond the intended design.

Trap Documenting prohibited connectors in the project wiki while leaving them available in the environment.

Evaluate the agent, its flows, connector tools, knowledge sources, and channel settings together because a policy can block a dependency anywhere in the path. Revalidate after policy changes before publishing.

Environment data policies govern what integrations can be built or used, while authentication and source permissions determine what a particular user can access. A secure design needs both layers.

Restrict the environment to agent authors and just-in-time administrators through Microsoft Entra group-based access. Broad tenant access isn't necessary for building or operating one governed solution.

Use group teams in each Dataverse environment to manage maker and operator role assignments. This centralizes membership and supports least-privilege access as personnel change.

An environment strategy isolates authoring and validation from live users and data. Package agent components in solutions and promote tested versions instead of rebuilding directly in production.

Trap Using the default environment for all lifecycle stages because its components can still be added to a solution.

A maker who can edit an agent can add a Message node that exposes a secret environment-variable value. Restrict editor permissions even when the underlying secret is stored in Azure Key Vault.

Share an agent for chat when a user only needs to converse with it; this grants no authoring permission. Share for collaborative authoring only with individual organizational users who need to view, edit, configure, share, and publish the agent, and ensure each coauthor has the Environment Maker security role.

Configure user identity, tool credentials, source permissions, and connector access so the agent can retrieve or change only what the scenario requires. A broad maker connection shouldn't silently become the authorization boundary for all users.

Evaluate runtime protection, authentication, sharing, data policy, source access, regional data handling, and compliance requirements as a release gate. Publishing successfully doesn't prove these controls meet project policy.

Regular monitoring and detailed audit logs are needed to detect incidents and trace data access or modifications. Define who reviews the evidence and how findings are escalated.

Trap Using aggregate Analytics charts as the sole audit record for individual sensitive transactions.

Test signed-in users with different source permissions to confirm the agent doesn't surface inaccessible content. Maker-only testing can hide an overprivileged connection or incorrect credential mode.

Assign the agent-specific Analytics Viewer role for read-only access to the Analytics page without edit or share permissions. If an Analytics drill-down exposes session-level data or conversation transcripts, the user also needs the environment-level Bot Transcript Viewer security role.

Solutions provide versioning, export, import, customization, and lifecycle management for agents and related components. Include required flows, connectors, prompts, and configuration references so the package can move between environments.

Although an agent-compatible flow can be stored in the Default Solution, a project solution gives explicit ownership and a controlled component boundary for export and promotion. Plan reusable assets in that project package from the start.

Trap Leaving shared flows only in the Default Solution because Copilot Studio can discover them there.

Treat flow, topic, prompt, and connector parameter names and types as contracts used by other components. Test dependents before changing those contracts and promote compatible versions together.

A Redirect node calls another topic and normally returns to the originating topic when the child finishes. Define topic inputs and outputs to pass a clear data contract across this boundary.

A published agent flow can be added as a tool and reused wherever its trigger and response contract applies. Keep fixed transaction logic in the flow instead of duplicating connector steps in multiple topics.

Trap Copying the same series of connector nodes into each topic to preserve explicit execution order.

An agent-level tool is available to the orchestrator across the agent, while a tool added within a topic is constrained to that authored path. Choose the narrowest scope that supports the reuse requirement.

Generative orchestration relies heavily on descriptions to select topics and tools. Overlapping descriptions can cause multiple modules to be invoked, so state each module's capability and exclusions precisely.

Unlike ordinary environment variables, a secret environment variable is read at runtime, so changing its Key Vault-backed value doesn't require republishing the agent. Access still depends on the required Key Vault roles and allowed environment or agent tags.

Global variables are writable conversation values available to all topics and persist only for the user session. Environment variables are the correct reusable mechanism for deployment settings shared through ALM.

An internal agent should authenticate employees and use their Entra identity for permission-aware resources. Native Microsoft channels can use Authenticate with Microsoft; an internal custom website requires manual Entra authentication.

Teams and Microsoft 365 Copilot support personal installation, shared-user availability, and organization app-store approval. Use staged distribution rather than exposing an internal agent through an anonymous public link.

Trap Using No authentication on a custom website because the website itself is reachable only from the corporate network.

Define employee roles, channels, multilingual needs, expected inputs, and fallback behavior during planning. Internal access alone doesn't make a customer-oriented conversation design suitable for employees.

An external agent can be anonymous only when public access is intentional and no capability requires caller-specific authorization. Use manual Entra ID or Generic OAuth 2 when external users must establish identity.

Document how unmatched, unsafe, and unsupported requests behave on every external channel. Provide a clear recovery or human handoff path rather than exposing implementation errors or repeatedly retrying.

Trap Using the generic On Error message as the customer-service escalation mechanism.

A flow used as an agent tool must use the When an agent calls the flow trigger. Creating a new agent flow as a tool in Copilot Studio supplies this trigger automatically.

An agent-callable flow requires a Respond to the agent action that defines its output contract. Add outputs there rather than expecting the agent to inspect intermediate action results.

The flow must be published before it can be selected and used as an agent tool. Saving a draft isn't sufficient for runtime availability.

Trap Publishing only the agent after editing a draft flow and expecting the tool definition to update.

Define trigger inputs, add the required connector and control actions, then map final values to Respond to the agent outputs. This keeps the agent-facing contract distinct from internal flow implementation.

Agent flows are deterministic: they execute authored actions along rule-based paths. Use them for processes whose sequence and branching must be predictable for the same inputs.

Copilot Studio can generate an agent flow from a description or makers can build it directly in the visual designer. In either path, inspect triggers, actions, conditions, and parameter mappings before publishing.

Place business-rule branches in the flow when outcomes must be controlled by known values. Don't rely on the language model to infer a fixed transactional branch that the flow can evaluate deterministically.

Trap Returning all raw records to the agent and asking the orchestrator to decide which transaction branch should run.

Every executed agent-flow action consumes capacity and adds latency. Keep the synchronous path focused on work needed to produce the agent response.

In an environment running on the new infrastructure, turn on Asynchronous response in the Respond to the agent action when a flow must continue beyond the normal two-minute limit and return its result after completion. The callback is fully supported in Microsoft Teams, but isn't supported in Microsoft 365 Copilot or telephony channels.

Trap Turn on Asynchronous response when the requirement is an immediate acknowledgement rather than a result callback after the flow finishes.

To be available to an agent, an existing flow must be in a solution in the same Power Platform environment. Add it to a solution after replacing incompatible trigger and response actions.

Synced Copilot connectors copy and semantically index non-Microsoft enterprise content into Microsoft Graph, making them suitable for broad searchable grounding with citations across Microsoft experiences.

Trap Use a Power Platform connector when the requirement is a reusable semantic index shared with Microsoft Search.

A tenant administrator must configure the Copilot connector before a maker can select it from Copilot Studio; unavailable configured sources can be sought under Add knowledge > Advanced.

Copilot connectors honor source-level access control lists so an agent surfaces only indexed items the requesting user is authorized to view.

From an agent Overview, Knowledge page, or generative answers node properties, select the configured Copilot connector and connection, then choose Add to agent.

Agents published to channels with Copilot connector knowledge require manual authentication that includes the ExternalItem.Read.All scope.

Trap Rely on the maker's connector sign-in as the published channel's end-user authorization.

Turn on Work IQ and configure authentication when using tenant graph grounding; missing grounding settings or scopes can explain a connector that works elsewhere but not in Copilot Studio.

Use a Copilot connector for large bodies of external documents, tickets, wikis, or knowledge articles that benefit from semantic ranking and indexed discovery.

When a requirement is to create or update records, use a Power Platform connector as a tool or action rather than a Copilot connector knowledge index.

Trap Index the transactional system with a Copilot connector and expect grounded answers to perform writes.

One agent can use Copilot connectors for evergreen indexed content and Power Platform connectors for current facts or transactional operations.

Copilot connector retrieval is served from a Microsoft Graph index, so choose a runtime Power Platform connection instead when every answer must reflect the source system at request time.

Select a synced Copilot connector when enterprise content should be semantically retrieved with citations and reused by Microsoft Search or Microsoft 365 Copilot, rather than queried only through live API calls.

Power Platform real-time knowledge indexes only metadata such as table and column names; each request queries the target system at runtime without moving its records into Microsoft 365.

Runtime knowledge calls use the requesting user's authentication token, preserving access controls configured in the enterprise source.

After signing in and choosing a target location, select the tables, name the knowledge source, and wait for metadata indexing to reach Ready before testing it.

Copilot Studio uses the same Power Platform connection for real-time knowledge and actions, so Power Platform data policies govern both uses.

On the agent Tools page, select Add a tool > Connector, choose a service operation, create or select its connection, and add and configure the operation.

A topic can invoke a connector deterministically through Add node > Add a tool > Connector, rather than leaving selection to generative orchestration.

Connector tools use end-user credentials by default, requiring users to authenticate to the associated service when the tool runs.

To run a connector with maker-provided credentials, configure an authenticated channel and set Credentials to use to Maker-provided credentials on the connector tool.

Trap Enable anonymous access and assume it protects a shared maker connection.

Set Ask the end user before running on the connector tool when execution requires explicit user confirmation; this option is No by default.

Under After running, choose Don't respond, Write the response with generative AI, Send specific response, or Send an adaptive card according to how the result should enter the conversation. Make only the output variables needed by the agent or downstream tools available.

Choose a prebuilt Power Platform connector when its operations cover the service because it provides built-in connections and prebuilt tools and triggers without requiring you to define a custom connector.

Power Platform connectors are reusable API wrappers that can be called at agent level, inside topics, or as actions in agent flows.

When integration requires a predefined sequence of connector actions, pass inputs to an agent flow and consume its outputs instead of exposing every operation independently to orchestration.

Keep each connector payload within the Copilot Studio limit for the deployment plan: 5 MB for public cloud plans and 450 KB for Government Community Cloud (GCC) plans.

Trap Apply the separate 512 MB file-upload limit to connector payloads.

Add Azure AI Search from Add knowledge and create a supported data connection; do not manually store an endpoint and key outside the connection workflow.

An Azure AI Search connection can use an access key, client certificate, service principal, or Microsoft Entra ID Integrated authentication, with details supplied for the chosen method.

After creating the Azure AI Search connection, specify the vector index; Copilot Studio permits one vector index for that added connection.

Azure AI Search data connections are managed at environment level, so a malformed connection can affect the connection dialog for multiple agents in that environment.

Copilot Studio supports vector indexes created with Azure AI Search integrated vectorization, which uses the embedding model associated with the index to vectorize incoming prompts at runtime.

If semantic ranking is required, enable and configure semantic ranker in Azure AI Search before adding the connection in Copilot Studio.

Copilot Studio can reach an Azure AI Search index through a private endpoint when virtual network support is configured for both the search service and the Power Platform environment.

The added Azure AI Search source should reach Ready before retrieval is tested; its own documentation page shows an In progress status while metadata is indexed, though the knowledge testing page scopes In progress to file upload and Dataverse sources - the Ready requirement is the settled rule.

For Azure AI Search citations, include metadata_storage_path or another index field containing a complete document URL; Copilot Studio interprets that value as the citation target.

Citation URLs do not grant access, so users still need permission to the website or repository referenced by the search index.

Add an Azure AI Search index as a knowledge source when the goal is document retrieval and grounded synthesis, not as a REST tool for performing business operations.

Trap Import the search service's REST endpoints as actions and expect generative answers to ground themselves automatically.

Computer use drives websites and Windows desktop applications with a virtual mouse and keyboard, making it appropriate for UI-based tasks when no direct API is available.

An agent can use the computer use tool only when generative orchestration is turned on.

A computer use tool requires a name, a routing description, an execution model, and natural-language instructions that identify the task, URLs, and application names.

Define inputs for values that change on each run; the tool combines those input values with its stable instructions during execution.

Use exact application names and full website URLs, and state consequential actions such as submitting a form or sending an email explicitly. Break complex interface work into clear steps and format longer tasks as an ordered list.

Use the computer-use Test experience to inspect the live reasoning and action log alongside the machine preview. Select Stop testing to halt incorrect or unsafe machine actions immediately, then refine the instructions before publishing.

The hosted browser needs no machine setup but is not Entra joined or Intune managed and does not support enterprise resources or custom desktop applications.

A Windows 365 for Agents Cloud PC pool supplies Microsoft-hosted, autoscaling machines that are Microsoft Entra joined and Intune enrolled without requiring the maker to maintain machines.

Trap Choose the hosted browser when organization policy requires Entra join, Intune enrollment, and custom Windows applications.

For a customer-managed Windows machine, install the supported Power Automate for desktop runtime, register the machine to the environment, and turn on Enable for computer use in its settings.

Requests targeting an occupied individual machine are queued and execute sequentially; monitor them from the machine's Run queue in Power Automate.

Run computer use on a dedicated machine with an account granted only the permissions required for its tasks. Restrict web access to vetted domains and install or allow only the desktop applications essential to the automation.

Select a computer use action in a run's Activity map to open session details containing replay screenshots, action coordinates and timestamps, inputs, machine information, and accessed applications.

The run Transcript view shows how the tool followed its instructions, including reasoning messages and screenshots for each action.

Computer use access control blocks interaction with websites or desktop applications outside its allow list, although it does not prevent those sites or applications from opening.

The Test panel's real-time activity map shows a run while it executes, whereas the Activity page provides historical maps and transcripts for past runs. On the Activity page, narrow sessions with status filters such as Failed, Blocked, In progress, Waiting for user, or Completed before opening a run's evidence.

Use maker-provided credentials for autonomous shared execution, recognizing that anyone using the shared agent can act with the maker's access on the configured machine. Use end-user credentials when each interacting user should run with their own machine access; every user must then have credentials for that machine.

Configure Human supervision with an authorized reviewer who has the context and activity visibility needed to assess potentially harmful instructions. If no response arrives within the configured response time limit, the request expires and the computer-use run stops.

Define website and desktop sign-in credentials under Stored credentials, using encrypted Power Platform internal storage or an Azure Key Vault secret for the password. Keep passwords out of natural-language tool instructions.

Turn on Enforce HTTPS when computer use must interact only with URLs that start with https://; with this setting enabled, the tool doesn't interact with HTTP sites.

On the agent Tools page, select Add a tool > Model Context Protocol, choose the MCP connector, authorize its connection, and select Add and configure.

Trap Upload MCP server documentation as knowledge and expect its operations to become callable.

Copilot Studio requires generative orchestration to use tools and resources exposed by an MCP server.

MCP connectivity relies on Power Platform connectors, so connector data policies can regulate access to an MCP server and its published capabilities.

When an MCP server requires authentication, authorize or create its connection while adding the server; merely publishing the agent does not establish credentials.

Although MCP can define resources, tools, and prompts, Copilot Studio currently consumes MCP tools and resources from connected servers.

The MCP server supplies names, descriptions, inputs, and outputs, and Copilot Studio dynamically reflects tools and resources that the server adds, updates, or removes.

For a Copilot Studio agent to use an MCP resource, the server owner must configure that resource as an output of an MCP tool.

All server tools start enabled; turn off Allow all to expose individual toggles, and note that later-added server tools then default to off.

Use MCP when an existing server publishes a changing collection of model-callable tools or contextual resources that the agent should discover through a standard protocol.

Use A2A for multiturn task delegation to an external reasoning agent; use MCP for invoking server-provided tools and reading resources.

Trap Model a specialist external agent as a transactional MCP tool when full conversation history and agent reasoning must be preserved.

When the external dependency is a conventional API described by OpenAPI rather than an MCP server, add REST API tools or a custom connector instead.

One connected MCP server can manage multiple tools and resources, avoiding separate static imports for every operation while retaining centralized server metadata.

A single-response evaluation sends one unconnected question at a time rather than continuing a conversation. Use it to test specific answers, capability selection, or required wording without carryover context.

Trap Use one long test-chat transcript when every question must begin without context from the previous question.

Quick question set uses the agent description, instructions, and capabilities to generate 10 questions. It is suited to a small fast evaluation or as the seed for a larger curated set.

Trap Choose Full question set when the requirement is specifically to generate the fixed ten-question starter set without selecting knowledge or topics.

A single-response import file must be CSV or text with Question and Expected response headings in that order; the expected response can be omitted during import. An expected response must later be supplied for methods that compare the answer with an expected result.

From the Analytics page, evaluate a selected theme to create cases from real user questions associated with that theme. Use the resulting set to track a focused area of the agent, such as billing, separately from other production use cases.

Full question set generates a chosen number of cases from either one selected knowledge source or the agent's topics. Choose Knowledge for a generatively orchestrated agent and Topics for a classically orchestrated agent.

AI test-case generation can fail when generated questions violate the agent's content-moderation settings. Before regenerating, adjust the selected knowledge source, the agent instructions, or the moderation settings that caused the flagged content.

Conversational evaluation models an ongoing interaction in which each response can depend on previous context. Choose it to assess clarification behavior, context retention, and completion of multistep tasks.

Trap Choose independent single-response cases to validate whether an agent carries information through a six-turn service workflow.

Quick conversation set generates 10 short conversations from the agent description, instructions, and capabilities, while Full conversation set can generate short or long conversations from knowledge or defined topics. Use your test chat instead when the latest authored interaction itself is the scenario to preserve.

Generating test cases from an existing knowledge source is useful for testing how the agent uses that source. It is not a sound method for discovering information absent from the selected knowledge source.

Trap Generate every case only from the current knowledge source when the test objective is to discover missing policy coverage.

Automated testing uses the selected test account to connect to the agent's knowledge sources and tools. Select an account with the intended permissions when responses or capability access vary by user.

Trap Use the maker's broad production connections to represent a restricted employee persona whose permitted sources differ.

When Copilot Studio generates cases, it can use the connected account's credentials to access agent knowledge and tools. Generated cases can therefore contain sensitive data that account can access, and makers with agent access can view the linked test sets.

A stable test set provides the same questions and scoring standard across agent revisions. Reusing it isolates changes in agent performance more reliably than ad hoc manual prompts.

Trap Replace all cases after every agent edit and then treat the resulting pass rates as a direct regression comparison.

Test chat provides interactive control over a live session, while agent evaluation runs repeatable sets and can simulate selected profiles. Use both when a release needs conversational exploration as well as comparable regression evidence.

In addition to the Copilot Studio interface, evaluations can be triggered through the Power Platform API or Copilot Studio connectors used in tools and automation flows. These interfaces support incorporating repeatable agent tests into CI/CD workflows.

General quality uses an LLM to assess relevance, groundedness, completeness, and whether the agent attempted an answer. A response must meet all key criteria to be considered high quality.

General quality is appropriate when there is no single exact answer and does not require expected answers in the test cases. It is available for both single-response and conversational sets.

Trap Choose Exact match for an open-ended grounded explanation that can be phrased correctly in many ways.

Compare meaning scores how closely the ideas and intent of an actual response align with the expected response rather than requiring identical wording. It is a single-response method with a configurable passing threshold, whose default is 50.

Trap Choose Text similarity solely because two semantically equivalent policy answers use different sentence structure.

Every case assessed with Compare meaning needs an expected answer. A case without one produces an Invalid result for that method rather than a semantic pass or fail.

For a multilingual agent, select the language in which to run the evaluation. If Compare meaning detects that the expected and actual responses are inconsistent in language, it fails the response for the language mismatch.

Exact match passes only when the actual answer exactly equals the expected response, character for character and word for word. Use it for short fixed outputs such as codes, numbers, or mandated phrases, not naturally variable prose.

Text similarity uses a cosine-similarity metric to score how closely actual wording and construction match the expected response. Choose it when near-exact legal or prescribed language matters but minor differences need not force the binary outcome of Exact match.

Trap Use Compare meaning alone when the acceptance criterion requires a generated notice to preserve prescribed sentence structure.

Keyword match passes according to whether the response includes Any or All configured words or phrases. It verifies token presence, not semantic equivalence or similarity of the complete answer's construction.

Exact match and Text similarity require an expected response, while Keyword match requires expected keywords or phrases. Missing the applicable expected content makes that case Invalid for the selected method.

Text similarity returns a continuous cosine-similarity score, and you configure the method's passing-score threshold to convert that score into a pass or failure. Microsoft publishes that score's scale both ways — out of 100% in the method table, 0 to 1 in the method's own section — so do not rely on either unit.

The Tool use method passes when a test case used the configured expected tools or topics and fails when it did not. Use it to validate orchestration choices independently of whether the final wording resembles a reference answer. Microsoft's method table lists it for single response sets while the conversation test set procedure also offers it, so do not rely on either scope.

Trap Choose General quality alone when the acceptance condition is that a refund action must invoke a specific authenticated tool.

A Custom method uses goal-oriented evaluation instructions and two or more labels, each assigned a Pass or Fail outcome. Use it for organization-specific criteria such as policy compliance that built-in graders do not directly express.

A test set can evaluate the same agent interactions with multiple methods in one run. Combine orthogonal checks, such as semantic correctness and required tool use, when passing one dimension does not prove the other.

General quality uses an LLM to evaluate relevance, groundedness, completeness, and abstention, and it is available for both single-response and conversation test sets. Use it when no exact answer is expected; this method does not require expected answers in the test cases.

Trap Choose Text similarity for open-ended answers with many valid phrasings because its score accepts meaning-preserving rewrites.

Agent evaluation measures correctness and performance, so even an agent that passes every evaluation can still produce unsafe or inappropriate content. Retain responsible AI reviews and content-safety filters because evaluation doesn't replace them.

In a single-response test set, use the Edit icon in the Tools column heading to assign the same expected tools and topics to every test case. Use the Select tools panel on an individual case when its expected capabilities differ.