Domain 2 of 3 · Chapter 15 of 15

Monitor agents by using Application Insights

Unlock the complete study guide + 1,040 practice questions across 16 full exams.

Bundled into the existing Designing and Building Integrated AI Agent Solutions in Copilot Studio premium course — no separate purchase.

14-day money-back guarantee — no questions asked.

Included in this chapter:

  • Where agent telemetry goes when it leaves Copilot Studio
  • Connect one agent: the connection string is the switch
  • What each of the four logging settings adds
  • Environment-level telemetry and its span model
  • Reading agent-level telemetry with Kusto
  • The Copilot Studio Dashboard workbook
  • Once the data lands, the privacy problem is yours
  • Blocking agent-level telemetry with a data policy
  • Exam patterns and the traps behind them

Microsoft's comparison of the two Copilot Studio telemetry scopes

AreaAgent-level telemetryEnvironment-level telemetry
Configuration scopeConfigured separately for each agentConfigured once for the environment
Configuration ownerMaker or developer responsible for the agentAdministrator, platform team, tenant admin, or Center of Excellence team
Privilege levelRequires access to the specific agent configurationRequires tenant-level Power Platform administrator or Dynamics 365 administrator privileges, plus an administrator role in the Dataverse environment
Primary audienceMakers and development teams that need agent-specific diagnosticsAdministrators and platform teams responsible for governance, operations, and centralized observability
Operating modelDecentralized diagnostics per agent or solution teamCentralized observability across agents in the environment
Telemetry modelEvent-based telemetryTrace or span-based telemetry aligned with OpenTelemetry concepts
Best suited forMessage activity, topic events, custom events, and single-agent troubleshootingAgent invocations, tool execution, outputs, dependencies, and cross-agent monitoring
Primary Application Insights tablescustomEventsdependencies
OpenTelemetry alignmentNoYes

Decision tree

Must the record leaveCopilot Studio?no, the product surfaces answer ityes, it must be exportedIn-product surfacesagent Activity page,agent Analytics pageOne agent's diagnostics,or the whole environment?one agent (maker)every agent (admin or CoE)Connection stringSettings > AdvancedAdmin export packagemanaged environments onlyAgent-level telemetry:what must it capture?Also need an eventper topic node?message text and identitysensitive valueswhere in a topic it stoppedyesnoEnable logging+ Log conversation detailsmessage text, user ID,user nameLog sensitiveActivity propertiesseparate opt-in, notimplied by the othersNode execution eventsone event each timea topic node runsAgent-leveltelemetry as wellspans omit topicevents entirelyEnvironment-leveltelemetrydependencies table,OpenTelemetry spans
Cheat sheet

Cheat sheet

  • Connect telemetry with the Application Insights connection string
  • Enable message and event telemetry explicitly
  • Control capture of conversation and tool details
  • Enable node execution events for topic tracing
  • Opt in separately before logging sensitive Activity properties
  • Query agent telemetry with Kusto
  • Filter test-canvas telemetry with designMode
  • Open the Copilot Studio monitoring workbook
  • Grant readers access to shared monitoring workbooks
  • Do not trust user counts from unstable anonymous IDs
  • Extend the monitoring workbook with KQL tiles
  • Recognize data-policy blocking of telemetry

Unlock with Premium — includes all practice exams and the complete study guide.

Also tested in

References

  1. Analytics overview - Microsoft Copilot Studio
  2. Telemetry with Application Insights overview - Microsoft Copilot Studio
  3. Environment-level telemetry with Application Insights (preview) - Microsoft Copilot Studio
  4. Agent-level telemetry with Application Insights - Microsoft Copilot Studio
  5. Connection strings in Application Insights - Azure Monitor
  6. Generative answers pointing to SharePoint sources don't return results - Copilot Studio
  7. Azure built-in roles for General - Azure RBAC
  8. Manage personal data in Azure Monitor Logs - Azure Monitor
  9. Monitor operations, compliance, and capacity - Microsoft Copilot Studio