Domain 3 of 3 · Chapter 3 of 7

Review test results

Unlock the complete study guide + 1,040 practice questions across 16 full exams.

Bundled into the existing Designing and Building Integrated AI Agent Solutions in Copilot Studio premium course — no separate purchase.

14-day money-back guarantee — no questions asked.

Included in this chapter:

  • What a finished evaluation run hands you
  • Read the set score before you open a single case
  • Inside one case: the detailed assessment
  • Rule out the run before you blame the agent
  • Which failures to open first
  • The five questions that test the test
  • Multi-turn cases fail late and break early
  • Telling a real change from grader noise
  • Checking the instrument that produced the score
  • From scores to a readiness decision
  • What the CSV export preserves
  • Who can see an evaluation run

Where a failing test case's fault actually lives, and the order you rule the homes out

Fault homeWhat the run looks likeThe check that identifies itWhere you actHow you verify the fix
Dependency health during the runFailures with no consistent behavioral pattern, often clustered in time, sometimes arriving as Error rather than FailPre-triage check: knowledge accessible and fully indexed, no connector or API errors, timeouts or rate limits, authentication valid throughout, evaluation environment matching the intended configurationThe dependency itself, outside the Evaluation pageRerun the evaluation before triaging anything else; results from an unhealthy run cannot be trusted
Evaluation setupThe agent's answer looks acceptable to a human reader, or the case carries no usable expectation and returns InvalidStep 1's five ordered questions: is the response acceptable, is the expected answer current, is the input realistic, does the grader allow valid alternatives, is the method right for the signalThe test set on the Evaluation page — the expected value, the keywords, or the test methodRerun the affected cases and confirm the remaining failures now reflect genuine agent behavior
Agent configurationThe evaluation is valid and the agent genuinely produced a wrong, ungrounded, misrouted, or unsafe responseStep 2's diagnostic questions per quality signal — which source was retrieved, which tool fired, which topic triggered, whether it escalatedInstructions, knowledge, tools, or topics — the agent's own configuration pagesRerun the affected test set, and spot-check an unrelated set for regressions; after a system prompt change, rerun the full suite
Platform limitationThe same failure survives multiple reasonable configuration variationsStep 3's indicators — retrieval consistently returns the wrong documents, the required reasoning or orchestration pattern is unavailable, the grader misclassifies despite rubric tuningA workaround, plus documentation and escalation with evidenceMark the case as a known limitation or adjust thresholds so it stops blocking unrelated progress, and re-evaluate when platform capabilities change

Decision tree

Is the result state Invalid or Errorrather than Fail?Not an agent defect: fix the case's expectation,or the run that produced no gradable responseYesWas any dependency unhealthyduring the run?Correct the dependency and rerun the evaluationbefore triaging any caseYesNoWould a real user acceptthe agent's response?Evaluation setup: the grader or theexpected value is wrongYesNoIs the expected value current, the inputrealistic, and the method right?Evaluation setup: fix the test set,then rerun the affected casesNoNoDoes the failure survive severalreasonable configuration changes?Platform limitation: work around, document,and escalate with evidenceYesYesAgent configuration: fix instructions, knowledge,tools or topics, then rerun the test setNo
Cheat sheet

Cheat sheet

  • Cases resolve to Pass, Fail, Invalid, or Error
  • Case details expose the evidence behind a score
  • The activity map reconstructs a test case's execution path
  • Result feedback rates the grader rather than the agent answer
  • Verify dependency health before diagnosing the agent
  • Validate the evaluation setup before changing the agent
  • Prioritize failures by consequence and systemic signal
  • Trace a multi-turn failure back to its first divergence
  • Sample passing cases to detect grader false positives
  • Run comparison requires two runs of the same test set
  • Comparison flags cases that changed pass state
  • Repeated case failures narrow remediation scope without proving cause
  • Set readiness thresholds by risk and quality signal
  • Baseline LLM evaluations with a three-run average
  • Interpret LLM grader variance before diagnosing a regression
  • CSV export preserves case inputs, outcomes, and analysis
  • Agent viewer grants evaluation access without authoring access
  • Restrict response evidence to the maker who initiated the run

Unlock with Premium — includes all practice exams and the complete study guide.

Also tested in

References

  1. About agent evaluation - Microsoft Copilot Studio
  2. Run evaluations and view results - Microsoft Copilot Studio
  3. Choose evaluation methods - Microsoft Copilot Studio
  4. Interpret evaluation scores and assess readiness - Microsoft Copilot Studio
  5. Improve agents using evaluation-driven triage and remediation - Microsoft Copilot Studio
  6. Triage agent failures: Diagnose and prioritize issues - Microsoft Copilot Studio
  7. Review agent activity - Microsoft Copilot Studio
  8. Create a conversational test set - Microsoft Copilot Studio
  9. Review the agent evaluation checklist - Microsoft Learn
  10. Map failure patterns to remediation strategies - Microsoft Copilot Studio
  11. Share agents with other users - Microsoft Copilot Studio