Domain 2 of 6 · Chapter 2 of 3

Analyze agent failures and identify their root causes

Unlock the complete study guide.

Bundled into the existing Developing in Agentic AI Systems premium course — no separate purchase.

14-day money-back guarantee — no questions asked.

Included in this chapter:

  • The surfaces that record what an agent was given and did
  • Reading a workflow run as failure evidence
  • Keeping workflow evidence usable
  • Upgrade records: what was analysed, what was decided, what was executed
  • Configuration gates that impersonate agent failure
  • Telling the tool, context, and reasoning layers apart
  • Applied reasoning: from symptom to root cause without over-claiming

Which surface answers which question

SymptomSurface to openWhat it can establishWhat it cannot establish
A pull request exists, but it is unclear whether a security finding was handledCloud agent session logThe analysis the agent performed and the actions it tookThat the change is correct — the log records activity, not quality
A pull request looks stalled after a 'Copilot started work' eventView session on that eventWhether the session is still streaming activityHow long the remaining work will take
One agent task needs watching from the terminal`gh agent-task view --log --follow`That task's session log as it is produced, by session ID or PRA cross-task overview — listing tasks is a different question
Copilot ignores the workspace and answers genericallyChat Debug view — Context sectionWhether the expected repository files actually reached the modelWhy they are missing: this section shows the absence, not its cause
Custom instructions appear to be ignoredChat Debug view — System prompt sectionWhich instructions and available tools were actually suppliedThat a configuration file on disk was the source of them
A tool ran but the result is wrongChat Debug view — Tool responses sectionThe actual arguments sent and the output returnedWhether the service itself is healthy for other callers
A workflow step failed`gh run view --log-failed`, with `--job` and `--attempt`The failing step's output, for the right job and the right attemptWhich revision was tested — that is `--json headSha`
An interrupted GitHub Copilot upgrade stopped somewhere unknown`.github/upgrades/{scenarioId}/tasks.md`Live per-step status and the validation criteria for eachWhy a task failed internally — that is `progress-details.md`
The GitHub Copilot upgrade approach itself looks wrong`assessment.md`, then `plan.md` and `upgrade-options.md`What the analysis found, and confirmed decisions versus detailed strategyHow far execution actually got

Decision tree

Did a configuration gate stopthe action or input by design? Configuration gate: approval, trigger filter,write access, closed or merged pull request,firewall, secret scope, content exclusion yes no Was the expected tool actuallyavailable and invoked? Tool availability: in VS Code, check theChat Debug system prompt, then selection no yes Did the call carry the intended argumentsand return a conforming result? Tool call: misuse or service failure;for MCP, reported via isError or JSON-RPC no yes Did the intended context actuallyreach the model, and only that? Context: indexing, stale records,or injected instructions no yes Does the response misread arequirement present in its inputs? Reasoning: correct inputs,wrong interpretation or plan yes no No layer confirmed: narrow the causesand name the evidence that settles them
Cheat sheet

Cheat sheet

  • The session log records the cloud agent's security analysis and actions
  • View session reveals live agent activity behind an apparently idle pull request
  • The GitHub CLI can stream a selected agent-task session log
  • The CLI diagnose command analyzes the current session log
  • Upgrade tasks.md records live execution progress and validation criteria
  • Upgrade assessment.md records the analysis that precedes execution
  • Upgrade plan.md and upgrade-options.md answer different planning questions
  • Per-task upgrade records distinguish scope from execution details
  • Failed-step logs isolate workflow failure evidence
  • Run-attempt selection prevents confusion between original and retried execution
  • Workflow run metadata identifies the revision that produced a result
  • Diagnostic artifact retention cannot exceed the enclosing retention limit
  • Artifact digest validation can flag changed diagnostic evidence
  • Retrieving artifacts from another run requires explicit run identity and authorization
  • UNKNOWN STEP can reflect a log-association limitation rather than agent behavior
  • Reasoning errors are diagnosed against the inputs the agent actually received
  • Tool inputs and outputs distinguish tool misuse from a valid call that failed
  • The raw context reveals whether expected repository files reached the model
  • The system prompt reveals which agent instructions were actually supplied
  • A tool that was not invoked may be absent from the model's available tools
  • An MCP error result and a protocol error are different reporting channels
  • A declared MCP output schema provides a contract for checking returned structured data
  • A cloud-agent firewall warning identifies a network-policy configuration gate
  • Missing CI execution can be the expected workflow-approval gate
  • A reviewer's write access determines whether cloud-agent feedback is delivered
  • Closed or merged pull requests do not accept new cloud-agent follow-up work
  • An Actions secret does not make a credential available to a cloud-agent session
  • The errorOccurred payload reports where a runtime error occurred
  • Comparing documented intent with produced changes locates the failure point
  • Stale or contradictory task records require context reconciliation
  • Workflow events and branch filters must match the intended validation activity
  • Content-exclusion policy can intentionally remove expected CLI or app context
  • Behavior redirected by untrusted instructions warrants a prompt-injection diagnosis

Unlock with Premium — includes all practice exams and the complete study guide.

Also tested in