Domain 4 of 4 · Chapter 1 of 3

Root Cause Analysis

Four signals, four questions: the one model for picking telemetry

A dashboard spike, a single failed transaction, slow latency across services, a config that changed with no deploy: four symptoms, and AWS answers each with a different signal. Naming which signal a DVA-C02 troubleshooting stem is really asking for, before you touch any code, is the entire skill here. Everything later on the page is a deeper look at one of these four signals.

The single mental model: AWS root cause analysis reads four complementary signals, and each answers exactly one question. Match the symptom in the stem to the question, and the service falls out.

  • Amazon CloudWatch Logs answers "what exactly happened in this one request?", the line-level event record. A log group is the container for a resource's log streams (for example /aws/lambda/<function-name>); the SDK and managed services write timestamped text or JSON events into it. Reach for logs when you have a single failed transaction and need its exact error.
  • Amazon CloudWatch metrics answer "how often, how much, and is it trending?", numeric time series such as error count or p99 latency, on which you set alarms. A metric is a published time series identified by a namespace plus a set of dimensions[1]. Reach for metrics when a dashboard shows a spike but no one request is obviously at fault.
  • AWS X-Ray answers "where in the call chain did time or errors accumulate?", one request followed across every service it touches. Reach for it when end-to-end latency degrades across microservices but no single component looks broken.
  • AWS CloudTrail answers "who called which AWS API, when, and from where?", the control-plane audit trail. Reach for it when behavior changed without a deployment you initiated, or a call started being denied.

The DVA-C02 reflex is this mapping itself. A spike on a dashboard is a metrics question; a single failed transaction is a logs question; slow end-to-end latency across services is a trace (X-Ray) question; an unexplained configuration or permission change is a CloudTrail question. The most common exam time-waster, and the most common distractor, is reaching for the wrong signal, such as grepping raw logs to judge whether errors are trending (a metrics job) or treating CloudTrail as an application debug log (it records API calls, not your code's internal events). The mechanics sections below define each signal's searchable key; the decision flow at the end of the page turns this mapping into a branch you can walk under time pressure.

A symptom maps to one question, and the question names the signal CloudWatch Logs What happened in this one request? Symptom: a single failed transaction CloudWatch Metrics How often / how much / is it trending? Symptom: a spike on a dashboard / alarm AWS X-Ray Where in the call chain did time/errors build up? Symptom: slow latency across many services CloudTrail Who called which API, when, from where? Symptom: a config or permission changed Pick the signal whose question matches the symptom; reaching for the wrong one is the common time-waster.
The four-signal model: each AWS observability service answers one distinct RCA question.

Logs and metrics: querying events vs. measuring trends

This section explains the first two signals in depth, what Amazon CloudWatch Logs and Amazon CloudWatch metrics let you do once data is flowing, and the two tools that bridge them. It assumes only that your function or service is already writing to a CloudWatch log group.

CloudWatch Logs Insights, the reactive hunt. Once logs are centralized, CloudWatch Logs Insights[2] is an interactive query engine with a purpose-built query language (the fields, filter, stats, sort, and limit commands). You run it ad hoc during an investigation to find the exact log events behind one failure, and it can query up to a number of log groups at once[2]; a query that has not finished is stopped after it runs for 60 minutes[3]. Use Logs Insights when the question is "what was the error on request X?"

Metric filters, turning log text into a measurable trend. A metric filter[4] is the opposite of an ad hoc query: it is a standing rule that scans every incoming log event for a pattern (for example the term ERROR or a 5xx status) and emits a numeric CloudWatch metric you can then alarm on. So Logs Insights is the reactive hunt; a metric filter plus a CloudWatch alarm is the proactive trigger that pages you before you start hunting. This is the resolution of a tension the exam likes: if the stem asks you to spot a trend or rate, do not grep logs by hand, convert the log pattern into a metric with a metric filter, then alarm on the metric.

CloudWatch metrics, namespaces, and dimensions. A metric is a numeric time series. It is identified by a namespace (a container that isolates one service's metrics from another's, for example AWS/Lambda) plus up to a fixed number of dimensions (name/value pairs such as FunctionName=checkout that scope the series), as described in the CloudWatch concepts[1]. Dimensions are how you narrow a metric to one resource on a dashboard.

Reading the Lambda metrics, the triage that decides which layer to debug. For AWS Lambda the published metrics[5] split the problem before you read any code:

  • Errors counts invocations that ended in a function error, an unhandled exception or a timeout raised inside your handler. A rising Errors line points at your code.
  • Throttles counts invocations rejected because concurrency was exhausted. A throttle is an invocation error that happens around your code, surfaced as TooManyRequestsException / HTTP 429, and it is counted in Throttles, not Errors. The reconciliation to hold onto: a flat Errors line with rising Throttles means concurrency was exhausted (raise reserved or provisioned concurrency, or the account limit); it does NOT mean your handler is broken.
  • Duration measures execution time; a climbing Duration that nears the configured timeout is what eventually turns into Errors (timeouts) and is the metric to watch for performance regressions.

The rule that ties logs and metrics together: metrics tell you that something is wrong and bound the time window; logs (via Insights) then tell you exactly what failed inside that window.

CloudWatch Logs log group events reactive Logs Insights query fields/filter/stats; find one request Exact failure found "what happened in request X" proactive Metric filter pattern -> CloudWatch metric Alarm pages you before you start hunting Lambda: function errors -> Errors metric; throttling (429) -> separate Throttles metric, not Errors
Two paths out of CloudWatch Logs: a reactive Logs Insights query for one request, and a proactive metric filter plus alarm for trends.

X-Ray and CloudTrail: locating the slow hop vs. auditing the change

This section covers the remaining two signals, AWS X-Ray for distributed tracing and AWS CloudTrail for the API audit trail, and why neither substitutes for the other. It assumes you have a multi-service request path (the case where logs and metrics alone are not enough to localize the fault).

X-Ray: segments, subsegments, and the service map. X-Ray records each request as a segment, the work done by one service for that request, and breaks the calls that service makes to downstream resources into subsegments. Segments that share one trace ID are stitched into a service map, the visual graph of services and the edges between them, color-coded for errors, faults, and throttles[6]. The service map is what lets you see where in the call chain time or errors accumulate, rather than just that they did.

Annotations are indexed and filterable; metadata is not. To make traces searchable you attach annotations, key-value pairs that X-Ray indexes (up to 50 per trace) and that you can use in filter expressions and the GetTraceSummaries API[6]. Metadata is also key-value, but it is NOT indexed, so it rides along with the trace for context yet cannot be searched on. The exam-load distinction, stated where it bites: put the field you will filter or group by (such as userId or orderType) in an annotation; put bulky diagnostic detail in metadata. If a stem asks you to filter traces by a value and proposes recording it as metadata, that is the wrong answer, metadata is not searchable.

Default sampling, why not every request is traced. To control cost and overhead, X-Ray does not trace every request. The default sampling rule records the first request each second plus five percent of any additional requests[7]. The consequence for RCA: a low-volume intermittent error may simply not have been sampled, so absence of a trace is not proof the request succeeded, you raise the sampling rate (or add a targeted rule) when you need fuller capture.

CloudTrail: the API-level audit, for the question the others can't answer. When a resource changed or a call was denied at the control plane (a security group edited, an IAM role's permissions changed, an unexpected API call), none of logs, metrics, or traces records it, because they observe your application, not the AWS control plane. CloudTrail[8] is the audit trail that does: it records management-event API activity, who (the principal), what (the API call), when, and from where (source IP). The reconciliation the exam tests: CloudTrail is not an application debug log. It tells you that someone called ModifySecurityGroupRules at 14:02; it does not contain your function's internal log lines or request payloads, for those you read CloudWatch Logs. Use CloudTrail only for the "who or what changed it" question.

X-Ray: one trace ID across the call chain (the service map) Client Service A segment Service B subsegment Database subsegment Annotations = indexed + filterable | Metadata = NOT indexed Default sampling: first request/second + 5% of the rest CloudTrail (separate audit trail) who / what API / when / from where changed config
X-Ray stitches segments and subsegments into a service map by trace ID; CloudTrail separately audits the control-plane API call.

Correlating signals and exam-pattern recognition

This section is the page's payoff: how to combine the four signals into one diagnosis, and how to map recurring DVA-C02 stem cues to the right answer. Each rule restates a fact from the sections above (see those for the full explanation). The decision flow below turns this into one branch you can walk under time pressure: name the question, take its branch to the one signal, then converge on correlating the rest by timestamp.

Correlate, never trust one signal alone. The disciplined RCA loop is to align timestamps across signals: when a metric spikes, note the exact time window, then pull the matching X-Ray traces, the CloudWatch Logs lines, and any CloudTrail event inside that same window. A 5xx spike whose window lines up with a CloudTrail ModifySecurityGroupRules event is a configuration regression, not a code bug, a conclusion no single signal could reach.

HTTP status triage, whose fault is it. A 4xx generally means the caller (your application or the client) sent a bad request; a 5xx means the serving side faulted. On a multi-service path, narrow a 5xx by which hop reported it on the X-Ray service map before assuming the fault is yours.

The stem-cue table:

  • Cue: "a specific request failed," "one transaction returned the wrong result," "find the exact error."CloudWatch Logs Insights to pull that request's log lines (and its X-Ray trace if instrumented). Distractor that fails: CloudTrail, it audits AWS API calls, not your application's internal events.
  • Cue: "error rate / latency / throttles jumped on the dashboard," "is it trending," "alert me automatically."CloudWatch metrics with an alarm, and a metric filter if the signal lives only in log text. Distractor that fails: grepping raw logs by hand to judge a trend, slow and error-prone.
  • Cue: "end-to-end latency is high across microservices," "which service is slow," "trace the request path."AWS X-Ray service map and trace timeline. Distractor that fails: reading a single service's logs, it cannot show where across hops the time accumulated.
  • Cue: "need to filter/group traces by a field (userId, orderType)." → record that field as an X-Ray annotation (indexed). Distractor that fails: recording it as metadata, which is not indexed and cannot be searched.
  • Cue: "something changed with no deployment," "a call started being denied," "who edited this resource."CloudTrail management-event history (principal, API, time, source IP). Distractor that fails: CloudWatch Logs, it does not record control-plane API activity.
  • Cue: "Lambda throttling," "TooManyRequestsException," "429," "errors metric is flat but invocations are failing." → read the Throttles metric, not Errors; the cause is exhausted concurrency, raise reserved/provisioned concurrency or the account limit. Distractor that fails: debugging handler code because you looked at the Errors metric, which a throttle does not increment.
  • Cue: "unhandled exception / timeout inside the handler." → a function error, counted in the Errors metric; fix the code, and watch Duration if a climbing runtime is hitting the timeout. Distractor that fails: raising concurrency, which addresses throttles, not a code fault.

The through-line: name the question the stem is really asking (what / how-often / where / who), pick the one signal that answers it, then correlate the others to confirm, exactly the flow the decision diagram above walks.

Which question is the stem asking? name it before you pick a signal what happened in one request? how often / is it trending? where in the call chain? who / what changed it? CloudWatch Logs Insights query CloudWatch metrics + alarm / metric filter AWS X-Ray service map CloudTrail API audit trail Correlate by timestamp align the other signals to confirm
Decision flow: the stem question picks one signal, then correlate the others by timestamp to confirm the diagnosis.

CloudWatch Logs vs CloudWatch Metrics vs AWS X-Ray vs CloudTrail: what each answers

DimensionCloudWatch LogsCloudWatch MetricsAWS X-RayCloudTrail
Question it answersWhat happened in this request? (event detail)How often / how much / is it trending? (numeric)Where in the call chain did time or errors accumulate?Who called which AWS API, when, from where?
Data shapeTimestamped text/JSON log eventsNumeric time-series metricsSegments/subsegments stitched into traces + service mapAPI activity audit records (management/data events)
Primary tool to investigateLogs Insights queries; metric filters to alarmDashboards and alarms on metricsService map, trace timeline, filter expressions on annotationsEvent history / lookup, log file delivery to S3
Search / filter keyLogs Insights QL over log fieldsNamespace + dimensionsAnnotations (indexed); metadata is NOT searchableEvent name, principal, source IP, time
Best for in RCAPinpointing one transaction's exact failureSpotting a spike and bounding the time windowIsolating the slow or failing hop across servicesTracing a config/permission change that caused the issue

Decision tree

What does the symptom ask? map the symptom to one signal One failed request Rate / trend Slow path Who / what changed? CloudWatch Logs Insights query (line-level event) CloudWatch Metrics dashboards + alarms (metric filter) AWS X-Ray service map + trace (slow / failing hop) CloudTrail management-event audit (who / what / when / where) Two secondary triage splits (not part of the main routing): Lambda triage Errors metric vs Throttles metric Errors metric function error: unhandled exception or timeout Throttles metric concurrency exhausted: TooManyRequests- Exception / HTTP 429 X-Ray trace fields which field to attach for filtering? Annotation indexed + filterable (userId, orderType) Metadata NOT indexed (context only, not searchable) Always: correlate signals on one timestamp window 4xx = caller (bad request) vs 5xx = server (serving side faulted)

Sharp facts the exam loves — give these one last read before exam day.

Cheat sheet

Sharp facts the exam loves — scan these before test day.

Match the symptom to the question, and the right telemetry signal falls out

AWS root cause analysis reads four complementary signals, each answering exactly one question: CloudWatch Logs answers "what happened in this one request?", CloudWatch metrics answer "how often, how much, is it trending?", X-Ray answers "where in the call chain did time or errors build up?", and CloudTrail answers "who called which AWS API, when, from where?". The DVA-C02 reflex is this mapping itself, so when a stem describes a symptom, name the question first and the service follows. A spike on a dashboard is a metrics question, a single failed transaction is a logs question, slow end-to-end latency across services is a trace question, and an unexplained config or permission change is a CloudTrail question.

Trap Grepping raw logs by hand to judge whether errors are trending up. That is a metrics job, and scanning log text for a rate is slow and error-prone.

Reach for CloudWatch Logs when you need the exact error on one request

CloudWatch Logs holds the line-level event record: the timestamped text or JSON your code and managed services emit into a log group such as /aws/lambda/<function-name>. It is the signal that tells you what exactly happened inside a single failed transaction, down to the stack trace or message. Use it when you already know which request failed and need its precise cause, rather than when you are trying to spot a trend across many requests, which is a metrics question.

Trap Reaching for CloudWatch Logs to spot a trend across many requests, when a rate or count over time is a metrics question and Logs answers what happened inside one specific request.

1 question tests this
Use Logs Insights as the reactive hunt for one request's failure

CloudWatch Logs Insights is an interactive query engine over your log groups with a purpose-built query language built from fields, filter, stats, sort, and limit commands, run ad hoc during an investigation to surface the exact events behind a failure. It can search multiple log groups in one query, and a running query is stopped after 60 minutes if it has not completed. Reach for it when the question is "what was the error on request X?", the reactive hunt, in contrast to a metric filter, which is a standing rule that watches every new log event.

Trap Standing up a Logs Insights query to continuously watch for and alert on new failures, when Insights is an on-demand reactive hunt and a metric filter plus an alarm is the proactive standing rule.

7 questions test this
Convert a log pattern into a metric with a metric filter, then alarm on it

A metric filter is a standing rule on a log group that scans every incoming event for a pattern (say the term ERROR or a 5xx status) and emits a numeric CloudWatch metric you can alarm on. This is how you turn unstructured log text into a measurable trend that pages you proactively, rather than discovering the problem only when you go hunting. Logs Insights is the reactive query you run after the fact; a metric filter plus an alarm is the proactive trigger set up in advance.

Trap Trying to detect a rising error rate by repeatedly running a Logs Insights query: use a metric filter feeding an alarm so the trend pages you instead of you polling for it.

A CloudWatch metric is identified by its namespace plus dimensions

A metric is a numeric time series (error count, p99 latency, throttle count) identified by a namespace that isolates one service's metrics (for example AWS/Lambda) plus a set of dimensions, which are name/value pairs such as FunctionName=checkout that scope the series to one resource. Dimensions are how you narrow a chart from "all functions" down to the single function that is misbehaving. You set alarms on metrics, and a metric spike bounds the time window you then drill into with logs and traces.

Trap Treating a namespace/dimension pair as interchangeable with another after the fact, forgetting that a dimension is part of a metric's identity, so an alarm created on a specific dimension combination does not retroactively cover series you never published with those exact dimensions.

4 questions test this
A flat Lambda Errors line with rising Throttles means concurrency is exhausted, not broken code

Lambda's Errors metric counts only function errors (an unhandled exception or a timeout raised inside your handler) so a rising Errors line points at your code. Throttles is a separate metric that counts invocations rejected because concurrency was exhausted, surfaced as TooManyRequestsException / HTTP 429, and a throttle does NOT increment Errors. So if Errors stays flat while Throttles climb, the fix is to raise reserved or provisioned concurrency or the account concurrency limit, not to read your handler code.

Trap Debugging handler logic because invocations are failing, when the Errors metric is flat and only Throttles is rising. A throttle is a concurrency limit, not a code fault.

Watch Lambda Duration climbing toward the timeout before it turns into errors

Lambda's Duration metric measures execution time per invocation, and it is the early-warning signal for a performance regression: a Duration line trending up toward the configured timeout is what eventually turns into function errors once invocations actually hit the ceiling and time out. Treat a rising Duration as the metric to investigate proactively, since by the time it shows up as timeouts in the Errors metric the user impact has already begun.

Trap Waiting for the Errors metric to rise before investigating a slowdown, since by then invocations are already timing out, whereas a Duration line climbing toward the configured timeout warns you while there is still headroom to act.

Distinguish a function error from an invocation error before you touch code

A function error is raised inside your handler (an unhandled exception or a timeout) and is the case where you actually debug your code. An invocation error happens before or around your code: a throttle when concurrency is exhausted (TooManyRequestsException / 429), or a permission/configuration failure such as the caller lacking lambda:InvokeFunction. Read the error type and the metric first, because the two classes point at different layers: fix the handler for a function error, but fix concurrency limits or IAM permissions for an invocation error.

Trap Reading application logs to fix a "not authorized to perform lambda:InvokeFunction" failure. That is an invocation-level permission problem, not a bug in your handler.

Use X-Ray to find where in a multi-service call chain the time or errors accumulate

X-Ray records each request as a segment (the work one service did) and breaks that service's downstream calls into subsegments, then stitches all segments sharing one trace ID into a service map, the visual graph of services and the edges between them, color-coded for errors, faults, and throttles. Reach for it when end-to-end latency is high across microservices but no single component looks broken: the service map shows which hop accumulates the time, which a single service's logs cannot reveal.

Trap Reading one service's logs to explain high end-to-end latency across services: logs from a single hop cannot show where across the chain the time was spent.

1 question tests this
X-Ray default sampling traces the first request each second plus 5% of the rest

To control cost and overhead X-Ray does not trace every request: the default sampling rule records the first request each second (the reservoir) plus five percent of any additional requests that second. The consequence for root cause analysis is that a low-volume intermittent error may simply not have been sampled, so the absence of a trace is not proof the request succeeded. When you need fuller capture, raise the sampling rate or add a targeted sampling rule for the affected path.

Trap Concluding an intermittent request succeeded because no X-Ray trace exists for it, when default sampling captures only the first request each second plus 5% of the rest, so a missing trace may just mean it was not sampled.

1 question tests this
Use CloudTrail to find who called which AWS API, when, and from where

CloudTrail is the control-plane audit trail: it records management-event API activity, the principal who made the call, the API action, the timestamp, and the source IP. Reach for it when behavior changed without a deployment you initiated, or a call suddenly started being denied, for example a security group was edited or an IAM role's permissions changed. Logs, metrics, and traces observe your application, so none of them captures a control-plane change; CloudTrail is the only one of the four signals that answers "who or what changed it?".

Trap Hunting through CloudWatch Logs or X-Ray for the cause of an unexplained permission or config change, when those observe your application and only CloudTrail records who called the AWS API that changed it.

CloudTrail is an API audit, not an application debug log

CloudTrail records that a principal called an API (say ModifySecurityGroupRules at 14:02 from a given IP) but it does not contain your function's internal log lines, error messages, or request payloads. For those you read CloudWatch Logs. Keep the boundary sharp: CloudTrail answers the "who changed the configuration" question only, and using it to debug application behavior will turn up nothing because it never sees inside your code.

Trap Searching CloudTrail for an application's runtime error or request payload: it logs AWS API calls, not your code's internal events; read CloudWatch Logs for those.

Triage a status code by side: 4xx blames the caller, 5xx blames the server

A 4xx status generally means the caller (your application or the client) sent a bad request (malformed input, missing auth, a 429 throttle), while a 5xx means the serving side faulted. This split tells you which direction to investigate first. On a multi-service path do not assume a 5xx is your code: narrow it by checking which hop reported the 5xx on the X-Ray service map, since the failing service may be a downstream dependency rather than your own.

Trap Assuming every 5xx on a multi-service path is your own service's fault, when the X-Ray service map may show the 5xx originated at a downstream dependency you call rather than in your code.

Correlate signals on a shared time window rather than trusting one alone

The disciplined RCA loop aligns timestamps across signals: when a metric spikes, note the exact window, then pull the X-Ray traces, the CloudWatch Logs lines, and any CloudTrail event inside that same window. A 5xx spike whose window lines up with a CloudTrail ModifySecurityGroupRules event is a configuration regression, not a code bug, a conclusion no single signal could reach. Metrics localize the blast radius and bound the window; logs and traces then explain exactly what failed inside it.

Trap Declaring a code bug from a 5xx spike alone, when correlating its time window against CloudTrail can reveal a ModifySecurityGroupRules event that points at a configuration regression rather than your code.

Read an ECS task failure from stopCode and stoppedReason first, and treat exit code 137 as out-of-memory

When ECS tasks start then stop, run describe-tasks and read the stopCode (the category, such as ResourceInitializationError or EssentialContainerExited) and stoppedReason (the human-readable detail) before anything else. The most common container-level cause is a SIGKILL: a container exit code of 137 (128 + 9) means the OOM killer terminated it for exceeding its task-definition memory limit, so the fix is to raise the allocated memory, not to debug code. A ResourceInitializationError points instead at setup failing before the app ran (logging-driver or secrets connectivity).

Trap Reading container application logs to explain a 137 exit when there are none: the process was killed by the OS for exceeding its memory limit, which surfaces in stopCode/stoppedReason, not in app output.

4 questions test this
A Fargate awslogs ResourceInitializationError is missing connectivity or task-execution-role permission, not bad code

Fargate tasks using the awslogs driver fail at startup with ResourceInitializationError / 'failed to initialize logging driver' for one of two reasons. First, connectivity: a task in a private subnet with no NAT gateway needs an interface VPC endpoint for CloudWatch Logs (com.amazonaws.region.logs), and that same private-subnet rule causes silently missing logs even when the task stays up. Second, permission: the task execution role must allow logs:CreateLogStream and logs:PutLogEvents, which the AmazonECSTaskExecutionRolePolicy managed policy grants. The identical 'unable to pull secrets or registry auth' failure needs the task's security group to allow outbound HTTPS (443) to Secrets Manager / ECR.

Trap Assuming a NAT gateway is enough on its own: the task's security group must still permit outbound 443 to reach CloudWatch Logs, Secrets Manager, or ECR endpoints.

6 questions test this
Lambda emits two X-Ray segments: AWS::Lambda is the service, AWS::Lambda::Function is your code

Active tracing makes Lambda record two segments per invocation, so the X-Ray service map shows two nodes for one function. AWS::Lambda is the Lambda service overhead (scheduling, creating or unfreezing the execution environment, and downloading function code) and an error or latency here is a platform/init problem (including throttling), not your code. AWS::Lambda::Function is the handler running, so an error or high latency on that node is in your application logic. Decide which node shows the symptom before you touch anything.

Trap Debugging handler code because the function's X-Ray node shows latency, when the slow node is AWS::Lambda, which is service overhead like cold-start init or throttling rather than your application logic.

5 questions test this
Create a field index to make Logs Insights lookups on a high-volume field fast and cheap

When Logs Insights queries that filter on a field such as requestId scan huge volumes and run slow and expensive, create a field index policy for that field (PutIndexPolicy / console) without changing the log format. Queries written as filter requestId = 'value' (equality) then skip log events known not to contain the indexed value, cutting scan volume and cost. Across hundreds of log groups, the filterIndex command restricts a query to only the groups indexed on that field.

Trap Expecting a field index to speed up a non-equality filter such as a wildcard or range on requestId, when the index only lets an equality match skip events known not to contain the indexed value.

5 questions test this
Build Logs Insights trend queries with stats and bin, and match timeouts on 'Task timed out'

To turn logs into a time series, use stats with aggregations grouped by bin(5m) (for example stats count(*) by endpoint, bin(5m)); bin() creates the time buckets a chart needs. Filter HTTP errors by numeric range (filter status>=400 and status<=499 for 4XX). Lambda timeouts do NOT log as ERROR (they emit a 'Task timed out' message) so capturing all failures needs a pattern like filter @message like /ERROR/ or @message like /Task timed out/.

Trap Filtering only on /ERROR/ to count all Lambda failures, when a timeout emits a 'Task timed out' message rather than ERROR, so it slips past that filter unless you match it explicitly.

6 questions test this
Use VPC Reachability Analyzer to find the blocking component without sending traffic

VPC Reachability Analyzer performs static configuration analysis between a source and destination (it sends no packets) making it the tool to proactively prove a path or pinpoint what blocks it (security group, network ACL, or route table) before any traffic flows. It reports the blocking component with explanation codes. Because it snapshots configuration at analysis time, after you change a rule you must run a NEW analysis; re-reading the old result still shows the stale, pre-change state.

Trap Expecting an existing Reachability Analyzer result to update after you fix a security group: it is a point-in-time snapshot, so you must analyze the path again.

6 questions test this
A VPC Flow Log REJECT means a security group or NACL denied the traffic

An action=REJECT record means a security group or network ACL blocked the traffic; check the security group first since it is evaluated first and is the most common cause. Because NACLs are stateless, return traffic is evaluated independently and needs explicit ephemeral-port rules. This is why one direction can show ACCEPT while the reply shows REJECT, or why an SG that allows a port still fails when the subnet's NACL does not. Flow logs only appear once there is real traffic on the interface (the log group is created on first traffic), and custom formats can add fields like instance-id, subnet-id, flow-direction, and tcp-flags.

Trap Blaming the security group for a REJECT on return traffic when the outbound request showed ACCEPT, when NACLs are stateless, so the reply needs its own explicit ephemeral-port rule that the SG, being stateful, does not require.

6 questions test this
API Gateway logging needs an account CloudWatch role, and some client errors are never logged

Before any API Gateway logs appear, the account must have an IAM role (trusting apigateway.amazonaws.com, with AmazonAPIGatewayPushToCloudWatchLogs) configured in API Gateway Settings; without it, requests succeed but produce no logs. Even with logging on, API Gateway does not write certain client errors to execution logs (such as 403 from a wrong resource path, and excessive 429 or 413 errors), so enable custom access logging to capture them. To surface integration failures behind 500 errors, add $context.integrationErrorMessage to the access log format.

Trap Assuming execution logging captures every failed request, then concluding a missing 403 means the call never reached API Gateway, when certain client errors like a wrong-path 403 are absent from execution logs unless you turn on custom access logging.

4 questions test this
Read RDS ReplicaLag carefully: -1 means replication is broken, and an undersized replica lags

A ReplicaLag value of -1 does not mean zero lag: it means replication is not active or the lag can't be determined (for MySQL/MariaDB, equivalent to Seconds_Behind_Master = NULL), so check the replication state and event log for errors. Sustained positive lag often comes from a read replica with a smaller instance class than the primary: a replica replays the primary's full write load and must match or exceed its compute/storage to keep up. For consistent reads, the app can check ReplicaLag and route time-sensitive queries only to replicas below an acceptable threshold.

Trap Reading a ReplicaLag of -1 as zero lag and a perfectly healthy replica, when the value actually signals replication is not active or cannot be measured, so the replica may be receiving nothing.

4 questions test this

Also tested in

References

  1. Amazon CloudWatch: Metrics concepts (namespaces, metrics, dimensions)
  2. Analyzing log data with CloudWatch Logs Insights
  3. CloudWatch Logs quotas (query timeout and limits)
  4. Creating metrics from log events using metric filters
  5. Using CloudWatch metrics with Lambda (Errors, Throttles, Duration)
  6. AWS X-Ray concepts (segments, subsegments, service map, annotations vs. metadata)
  7. AWS X-Ray: Configuring sampling rules (default reservoir + 5% rate)
  8. What Is AWS CloudTrail? (management-event API audit trail)