Apply Responsible AI Principles to Business Decisions
The ten responsible AI dimensions
Ask whether a customer-service chatbot that answers questions on returns is responsible and you will not get one answer, because the question splits into several: does it treat customer groups fairly, does it state the return policy correctly, does it leak personal data, and can staff steer it when it misbehaves? Responsible AI is the discipline of designing, developing, and using AI technology with the goal of maximizing benefits and minimizing risks[1], and AWS breaks it into named dimensions that a business leader applies to one use case at a time.
The AWS AIB-C01 exam guide[2] lists "Task 3.1: Apply responsible AI principles to business decisions." under AI Governance and Responsible AI Leadership. Its skills are to apply responsible AI principles and dimensions to business scenarios, navigate tradeoffs when business objectives conflict with responsible AI principles, identify when responsible AI practices should be integrated into AI project planning to ensure governance by design, and recognize circumstances when AI systems require human oversight and identify appropriate safeguards (for example, hallucination detection, guardrails, escalation criteria). The same domain lists "Task 3.2: Establish AI governance structures and ensure regulatory compliance." (see Establish AI governance and ensure regulatory compliance) and "Task 3.3: Identify enterprise AI risks and direct mitigation strategies." (see Identify enterprise AI risks and direct mitigation strategies). The skill verbs separate the three tasks: Task 3.1 asks you to recognize when an AI system requires human oversight and to identify appropriate safeguards, such as hallucination detection; Task 3.3 asks you to identify the need for risk controls and monitoring mechanisms for AI systems in production and to mitigate reliability risks such as hallucinations; Task 3.2 asks you to establish governance structures with clear accountability and to apply AI risk classification frameworks to prioritize governance and compliance decisions.
The dimensions in the Responsible AI Lens
The AWS Well-Architected Responsible AI Lens (AWS guidance for teams that build and operate AI for a specific use case; "the Lens" from here on) lists ten dimensions[1]:
| Dimension | Lens definition | Business question it raises |
|---|---|---|
| Controllability | Having mechanisms to monitor and steer AI system behavior | Can we adjust, override, or roll back the system when needed? |
| Privacy | Appropriately obtaining, using, and managing data | Is this use of the data permitted? |
| Security | Protecting data and models from exfiltration and adversarial inputs | Are data and models protected from theft and attack? |
| Safety | Blocking harmful system output and misuse | Could an output or a use cause harm? |
| Veracity | Achieving factually correct system outputs | Are the outputs true? |
| Robustness | Achieving correct system outputs for both expected and unexpected inputs | Does it keep working as real inputs vary? |
| Fairness | Considering impacts on different groups of stakeholders | Who is affected, and is any group disadvantaged? |
| Explainability | Having mechanisms to understand system behavior | Can people understand why it produced an output? |
| Transparency | Enabling stakeholders to make informed choices about their engagement with an AI system | Do people know when and how they rely on AI? |
| Governance | Incorporating best practices into the AI supply chain, including providers and deployers | Are good practices built into how the system is sourced, built, and run? |
The AWS responsible AI page[3] groups the same concerns into eight, pairing privacy with security and veracity with robustness. This topic uses the Lens's ten because business decisions treat the members of each pair differently: a privacy review and a security review ask different questions, as the section Dimensions that get merged with a neighbor below shows.
Separate tests, not one score
A strong result on one dimension does not by itself establish acceptable performance on every other dimension. A lending model with high accuracy can still disadvantage one applicant group (fairness), fail on applications written in plain everyday language instead of formal language (robustness), or give declined applicants no reasons (explainability). The Lens uses the dimensions to identify benefits and risks inherent in an AI use case, and to organize their decision-making[1] about design and evaluation. It is guidance, not a legal standard: the Lens says do not use this guidance as a compliance or assurance checklist[1], and each organization decides how the regulations that apply to it are met.
The takeaway: name the dimensions a use case depends on, and ask for evidence on each one separately.
Fairness: choosing the goal before reading the metric
This section covers fairness, which should not be reduced to a single number. The Lens defines it as considering impacts on different groups of stakeholders[1], which is wider than treating every input identically.
Fairness starts from who is affected
A fairness assessment may consider harms to individuals (for example, wrongful denials) and to groups (for example, performance variations across demographic groups)[4]. Those are different concerns. Individual fairness asks whether similar individuals get similar treatment regardless of their demographic characteristics[5]; group fairness compares the system's performance across stakeholder groups identified when benefits and risks are assessed (Map stakeholders beyond direct users, below). Which concern matters depends on the use case, so the business chooses the fairness goal together with the affected stakeholders and records any conflict between goals.
Identical treatment is not automatically fair. A service that requires every customer to interact the same way can shut out people with disabilities, which is why the Lens asks teams to identify accessibility requirements for people with different capabilities and disabilities[6]. Balanced prediction rates across groups do not make a service fair if some people cannot use it at all.
Two fairness measures that compare different populations
Demographic parity and equal opportunity are two of the fairness measures the Lens lists, and they reveal different types of bias[5]. "Equal opportunity" here names a statistical measure, not the legal concept of equal employment opportunity, and meeting it is not a compliance finding: the Lens says do not use this guidance as a compliance or assurance checklist[1]. Take a hiring tool that recommends applicants for interview:
- Demographic parity compares positive decisions across groups: whether different groups receive similar positive prediction rates[5]. Here it compares, for each group, the share of all applicants who are recommended.
- Equal opportunity compares correct recognition of the people who should get the positive decision: whether the system correctly identifies positive cases at similar rates across groups[5]. Here it compares, for each group, the share of qualified applicants who are recommended.
The first compares all applicants and the second compares only qualified applicants, as the figure at the end of this section shows, so meeting one does not establish the other; the Lens notes that systems can appear fair on one measure but not on another[5]. An illustration with invented numbers: if 60 percent of group A's applicants are qualified but only 30 percent of group B's, recommending the same share of each group satisfies demographic parity but does not establish equal opportunity. The Lens advises to make explicit decisions about which to prioritize[5] and record the reasoning.
Similar overall accuracy across groups is not equal opportunity either. Overall accuracy also counts the correct rejections, so a model can score the same accuracy for two groups while recognizing qualified people in one group less often. Tools such as Amazon SageMaker Clarify, which the Lens lists as a related tool for this practice[5], can compute these measures; choosing which one the business commits to stays a business decision.
The takeaway: agree on which fairness goal matters, and for whom, before judging any fairness result.
Explainability: matching the explanation to the decision
This section covers explainability, which the Lens defines as having mechanisms to understand system behavior[1]. The business decision is which kind of explanation a use case needs, and whether the people who receive it can act on it.
One decision versus overall behavior
Explanations answer different "why" questions. The Amazon SageMaker Clarify documentation gives examples such as why did the model predict a negative outcome such as a loan rejection for a given applicant[7], and which features have the largest influence on the behavior of the model[7]. The first asks for an explanation of one decision (also called a local or per-instance explanation); the second asks for an explanation of overall behavior (a global explanation: which factors drive predictions across all cases). Clarify's feature attribution, which assigns each feature an importance value for a particular prediction[7], can serve both, including per-instance explanation during inference[7]. Clarify is one option among others: the Lens also names explanation tools (like LIME or SHAP)[8] and, when explanations are critical, interpretable models like decision trees that are simple to understand[8].
The two kinds of explanation, one decision versus overall behavior, are not interchangeable. A declined applicant needs the first kind: a summary of the factors that matter across the whole loan portfolio does not show what drove that applicant's decision. Privacy regulators make the same point; Australia's Office of the Australian Information Commissioner (OAIC) states that organisations must ensure that the output of AI systems, including any decisions made using AI, can be explained to individuals affected[9]. An oversight team checking whether a model relies on sensible factors needs the second kind.
Set the required explanation level with business and compliance stakeholders before committing to a solution, because the requirement can rule options in or out: NIST's AI Risk Management Framework notes that organizations might face a tradeoff between predictive accuracy and interpretability[10].
Explanations must improve judgment, not just confidence
Explanations and uncertainty information (confidence scores, known limitations) help only if users can tell when to rely on an output. The Lens asks teams to help users identify when to trust system outputs, when to seek additional verification, and when to override or ignore system recommendations[11].
Measuring this takes more than a satisfaction survey. The Lens combines objective measures (such as task completion when acting on explanations) with subjective ones (such as trust ratings) and says to pay particular attention to whether users can accurately identify when to rely on or question the system's outputs[12], testing across user groups and expertise levels. Trust ratings alone can rise because an interface is persuasive while decisions get no better; the test is whether users correctly rely on good outputs and override poor ones, which catches cases where users might over- or under-trust your system[12].
The takeaway: specify which question the explanation must answer and for whom, then check that people actually make better calls with it.
Transparency: disclosure, system cards, and user guidance
This section covers transparency, which the Lens defines as enabling stakeholders to make informed choices about their engagement with an AI system[1]. It differs from explainability (previous section): explainability helps people understand an output, while transparency tells them that AI is involved and what it can and cannot do, so they can decide whether and how to rely on it. A system can be transparent about its presence and limits and still give a decision nobody can explain.
Disclose where people engage and where they rely
Transparency harms include situations where users are unaware of AI system presence, or may not realize that an output is AI-generated[13]. The Lens's remedy is to map the user journey in four phases and mark specific moments in each phase where guidance is most valuable[14], the transparency touch points:
- Pre-interaction: help users learn the system's capabilities and limitations[14] before they engage.
- Initial interaction: identify the first touchpoints where users provide input, the guidance they need, and their accessibility needs.
- Processing: maintain transparency about AI decision-making and provide appropriate progress indicators[14].
- Post-interaction: enable users to understand, challenge, and provide feedback on AI outputs[14].
Disclosure therefore belongs where users first meet the system and where they act on its outputs, and it deserves more prominence where much depends on the output: the Lens asks teams to identify decision points where users rely on AI outputs[13] and weigh the impact if users are unaware of the system's limitations. A notice in an annual corporate report alone may not inform users at those moments. The figure at the end of this section traces the four phases and their touch points.
System cards describe intended use; they are not proof for your context
A system card (the Lens also calls it an AI service card, or service card) is documentation that gives stakeholders a single place to find information on the intended use cases and limitations, responsible AI design choices, and deployment and performance optimization best practices[15]. The Lens's practice for writing one for your own system, create a system card that communicates intended usage and limitations[15], plans a specific set of evaluations for the AI service card[15]. AWS publishes the system cards for its own AI services as AWS AI Service Cards, for example the one for Amazon Bedrock Guardrails[16]. Use a system card to check whether a proposed application fits the documented boundaries and to prepare guidance for your own users.
A system card does not establish that a service suits your inputs. System cards do not provide guidance on expected performance of the AI system on the specific inputs the deployer may provide; that testing is the responsibility of the deployer[15], and AWS recommends that customers assess the performance of any AI service on their own content for each use case[16]. A supplier's general system card is not go-live evidence for a new population.
Tailor transparency without exposing protected information
Different audiences need different detail. The Lens recommends choosing a format for each stakeholder group, for example a blog post, user guide, FAQs, system documentation, service card[17], matched to that group's needs and technical capabilities. Customers may need plain-language capabilities and limitations, while authorized reviewers need detailed evidence.
More openness is not automatically better. The Lens asks leadership to separate information appropriate for public sharing versus proprietary details restricted to internal audiences[17] through a publication review and approval process, and its transparency measures check for unintended information leakage or misuse[18]. Publishing raw internal evaluation records can expose personal data or exploitable security details without helping a customer understand anything.
Keep user guidance current
Guidance goes stale when the system or its uses change. The Lens says to update documents based on stakeholder feedback, new use-cases, new system releases, and industry best practice developments[19] and to dedicate an owner[19] who runs review cycles, versioning, and approval chains. Editing an internal document is only half the job: the plan includes publishing the updates and making them accessible to the stakeholders[19], and catching when outdated information is still circulating[17]. Engineering release notes alone may not reach the business users who need the revised guidance.
The takeaway: transparency is designed per audience and per touch point, and it is maintained like a product, not published once.
Dimensions that get merged with a neighbor
For example, suppose a review approves a customer-data assistant because access is encrypted, its answers are factually accurate, its uptime is high, and a monitoring dashboard exists. Each of those sign-offs checks a neighbor of a dimension the Lens lists separately[1], so privacy, safety, robustness, and controllability remain unchecked. This section covers those four dimensions and the neighbor each gets merged with: privacy with security, safety with veracity, robustness with availability, and controllability with monitoring.
Privacy is about permitted use; security is about protection
Privacy is appropriately obtaining, using, and managing data[1]; security is protecting data and models from data theft (exfiltration) and from adversarial inputs[1] designed to manipulate the system. They answer different questions. An employee with authorized, encrypted access can still use customer personal data for a secondary purpose without consent or another lawful basis: a privacy failure despite secure access. The OAIC notes that it may be difficult to establish reasonable expectations for using personal information for a secondary, AI-related purpose[9], and that an organization that cannot establish this should seek consent or offer a meaningful opt-out. In the other direction, a permitted purpose still needs protection against theft and tampering. Encryption approval therefore cannot authorize a new use of personal data; that takes a privacy review.
Safety is about harm; veracity is about truth
Safety is blocking harmful system output and misuse[1]; veracity is achieving factually correct system outputs[1]. Safety harms arise when outputs, which can be content or actions, create unintended impacts on the health or well-being of individuals, groups, society or the environment and can be misused in ways that could cause harm[20]. The two separate in both directions. Accurate step-by-step instructions can still enable harm, so a factual-accuracy review is not a safety approval. A harmless-sounding answer can still be wrong, so passing a harmful-content check is not a veracity approval.
Robustness is about input variation, not uptime
Robustness is achieving correct system outputs for both expected and unexpected inputs[1]. The Lens distinguishes two kinds of input variation: intrinsic variations are differences in input data to which an AI system must attend to succeed. Confounding variations are differences in input data that an AI system must ignore to succeed[21]. Whether a caller asks to cancel or to upgrade is intrinsic variation; the same request spoken in a regional dialect over background noise is confounding variation.
Robustness evaluation therefore uses the variations real users produce, such as different writing styles, dialects, image qualities, or audio conditions[22], including combinations of multiple input variations together, such as processing accented speech with background noise[22]. A voice assistant that fails on one dialect has a robustness problem; raising service uptime measures availability and leaves the errors untouched.
Controllability needs interventions that work
Controllability is having mechanisms to monitor and steer AI system behavior[1]. Monitoring gives visibility; steering needs a way to intervene. The Lens measures whether users can effectively override, adjust, or roll back its behavior[23], how quickly controls take effect, whether controls fail to work as intended or when they produce unexpected side effects[23], and whether the system gives clear feedback when controls can't work[23]. Judge a control by whether it produces the intended change quickly enough for the business risk. A monitoring dashboard with no effective intervention behind it is visibility, not controllability.
The takeaway: each pair needs two reviews, and passing one says nothing about the other.
Governance by design: responsible AI from the first plan
This section covers when responsible AI enters a project. Governance by design means responsible AI requirements are part of the use-case definition and stay with the project through operation, instead of arriving as a review just before launch. It concerns when requirements enter a project, not who governs: it is neither the Lens's Governance dimension (supply-chain best practices) nor the governance structures of the exam guide's Task 3.2 (see Establish AI governance and ensure regulatory compliance). AWS states this as the design principle responsible by design[24]: best practices throughout the AI lifecycle from design through operations[24], with the emphasis on identifying and resolving potential issues in design[24].
Governance by design is compatible with rapid prototype-and-release development. The principle does not preclude the use of rapid prototype-and-release development processes, but it does encourage open, transparent iteration[24]. A fast pilot is fine, provided its boundaries and required approvals are set when it starts.
Why the first planning step matters
Early decisions lock in risks: which data is used, which vendor is chosen, how the workflow changes. If responsible AI review waits until a pilot is selected for production, those decisions have already been made without the requirements.
The Lens is organized as eight focus areas, each of which aligns with a phase in the machine learning lifecycle[1]: use case, benefits and risks, release criteria (pass-or-fail release tests, defined below), dataset planning, system planning, evaluate and release, user guidance, and monitoring. The first three establish requirements to guide data and system design; the figure at the end of this section lists the eight in presentation order. The order is a guide, not a strict sequence: the Lens says AI development can be iterative and nonlinear[1] and teams work through the focus areas in an order that suits them. AWS also favors narrow use cases because the use case directly determines potential risks and release criteria[24].
Map stakeholders beyond direct users
A stakeholder is anyone involved in, or affected by, the development or operation of the system. The Lens sorts downstream stakeholders into primary users, secondary users whose data may be used as input, and indirect stakeholders affected by system operations[25], asks teams to consider vulnerable populations, and to seek individuals with different perspectives from those of the builder team[25]. Upstream stakeholders count too, including vendors who supply data or model components[26] and oversight functions such as compliance and risk. The map is not permanent: review and validate the stakeholder list periodically[25], especially when the workflow or audience changes. Surveying only paying users misses the people whose data is processed, the people affected without using the system, and the suppliers.
Turn principles into pass-or-fail release criteria
Principles become decisions only when they can be tested. A release criterion is a binary test that consists of a quantitative assessment and a decision threshold[27], and together the criteria form a high-level specification for the design of system datasets and the AI system[27], which is why they come before building or buying.
- Write each benefit and harm as a yes-or-no question. "Users might get biased recommendations" becomes Does the system mitigate unwanted bias for each user group?[28], answered from measurable data, not opinion.
- Record who is affected by each failure. The Lens says to document the stakeholders who would be impacted if the system failed to meet the release criterion[28].
- Set thresholds from evidence. Good targets are based on real data, not guesswork[29], drawn from existing performance, stakeholder needs, and risk tolerance; no single universal accuracy percentage establishes readiness.
- Split complex promises. "Does the system provide a good user experience?" becomes separate yes or no questions[30] about response speed, accurate results, and an intuitive interface.
The takeaway: responsible AI requirements are written into the use case, the stakeholder map, and the release criteria before the solution exists.
Evaluation evidence that supports a release decision
This section covers the evidence behind a release decision. Release criteria (previous section) say what must be true; evaluation shows whether it is. Three properties make that evidence trustworthy: independence for the judgments that matter most, a tie to the exact version being approved, and evaluation methods that have themselves been checked.
Independent corroboration for critical or subjective judgments
For criteria that are highly critical or subjective, such as safety assessments, unwanted bias evaluations, or user experience judgments[31], get a second opinion from qualified reviewers outside the delivery team's incentives. For high-risk systems, the Lens suggests evaluators who have no stake in your project's success[31], and to maximize independence it suggests they build or acquire their own evaluation datasets, using the same information about the intended use case(s)[31] that will be shared with downstream users, rather than cases chosen by the supplier or project team. Investigate significant disagreements before making release decisions[31] and keep both sets of findings as decision evidence. Rerunning the delivery team's own demonstration does not by itself establish independent review: the reviewers' independence matters, not just the cases used.
Evidence tied to the exact version
An approval applies to the system that was tested. For each system update, the Lens says to rerun the evaluation and log test conditions, system configurations, data inputs, raw results, and methodological notes[32], and to link the evaluation materials to the system and dataset records so that it is clear which data and system versions led to the evaluation results[32]. That lets an auditor reconstruct what was tested and why that version was accepted. At the start of each release evaluation, check that the criteria still cover the system: if your system includes capabilities that were not considered when you last updated your criteria[33], add tests and revisit the risk and benefit assessment. A previous version's safety report is evidence about a different system.
Automated evaluation, validated against people
Automated evaluation scales; qualified human assessment catches context that automation misses. The Lens suggests LLM-as-a-judge[30], where a large language model (LLM) scores outputs against a rubric, for criteria that need judgment of context or quality, and human evaluation where neither automated metrics nor an LLM judge captures the property. Amazon Bedrock Evaluations[34] supports both approaches, including evaluation jobs that use a judge model and jobs that use human workers.
Neither approach validates itself. The Lens says to validate model-derived metrics (such as LLM-as-a-judge or -jury) through correlation with human assessors[35] before relying on them, and to evaluate metrics derived from human assessors and annotators for unwanted bias, assessor variance, and consistency[35]. High scores from an AI evaluator are approval evidence only after its judgments have been shown to match the intended quality standard.
The takeaway: evidence counts when it is independent where it matters, tied to the version being approved, and produced by a validated method.
Baking, filtering, and guiding: mitigation strategies
Suppose a healthcare chatbot should not give legal advice. In the Lens's own example of that harm, a team can reduce the risk by either customizing the underlying model or guardrails, or by warning users not to request legal advice, or both[36]: change the model, screen what goes in and out, or warn users. This section covers those mitigation strategies. The Lens describes an AI system as a core AI system and an optional set of filters (also referred to as guardrails)[37] that block or modify data flowing into or out of it, and it names three design strategies, each acting at a different point on a request, as the figure at the end of this section shows:
- Baking: design choices that directly impact the core AI system[37], such as choosing, customizing, or retraining the model so that its own behavior changes.
- Filtering: designing data filters, the guardrails that screen requests going in (an input filter) and responses coming out (an output filter).
- Guiding: communicating proper usage to users through documentation, legal terms of usage, classes, or similar channels[37].
Assign each harm to a strategy, or to several
For each identified harm, decide whether it can be addressed through technical features or stakeholder guidance[36]; the healthcare chatbot example above can use either route, or both.
Match the strategy to how the harm arises and how much users control it. Guiding works particularly well for criteria that depend on how people choose to interact with your system[38]. A boundary the system must never cross needs baking or filtering as well: a disclaimer asking users not to request a prohibited action leaves the system able to perform it. Whatever the mix, evaluate the protection that results instead of assuming one strategy fits every harm; when a release criterion fails, the Lens says to re-evaluate the original implementation strategy[38].
Guardrails: the enforcement layer
Guardrails are safeguards that enforce permitted boundaries on AI inputs, outputs, or actions: they block prohibited content, or stop an action until a reviewer approves it. They are the enforcement layer behind the organization's mitigation policy, not another name for user guidance or for human judgment. The Lens's filters on inputs and outputs are the content kind of guardrail; holding an agent's action until a reviewer approves applies the same enforcement idea to an action (see Bounding the autonomy of AI agents below).
Amazon Bedrock Guardrails is AWS's managed example of the content kind. It provides configurable safeguards that detect and filter undesirable content and protect sensitive information that might be present in user inputs or model responses[39], including:
- Content filters for harmful categories such as hate, insults, sexual content, violence, misconduct, and prompt attacks.
- Denied topics the application must not discuss.
- Sensitive information filters that block or mask personal data.
- Contextual grounding checks, which detect hallucinations in model responses if they are not grounded[39] in the source or are irrelevant to the user's query. When a model answers from a source the application supplies, a hallucination can state something that source does not support.
- Automated Reasoning checks, which validate the accuracy of foundation model responses against a set of logical rules[39].
Know each safeguard's bounds. Guardrails reduce but do not eliminate hallucinations[16], and grounding checks evaluate consistency with the source the application provides, not truth in general[16]. Its AWS AI Service Card also says Bedrock Guardrails may not by itself suffice to implement all customer responsible AI policies[16].
A system prompt (the standing instructions given to the model) is not a substitute. It asks the model to behave but does not independently check what the model produced, whereas protection in a guarded application comes from two layers: customer-defined policies configured in Bedrock Guardrails and the native safety controls of the foundation model[16].
Guardrails complement human oversight instead of replacing it. They enforce known boundaries automatically, while AWS guidance keeps a human in the loop for high-risk or unfamiliar scenarios that are not previously covered in testing scenarios[40].
The takeaway: assign every harm a strategy, enforce must-hold boundaries with baking or filtering, and use guiding for the choices users make.
Navigating tradeoffs when objectives conflict
This section covers the moment a business objective, such as accuracy, speed, or savings, conflicts with a responsible AI principle, or two principles conflict with each other. Rating a use's risk is a different decision: NIST says policies and resources should be prioritized based on the assessed risk level[10] and expects evidence that for a system to be deployed, its residual negative risk does not exceed the risk tolerance[10], so a rating sets how much scrutiny a use gets and, when residual risk (risk left after treatment) exceeds the organization's tolerance, whether it goes ahead at all (see Establish AI governance and ensure regulatory compliance). A tradeoff instead chooses among options for one use; it defers the use when no option meets a mandatory obligation (Step 1) and stops it when no option or narrower scope controls the harm (Step 3). Conflicts are normal. NIST's AI Risk Management Framework says tradeoffs are usually involved, rarely do all characteristics apply in every setting[10], and that analysis can reveal a tradeoff but does not answer questions about how to navigate the tradeoff[10]; the answer depends on context and values and should be resolved in a manner that is both transparent and appropriately justifiable[10], in the order the figure at the end of this section traces.
Step 1: Separate mandatory obligations from tradeable objectives
A mandatory obligation, such as a legal requirement to explain decisions or to protect personal data, is a condition of proceeding, not one weight among many. NIST's framework expects that legal and regulatory requirements involving AI are understood, managed, and documented[10]. The Lens gives the pattern: when privacy and accuracy compete, you may opt for privacy requirements even if it means accepting lower accuracy within acceptable bounds[41].
So the feasible option that meets the obligation wins over a higher-performing option that misses it, provided it still meets the necessary operating standards its release criteria set. If no option meets the obligation, narrow the use case (change its scope, Step 3) or defer the use. Choosing the highest-accuracy option with a compliance retrofit planned for later leaves the obligation unmet from the first decision the system makes.
Step 2: Weigh who benefits and who bears the harm
Among the options that meet every mandatory obligation, compare benefits and harms per stakeholder group: the Lens says in most cases, benefits and risks should be assessed for each stakeholder group[42]. Include people affected indirectly and groups that may suffer disproportionately severe harm; severity includes the reversibility of harm[43]. Total company savings is a benefit to one stakeholder and says nothing about who pays. Use explicit criteria, which the Lens describes as explicit trade-off criteria (like tenets) that reflect organizational policies and stakeholder needs[44], and document threshold decisions and rationale[41] so that later decisions apply the same standards.
Two tradeoffs recur:
- Privacy protection versus utility. Data-minimizing methods such as de-identification and aggregation[10] reduce exposure, but under certain conditions such as data sparsity, privacy-enhancing techniques can result in a loss in accuracy, affecting decisions about fairness[10]. Aggregation can erase the detail a small group needs to be served well. Choose the least sensitive data that supports the authorized purpose, check the effect on the task and on smaller groups, and document the accepted utility cost; the mandatory privacy obligations from Step 1 do not relax.
- Protection versus legitimate use. Safety filters (the filtering strategy from the previous section) make two kinds of error: missing harmful content and blocking acceptable requests. The AWS AI Service Card for Amazon Bedrock Guardrails notes that over-detecting harmful content can block benign content[16] and advises customers to evaluate both missed and unnecessary interventions against their own acceptance criteria[16], which belong in the deployer's release criteria, a single list of criteria[27] for release readiness. A stricter setting can be justified for severe harms, while ambiguous but legitimate requests may go to human review instead of automatic rejection; track useful service delivery alongside protection. A success measure based only on the share of requests blocked rewards over-blocking as much as protection.
Step 3: Narrow the scope instead of lowering the standard
When a system cannot meet the essential criteria (its release criteria) across the whole use case, restrict it to the domains, inputs, locations, or users where it can. The Lens says to narrow the use case to a specific domain, industry vertical, geography, or user segment[45] and, when release criteria cannot be met, rather than compromising on safety or performance standards, narrow your use case[46], then revisit the benefits and risks and update the criteria for the narrower scope. Narrowing is a valid treatment while meaningful value remains. Lowering a safety threshold to keep the original rollout changes the protection standard instead of fixing the unsupported use.
When the harm cannot be controlled at all, stop that use. NIST is direct: where risks are unacceptable, development and deployment should cease in a safe manner until risks can be sufficiently managed[10], the same outcome the risk-rating decision reaches when residual risk exceeds tolerance.
If the release criteria are met, proceed, and document why the chosen balance is acceptable.
The takeaway: obligations first, then a per-stakeholder weighing against stated criteria, then narrow or stop, never lower the bar.
Placing human oversight in the workflow
This section covers where people review AI work and what makes that review real. The Lens rule is to place human review at points where the quality of system inputs or outputs can be harder to judge[47], and at moments where human expertise adds unique value or assists with consequential decisions[47].
Placement by decision, escalation by case
Map the business workflow and, for each decision type, ask what an error would cost and whether it can be undone; the Lens treats the reversibility of harm[43] as part of severity. The answer places review when the workflow is designed:
- Human review before the effect: a designed checkpoint for decisions whose errors are serious or hard to reverse, such as an individual credit denial. The review happens before the decision takes effect, because correcting it afterward does not undo the harm.
- Automated handling with supervisory review: for well-defined, low-risk, reversible activity, where the system acts without a per-case checkpoint and people review a sample of its results afterward to audit its predictions on an ongoing basis[48]. AWS guidance on operating generative AI presents, as one approach to balancing an application's autonomy, allowing automated actions only in well-defined and low-risk scenarios[40] and using a human in the loop for high-risk or unfamiliar scenarios[40].
Escalation works per case at run time instead: when a single case on the automated path meets a predefined trigger, it leaves that path and goes to a named person with authority to decide. The Lens asks teams to configure human review triggers in monitoring systems[49] for this. The same Lens best practice recommends sampling-based human review processes that validate the accuracy and effectiveness of automated monitoring systems[49], a human check on the monitoring that raises those triggers.
Placement follows the consequence of an error and the value of human expertise, not whether the system is customer-facing: an internal tool that sets an employee's pay may need review before the effect, while a public assistant answering general questions may not. A monthly retrospective sample cannot be the only oversight for irreversible individual decisions, because it reviews after the point where it could prevent harm.
Amazon Augmented AI (Amazon A2I) is an AWS service for human review of machine learning predictions. For existing customers, it can send low-confidence predictions or random prediction samples[48] to human reviewers, letting reviewers step in when a model is unable to make a high-confidence prediction or to audit its predictions on an ongoing basis[48]; A2I is no longer open to new customers. The first pattern is a confidence-based review trigger: a flagged case goes to human review, because many ML applications require humans to review low-confidence predictions to ensure the results are correct[48]. The second supports supervisory review. A2I is one AWS implementation of a practice the Lens sets out, human review triggers[49], and the Lens names it as a tool for human review management[49], so the patterns apply whatever tool carries them.
What makes review meaningful
A reviewer who cannot inspect the evidence or overturn the recommendation provides nominal oversight, not effective oversight. Effective review needs three things:
- Expertise and training. The OAIC says staff should be trained and have the necessary resources to verify input data, critically assess outputs[9].
- Information. The Lens lists system output explanations and contributing factors, confidence scores, similar case examples[47] among the things reviewers need, not just an approve button.
- Authority. A human should be responsible for verifying the output and able to overturn decisions made[9].
Evaluate the combined human-and-AI workflow, not only the model. Without that, staff can come to over-rely on the assistant and overestimate its accuracy[9], and mandatory approval clicks turn into routine acceptance of unreliable outputs. NIST likewise says accuracy measures should consider human-AI teaming[10].
The takeaway: put review where it can still prevent the consequential error, and give reviewers the expertise, evidence, and authority to change the outcome.
Escalation: triggers and decision rights
This section covers escalation, the route that takes a single exception from the automated path to a person who can resolve it. An escalation rule has two halves, and both must be specified: the trigger that starts it and the decision rights at the end of it.
The trigger
The Lens says to configure human review triggers in monitoring systems based on alert severity, confidence thresholds, and business impact[49]. Typical triggers are harm severity, uncertainty (low confidence), a policy violation, and business impact. Confidence is not the only signal: a high-impact policy violation escalates even when the output appears confident, while an ambiguous, lower-risk case may follow a lighter review path.
The route and its decision rights
The Lens asks for oversight workflows that provide escalation paths, decision-making authority, and documentation requirements[49], with clear authority levels[49] for critical decisions. In practice the route names:
- the reviewer: a specific role, with backup contacts, as in the Lens's stakeholder contact list that includes backup contacts and escalation paths for important decisions[26];
- the required response: what the reviewer must do and by when;
- the decision authority: what the reviewer may approve, block, or override;
- the record: the Lens's example documents timestamps, reviewer identity, decision rationale[49] and the resulting change in system behavior.
Routing every exception to a general inbox with no decision rights supplies a notification but no accountable resolution.
The takeaway: an escalation rule is incomplete until it states both what triggers it and who is authorized to decide.
Recourse for people affected by AI decisions
This section covers recourse: a process that lets people affected by an AI decision report a problem, challenge the outcome, and have incorrect information or decisions reviewed and corrected. Human oversight (two sections above) is the organization checking its own work; recourse is the affected person's route in.
The frameworks agree on it. NIST's AI Risk Management Framework calls for feedback processes for end users and impacted communities to report problems and appeal system outcomes[10] that are integrated into AI system evaluation metrics, and its post-deployment monitoring plans include appeal and override[10]. The Lens's user journey ends with the post-interaction phase that should enable users to understand, challenge, and provide feedback on AI outputs[14]. The OAIC notes that without understanding how an AI product works, it is difficult for individuals to understand or challenge decisions, or request corrections[9].
Design recourse with three properties:
- Understandable and accessible. People must be able to find the route and use it, including people with disabilities.
- Able to remedy. The reviewers behind it need the authority to correct the record or reverse the decision, not just to acknowledge the complaint.
- Connected to evaluation. Feed validated complaints back into system evaluation. Resolving one person's case and spotting a recurring defect serve different needs, and identifying recurring defects and implementing corrective changes helps prevent repetition.
A satisfaction survey sent after an adverse decision gathers opinions; it neither reviews nor corrects that decision, so it is not recourse.
The takeaway: every consequential AI decision needs a way for the affected person to challenge it and seek correction when an error is found, and the complaints feed the next evaluation.
Bounding the autonomy of AI agents
This section covers AI agents, AI systems that can plan and carry out actions through tools instead of only answering questions. Once AI can act, oversight has to cover what it does, not only what it says.
Permission to inform is not permission to act
Approval for an information assistant does not extend to making commitments or changing business records on the organization's behalf. AWS guidance on governing AI adoption describes the risk of an agent connected to corporate systems without review: an autonomous process that acts on its behalf without oversight[50]. Deciding how far an autonomous agent can act before a human approves[50] is a judgment call for the organization's AI governance body, not a configuration detail.
Define the boundary before the agent runs
The same guidance gives the response: scope the autonomy by defining which agents can act on behalf of your organization and what approval thresholds apply before irreversible actions run[50]. In business terms, that means:
- List what the agent may do alone. Reversible, low-risk, well-tested actions, matching the placement rule in Placing human oversight in the workflow above.
- List what needs human approval. Irreversible or consequential actions, such as payments, contract commitments, or changes to customer records, with the approval required before the action runs. A guardrail on the action, not an input or output filter (the content kind of guardrail), can enforce this by stopping the action until a reviewer approves.
- Extend the records. Oversight covers agent registries, human-in-the-loop thresholds, and audit trails[50]. For agents, the Lens points to traces that show which knowledge sources or tools were consulted[11], so records cover the agent's actions and external interactions, not just its chat responses.
AWS guidance frames the balance: applications should be able to work autonomously, but humans should retain control over how they achieve their objectives, especially for high-stake decisions[40].
The takeaway: grant an agent autonomy action by action, with human approval set before anything irreversible happens.
Exam-pattern recognition: responsible AI scenarios
This section covers how responsible AI decisions show up as exam scenarios. The AWS AIB-C01 exam guide[2] lists Skill 3.1.1, "Apply responsible AI principles and dimensions (for example, fairness, explainability, privacy, safety, transparency, robustness) to business scenarios." That skill's core test: look for an answer that addresses the specific dimension or decision the scenario raises, rather than evidence or a control that addresses a different one. The same exam guide[2] also lists Skill 3.1.3, "Identify when responsible AI practices should be integrated into AI project planning to ensure governance by design," and Skill 3.1.4, "Recognize circumstances when AI systems require human oversight and identify appropriate safeguards (for example, hallucination detection, guardrails, escalation criteria)."
| Scenario cue | Right direction | Tempting wrong answer, and why it fails |
|---|---|---|
| A declined applicant asks why | An explanation of that individual decision | A portfolio-wide factor summary: it explains overall behavior, not this decision |
| A new use of customer personal data | A privacy review of whether the use is permitted | Encryption sign-off: it evaluates protection, not permission |
| A voice assistant fails for one dialect | Robustness testing on real input variations | Higher uptime: it measures availability, not performance under variation |
| Highest-accuracy option lacks a required explanation | The compliant option, or narrow or defer the use | Launch now and retrofit compliance: the obligation is unmet from day one |
| A rollout cannot meet a safety criterion | Narrow the use case; stop if harm stays uncontrolled | Lower the safety threshold: it changes the standard, not the risk |
| A supplier's system card | Use it to check intended use, then test on your own inputs | Treat the system card as go-live evidence for your population |
| Responsible AI review planned at production selection | Requirements at use-case definition (governance by design) | Review after the pilot: data and workflow decisions already locked in |
| A prohibited action the system can still perform | Enforce the boundary with filtering or baking | A user disclaimer alone: guiding does not enforce |
| Irreversible decisions about individuals | Human review before the effect, with authority to overturn | A monthly retrospective sample: it reviews too late to prevent harm |
| Exceptions with no owner | A trigger plus a named reviewer with decision rights | A shared inbox: notification without resolution |
| A chatbot approved for answers gains the ability to transact | New approval boundaries before irreversible actions | Extending the chatbot's approval to actions |
| An AI judge gives high scores | Validate the judge against human assessment first | Treat its scores as self-validating |
Reading the options
- Skill 3.1.1, dimension: does the option address the dimension in the scenario? Accuracy, uptime, encryption, and trust ratings are real measures, but an option that answers a fairness, privacy, or explainability question with one of them is insufficient if it does not test the specific requirement.
- Skill 3.1.3, timing: does it happen in time? Review after an irreversible effect, compliance after launch, and responsible AI review after the pilot all arrive too late.
- Skill 3.1.4, oversight and safeguards: does it enforce or only inform? Guidance, dashboards, notifications, and surveys inform. When the scenario needs a boundary held or a decision changed, look for a control or a person with authority.
The takeaway: match the remedy to the dimension, place it before the harm, and prefer enforcement plus authority over notices and visibility.
Baking vs filtering vs guiding: the three mitigation strategies
| Question | Baking | Filtering | Guiding |
|---|---|---|---|
| What it changes | The core AI system's own behavior (model choice, customization, design) | What passes into or out of the core AI system (guardrails block or modify content) | How people use the system (documentation, user guidance, terms of use, training) |
| Enforced by the system? | Yes, within the behavior the system was evaluated on | Yes, for the boundaries the filters are configured and tested to catch | No; it depends on people following it |
| Typical fit | Persistent problems in the system's own behavior | Prohibited topics, harmful content, sensitive data, ungrounded answers | Appropriate use, known limitations, when to verify an output |
| AWS example | Choosing or customizing the model the application uses | Amazon Bedrock Guardrails | A system card, such as an AWS AI Service Card, and user guides |
Decision tree
Sharp facts the exam loves — give these one last read before exam day.
Cheat sheet
Sharp facts the exam loves — scan these before test day.
- Fairness goals must reflect the people and outcomes a use case affects
Fairness considers impacts on different stakeholder groups, not just whether an AI system treats every input identically. Similar treatment of comparable individuals and appropriate outcomes across groups answer different fairness concerns; choose the goal with affected stakeholders and document conflicts between goals. Access also matters: balanced prediction rates do not make a service fair if people with disabilities cannot use it.
Trap Using identical interaction requirements for every customer as the fairness standard, which overlooks accessibility and differences in stakeholder needs.
5 questions test this
- A delivery platform's AI assigns jobs with one formula that it applies identically to every courier. Leaders call the formula fair for that reason. Data show that couriers based in outer suburbs recei
- A bank is launching an AI voice assistant for account servicing. To be fair, the project team proposes that every customer must use the same voice-only interaction, with no exceptions. Some customers
- An insurer uses AI to set renewal premiums. Its group-level metrics show similar average premiums across demographic groups. However, two customers with nearly identical risk profiles, who differ only
- A retailer's AI hiring screener does not use applicants' race or gender as inputs. The vendor therefore claims that the screener is fair. However, the model uses postal codes and school names, which c
- A university's AI tool ranks scholarship applicants. Analysts confirm that its goal of equal selection rates across applicant groups conflicts with its goal of giving similar scores to applicants with
- An explanation of one decision serves a different need from an explanation of overall behavior
Explainability helps people understand why an AI system produced an output. When an applicant needs reasons for one denial, require an explanation of that individual decision; when an oversight team needs to understand general behavior, require information about the factors influencing predictions across cases. Set the required explanation level with business and compliance stakeholders before committing to a solution.
Trap Providing a portfolio-wide factor summary to explain an individual denial, which does not establish what influenced that applicant's decision.
3 questions test this
- A credit union must tell each declined loan applicant the main reasons for the decision. The vendor offers a report that lists the five factors that matter most across the whole loan portfolio. The cr
- An insurer's model risk committee must approve a claims-triage model. The committee wants to understand which factors drive the model's predictions across all claims and to check that the overall patt
- A pharmaceutical company deploys a generative AI assistant that answers researchers' questions from internal study reports. Researchers say that they cannot tell which reports shaped an answer, so the
- Privacy governs appropriate data use while security protects data and systems
Privacy concerns whether information is appropriately obtained, used, and managed; security concerns protection against unauthorized access, manipulation, and disclosure. An authorized employee can still use personal information for an inappropriate purpose, so a privacy review is needed even when access is secure. Conversely, a permitted business purpose still requires protection against theft and tampering.
Trap Using encryption approval as the authorization for a new personal-data purpose, which evaluates protection rather than whether that use is permitted.
4 questions test this
- A bank's customer chatbot can look up account details for authenticated users. Testers find that a logged-in user can type hidden instructions that make the chatbot reveal another customer's account s
- A hospital's privacy office has approved using de-identified patient records to train an AI model that predicts readmissions. The approved purpose is clear and documented. The project team now propose
- A retailer collected customer purchase histories for order fulfillment. A product team now wants to use the same data to train an AI model that predicts customers' health conditions for targeted ads.
- A call center records customer calls for quality and training purposes, as its notice states. Its analytics team, which has approved access, wants to use the transcripts to train an AI that flags call
- Factual correctness does not establish that an AI output is safe
Safety addresses harmful outputs, actions, and misuse, whereas veracity addresses factual correctness. Accurate instructions can still enable harm, so a business must assess permitted uses as well as truthfulness. A harmless-sounding answer can also be factually wrong, requiring factual validation even when it passes a harmful-content check.
Trap Using a factual-accuracy review as the safety approval, which does not assess harmful uses of accurate information.
4 questions test this
- A media company uses a generative AI tool to draft investment news summaries. Every draft passes the company's harmful-content filter. An editor later discovers that a published summary cited a market
- A law firm's AI assistant drafts answers to client questions based on the firm's policy library. Evaluators currently score drafts only for fluent writing and absence of offensive language. Partners w
- An education company is building an AI tutor for children aged 8 to 12. A test answer about historical battles was factually accurate but included graphic descriptions of violence. The product team ar
- A chemical supplier is launching an AI assistant that answers customers' product questions. The launch plan treats a factual-accuracy review by chemists as the assistant's safety approval. Testers fou
- Robustness means maintaining appropriate performance as inputs vary
Robustness concerns correct outputs for expected and unexpected inputs, including ordinary changes in language, image quality, or environmental conditions. Require evaluation on the variations actual users will encounter, not only clean examples. Distinguish changes that should affect the answer from irrelevant variations the system should tolerate, and assess combinations such as accented speech with background noise.
Trap Using higher service uptime as the remedy for dialect-dependent errors, which measures availability rather than performance under input variation.
4 questions test this
- A pharmacy chain is building an AI assistant that answers medication questions typed by customers. Customers use varied spellings, abbreviations, and casual wording, while some changes in wording, suc
- A utility's AI voice assistant misunderstands customers who speak with certain regional dialects, which causes wrong account actions. The operations team proposes buying a higher-availability hosting
- A call center's AI transcription tool passed separate tests for accented speech and for background noise. In production, accuracy drops sharply for accented callers in noisy environments such as train
- An insurer is testing an AI tool that reads photographed claim forms. All test images were high-resolution scans prepared by the vendor. In production, customers will submit phone photos taken in poor
- Controllability requires interventions that reliably change AI behavior
Controllability means having mechanisms to monitor and steer system behavior, including adjustment, override, or rollback where appropriate. Judge a control by whether it produces the intended change quickly enough for the business risk, and test whether it creates unintended effects elsewhere. A visible control is insufficient evidence unless users can operate it effectively and understand when it cannot work.
Trap Using a monitoring dashboard as the complete controllability mechanism, which provides visibility without an effective way to intervene.
4 questions test this
- A bank's AI fraud system can be paused by operations staff if it starts wrongly blocking legitimate card payments. A drill showed that the pause took six hours to take effect because of manual approva
- A marketing team's AI writing tool lets users set a tone control from formal to casual. User tests show that the setting changes the tone for short emails but is silently ignored for long reports, so
- A social platform lets moderators raise the strength of its AI content filter during breaking news events. After a recent increase, harmful posts dropped, but posts from news organizations reporting o
- A retailer's AI pricing system adjusts online prices every hour. The governance team approved a real-time dashboard that shows price changes as its controllability mechanism. During a test, the system
- Equal approval rates and equal recognition of qualified people are different fairness goals
Demographic parity asks whether groups receive positive decisions at comparable rates, such as similar shares of all applicants being recommended for a job. Equal opportunity asks whether genuinely qualified applicants are correctly recommended at comparable rates across groups. Because the first comparison includes all applicants and the second includes qualified applicants, meeting one goal does not establish the other; agree which concern matters in the business context before judging the results.
Trap Using similar overall accuracy across groups to assess equal recognition of qualified candidates, because overall correctness also counts negative cases and can conceal differences in identifying qualified people.
- AI disclosure should let people make informed choices before relying on the system
Transparency enables people to make informed choices about engaging with AI. Identify where users first encounter the system and where they act on its outputs, then disclose AI involvement and relevant limitations at those points. Match the prominence and frequency of disclosures to the context and risk rather than relying only on a distant policy page.
Trap Placing the only AI notice in an annual corporate report, which does not inform users at the points where they engage with or rely on AI.
4 questions test this
- A tax software company adds an AI feature that suggests deductions. The feature cannot handle some complex situations, such as income earned in several countries. Users currently learn about this limi
- A retailer added an AI shopping assistant with a short label that says it uses AI. Leaders believe the disclosure requirement is complete. A survey finds that half of the customers still think a human
- A bank uses AI in two places: a feature that recommends articles on its website, and a tool that pre-screens mortgage applications. The compliance team proposes the same small footnote for both. Mortg
- An insurer uses AI to draft responses in its customer chat service. The only statement that AI is involved appears in the company's annual corporate report. Customers act on chat answers to decide whe
- System cards communicate intended uses and limitations but do not replace local evaluation
An AI system card brings together intended uses, limitations, responsible AI design choices, and relevant performance information. Use it to judge whether a proposed application fits the documented boundaries and to prepare guidance for downstream users. The deploying organization must still evaluate performance on its own inputs and workflow; supplier documentation does not establish suitability for every customer context.
Trap Using a supplier's general service card as the go-live evidence for a new population, which omits evaluation of the deployer's specific inputs and conditions.
4 questions test this
- A healthcare provider plans to use a vendor's speech-to-text AI service to transcribe doctor visits. The vendor's service card reports strong accuracy on general business speech. The provider's visits
- A logistics company deploys a vendor's AI route-planning service for its dispatchers. The vendor's service card states that the service is not designed for hazardous-material shipments, which have spe
- A bank is considering an AI document-processing service for two uses: extracting fields from standard loan forms and reading handwritten notes from branch staff. The service card lists typed forms as
- A company built an internal AI model that drafts first responses to supplier contract questions. Several business units now want to use it. Leaders ask the team to create a system card. A team member
- Useful transparency is tailored to the audience and protects sensitive information
A transparency strategy should match each stakeholder's information needs, expertise, accessibility requirements, and communication channel. Customers may need plain-language capabilities and limitations, while authorized reviewers need more detailed evidence. Review public disclosures separately from restricted materials so that openness does not reveal personal data, proprietary information, or exploitable security details.
Trap Publishing raw internal evaluation records to maximize transparency, which can expose protected information without giving users an understandable explanation.
4 questions test this
- A card issuer's AI fraud system blocks suspicious transactions. Customers ask how it decides, and a product manager proposes publishing the exact score thresholds and detection rules online. The secur
- A lender wants to show that its AI credit model is transparent. A manager proposes publishing the full internal evaluation records online, including test datasets with customer details and notes on kn
- A tax authority's AI voice assistant answers callers on its phone helpline. Information about what the assistant can and cannot do appears only in a technical PDF on the authority's website. Most help
- A hotel chain is preparing a public explanation of its AI booking assistant. A draft includes example conversations copied from production logs, with guest names and room numbers, and a list of known
- AI user guidance must change when capabilities, limitations, or intended uses change
AI documentation needs an owner, review process, version history, and a way to reach affected users. Trigger review after system releases, new uses, significant stakeholder feedback, or relevant changes in expectations. Updating an internal document is only part of the task: publish the approved guidance through the channels users actually rely on and address outdated copies.
Trap Updating the engineering release notes alone, which does not revise or distribute the guidance business users need for responsible use.
4 questions test this
- An insurer revised the guidance for its AI claims-estimate tool after finding that the tool is unreliable for flood damage. The new guidance is published on the intranet. However, adjusters still use
- A retailer's AI pricing assistant was approved for setting prices on in-store items. Managers now want to use it for online flash sales, which it was not designed for. Store staff have also given feed
- A bank updated its AI expense-review assistant so that it no longer checks receipts in foreign currencies. Engineers recorded the change in their release notes. Finance staff, who rely on a user guide
- A manufacturer has published guidance for its AI quality-inspection tool, but no one is responsible for keeping it current. Different plants use different versions of the guidance, and nobody can tell
- Uncertainty information is useful only if users know when to rely on or question an output
AI guidance should explain uncertainty, known limitations, and the circumstances that call for additional verification. Evaluate whether different user groups actually recognize when an output deserves trust, further checking, or rejection. Measure comprehension and decision quality as well as reported trust, because a persuasive interface can increase confidence without improving judgment.
Trap Using higher user trust ratings as the sole proof of effective explanations, which does not show that users can identify unreliable recommendations.
4 questions test this
- A utility's AI tool gives call center agents repair-time estimates for power outages. The estimates are reliable for routine outages but often wrong during major storms, and agents pass the wrong time
- A legal research tool shows paralegals a confidence label with each AI-suggested case citation. After a redesign, average trust ratings rose from 3.1 to 4.4 out of 5. Partners will expand the tool to
- A bank redesigned the explanations in its AI fraud-alert tool, which flags about 2,000 card transactions each day for analyst review. After the redesign, analysts rated their trust in the tool much hi
- A clinic's AI tool suggests likely diagnoses with a confidence score for each one. Nurses say they do not know what to do with a score of 60 percent, so some ignore all suggestions while others follow
- Business performance improvements cannot substitute for mandatory obligations
When an AI use case has a mandatory explanation, privacy, or other compliance requirement, treat that requirement as a condition of proceeding. Accept a lower business-performance result if it is the feasible compliant option and still meets necessary operating standards. If no option meets the obligation, change the scope or defer the use rather than promising compliance after release.
Trap Selecting the highest-accuracy option with a later compliance retrofit, which leaves the mandatory obligation unmet when the system begins making decisions.
4 questions test this
- A fitness app must launch an AI coaching feature within six weeks under a signed partnership contract. Local law requires explicit consent before health data is used to train models. A model trained o
- An auto insurer must replace its retiring claims system within three months. Regulation requires a licensed adjuster to approve every claim denial before it takes effect. Option A denies claims automa
- A bank wants an AI assistant to recommend investment products to retail customers. Local rules require each recommendation to include an explanation of its suitability for the customer. After several
- A city housing agency wants AI to decide rental-assistance eligibility and to sort incoming applications by missing documents. Local law requires every eligibility denial notice to state the specific
- A responsible AI tradeoff considers who receives the benefit and who bears the harm
Evaluate benefits and harms separately for affected stakeholder groups rather than treating company-wide gains as a complete justification. Include people indirectly affected by decisions and groups that may experience disproportionately severe harm. Use explicit organizational principles and stakeholder needs to prioritize competing outcomes, and document the rationale so later decisions use the same standards.
Trap Using total company savings as the complete tradeoff criterion, which leaves the distribution and severity of stakeholder harms unexamined.
4 questions test this
- A delivery company's AI route optimizer cuts fuel costs by 12 percent company-wide. The new routes send heavy trucks past two primary schools during morning drop-off. The business case counts only com
- A bank's AI collections tool raised loan recoveries by 9 percent by contacting overdue customers more often. Complaints across the whole customer base stayed low, but customers in documented financial
- A property management company sells an AI tenant-screening tool to landlords. Its business case lists benefits for landlords, such as faster approvals and fewer defaults. Rental applicants never use t
- A utility plans to replace phone agents with an AI voice system, which would save USD 5 million each year. Analysis shows that many older customers and customers with hearing impairments cannot comple
- Privacy-enhancing choices must be evaluated alongside utility and fairness
Privacy protections such as minimizing, de-identifying, or aggregating information can reduce exposure but may also reduce useful detail. Evaluate the effect on the business task and on different groups, particularly where data is sparse. Choose the least sensitive information that can support the authorized purpose, and document acceptable utility tradeoffs without relaxing mandatory privacy obligations.
Trap Maximizing data aggregation as the only design goal, which can remove detail needed for useful or fair outcomes in smaller groups.
4 questions test this
- A clinic must de-identify patient notes before using them to train an AI documentation assistant. De-identification lowers the assistant's accuracy for medication names. The project lead proposes skip
- A telecom company wants to use an AI service to draft personalized retention offers. The marketing team proposes sending the service each customer's full record, including payment history, location hi
- A bank plans to protect privacy by aggregating customer data to the regional level before training a credit model. Testing shows that the model's accuracy drops sharply for a small rural customer grou
- A hiring platform removed age, gender, and ethnicity fields from its candidate data to protect privacy before training a screening model. The fairness team now reports that it cannot measure whether o
- Safety controls should be assessed for missed harms and unnecessary blocking
Protective filters can miss harmful material and can also block legitimate requests. Assess both kinds of error against the use case, user population, and harm severity before agreeing acceptance criteria. A stricter control may be justified for severe harms, while ambiguous legitimate requests may need review rather than automatic rejection; monitor useful service delivery alongside protection effectiveness.
Trap Using the percentage of blocked requests as the sole safety success metric, which rewards unnecessary blocking as well as genuine protection.
4 questions test this
- A bank deployed content filters on its customer chatbot. The safety team reports success because the filters block 18 percent of all customer requests. Customer complaints show that many blocked reque
- An insurer's AI assistant answers policyholder questions. Its filter automatically rejects any question that mentions injuries. As a result, legitimate questions about how to file an injury claim are
- A mental health support app uses AI to respond to user messages. Filters that detect messages about self-harm sometimes flag ordinary conversations about stress. Product managers want to relax the sel
- A travel company launched an AI booking assistant with content filters that met its acceptance criteria before launch. Six months later, the company tracks only the number of harmful responses that th
- Narrowing an AI use case can preserve value when broad deployment is not supportable
When a system cannot meet essential criteria across a broad application, restrict it to domains, inputs, locations, or users for which it can meet the required standards. Reassess benefits and risks and update the criteria for that narrower use. Scope reduction is a valid risk treatment when meaningful value remains; persistent inability to control harm is a reason to stop that use, not lower essential standards.
Trap Lowering the safety acceptance threshold to preserve the original rollout scope, which changes the protection standard instead of resolving the unsupported use.
4 questions test this
- A pharmacy narrowed its AI assistant from all medication questions to dosage reminders for one chronic condition. Even in this narrow scope, testing shows that the assistant sometimes gives wrong dosa
- A bank narrowed its AI advisor from all investors to retail investors who use standardized products, because the broader scope failed its release criteria. The team plans to launch the narrower versio
- A logistics company's AI support agent gives unreliable answers when customers type long free-text messages with several requests. It performs reliably when customers choose a request type and enter a
- A company built a general-purpose AI help assistant. Release testing shows that it consistently fails the safety criteria on medical questions, despite several redesigns, but it meets all criteria on
- Responsible AI belongs in initial planning and continues throughout the lifecycle
Governance by design incorporates responsible AI requirements when defining the use case, before data and solution commitments lock in avoidable risks. Carry those requirements into evaluation, user guidance, production monitoring, and retirement. Iterative pilots can still move quickly, but their boundaries and required approvals must be clear from the start rather than postponed until production.
Trap Scheduling responsible AI review only after the pilot is selected for production, which allows consequential data and workflow decisions to precede the relevant requirements.
5 questions test this
- A media company wants to pilot a generative AI tool that drafts article summaries within six weeks. Leaders want fast iteration during the pilot. The compliance lead worries that the pilot will quietl
- A hospital network wants an AI assistant for its staff. The innovation team proposes building one assistant that can answer any staff question and deciding later which uses are acceptable. The governa
- A bank plans to retire an AI model that scores small-business loan applications. Two internal workflows depend on the model's scores, and its training dataset contains applicant records. Leaders want
- A retail company will deploy an AI pricing assistant in three countries next year. The product team has drafted the use case and wants to sign a vendor contract next week. Nobody has yet checked which
- An insurance company launched an AI assistant that helps adjusters estimate repair costs. The launch passed every responsible AI release criterion. The project sponsor now proposes closing the respons
- AI planning must include upstream contributors and people indirectly affected by outputs
Stakeholder mapping should cover direct users, people whose information is processed, people affected without using the system, and upstream contributors such as suppliers. Seek perspectives outside the project team, including vulnerable populations where relevant, to reveal missed requirements and harm scenarios. Revisit the map when the workflow or audience changes rather than treating the initial list as permanent.
Trap Surveying only paying users to establish stakeholder requirements, which omits data subjects, indirect stakeholders, and upstream dependencies.
5 questions test this
- A retailer is planning an AI demand forecasting system. The forecasts will depend on weather data from an external supplier and on a model component licensed from a vendor. The project's stakeholder l
- A telecom company is planning an AI system that flags accounts for possible fraud by analyzing call records. The product team proposes surveying paying subscribers to collect all stakeholder requireme
- A bank deployed an AI assistant that answers employees' policy questions. The bank now plans to let customers use the same assistant through its mobile app. The stakeholder map was created two years a
- A pharmaceutical company's AI team built a stakeholder list for a drug-safety report summarizer. The list names data scientists, engineers, and the safety analysts who will use the tool. Late in testi
- A city utility is planning an AI chatbot that will help residents apply for bill-payment assistance. The project team consists of engineers and billing analysts. Some likely applicants are elderly or
- Responsible AI requirements need measurable pass-or-fail release criteria
Translate expected benefits and identified harms into clear release questions, measurements, and acceptance thresholds before building or buying around the use case. Use existing performance evidence, stakeholder requirements, and risk tolerance to set realistic targets, and identify who is affected by each failed criterion. A complex promise such as a good user experience may need separate criteria for correctness, response time, and usability; no single universal accuracy percentage establishes responsible readiness.
5 questions test this
- A recruiting platform has listed benefits and harms for an AI tool that ranks job applicants. Its draft release criteria measure only benefits, such as recruiter time saved per hire. The harm list inc
- A travel company is defining release criteria for an AI booking assistant. The business sponsor's only stated criterion is that the assistant must deliver a good user experience. The product team need
- An insurer is setting a release threshold for an AI model that triages incoming claims. A consultant recommends a 99 percent accuracy target because it sounds safe. The insurer's current manual triage
- A bank is comparing three vendor proposals for an AI tool that summarizes customer complaints. Each vendor reports its own accuracy figure from different test data. The bank has identified expected be
- A legal services firm plans to release an AI tool that drafts contract summaries. The draft release criterion reads: leadership is comfortable with summary quality after reviewing examples. Two review
- Risk mitigation can combine safer system behavior, protective filtering, and user guidance
Assign each identified harm to a mitigation strategy: improve the underlying behavior, screen inputs or outputs, guide users, or combine these approaches. Match the strategy to how the harm arises and how much users can reasonably control. Guidance is valuable for appropriate use, while risks requiring enforced boundaries need system safeguards as well; evaluate the resulting protection rather than assuming one strategy fits every harm.
Trap Using a user disclaimer as the sole control for a prohibited system action, which asks users to respect a boundary that the system still permits.
5 questions test this
- A bank's customer chatbot must never provide personalized investment advice, which bank policy prohibits. The project team proposes a banner telling customers not to ask for investment advice. Testing
- A software company's AI coding assistant keeps generating code that uses a deprecated internal library. The team has already added user warnings and an output filter, but the problem persists across m
- An insurer's AI assistant answers customers' coverage questions. Testing shows that it sometimes states coverage terms that do not appear in the policy documents. Most customers cannot tell whether an
- A bank's internal AI assistant summarizes customer emails for service agents. Agents sometimes paste full account numbers into their requests, and bank policy prohibits sending account numbers to the
- A healthcare provider assigned two mitigations for its patient chatbot. A guardrail blocks dosage instructions, and on-screen guidance tells patients to contact a clinician about medications. The proj
- Critical or subjective AI evaluations benefit from an independent second assessment
When safety, fairness, or another consequential judgment depends heavily on interpretation, obtain corroboration from qualified reviewers outside the delivery team's incentives. For higher-risk uses, an independent team can develop evaluation cases from the documented intended use rather than relying entirely on cases chosen by the supplier or project team. Investigate material disagreements before approval and keep both findings as decision evidence.
Trap Repeating the delivery team's own demonstration as an independent review, which retains the same case-selection assumptions and incentives.
5 questions test this
- An insurer ran parallel safety evaluations of its AI claims assistant. The delivery team found no serious issues. An independent quality assurance team, using the same criteria, found several cases of
- A retailer's AI team has a limited budget for independent evaluation of a new shopping assistant. Planned evaluations include response latency, uptime, a safety assessment of harmful advice, and an as
- A bank's AI assistant team and an independent research team will both evaluate whether the assistant's answers show unwanted bias. The research team plans to use its own criteria and a different datas
- A healthcare company is buying an AI triage system that the supplier will help deploy. The supplier offers its own test cases, which show excellent results. Because the use is high risk, leaders want
- A public agency must choose independent reviewers for a high-risk AI eligibility system. Candidates include the system's original developers, a vendor that hopes to win the maintenance contract, and a
- Evaluation evidence must identify the system and data versions actually being approved
System updates require renewed evaluation and traceable records of the version, evaluation data, conditions, methods, and results. Recheck that criteria still cover current capabilities and relevant expectations instead of carrying forward approval for a materially different system. Link evaluation evidence to the release decision so an auditor can reconstruct what was tested and why that version was accepted.
Trap Using the previous version's safety report for an expanded release, which provides evidence about a different system and possibly different risks.
5 questions test this
- A manufacturer keeps evaluation reports for its AI defect-detection model in a shared folder, separate from its model and dataset records. A regulator asks the company to show which model build and da
- A company's AI support agent was approved last year based on a safety report for version 1, which could only answer questions. Version 2 can now issue refunds and change customer orders. The product t
- A bank's governance board approved the release of an AI document assistant in a meeting, but no one recorded why it was approved, who signed off, or which conditions applied. Six months later, an inte
- A retailer is preparing to release version 3 of its AI recommendation assistant. Version 3 adds image search, which did not exist when the release criteria were written. The team plans to reuse the ex
- An insurer evaluates its AI claims model on a public benchmark that it downloads from an external site for each test run. The benchmark's maintainers sometimes update the data without notice. Results
- Automated evaluation scales assessment but its judgments still need validation
Choose automated evaluation for repeatable measurements it can capture reliably, and use qualified human assessment for context or experience the automated method misses. An AI system used to judge another AI system should be checked against human judgments before its scores become approval evidence. Human assessment also needs clear criteria and consistency checks; neither automation nor expert involvement makes an evaluation method automatically reliable.
Trap Using an AI evaluator's high scores as self-validating approval evidence, which omits whether its judgments match the intended quality standard.
5 questions test this
- A bank's AI assistant passes an automated accuracy metric for answers about account fees. However, customer feedback shows that some technically correct answers are confusing and do not resolve the cu
- A healthcare company asks three clinicians to rate an AI tool's patient-education answers for accuracy. Review of the ratings shows that one clinician scores almost every answer as excellent, while th
- A bank is planning evaluations for an AI mortgage assistant. Two release criteria are under discussion: whether responses arrive within two seconds, and whether first-time buyers find the application
- A company uses a large language model as a judge to score its customer service assistant's answers for helpfulness. The judge gives average scores of 4.8 out of 5, and the product team wants to use th
- An insurer's AI assistant writes summaries of claim files. Release criteria require that summaries be coherent and follow the insurer's style guidelines. The team must evaluate about 50,000 summaries
- Human oversight should be placed where expertise can prevent consequential errors
Map the business workflow to identify decisions whose errors are hard to judge or have serious consequences. Require human review before an effect occurs when later correction would be inadequate; lower-risk, reversible activity may support automated handling with supervisory review. Oversight placement depends on the actual consequence and the value of human expertise, not merely whether a system is customer-facing.
Trap Using a monthly retrospective sample as the only oversight for irreversible individual decisions, which places review after the point at which it could prevent harm.
5 questions test this
- A mortgage lender uses AI to extract income figures from scanned application documents. Most scans are clear, but some are low quality or handwritten, and extraction errors then become hard to spot la
- A manufacturer has two AI tools. A customer-facing chatbot answers questions about store hours and product colors. An internal tool recommends which employees to lay off during restructuring. The over
- A health insurer uses an AI system to recommend denying coverage for some specialist treatments. Denials take effect immediately and are hard to reverse once a patient cancels treatment. The current o
- A professional services firm uses AI to tag incoming internal documents by topic for its search portal. Wrong tags are easy to correct and cause only minor search delays. The firm's governance lead pr
- A bank uses an AI tool that summarizes customer complaint emails for internal trend reports, with no human review. The bank now wants to use the same summaries to decide which customers receive compen
- Human review is meaningful only when reviewers can assess and overturn AI recommendations
Human reviewers need relevant expertise, training, sufficient information, and authority to reject or correct an AI recommendation. Give them the input evidence, explanations, and limitations needed for independent judgment, not just an approve button. Evaluate the human-and-AI workflow as well as the system itself so that nominal review does not become routine acceptance of unreliable outputs.
Trap Adding mandatory approval clicks for staff who cannot inspect the evidence or overturn the recommendation, which creates nominal rather than effective oversight.
4 questions test this
- An insurer added a mandatory human approval step for AI-recommended claim rejections. Reviewers see only the recommendation and an Approve button. Rejecting a recommendation requires a manager's writt
- A retailer's fraud team reviews AI alerts before blocking customer orders. The vendor reports 96 percent model accuracy. However, the team's combined decisions have caused rising wrongful order blocks
- A tax agency's caseworkers review AI recommendations to audit specific returns. Caseworkers see only a risk score from 1 to 10. They report that they cannot tell why a return scored high, so they eith
- A hospital uses AI to flag chest scans that might show early lung disease. The current reviewers are administrative staff who confirm flags by checking that the scan file opened correctly. The hospita
- Escalation rules must specify both the trigger and the person authorized to act
Define escalation criteria using factors such as harm severity, uncertainty, policy violations, and business impact. Specify the reviewer, required response, decision authority, and documentation so that an exception reaches someone able to resolve it. High-impact policy violations may require escalation even when an output appears confident, while ambiguous lower-risk cases may follow a different review path.
Trap Routing every exception to a general inbox without decision rights, which supplies notification but no accountable resolution path.
4 questions test this
- A payments company escalates AI fraud alerts to analysts when certain criteria are met. Six months after launch, analysts report that many escalations are false alarms, while some real fraud patterns
- A social media company uses AI to moderate user posts. Some flagged posts are clearly violent threats, while others are ambiguous jokes with low risk. All flags currently follow one review path, and t
- A grocery chain's AI system raises about 150 food safety flags from supplier data each day, most of them minor. Flags are emailed to store managers, who cannot remove products from other stores. Last
- A telecom company's AI system flags billing anomalies that may overcharge customers. All flagged exceptions go to a shared mailbox that nobody owns. Exceptions often sit for weeks, and no one has auth
- People affected by AI decisions need a way to challenge and correct outcomes
A recourse process lets affected people report problems, challenge an outcome, and obtain review of incorrect information or decisions. Make the route understandable and accessible and give reviewers the ability to provide an appropriate remedy. Feed validated complaints into system evaluation as well, because resolving one case and identifying a recurring problem serve different needs.
Trap Offering only a satisfaction survey after an adverse decision, which gathers opinions without providing review or correction of that decision.
4 questions test this
- A government agency uses AI to help decide eligibility for housing benefits. Applicants who are denied receive only a notice that says they are ineligible. Advocacy groups say that applicants cannot c
- A utility company uses AI to decide which customers qualify for a discounted rate. Customers can dispute a decision only by mailing a form described on page 14 of the terms of service. Many affected c
- A credit card issuer's AI system lowers credit limits automatically. Customers who appeal speak to an agent who can explain the decision but cannot change it or fix incorrect data in the customer reco
- A rental marketplace uses an AI tool to screen tenant applications. Rejected applicants currently receive only a satisfaction survey about the application experience. Several applicants say that the r
- Agent autonomy requires explicit boundaries on actions taken for the organization
When AI can act rather than merely recommend, define which activities it may perform autonomously and which require human approval. Set approval boundaries before irreversible or consequential effects and extend oversight records to actions and external interactions, not just chat responses. An approved information assistant does not automatically have approval to make commitments or change business records on the company's behalf.
Trap Extending a chatbot's approval to autonomous business transactions, which treats permission to provide information as permission to act.
4 questions test this
- A retailer approved an AI chatbot last year to answer customer questions about orders and return policies. The product team has now connected the same chatbot to the payment system so that it can issu
- A sales manager connected an unvetted AI agent to the corporate mailbox so that it can reply to customers and promise delivery dates automatically. No one approved the agent's ability to make commitme
- A telecom company's AI agent can update customer plans, apply credits, and cancel contracts. Cancelling a contract ends service and releases the customer's phone number, which cannot always be recover
- A bank's AI agent schedules payments and sends emails to suppliers on the bank's behalf. Its oversight records contain only the chat conversations between staff and the agent. When a supplier disputes
- Guardrails turn AI policy boundaries into enforced checks
Guardrails are safeguards that enforce permitted boundaries on AI inputs, outputs, or actions, such as blocking prohibited content or preventing an action until a reviewer approves it. Recognize them as the enforcement layer that supports the organization's mitigation policy, rather than as another name for user guidance or human judgment. Guardrails complement human oversight: unfamiliar or consequential cases still need the appropriate review and escalation path.
Trap Using a system prompt as the sole policy-enforcement control, because instructions to the model do not independently verify compliance.
4 questions test this
- A bank's AI assistant must never give personalized investment advice. The team's only control is a system prompt that tells the model not to give such advice. Testers found that rephrased requests som
- A bank's policy prohibits storing full card numbers from its AI support chat. Customers sometimes type card numbers, and the assistant has repeated them back. Auditors require a control that checks ev
- A health insurer's policy prohibits its AI assistant from giving diagnoses. A project manager proposes meeting this policy by adding a sentence to the user guide that asks members not to request diagn
- A media company configured guardrails for its AI writing assistant to enforce a policy against defamatory claims about real people. The configuration was based on the policy's general wording, and no
Also tested in
References
- https://docs.aws.amazon.com/wellarchitected/latest/responsible-ai-lens/responsible-ai.html
- https://docs.aws.amazon.com/aws-certification/latest/ai-business-strategist-01/ai-business-strategist-01-domain3.html
- https://aws.amazon.com/ai/responsible-ai/
- https://docs.aws.amazon.com/wellarchitected/latest/responsible-ai-lens/raibr02-bp01.html
- https://docs.aws.amazon.com/wellarchitected/latest/responsible-ai-lens/rairc03-bp02.html
- https://docs.aws.amazon.com/wellarchitected/latest/responsible-ai-lens/raiuc04-bp03.html
- https://docs.aws.amazon.com/sagemaker/latest/dg/clarify-model-explainability.html
- https://docs.aws.amazon.com/wellarchitected/latest/responsible-ai-lens/raisp02-bp07.html
- https://oaic.gov.au/privacy/privacy-guidance-for-organisations-and-government-agencies/guidance-on-privacy-and-the-use-of-commercially-available-ai-products
- https://nvlpubs.nist.gov/nistpubs/ai/NIST.AI.100-1.pdf
- https://docs.aws.amazon.com/wellarchitected/latest/responsible-ai-lens/raigt01-bp04.html
- https://docs.aws.amazon.com/wellarchitected/latest/responsible-ai-lens/rairc03-bp08.html
- https://docs.aws.amazon.com/wellarchitected/latest/responsible-ai-lens/raibr02-bp08.html
- https://docs.aws.amazon.com/wellarchitected/latest/responsible-ai-lens/raiuc04-bp01.html
- https://docs.aws.amazon.com/wellarchitected/latest/responsible-ai-lens/raigt01-bp02.html
- https://docs.aws.amazon.com/ai/responsible-ai/bedrock-guardrails/overview.html
- https://docs.aws.amazon.com/wellarchitected/latest/responsible-ai-lens/raigt01-bp01.html
- https://docs.aws.amazon.com/wellarchitected/latest/responsible-ai-lens/rairc03-bp10.html
- https://docs.aws.amazon.com/wellarchitected/latest/responsible-ai-lens/raigt01-bp03.html
- https://docs.aws.amazon.com/wellarchitected/latest/responsible-ai-lens/raibr02-bp05.html
- https://docs.aws.amazon.com/wellarchitected/latest/responsible-ai-lens/raibr02-bp03.html
- https://docs.aws.amazon.com/wellarchitected/latest/responsible-ai-lens/rairc03-bp04.html
- https://docs.aws.amazon.com/wellarchitected/latest/responsible-ai-lens/rairc03-bp07.html
- https://docs.aws.amazon.com/wellarchitected/latest/responsible-ai-lens/design-principles.html
- https://docs.aws.amazon.com/wellarchitected/latest/responsible-ai-lens/raiuc02-bp01.html
- https://docs.aws.amazon.com/wellarchitected/latest/responsible-ai-lens/raiuc02-bp02.html
- https://docs.aws.amazon.com/wellarchitected/latest/responsible-ai-lens/release-criteria.html
- https://docs.aws.amazon.com/wellarchitected/latest/responsible-ai-lens/rairc01-bp01.html
- https://docs.aws.amazon.com/wellarchitected/latest/responsible-ai-lens/rairc04-bp01.html
- https://docs.aws.amazon.com/wellarchitected/latest/responsible-ai-lens/rairc02-bp01.html
- https://docs.aws.amazon.com/wellarchitected/latest/responsible-ai-lens/raier01-bp02.html
- https://docs.aws.amazon.com/wellarchitected/latest/responsible-ai-lens/raier01-bp03.html
- https://docs.aws.amazon.com/wellarchitected/latest/responsible-ai-lens/raier01-bp01.html
- https://docs.aws.amazon.com/bedrock/latest/userguide/evaluation.html
- https://docs.aws.amazon.com/wellarchitected/latest/responsible-ai-lens/rairc02-bp02.html
- https://docs.aws.amazon.com/wellarchitected/latest/responsible-ai-lens/raibr04-bp03.html
- https://docs.aws.amazon.com/wellarchitected/latest/responsible-ai-lens/system-planning.html
- https://docs.aws.amazon.com/wellarchitected/latest/responsible-ai-lens/raier03-bp01.html
- https://docs.aws.amazon.com/bedrock/latest/userguide/guardrails.html
- https://docs.aws.amazon.com/prescriptive-guidance/latest/gen-ai-lifecycle-operational-excellence/prod-monitoring-feedback.html
- https://docs.aws.amazon.com/wellarchitected/latest/responsible-ai-lens/rairc04-bp02.html
- https://docs.aws.amazon.com/wellarchitected/latest/responsible-ai-lens/benefits-and-risks.html
- https://docs.aws.amazon.com/wellarchitected/latest/responsible-ai-lens/raibr03-bp02.html
- https://docs.aws.amazon.com/wellarchitected/latest/responsible-ai-lens/raibr04-bp02.html
- https://docs.aws.amazon.com/wellarchitected/latest/responsible-ai-lens/raibr04-bp01.html
- https://docs.aws.amazon.com/wellarchitected/latest/responsible-ai-lens/raier03-bp02.html
- https://docs.aws.amazon.com/wellarchitected/latest/responsible-ai-lens/raiuc04-bp02.html
- https://docs.aws.amazon.com/sagemaker/latest/dg/a2i-use-augmented-ai-a2i-human-review-loops.html
- https://docs.aws.amazon.com/wellarchitected/latest/responsible-ai-lens/raimon01-bp05.html
- https://aws.amazon.com/blogs/migration-and-modernization/govern-ai-adoption-before-it-governs-you/