AIB-C01 Cheat Sheet
AI Fundamentals and Literacy
Describe core AI concepts and terminology
Read full chapterCheat sheet
Sharp facts the exam loves — scan these before test day.
- AI, machine learning, and generative AI describe related but different capabilities
Artificial intelligence is the broad category of systems performing tasks such as perception, reasoning, and prediction; machine learning is an approach within AI that learns patterns from data. Deep learning is a form of ML using multilayer neural networks, and modern generative AI uses learned patterns to produce content such as text, images, or audio. A system can classify transactions or recommend products without generating new content, while a generative model can also perform tasks such as classification when prompted.
Trap Using content generation as the defining requirement for every AI application, which excludes predictive and recognition capabilities.
4 questions test this
- A manufacturer runs two AI review tracks. Systems that produce new text, images, or audio go through a generative AI track with extra content-safety review. Predictive and recognition systems go throu
- An insurance company wants to automate the first review of auto claims. It needs a capability that assigns each claim a likelihood of fraud from past claim outcomes, and it wants to avoid paying for f
- A bank's innovation board funds only projects it can categorize accurately for its AI portfolio report. A team proposes a system that scores each card transaction in real time and flags likely fraud f
- A retailer's leadership team wants a single sentence in its AI policy that defines which projects fall under AI governance. Draft wording says that AI means any system that generates new content. The
- An algorithm is a method, while a trained model contains learned relationships
A learning algorithm is the procedure used to discover patterns, whereas a trained model is the resulting representation of those patterns and learned parameter values. The same learning procedure can produce different models when trained on different business data. When discussing an AI proposal, distinguish changing the learning method from updating the learned model with new training evidence.
5 questions test this
- A bank's model governance policy requires full validation whenever the learning method behind a model changes. A lighter performance check applies when the same method is retrained on newer data. The
- A retailer is ending a contract with a consultancy that built its demand forecasting system. The retailer must keep producing forecasts on day one after handover without retraining anything. The consu
- An insurer's regulator can ask it to reproduce any automated claim decision from the past seven years. The insurer retrains its claims model every quarter with the same learning method, and each retra
- A payments company built a fraud model on transactions from its home country and now plans to launch in a second country with different shopping habits. A product manager says that because the learnin
- A hotel group's pricing model was trained four years ago, and its rate recommendations now miss current booking patterns. The data team still trusts the learning method and wants to keep costs low. A
- Training learns patterns, while inference applies a model to a new request
Training uses examples to learn model parameters; inference uses an already trained model to produce a prediction or response for an input. Forecasting next month's demand or generating a reply is inference, even when the input contains information the model has not previously encountered. Ordinary inference does not itself update the model's trained parameters; learning from newly collected examples requires an explicit training or updating process.
Trap Treating a growing archive of prediction requests as automatic retraining, which confuses collecting examples with updating model parameters.
5 questions test this
- A logistics company is planning a parcel delivery-time model. The project sponsor must assign two budgets: one for gathering five years of delivery records with actual arrival times, and one for syste
- A lender's credit model, trained on ten years of applications, receives an application from a profession that rarely appears in its training data. A loan officer insists the model must be retrained be
- A bank's chatbot answers customer questions with a deployed language model. Compliance staff are worried that customer messages could teach the model to repeat one customer's details to others. The ba
- A software company plans a generative AI assistant for its 40,000 customers. Finance must estimate which costs will rise as customers send more questions each month. The company will use an existing p
- An email security vendor's phishing filter was trained two years ago and has scored millions of messages since. Attackers now use new techniques, and more phishing emails slip through. The vendor's ac
- Foundation models provide broadly reusable capabilities rather than one narrowly trained task
A foundation model is pretrained on broad, diverse data and can be adapted or prompted for multiple downstream tasks. This allows a business to reuse an existing capability instead of funding training from scratch for every application. Broad pretraining does not establish that the model knows a company's private information or meets its specific quality requirements, so business-specific evaluation and, where necessary, adaptation remain important.
Trap Commissioning a new model from scratch for each text task, which overlooks the reusable capabilities of pretrained foundation models.
4 questions test this
- A hospital network plans several generative AI tools for patient communication, appointment summaries, and staff training content. The CIO worries that each tool would need its own costly model build.
- A retail company is evaluating a foundation model to answer customer questions about its return policies. In a demo, the model writes fluent general answers but cites return windows that differ from t
- A regional publisher wants AI to draft article summaries and headline options in its house style within six months. A product lead proposes training a new language model from scratch so the output ref
- A legal services firm wants AI to summarize contracts, draft client emails, and classify incoming requests. The IT director proposes training three separate models from scratch, one per task. The mana
- Language, vision, and speech capabilities address different forms of business information
Natural language processing interprets or generates human language, supporting sentiment analysis, intent recognition, and translation. Computer vision extracts information from images or video, supporting visual inspection and object recognition. Speech recognition converts spoken words into text, which can then be analyzed for meaning; transcribing a call and determining customer sentiment are different tasks that can be combined.
Trap Using speech transcription alone to measure customer sentiment, which produces words rather than an interpretation of their meaning.
4 questions test this
- A bank wants to monitor recorded advisor calls for signs of customer dissatisfaction and needs results in a compliance dashboard. The project sponsor wants to buy only what is necessary. One vendor se
- A quick-service restaurant chain wants drive-through customers to speak their orders to an automated assistant. The assistant must enter each requested item into the point-of-sale system and then read
- An online marketplace wants to flag product photos that show prohibited items and to translate seller descriptions into five languages. The trust and safety lead and the localization lead are sharing
- A contact center records every customer call. Leadership wants a weekly report showing how frustrated customers feel about a new billing change. The team has deployed speech recognition and now has ac
- A large language model generates language by predicting successive tokens
A large language model (LLM) is a model trained on extensive language data to learn patterns that support understanding and generating text. A generative LLM produces a response by predicting successive tokens from the supplied context and the tokens already generated. LLMs are language-focused members of the broader foundation-model category, which also includes models for non-language content such as images; foundation model and LLM are therefore not interchangeable names.
- A learned prediction is an estimate rather than a guaranteed outcome
A learned prediction is a probabilistic estimate based on patterns, not a promise that the predicted event will occur. A demand forecast can therefore be useful even though actual demand differs from its predicted value. Generative responses can also vary across repeated requests with the same prompt because generation can select among possible next tokens; identical wording is not guaranteed simply because the request is unchanged.
4 questions test this
- A telecom company's support chatbot uses a generative model to answer billing questions with each customer's account details. QA automation sends the same test prompt five times and fails the build wh
- A marketing team uses a generative AI model to draft product descriptions. When a manager submits the same prompt twice, the model returns two differently worded drafts. The manager reports a defect a
- An insurer uses a generative AI assistant to draft claim denial letters. Compliance discovers that letters for the same claim facts are worded differently each time they are generated, and regulators
- A beverage distributor's demand model forecast 12,000 cases for July, and actual sales were 11,200. A board member argues that the model is broken because its prediction was not exact and proposes can
- Broad task capability does not by itself establish artificial general intelligence
Artificial general intelligence (AGI) denotes the theoretical goal of generalized human-level cognitive ability, including learning and solving problems across domains beyond the tasks a system was developed for. Task-bounded AI operates within capability limits even when it performs many useful activities. Reusing a foundation model for several business tasks or letting an agent complete a workflow autonomously does not by itself demonstrate AGI.
- Recommendation engines select relevant items rather than merely personalize wording
A recommendation engine uses evidence such as customer interactions, preferences, and item information to select or rank relevant products or content. Purchase history and product descriptions can help determine which existing items to show a customer. Generating a personalized message about an item is a different capability: it creates wording rather than performing the selection or ranking, although a business experience can combine both.
3 questions test this
- A clothing retailer's marketing lead says the company already has recommendations because a generative model writes a personalized greeting with each customer's name in every email. Click-through rate
- An online bookstore wants each customer's home page to show the titles they are most likely to buy, chosen from its existing catalog. A product manager proposes using a generative model to write a per
- A travel company wants its app to show each user the five hotels they are most likely to book in a destination, and to add a short generated summary of each hotel's amenities. The product team debates
- Amazon Bedrock, Amazon SageMaker AI, and Amazon Quick serve different business capability needs
Amazon Bedrock provides managed access to foundation models and capabilities for creating generative AI applications and agents. Amazon SageMaker AI provides a managed environment for technical teams to develop, customize, train, deploy, and manage models across the custom-ML lifecycle. Amazon Quick is a ready-to-use business AI service combining assistance grounded in connected data, analytics, and workflow automation. These categories distinguish application-building capabilities, a model-development environment, and business-facing assistance; overlapping features do not make the offerings interchangeable.
4 questions test this
- A manufacturer has three AI needs. Its data science team must build and train a custom defect-detection model on its own images. Its product team wants a generative AI assistant in the customer portal
- A retailer's business analysts want to ask natural-language questions about sales dashboards and automate routine reporting tasks. They have no ML engineers and want a ready-to-use service grounded in
- An insurer's data science team uses Amazon SageMaker AI to train and deploy custom claims-fraud models. After adopting Amazon Quick for business analytics, the CFO proposes retiring SageMaker AI becau
- A bank's data science team needs to build a credit risk model from its own labeled loan data, tune it, deploy it, and manage it across the model lifecycle. The CIO asks whether Amazon Bedrock or Amazo
- Supervised learning uses known outcomes to learn categorical or numerical predictions
Supervised learning learns from inputs paired with known target outcomes, such as historical transactions marked fraudulent or genuine. Classification predicts a category, including two-category and multiple-category outcomes; regression predicts a numerical value such as a selling price. A business must identify both the outcome it wants predicted and whether reliable historical examples of that outcome exist.
Trap Using customer segmentation as the training objective for an exact revenue forecast, which discovers groups rather than learning the required numerical target.
4 questions test this
- A real estate company wants to predict the selling price of each listed home from features such as size, location, and age. It has ten years of completed sales with final prices. An analyst proposes g
- A bank wants to sort incoming customer emails into five service categories, such as billing, cards, and loans. It has two years of emails already tagged with the correct category by agents. The operat
- A utility company wants to forecast each customer's electricity use for next month in kilowatt-hours. It has five years of monthly meter readings. A consultant suggests a classification model that lab
- A subscription company wants to know which customers will cancel next month so it can offer retention discounts. It has three years of customer records showing who cancelled and who renewed. The marke
- Unsupervised learning discovers relationships without examples of a required answer
Unsupervised learning finds patterns in data without supplied target labels. Clustering groups similar customers or documents, while association discovery identifies relationships such as products frequently purchased together. These findings support exploration, but the resulting groups need business interpretation rather than being treated as predefined outcomes the system was taught to predict.
Trap Treating newly discovered customer clusters as verified churn labels, which substitutes similarity groups for observed business outcomes.
4 questions test this
- A bank wants to design product bundles, such as a checking account with a credit card, but has never offered bundles before. It holds five years of records showing which products each customer holds.
- A telecom company clustered two million customers by usage and found seven segments. The marketing director plans to use the cluster output as each customer's probability of buying a new premium plan
- An insurer ran clustering on five years of policyholder data and found a group with unusual coverage combinations. A product manager wants to label this group as high-risk and raise their premiums at
- A city transit agency wants to discover distinct rider travel patterns in two years of fare-card trip records to guide a route redesign. Riders are tagged only by fare type, such as student, senior, o
- Limited manual labels do not mean all learning must stop
Semi-supervised learning uses a small labeled dataset together with a larger unlabeled dataset, which can be useful when expert labeling is costly. Self-supervised learning derives learning targets from the data itself, as in learning to predict missing or subsequent text, and is widely used in foundation-model pretraining. These approaches differ from collecting a human-provided correct answer for every training example; neither removes the need to evaluate results against the intended business task.
Trap Requiring experts to label every document before any pretraining can begin, which overlooks learning signals obtainable from unlabeled data.
4 questions test this
- A media company wants to build a language model for its archive. A project manager insists that editors must label every article with the correct next sentence before any training can start, which wou
- A software company is evaluating a foundation model for internal search. An executive objects that the model cannot be trustworthy because no human labeled the billions of documents it was pretrained
- A support center wants a model that sets ticket priority. Senior agents labeled 2,000 tickets by priority, and 400,000 older tickets have no labels. A vendor proposes semi-supervised learning, and the
- A hospital wants to classify radiology reports by urgency. Radiologists can label only a few hundred reports, and the hospital has tens of thousands of unlabeled reports. The chief medical officer wan
- Reinforcement learning learns actions from rewards rather than paired correct answers
Reinforcement learning improves a strategy through actions and feedback expressed as rewards, with the aim of improving cumulative outcomes over time. A locally attractive action can be less useful than a sequence that earns a better long-term result. Business leaders must understand what the reward encourages, because a system can learn to pursue the rewarded outcome even when that outcome is a poor proxy for the real business goal.
Trap Rewarding only immediate engagement for a long-term retention goal, which teaches the system to optimize a different outcome.
3 questions test this
- A courier company uses reinforcement learning to assign drivers to orders. The reward counts deliveries completed each hour, and drivers now skip distant high-priority orders that carry service-level
- A logistics company is piloting reinforcement learning to schedule warehouse shifts. A product manager says the system will work out what the business wants on its own, so leaders do not need to agree
- A streaming service uses reinforcement learning to choose which shows to recommend. The current reward is the number of clicks in each session. Leadership's real goal is long-term subscriber retention
- Anomaly detection identifies unusual behavior rather than proving its cause
Anomaly detection identifies observations that differ from expected patterns, which can help surface unusual transactions or equipment behavior when exhaustive examples of failures are unavailable. An unusual observation is a signal for investigation, not by itself proof of fraud or a particular equipment fault. A known-outcome classifier instead learns the categories represented in its labeled examples, so the two capabilities answer different business questions.
Trap Treating an unusual transaction as a confirmed fraud label, which turns a deviation signal into an unverified conclusion.
3 questions test this
- A manufacturer's inspection logs contain thousands of labeled examples of three known defect types on its packaging lines. It now wants to name those known defects automatically and also surface unusu
- A manufacturer wants to detect equipment problems on a new production line. It has no history of failures for this equipment, but it does have sensor data from normal operation. The plant manager also
- A payments company has few confirmed fraud labels and deploys anomaly detection to surface unusual transactions. A manager proposes automatically blocking every flagged transaction and recording each
- Data structure depends on its schema, not whether it contains numbers or words
Structured data follows a predefined schema, such as transaction rows with named fields for customer, date, and amount. Unstructured data includes narrative documents, images, and recordings whose meaning is not organized as fixed record fields. Short text fields can be structured, and a digital document can remain unstructured; the distinction helps determine whether a business needs record analysis or content understanding.
Trap Classifying all text as unstructured, which ignores schema-defined fields such as customer identifiers and product categories.
4 questions test this
- A hospital's operations team wants to forecast bed demand from admission records that include patient ID, ward, admission date, and diagnosis code. A data vendor insists the records need natural langu
- An insurance company stores adjuster reports as free-form narrative documents and also keeps a claims table with policy number, claim date, and status. Leadership wants to know which source needs cont
- A utility company wants to analyze two sources: a meter table with account number, reading date, and kilowatt-hours, and thousands of scanned letters in which customers describe billing disputes. A ma
- A bank's product team is building a dashboard of complaint trends. The complaint system records complaint category, product, branch, and date in named fields, and it also stores a long free-text descr
- Metadata organizes content without making all of its meaning structured
Semi-structured information combines organizational cues with content that does not fit a rigid table, such as a media collection with dates and topic tags. Those tags can support filtering and discovery, while understanding what a recording actually says still requires analysis of its content. A business should distinguish searching known metadata from extracting new meaning from the underlying material.
Trap Using document filenames as substitutes for analyzing contract obligations, which searches labels rather than the terms inside the documents.
4 questions test this
- An e-commerce company stores product photos in a library where each image has metadata for product ID, upload date, and photographer. The merchandising team wants to find photos that show a product be
- A pharmaceutical company keeps research reports in a repository where each report has tags for project code, author, and approval date. A compliance officer must confirm which reports mention a specif
- A hotel group stores two years of guest reviews as JSON records with a star rating, stay date, property code, and a free-text comment. Leadership asks how often guests at its downtown property complai
- A media company's archive of interview videos is tagged with recording date, location, and topic. Editors can already filter by those tags. The content director now wants to find every clip in which a
- An AI solution must support the information modalities the business needs it to use
A modality is a type of information, such as text, images, audio, or video; a multimodal capability can process more than one type. A text-only capability cannot directly interpret visual evidence in a product photograph or a scanned chart merely because a user mentions it. For a mixed-information task, establish which modalities must be understood and whether any required conversion preserves the evidence needed for the decision.
Trap Using a text-only summary to judge visual damage that was never described in text, which removes the evidence needed for the assessment.
4 questions test this
- An auto insurer wants AI to help adjusters assess vehicle damage from photos that policyholders upload. A proposed tool accepts only text, so the team plans to send it the claim form's written descrip
- A manufacturer wants AI to triage maintenance requests. Technicians submit a typed note, a photo of the faulty part, and sometimes a short audio clip of machine noise. Plant leadership wants triage de
- A property management company wants AI to summarize move-out inspections. Inspectors record a video walkthrough of each unit and write brief notes, and deposit deductions depend on visible damage such
- A retailer is evaluating an AI assistant for store managers. Managers want to upload shelf photos and ask whether displays follow the planogram, and they also want to type questions about stock levels
- Recognizing text and extracting business fields are different document-processing capabilities
Optical character recognition converts an image of text into machine-readable text. Intelligent document processing goes further by extracting, classifying, and validating business information such as invoice numbers, totals, and payment terms for downstream workflows. When the goal is to populate a business record rather than simply make a document searchable, assess field-level extraction and validation as well as text recognition.
Trap Accepting a searchable invoice as proof that its payment fields were correctly captured, which confuses text recognition with business-data extraction.
5 questions test this
- An accounts payable team scanned all supplier invoices and ran optical character recognition, so the invoices are now searchable in the document system. The controller now wants invoice numbers, total
- An insurer digitized its paper claim forms with OCR, and adjusters can now search for any word in a form. The claims director wants claim amounts and policy numbers loaded into the claims system witho
- A bank's mortgage team receives pay stubs and tax forms as images. A completed pilot made them searchable, and underwriters still retype income figures into the loan system. Leadership wants income va
- A logistics company receives thousands of bills of lading as scanned PDFs. Finance wants shipment weights and freight charges entered into the billing system automatically, and audit requires that tot
- A hospital wants patient intake forms to update its records system automatically. The forms are scanned, and a pilot made them searchable as text. The records team notices that the system still has no
- Incorrect and missing records are data-quality problems before they are model problems
When predictions rely on inaccurate records or missing business information, prioritize correcting the source data and assessing its suitability before retraining. Accuracy means that recorded values reflect reality, while completeness concerns whether required information is present. Timeliness concerns currency and relevance, so an accurate old record may still be unsuitable for a decision that needs current information. Adding more examples from the same faulty process can enlarge the problem rather than supply better learning evidence.
Trap Funding a more sophisticated model for contradictory customer records, which leaves the incorrect learning evidence unchanged.
5 questions test this
- An online retailer's product recommendation model suggests discontinued items. The catalog feed marks discontinued products inconsistently, and many product records have no category. Merchandising wan
- A grocery chain trains a replenishment model on supplier delivery records. Suppliers updated delivery schedules three months ago, but the records used for training are from last year. The records are
- A telecom company's churn model gives poor predictions. An audit finds that many customer records list the wrong contract end date and that a third of records lack a plan type. The CTO proposes buying
- A bank's credit risk model was trained on income values that branch staff typed by hand, and an audit shows many entries are off by a factor of ten. The model team proposes adding records from three m
- A health insurer's claims model flags too many valid claims for review. Investigation shows that a legacy intake system stored some procedure codes incorrectly and left provider specialties blank on m
- Consistent definitions and valid values make data comparable across business sources
Data consistency requires equivalent business information to have compatible meanings and representations across records and sources. Validity checks whether values follow required formats or acceptable ranges; a value can pass a format check and still be factually wrong. When stores record the same business event differently, standardize collection definitions and quality checks rather than merely standardizing the AI's final reports.
Trap Standardizing forecast report layouts to reconcile incompatible store inputs, which changes presentation without making the underlying measurements comparable.
4 questions test this
- A national retailer trains a returns forecast on data from its stores. Some stores log a return when the customer drops off an item, and others log it only after inspection approves a refund, so retur
- A logistics company merges shipment records from two acquired carriers. One carrier records weight in pounds and the other in kilograms, and both fields pass the format check for a numeric value. The
- A restaurant chain wants one demand forecast across all locations. Some stores record sales by order and others by individual menu item, and some log times in local time while others use headquarters
- A hospital group combines readmission data from five facilities to train a risk model. Each facility defines a readmission differently, using windows of 14, 30, or 60 days. All records pass the data w
- Large historical datasets can still omit the people and situations that matter
Training evidence must represent the intended users and operating conditions, not simply contain many records. Underrepresented populations, rare outcomes, or missing business contexts can lead to uneven performance even when overall results look strong. Seek relevant coverage of those gaps and document exclusions; collecting more of the already dominant examples does not necessarily address them.
Trap Expanding only the majority customer segment to fix weak minority coverage, which increases volume without filling the missing experience.
4 questions test this
- A hospital wants a model to predict patient deterioration. The training data comes from adult wards, but the hospital plans to use the model in its pediatric ward as well. A vendor notes that the data
- A food delivery company trained an arrival-time model on four years of orders, nearly all from its southern cities in mild weather. It will launch in northern cities this winter, where snow is common.
- A bank trained a loan approval model on ten years of applications, almost all from urban customers. The model performs well overall but poorly for rural applicants, a segment the bank is expanding int
- An e-commerce company's support chatbot was trained on English-language tickets from its largest market. The company is now expanding to Spanish-speaking customers and plans to add two more years of E
- Training labels require quality control whether people or AI produced them
A label tells a supervised-learning system what answer to learn, so inconsistent or biased labels can teach the wrong associations. Require clear labeling guidance, checks against known examples, and review of disagreements by appropriate domain experts. Machine-generated labels also need validation against human judgment, especially for underrepresented groups and difficult cases; automation changes how labels are produced, not the need to establish their reliability.
Trap Treating agreement among automatically generated labels as independent proof of correctness, which can reproduce the same underlying labeling error.
4 questions test this
- An insurer is training a claims triage model and uses an automated tool to pre-label claims as simple or complex. Adjusters review only claims the tool marks uncertain. The model performs worse on cla
- A bank uses a language model to label thousands of transaction descriptions by spending category. Three different prompts produce the same label for most transactions, and the project lead argues that
- A medical imaging startup has radiologists label scans as normal or abnormal. Two radiologists often disagree on borderline scans, and one consistently labels more scans as abnormal than colleagues. T
- A retailer uses contractors to label product reviews as positive, negative, or neutral for a sentiment model. Early results show inconsistent labels for sarcastic reviews. The product owner wants reli
- Synthetic examples are useful only when they preserve the business phenomena being learned
Synthetic or augmented data can supplement scarce examples, but its usefulness depends on realism, relevant relationships, and coverage of important edge cases. Require domain validation and checks for bias rather than accepting generated volume as evidence of quality. Document where synthetic examples can substitute for real observations and where their limitations make real data necessary.
Trap Replacing rare real incidents with generic generated examples, which may remove the unusual relationships the system needs to recognize.
4 questions test this
- A payments company has only a few hundred confirmed cases of a rare fraud pattern. A vendor offers to generate one million synthetic fraud transactions and replace the real cases with them, saying vol
- A bank plans to train its support chatbot on 200,000 conversations generated by a language model because it has only 5,000 real chat transcripts. In a pilot, the bot handles routine questions but stum
- A bank wants to train a credit model for a new small-business product but has little real data. A team proposes using a generative model to create synthetic applicants. The chief risk officer is conce
- A manufacturer wants a model to detect a rare equipment failure. It has 40 real failure records and plans to generate synthetic sensor data to augment them. The engineering lead wants the model to wor
- Data profiling reveals issues, but business expertise determines what they mean
Data profiling examines characteristics such as missing values, distributions, unusual observations, and inconsistent categories before training. Subject matter experts must interpret those findings because an unusual value may be either a recording error or a valid business event. Establish quality expectations and record the rationale for remediation instead of blindly accepting an automated suggestion or removing every unusual case.
Trap Removing every unusually large transaction before fraud training, which may discard legitimate examples of the events being investigated.
4 questions test this
- A utility company profiles smart-meter data before training a consumption forecast. The tool flags extreme usage spikes on several days. Engineers know that some spikes were caused by a heatwave and o
- An insurer profiles policy data before training a pricing model. The tool reports that a vehicle category appears under three spellings and that some drivers are listed as 110 years old. The pricing m
- A retail company profiles customer data for a churn model and finds many missing ages and a few transactions with negative amounts. The data engineer proposes accepting the profiling tool's automatic
- A card issuer is preparing data to train a fraud model. Automated profiling flags a cluster of very large transactions as outliers and suggests removing them. Some may be data entry errors, but some m
- Duplicate records can distort evidence even when every field is correct
Uniqueness is the data-quality dimension concerned with avoiding unintended duplicate records for the same business entity or event. Recording the same customer twice can inflate customer counts even when both records are accurate and complete. Assess whether repeated records represent genuinely separate observations before accepting record volume as evidence of the size of the business population.
Trap Using a required-field completeness check as the sole acceptance criterion for customer counts, because duplicate records can contain every required field.
- Training, validation, and testing provide different kinds of evidence
Training data teaches the model, validation data supports development decisions, and a separate test set assesses performance after those decisions are made. A business approval should rely on examples not used to fit or repeatedly refine the model. The same independence principle applies to prompt engineering: use development cases to refine instructions and held-out cases to verify the final prompt. This distinguishes a reusable improvement from a result tailored to familiar examples.
Trap Using the final test set for every development revision, which makes the claimed independent assessment part of the development process.
3 questions test this
- An insurer is building a claims severity model. The team plans to use one dataset both to tune model settings and to report final accuracy to the regulator, because splitting the data would leave fewe
- A software company tested several large language model prompts for drafting sales emails. The product manager picked the winning prompt based on scores from a test set and then made six more edits, ch
- A retailer's data science team tuned a demand model over twenty iterations, checking each revision against the same test set to pick the best version. They now present that test score to the steering
- Information unavailable at decision time must not inflate historical evaluation results
Data leakage occurs when learning or evaluation uses information that would not legitimately be available for the real prediction. Examples include later business outcomes present in historical records and duplicate examples appearing across training and testing. When reviewing impressive results, ask whether the evidence reflects the information available at the actual decision point and whether evaluation examples are genuinely separate.
Trap Including completed-claim settlement information in an initial claim-risk prediction, which supplies knowledge the business would only obtain later.
2 questions test this
- A retailer's product review classifier shows near-perfect test accuracy. An analyst finds that many reviews were posted on several sites and appear in both the training and test sets. The product owne
- A subscription company trained a churn model using a feature that records whether a customer called to cancel. Test accuracy is excellent. The marketing director plans to use the model to target reten
- Training success must be compared with performance on unfamiliar examples
Overfitting occurs when a model learns training-specific detail or noise and performs well on training examples but poorly on new examples. Underfitting means the model has not learned the relevant relationship and performs poorly on both training and test examples. These patterns call for different technical investigations; business leaders should request evidence of generalization rather than assuming longer training or higher training accuracy is always better.
Trap Demanding more training solely to maximize an already strong training score, which can deepen memorization instead of improving generalization.
4 questions test this
- A logistics company's delivery time model performs poorly on both its training data and its test data. A manager suggests that the problem is overfitting and wants to reduce the amount of training to
- A bank's fraud model was retrained with many more features and now shows 100 percent accuracy on training data. Its accuracy on a held-out test set dropped from 92 to 80 percent. The project sponsor c
- A marketing analytics team reports that its new lead scoring model achieves 99 percent accuracy on training data but only 70 percent on a held-out test set. The sales director asks the team to keep tr
- A retailer is contracting a vendor to build a weekly demand forecasting model. The vendor proposes that the retailer accept the model once training accuracy reaches 99 percent. The merchandising direc
- Historical evaluation must address the specific context in which the model will be used
A national or historical average test score does not establish performance for a particular city, customer group, or newly changed market. Request evaluation on relevant subsets and, after a major business shift, suitably recent examples before relying on the broader result. The decision is whether the evaluation evidence supports the intended application, not whether a single aggregate score is impressive.
Trap Accepting a national average test result as sufficient evidence for entry into a poorly represented city, which leaves local performance unexamined.
3 questions test this
- A retail chain's markdown model was tested on three years of data and achieved strong overall accuracy. The chain recently launched a new online-only product line, and the test data contains almost no
- A health insurer's claims model performs well on its overall test set. The insurer plans to use it for a newly acquired group of members over age 80, who make up less than 1 percent of the test data.
- A national bank's credit model has a strong average test score across the country. The bank plans to expand into a city where few customers appear in the training and test data. The expansion lead wan
- ISO/IEC 22989 and ISO/IEC 23053 support different levels of shared AI understanding
ISO/IEC 22989 establishes AI concepts and terminology for communication among diverse stakeholders. ISO/IEC 23053 provides a framework for describing the components and functions of AI systems using machine learning, including deep learning. Use shared vocabulary to resolve inconsistent terminology and the ML-system framework to explain how system elements relate; neither serves the same purpose as an organizational AI management system.
Trap Using ISO/IEC 42001 as the primary glossary for a terminology dispute, which selects a management-system standard rather than the dedicated vocabulary standard.
4 questions test this
- A healthcare technology company is preparing training materials so clinicians, engineers, and lawyers can discuss AI projects consistently. The training must first align everyone on basic AI terms and
- A bank's architecture review board must describe its machine learning credit systems consistently, covering components such as data, the trained model, and processing, and how they function together.
- A manufacturer has adopted ISO/IEC 22989 terminology across its AI teams and uses ISO/IEC 23053 to document its ML pipelines. The CEO now wants organization-wide policies, objectives, and continual im
- A global insurer's business and technical teams use terms such as model, training data, and AI system inconsistently, which slows approvals for AI projects. The chief data officer wants an internation
- ISO/IEC 42001 addresses organizational AI management rather than an individual model's accuracy
ISO/IEC 42001 specifies requirements for establishing, maintaining, and continually improving an AI management system. It applies to organizations that provide or use AI, not only those that develop models, and uses a Plan-Do-Check-Act approach to managing risks and opportunities. Its focus is coordinated policies, objectives, and processes across the organization rather than a substitute for evaluating an individual AI application's outputs.
Trap Using a model benchmark report as the organization's AI management system, which measures outputs without establishing management policies and improvement processes.
3 questions test this
- A retailer uses third-party AI services for pricing and customer support but builds no models itself. The compliance director wants to know whether ISO/IEC 42001 applies to the company. A manager argu
- A bank's AI team reports that its fraud model achieved excellent accuracy on an industry benchmark. The chief risk officer wants to align the bank's AI program with ISO/IEC 42001, and the AI team prop
- A logistics company plans to deploy several AI tools from different vendors and to build one forecasting model in-house. The CEO wants a single organizational approach for managing AI risks and opport
- The NIST AI RMF provides a voluntary structure for managing AI risks across the lifecycle
The NIST AI Risk Management Framework is voluntary, use-case agnostic guidance for incorporating trustworthiness into AI design, development, use, and evaluation. Its core functions are Govern, Map, Measure, and Manage: establish governance, understand context and risks, assess those risks, and prioritize responses. Govern is cross-cutting rather than a one-time first step; organizations tailor the risk-management outcomes and approaches to their application and context.
Trap Treating adoption of the NIST AI RMF as regulatory permission to deploy, which confuses voluntary risk guidance with applicable legal requirements.
3 questions test this
- A health insurer has adopted the NIST AI Risk Management Framework for a new claims AI tool. A product manager says that adopting the framework gives the company regulatory permission to deploy the to
- A bank's AI program office wants to use the NIST AI RMF to structure risk work for a new loan assistant. The program manager asks which functions to use to understand the system's context, assess its
- A retailer is applying the NIST AI RMF to a pricing AI system. The project team treats Govern as a step to finish in the first week and then plans to move on to Map, Measure, and Manage without revisi
- AWS CAF for AI treats adoption as an organizational capability challenge
AWS CAF for AI applies six perspectives to AI adoption: Business, People, Governance, Platform, Security, and Operations. These connect business outcomes and workforce capability with oversight, enabling technology, protection, and reliable operation. A technology assessment alone therefore cannot establish overall AI readiness; the perspectives provide a common way for business and technical stakeholders to discuss gaps without prescribing one executive owner for every organization.
Trap Using platform capacity as a complete AI-readiness assessment, which leaves people, governance, and other organizational capabilities unassessed.
3 questions test this
- A retailer wants to scale generative AI from pilots to production. Its platform team has delivered strong infrastructure, and leadership wants to confirm readiness before committing budget. The COO as
- A bank's AI program is stalling. The data science team has strong tools, but business units do not sponsor use cases, staff lack AI skills, and no one owns AI risk oversight. The chief transformation
- A manufacturer's CIO assessed AI readiness by reviewing GPU capacity, data platforms, and cloud tooling, and concluded the company is fully ready to scale AI. The CEO notices that business sponsors, w
- AWS responsible AI dimensions describe distinct concerns that a single quality score cannot replace
AWS describes responsible AI through fairness, explainability, privacy and security, safety, controllability, veracity and robustness, governance, and transparency. Explainability concerns understanding and evaluating outputs, while transparency enables informed stakeholder choices; controllability concerns mechanisms to monitor and steer behavior. Fairness considers impacts on different groups, and veracity and robustness address correct outputs even with unexpected inputs, so strong performance on one dimension does not establish all the others.
Trap Using a transparency disclosure as evidence that individual decisions are understandable, which substitutes stakeholder information for output explainability.
3 questions test this
- An insurer's AI claims assistant passed a bias review across customer groups. The board wants assurance on other responsible AI concerns before launch, specifically whether adjusters can understand an
- A retailer's chatbot scores highly on answer accuracy in testing. Before launch, the responsible AI lead finds that the bot gives harmful advice when users enter unexpected adversarial prompts, and th
- A bank publishes a notice telling loan applicants that AI is used in credit decisions and describing the system's intended use and limitations. A regulator later asks the bank to show why a specific a
Identify and select appropriate AI solutions
Read full chapterUnlock with Premium — includes all practice exams and the complete study guide.
Apply generative AI concepts and techniques
Read full chapterUnlock with Premium — includes all practice exams and the complete study guide.
AI Strategy and Business Value Creation
Align AI strategy with business goals
Read full chapterCheat sheet
Sharp facts the exam loves — scan these before test day.
- A useful AI strategy starts with a business problem and a measurable outcome
AI use-case discovery starts by identifying a customer or business problem, the decision or activity to improve, and the outcome that would justify investment. Map the required capability to that activity and define success before choosing a tool. Involve the business owner and affected stakeholders early so the proposal includes the process changes needed to realize the value.
Trap Using a technology demonstration as the investment brief, which proves a capability without establishing the business problem it should solve.
5 questions test this
- A claims department plans an AI tool that drafts settlement recommendations. The business case shows strong time savings but assumes that adjusters keep their current approval steps and workloads. The
- A hospital group's AI proposal states the goal as using AI to improve patient scheduling. The finance committee will approve only proposals whose results can be judged later. Scheduling staff report t
- A bank's data team is drafting an AI proposal to speed up small-business loan decisions. So far, only data scientists have worked on it. Loan officers, credit risk managers, and the lending business o
- An innovation team at a property insurer built an impressive demo in which a generative AI model writes poems about home safety. The team asks the executive committee to fund the demo as the company's
- A manufacturer's chief executive wants to sign a contract with a popular AI model provider this month and then find uses for it. The operations team has not yet agreed on which process to improve or h
- Customer operations AI should address the actual service bottleneck
When customers need consistent answers to routine inquiries, use AI-assisted self-service grounded in approved information with a route to human support. When agents spend time searching, triaging, or documenting cases, agent assistance and summaries address that internal workload instead. Match the intervention to the bottleneck and assess resolution, recontacts, and customer experience rather than simply counting automated responses.
Trap Using a generic FAQ chatbot to resolve account-specific billing disputes, which lacks the account context and exception-handling authority that the work requires.
5 questions test this
- A utility's contact center analysis shows that customer questions are varied and often complex. Agents spend most of each call searching several systems for account history and then writing case notes
- A telecom provider plans to use its website FAQ chatbot to resolve billing disputes. Each dispute depends on the customer's account history, plan changes, and credits that require approval. The billin
- An online furniture retailer receives thousands of daily questions about store hours, delivery windows, and return rules. Customers wait hours for answers outside business hours. The answers are well
- Three months after launch, a travel company's support chatbot sends 40,000 automated responses each week, and the vendor presents this as proof of success. The customer experience director suspects th
- An electronics retailer's self-service assistant hands customers to a live agent when it cannot help. Customer surveys show frustration because agents ask customers to repeat everything they already t
- Recommendations improve relevance while content generation improves creative throughput
When the business needs more relevant offers or product discovery, use personalization and recommendations based on customer preferences and context. When the bottleneck is producing campaign variants or product descriptions, use generative content assistance with brand review. The first intervention targets engagement, conversion, and upsell; the second targets production cycle time and the effort required to produce acceptable content.
Trap Using campaign draft volume as evidence that product recommendations are relevant, which measures content production rather than customer-item matching.
3 questions test this
- A cosmetics company uses generative AI to draft social media posts for new products. Some drafts recently included product claims that the company's regulatory team had not approved. The marketing dir
- An online bookstore has a large catalog, but visitors browse many pages and leave without buying. Analysis shows that every visitor sees the same generic best-seller lists whatever their reading histo
- A consumer electronics brand launches products in 12 markets. Its marketing team needs dozens of campaign variants for each launch, and copywriters cannot keep up, so launches slip by weeks. Every pub
- AI can accelerate research by expanding the ideas and designs experts can evaluate
Research and development teams can use AI to generate design candidates, summarize extensive research, and explore alternatives under stated constraints. This creates value by accelerating ideation and narrowing the work experts must investigate, not by establishing that a proposed design is already valid. Keep domain validation in the process and judge the initiative by development time, useful candidates, and downstream quality.
Trap Using the number of generated prototypes as the release criterion, which measures exploration volume rather than validated product fitness.
5 questions test this
- A furniture maker's engineers ask a generative AI tool for new chair-frame designs by using open-ended requests. Most results are creative but too heavy, too costly, or impossible to produce on the co
- A medical device company's R&D engineers spend weeks reading hundreds of studies before each design review. They often miss relevant findings, and reviews are delayed. The R&D director wants AI to red
- An automotive supplier used generative AI to produce 300 bracket design prototypes in one month. The engineering manager proposes releasing the program to production because the prototype count far ex
- A materials science lab uses AI to propose new coating formulas. One AI proposal appears to outperform every known formula, and a manager wants to announce it to customers immediately. The lab's proto
- A biotechnology company's research team designs enzyme candidates by hand, and each researcher can evaluate only a few designs per month. Leadership wants AI to speed up discovery. Senior scientists w
- Software development AI creates value through delivery improvements rather than code volume
AI assistance can accelerate drafting code, creating tests, and understanding existing software, but the business benefit depends on the developer activity and surrounding workflow. Retain normal security, testing, and review requirements and evaluate time to delivery alongside quality and rework. An assistant that speeds one task may need workforce enablement and process changes before the team can deliver more useful software.
Trap Using generated lines of code as the productivity target, which rewards output quantity without showing usable software delivered.
5 questions test this
- A payments company's developers now use an AI assistant to draft code. Some team leads propose letting AI-generated code skip peer review and security scanning, arguing that the assistant already writ
- A logistics company plans to introduce an AI coding assistant to its development teams next quarter. Leadership will later ask whether the assistant improved delivery. The company has not recorded how
- A retailer's developers write code faster with an AI assistant, but release dates have not moved. Analysis shows that finished work waits days in a shared testing queue and in approvals across several
- Three months after adopting an AI coding assistant, an insurer's development teams ship features 25 percent faster. Production incidents and rework requests have also risen sharply. The chief technolo
- A software company rolled out an AI coding assistant to 200 developers. The engineering vice president proposes a quarterly target of 30 percent more lines of code per developer to prove the investmen
- Knowledge assistance is valuable when employees struggle to find trusted company information
When employees repeatedly search policies, procedures, and product documents, use knowledge assistance over approved company content to shorten time to an answer. Identify which repositories contain the needed information and preserve users' access boundaries. Useful outcomes include faster onboarding, fewer unnecessary escalations, and less search effort; automating a transaction is a different business requirement from finding information.
Trap Using a general public chatbot as the company policy authority, which does not establish access to the organization's approved and permission-controlled information.
4 questions test this
- New associates at a wireless carrier's stores spend hours each week searching for the right device-return procedure and often escalate questions to store managers. Return transactions already run in a
- A bank's new employee knowledge assistant draws on the approved policy library and also on a team wiki full of unreviewed drafts and outdated pages. Staff report answers that mix current policy with o
- A company's HR knowledge assistant answers leave-policy questions accurately from approved documents. Managers now ask the project team to have the same assistant approve leave requests and update rec
- Employees at a manufacturing company often ask a free public AI chatbot about company travel and safety policies. Some answers contradict the official policies, and several restricted documents must r
- Document extraction and classification address manual entry and document-handling bottlenecks
When employees must read forms or invoices and re-enter their contents, use AI-assisted document extraction to turn the contents into fields usable by the business process. Document classification can identify the document type so it reaches the appropriate workflow. Evaluate the intervention through reduced manual entry, processing time, and correction effort, while preserving required checks on the extracted information.
Trap Using a knowledge-search assistant to address an invoice-entry backlog, because finding answers does not itself populate the accounting workflow.
5 questions test this
- A distributor's accounts payable team has a backlog of 8,000 supplier invoices that staff must read and key into the accounting system. A manager proposes a knowledge-search assistant that answers que
- A wholesale food distributor's pilot of AI document extraction writes supplier delivery-note data into a spreadsheet. Warehouse staff then retype the spreadsheet into the warehouse system, so processi
- A logistics company's mailroom receives a mixed stream of bills of lading, customs forms, invoices, and delivery receipts. Staff sort each document by hand before it reaches the right team, and misrou
- A travel insurer receives 2,000 claim forms a week, but after storms the volume rises to 20,000. Staff read each form and retype it into the claims system, and backlogs last weeks after every storm. S
- A property insurer uses AI to extract data from contractor repair invoices. Policy requires each invoice's bank details to match the approved contractor record before payment. Staff now check every ex
- Transaction risk detection creates value by directing attention to potentially fraudulent activity
When transaction patterns can reveal potentially fraudulent activity, use predictive fraud detection or anomaly detection to identify transactions needing proportionate investigation or additional checks. The business purpose is loss prevention, not generating explanations or content. Evaluate avoided fraud losses together with investigation effort and the friction imposed on legitimate customers; an unusual transaction is a reason to assess risk, not proof of fraud.
Trap Applying the same additional verification to every transaction, which adds customer effort without using transaction risk to target the checks.
4 questions test this
- An online retailer adds a one-time passcode check to every checkout to reduce fraud. Fraud losses fell slightly, but cart abandonment rose sharply among legitimate customers. Most orders come from rep
- An online marketplace's fraud model flags an unusually large payout request from a seller with eight years of clean history. A manager wants to freeze the seller's account immediately and report the s
- A bank deployed a fraud detection model on card transactions one year ago. The fraud team reports that the model flagged thousands of transactions. The model cost 2 million dollars. The chief risk off
- An insurer wants to reduce losses from fraudulent claims. A vendor proposes a generative AI tool that writes detailed narrative explanations of each claim for adjusters. Historical claim patterns are
- Build versus buy depends on differentiation, fit, and the ability to sustain delivery
When a mature commercial solution meets a common business need and time or specialist capacity is constrained, buying can accelerate value. Custom development is justified when distinctive requirements or competitive differentiation cannot be adequately met by available offerings and the organization can support the lifecycle. Compare functional fit, time to value, control, and ongoing capability rather than treating internal ownership as an advantage by itself.
Trap Building a commodity capability to preserve internal ownership, which consumes scarce delivery capacity without demonstrating a distinctive requirement.
5 questions test this
- A specialty chemicals manufacturer wants AI to recommend reactor settings by using decades of proprietary process data. A market review found no commercial product that models its patented process. Th
- A hospital network's IT director proposes building an in-house AI tool that drafts staff meeting notes. The director argues that owning the code is strategically safer than any vendor product. Several
- A retail bank's innovation team has started designing a custom model to classify customer complaints. The team has not checked whether any commercial products already handle complaint classification.
- A regional insurer wants AI-generated summaries of recorded customer calls for its quality team. Several mature commercial products already provide call summarization that meets the insurer's document
- A logistics company is deciding whether to build or buy an AI route-exception assistant. The internal team favors building, and a vendor has presented a polished proposal. The steering committee wants
- A partner can close a delivery capability gap while helping the organization learn
When an AI opportunity fits the strategy but internal expertise or delivery capacity is insufficient, a specialist partner can help co-build, integrate, and support the solution. Define deliverables, business acceptance criteria, knowledge transfer, and the eventual operating responsibilities. Partnership supplements capabilities; the organization still needs an accountable business owner and a plan for ongoing use.
Trap Treating a consulting engagement as the business ownership model, which leaves outcome accountability and the post-engagement handover undefined.
5 questions test this
- A university is finalizing a statement of work with an AI consulting partner for a student-services assistant. Past projects ended with code handed over but nobody able to maintain it. The provost wan
- A food distributor hired an AI partner to deliver a demand-sensing solution for its sales planners. The partner's lead consultant now chairs the weekly steering meeting. Internal leaders assume the pa
- A property management firm contracted an AI partner to deliver a tenant-inquiry assistant. The partner reports that every technical milestone is complete and asks for final payment. The operations dir
- A mid-sized utility wants a generative AI assistant that helps field crews troubleshoot equipment from manuals and work orders. The use case fits the company's strategy. The internal IT team has no ge
- A media company's AI partner will finish building a content-tagging solution in three months. The company's AI center of excellence will own the solution afterward. Its members have watched demos but
- Using an existing model does not prevent a business from creating a customized solution
A hybrid AI strategy can combine a provider's pretrained capabilities with company data, workflows, and a tailored application. This can preserve business-specific differentiation without the investment and specialist capability required to create a foundation model from scratch. Evaluate whether existing capabilities can satisfy the use case before funding foundational research and development.
Trap Treating a company-specific application as a requirement to create a new foundation model, which confuses application differentiation with foundational model development.
5 questions test this
- A competitor announced its own foundation model, and an insurer's chief executive wants a similar program. The insurer's two-year AI goal is a claims-intake assistant that uses its policy rules and cl
- A wealth management firm's board worries that using a provider's pretrained model will make its client advisory tool identical to competitors' tools. The firm holds years of proprietary research notes
- A telecom company's billing team wants an assistant that classifies incoming billing inquiries into eight fixed categories. Pilot tests show that an existing model with a clear prompt already meets th
- A pharmaceutical company's research division requests funding for foundational model research to support a literature-summary tool for its medical writers. Nobody has yet tested whether existing model
- An online travel company is building a tailored trip-planning assistant on its own booking data and itinerary workflow. Foundation models are improving quickly. The product director wants the company'
- A ready-to-use AI application and a managed AI development service solve different delivery needs
When a business needs ready-to-use workplace assistance, research, analytics, and workflow capabilities, evaluate an application such as Amazon Quick or suitable embedded AI in existing software. Amazon Bedrock provides managed foundation-model capabilities for teams creating tailored generative AI applications, while Amazon SageMaker AI supports teams developing and managing AI models. A managed development service reduces some technical burden but does not supply a finished business workflow by itself.
Trap Purchasing access to Amazon SageMaker AI as an immediately usable employee assistant, which provides model-development capabilities rather than the requested finished workplace application.
4 questions test this
- A professional services firm bought Amazon SageMaker AI access for 800 consultants after leadership asked for an AI assistant for research, internal document questions, and dashboards. Weeks later, co
- A hotel group selected Amazon Bedrock for a guest-request assistant. The operations manager expected Bedrock to arrive as a finished workflow that routes requests to housekeeping and maintenance. The
- A consulting firm wants its 3,000 employees to get AI help with research reports, answers grounded in internal documents, and dashboard analysis. The firm has no developers available to build or host
- A grocery chain's data scientists need to build, train, and deploy a custom demand forecasting model on the chain's sales history. They also need to manage the model's lifecycle in production. Busines
- Sourcing proposals must be compared over the same business scope and lifecycle
AI sourcing comparisons should include setup, integration, data preparation, licenses or usage, internal labor, training, human review, and ongoing support over a common evaluation period. A platform switch also adds transition work, temporary overlap, and any exit charges, which can delay or outweigh operating savings. Compare delivery commitments against the same must-have requirements and include all work needed to realize the proposed benefits; a low recurring price is not a whole-lifecycle business case.
Trap Comparing a vendor's subscription fee with a fully staffed internal implementation budget, which compares different cost boundaries and responsibilities.
5 questions test this
- A bank plans to move its AI customer-service assistant to a new platform that has lower monthly usage charges. The business case shows savings from the first month. It leaves out the migration project
- An insurer received two AI claims-processing proposals. Vendor A quoted costs for one year. Vendor B quoted costs for five years, including model updates and support. The procurement lead wants to pic
- A university is comparing two AI student-advising platforms. Platform A costs less but meets only eight of the ten must-have requirements. Platform B costs more and meets all ten. The provost wants th
- A retailer is choosing between a vendor's AI demand-planning service and an in-house build. Finance compared the vendor's annual subscription fee with a fully staffed internal project budget that incl
- A regional bank is comparing two AI proposals for loan-document review over the same three years, and both meet every must-have requirement. Vendor A's quote covers setup, integration, data preparatio
- A proposed AI solution must be validated against the actual business workflow
When evaluating an AI purchase or platform replacement, validate representative tasks, permitted data, intended users, and agreed success criteria. Assess quality, usability, response time, volume requirements, integration effort, and operating cost in the complete workflow. During a platform switch, reassess existing prompts and integrations because equivalent task descriptions do not guarantee equivalent behavior. A controlled trial establishes business fitness more directly than a curated demonstration or a provider's general benchmark.
Trap Using a vendor's curated demonstration as the acceptance test, which does not establish fitness for the buyer's data and workflow.
5 questions test this
- A tax preparation firm is evaluating an AI assistant for client questions. Pilot users were satisfied, but the pilot handled about 50 questions a day. During filing season the firm expects 20,000 ques
- A software company plans to move its customer-support assistant to a different foundation model that costs less. The new provider says its model handles the same task descriptions, so the existing pro
- A bank wants to trial a vendor's AI assistant for drafting loan-review summaries. The bank's AI policy prohibits testing external models against production customer data. The credit team still needs t
- A distributor trialed an AI order-entry assistant on real customer emails, and answer quality met the agreed target. The procurement lead wants to sign next week. The operations director notes that th
- A health insurer is evaluating a vendor's AI tool that drafts responses to member appeals. The vendor proposes a scripted demonstration on sample appeals that the vendor prepared. The insurer must dec
- AI procurement must establish what the provider may do with business inputs and outputs
AI vendor due diligence must examine how prompts, source data, and outputs are stored, accessed, retained, and reused, including whether they may be used for model training. Establish the permitted data classifications and review ownership, commercial-use rights, and liability with legal stakeholders. Continue reviewing material contractual changes because a previously acceptable service can change its data-use or licensing terms.
Trap Using a paid consumer subscription as evidence of enterprise data protections, which does not establish the negotiated data-use and ownership terms the business requires.
5 questions test this
- A law firm is reviewing an AI research service for associates. The provider's terms state that customer prompts may be used to improve its models unless the customer takes action. Partners are concern
- A financial services company approved two third-party AI applications last year after reviewing their terms and privacy policies. Since then, one provider has changed how it processes customer data. N
- A consulting firm wants to approve a third-party generative AI writing tool for client deliverables. Client documents will be pasted into prompts. The general counsel wants due diligence to cover the
- A marketing agency's staff use a popular AI chat service on individual paid consumer plans. The agency now wants to use the service with confidential client briefs. A manager argues that because the a
- A consumer goods company wants to use images from a third-party AI image generator in a national advertising campaign. The marketing team has accepted the tool's online terms. The chief marketing offi
- Vendor compliance fit depends on actual data flows and continuing evidence
When a use case has jurisdictional or regulatory constraints, verify where the vendor and its subprocessors store and process data and who can access it. Review relevant assurance evidence and contractual commitments against the organization's specific obligations. Procurement and compliance teams should reassess material changes rather than treating onboarding approval as a permanent guarantee.
Trap Using a vendor's general certification badge as proof of permitted cross-border processing, which does not establish the actual data locations and access arrangements.
6 questions test this
- A European bank must keep customer data within the EU. An AI vendor for complaint handling shows an ISO 27001 certificate and says that the certificate proves compliance. The bank's compliance team mu
- A national lottery operator is negotiating a contract with an AI vendor that will process player data. Regulations require approved storage locations, prompt breach reporting, and visibility into any
- A government contractor has confirmed that an AI vendor stores and processes its data only in approved Regions. The contractor's obligations also restrict which locations people may access regulated d
- An insurer onboarded an AI claims vendor two years ago after a thorough compliance assessment. No review has happened since. In that time, regulations in the insurer's market have changed, and the ven
- A pension fund sent a detailed questionnaire to a prospective AI vendor. The vendor answered yes to every security and data residency question but supplied no documents. The fund's regulator expects t
- An EU bank hosts AI workloads on AWS, which the European Supervisory Authorities have designated as a critical third-party provider under DORA. A manager proposes ending the bank's own monitoring of t
- An AI exit strategy must account for portability, dependencies, and the cost of leaving
When evaluating a platform, establish whether business data and prompts can be exported in usable formats and identify provider-specific dependencies that would need replacement. Review exit procedures, assistance, associated charges, and the obligations that continue after termination. These findings inform both the initial sourcing choice and the feasibility of a later transition; a lower operating price alone does not establish a cheaper switch.
Trap Using a cancellation clause as evidence that a platform is portable, which addresses contract termination but not usable exports or replacement dependencies.
4 questions test this
- A credit union is comparing two AI platforms for member service. Both meet the functional requirements, and one is slightly cheaper. The board wants to avoid being locked in later and asks when exit c
- A retailer is choosing an AI customer-service platform. The vendor highlights a 30-day cancellation clause as proof that the retailer can leave at any time. The retailer's AI lead worries that years o
- A logistics company runs an AI dispatch assistant on Platform A. Platform B offers a 25 percent lower operating price, and the finance team proposes switching immediately to save money. The assistant
- A media company built a content assistant that depends on one vendor's proprietary agent framework, prompt templates, and built-in search index. The vendor plans a large price increase. The chief tech
- AWS Marketplace simplifies finding and procuring AI offerings without replacing business evaluation
AWS Marketplace provides a catalog of software, data, and services with options for trials, product comparison, procurement, and consolidated AWS billing. Private offers can provide negotiated pricing and terms when public terms do not fit the requirement. The buyer must still assess suitability, data handling, and compliance; streamlined purchasing is not a substitute for due diligence.
Trap Treating a standard Marketplace listing as a negotiated agreement for bespoke commercial terms, which confuses a public offer with a seller's private offer.
5 questions test this
- A media company runs its workloads on AWS and wants to buy several third-party AI tools. It has eight business units. Finance does not want to set up a new payment relationship and invoice process wit
- A marketing agency is choosing between two AI image-editing products in AWS Marketplace. Neither has been tested on the agency's own client work, and each listing offers a free trial before an annual
- A logistics firm negotiated an AWS Marketplace private offer for an AI document-translation product, with a discount and custom EULA terms. The procurement lead believes that the private offer also un
- A retailer accepted an AWS Marketplace private offer for an AI forecasting product with discounted pricing. The agreement ends in two months. The finance lead assumes that the negotiated price will co
- A healthcare company found an AI transcription product listed in AWS Marketplace. The public listing's price and standard license terms do not fit the company's volume discount needs or its contract r
- High potential value is insufficient when an AI initiative is not feasible or strategically relevant
Use a common prioritization matrix to compare business impact, strategic alignment, feasibility, risk, and delivery requirements. A high-value opportunity with usable data and manageable integration is a strong early candidate; one blocked by missing prerequisites needs enabling work before expansion. An easy technical demonstration should not outrank a better business opportunity solely because it is easier to show.
Trap Ranking projects by expected revenue alone, which ignores whether the organization can deliver the proposed outcome.
5 questions test this
- A bank's AI steering group scored a proposal to automate small-business loan document checks as high value and well aligned with strategy. The data is ready. However, the lending operations team has n
- A retail chain's AI steering committee ranked five proposals by expected revenue alone. The top proposal needs customer data the company does not yet collect and an integration the IT team cannot deli
- A regional airline's three-year strategy focuses on on-time performance. A vendor pitches an AI tool for a hotel-booking business that the airline has no plans to enter, with large projected revenue.
- A hotel group's digital team can build a generative AI tool that writes playful room descriptions within a week. A harder proposal would cut the time front-desk staff spend on billing disputes, which
- A logistics company can fund one AI initiative this year. Proposal A forecasts delivery delays and uses clean shipment data that the operations team already owns. Proposal B promises slightly higher v
- AI portfolio sequencing must fund capabilities that later initiatives depend on
Some AI initiatives depend on earlier capabilities, such as reliable customer information or transaction categorization before more advanced customer recommendations. Sequence the enabling work before dependent outcomes and recognize its contribution to the wider portfolio. Evaluate dependencies across projects so an attractive downstream opportunity does not receive expansion funding while its essential foundation remains unavailable.
Trap Funding only the highest standalone ROI project, which can leave the prerequisites for realizing that project's benefits unfunded.
4 questions test this
- An insurer's portfolio committee funds AI projects one at a time by standalone ROI. This year's top project, claims fraud scoring, depends on a governed claims history service that scored low on stand
- A credit card issuer wants AI next-best-offer recommendations for its customers. Card transactions are not yet categorized reliably, so offer models would have no dependable signal of what matters to
- A telecom company's AI pilot for proactive outage alerts showed strong results with manually prepared network data. The automated network data feed that a full rollout needs will not be available for
- A manufacturer's data team built a clean, governed product master data service. Its own ROI looks small, so the finance team proposes cutting its budget. Three planned AI initiatives in demand plannin
- A balanced AI portfolio combines near-term results with longer-term strategic options
AI portfolio management should balance small, demonstrable improvements with initiatives that may create new products or business models over a longer horizon. Quick wins build experience and confidence, while more uncertain opportunities need bounded learning investments and appropriate expectations. Compare initiatives within their intended horizons instead of requiring every strategic experiment to resemble an immediate cost-saving project.
Trap Applying an immediate operating-cost payback requirement to every exploratory investment, which can exclude strategically valuable options before their hypotheses are tested.
4 questions test this
- A regional retailer is starting its first AI program. Store managers are skeptical, and the leadership team has little AI experience. The candidates are a small markdown-pricing improvement that can s
- After a competitor launched an AI-designed product line, an appliance maker's chief executive proposes spending the entire AI budget this year on three long-horizon new-product ventures. The company h
- A consumer goods company requires every AI investment to pay back its cost within 12 months through lower operating costs. A team proposes a bounded experiment on an AI-designed product line that coul
- A logistics company's AI portfolio contains eight initiatives, and all of them reduce operating costs within a year. The board has asked how AI will create new services or revenue streams over the nex
- Scale, pause, and termination decisions should follow explicit evidence-based gates
Set business success thresholds, quality limits, cost expectations, and readiness criteria before a pilot or release. Scale when the evidence meets the relevant gates, pause for a remediable gap with a defined next test, and stop or redirect when the use case cannot justify further investment. A convincing prototype is not sufficient evidence of business viability or readiness for full production.
Trap Using an executive deadline as the release criterion, which supplies schedule pressure rather than evidence that the agreed value and readiness gates were met.
4 questions test this
- A restaurant franchisor piloted an AI tool that drafts local promotions for franchise owners for ten months. Sales lift stayed far below the agreed gate, few owners used the drafts, and the team has i
- A logistics company's AI route-exception assistant impressed executives in a prototype demonstration that used a small set of sample cases. The company has not yet tested it under realistic peak load
- An insurer's AI pilot for claims-intake triage met its accuracy and value gates. It failed the readiness gate because contact center staff have not yet been trained on the new escalation process. The
- A telecom company's chief executive promised the board that an AI retention-offer engine would launch company-wide next month. The agreed gates require a three percent churn reduction and an offer err
- Sustainability is an AI investment consideration alongside business value and feasibility
AI proposals should account for the environmental resources required to deliver the intended benefit and the organization's sustainability goals. Compare alternatives that meet the business requirement and avoid funding unnecessary processing or capacity simply to maximize AI use. Track relevant efficiency or environmental measures alongside the outcome so a business improvement can be assessed against its resource footprint.
Trap Using financial ROI alone to rank proposals against an explicit environmental objective, which leaves the required sustainability outcome unmeasured.
4 questions test this
- A logistics company approved an AI route-optimization initiative partly because it supports the company's emissions targets. The quarterly dashboard tracks only cost savings and on-time deliveries. Th
- A retailer's AI demand forecasting workload runs once each night, but the proposal reserves dedicated compute that stays running all day at full capacity. The retailer has a corporate goal to reduce e
- A bank needs an AI classifier that routes customer emails into ten categories. Testing shows that a small, right-sized model meets the accuracy target, and the largest available model is only slightly
- A utility has a board-approved goal to cut the carbon footprint of its IT operations by 30 percent. Three AI proposals are being ranked. The finance team proposes ranking them by financial ROI only, b
- An AI portfolio must be reprioritized as business needs and observed results change
Review the AI portfolio regularly against current strategy, demonstrated outcomes, and resource constraints. Redirect future funding when an initiative does not address a priority or another opportunity provides stronger value for the available capacity. Prior investment explains history, but continued investment needs a current business rationale and realistic remaining benefits.
Trap Using the amount already spent as the reason to continue funding, which substitutes historical expenditure for the value of the next investment.
4 questions test this
- A manufacturer approved its AI portfolio three years ago and has not revisited it since. Several initiatives now report results well below plan, the company strategy has shifted toward services, and e
- A hospital network's five-person analytics team supports Initiative A, which meets its targets but has only small remaining benefits from optional enhancements. Initiative B fits the strategy and now
- A bank has spent 4 million dollars over two years on an AI wealth-advice platform. A current review shows low adoption and modest remaining benefits. A new fraud initiative offers stronger value for t
- An airline funded an AI initiative to optimize in-flight duty-free sales. The airline has since decided to exit duty-free retail and focus on loyalty and premium travel. The initiative is on schedule
- Stable rules and well-defined paths usually favor deterministic automation
When a process has explicit rules, predictable inputs, and defined paths, use conventional automation if it satisfies the requirement. AI becomes useful when the work needs learned patterns, interpretation of varied information, or adaptation to context. Compare the flexibility actually needed with the additional cost, latency, and oversight of an AI approach.
Trap Using an autonomous agent for a fixed approval routing table, which adds reasoning and governance overhead to a process with fully specified transitions.
4 questions test this
- A manufacturer routes purchase requests for approval by using a fixed table: amount ranges map to specific approvers, and the table rarely changes. An IT architect proposes an autonomous AI agent that
- An insurer processes policy renewals through five fixed steps that have not changed in years. A vendor pitches an AI agent to run the renewals, citing flexibility. The renewals team has no need for fl
- A payroll provider copies employee hours from a standard time-tracking export into its payroll system millions of times each month. The format never changes, and no judgment is required. A consultant
- A distributor receives 4,000 customer emails a day about orders, quotes, and complaints. Customers write free text with no standard format and often mix several requests in one message. A keyword-rule
- Finding existing information does not automatically require generative AI
When users only need to locate an existing item or retrieve an exact approved value, a suitable search or lookup function may meet the need with fewer resources. Generative AI is more appropriate when the requirement includes synthesizing, explaining, or drafting content from information. Evaluate the required outcome rather than replacing a working retrieval task merely because generation is available.
Trap Using generated prose as an exact account-balance lookup, which adds probabilistic generation to a requirement for an authoritative stored value.
4 questions test this
- A consulting firm's analysts must produce a two-page briefing for each client meeting that synthesizes findings across dozens of internal reports. Today they search for reports and then write the synt
- A bank's mobile app shows customers their account balance. The app has 2 million users. A product manager proposes replacing the balance screen with a generative AI chat that tells customers their bal
- A retailer's service agents send customers letters that explain refund decisions. Each letter must state the exact refund amount stored in the order system and explain the decision clearly in the cust
- Employees at a large company struggle to find existing HR policy documents on the intranet. Surveys show that they only want to locate and open the right document quickly. An HR leader proposes a gene
An AI proposal needs data or knowledge sources appropriate to the intended task, with adequate quality and permitted access. When these prerequisites cannot be met, first remediate the constraint, narrow the use case, or select a workable non-AI process. A sophisticated solution cannot establish missing business facts or make prohibited data use acceptable.
Trap Collecting a larger volume of the same unreliable records as the remedy, which increases quantity without fixing the essential quality or access gap.
4 questions test this
- An industrial parts seller wants an AI assistant to promise delivery dates to customers. Dates depend on supplier lead times, which the company records nowhere because buyers keep them in their heads.
- A manufacturer wants AI-based quality predictions across five product lines. Sensor data is complete and well governed for one line. The other four lines have gaps and inconsistent formats that will t
- A subscription retailer wants an AI model that predicts which customers will cancel, from recorded support calls. Its privacy notice says recordings are used only for staff quality training, and couns
- A distributor wants AI to forecast product returns. Its return records have missing reason codes, inconsistent product identifiers, and duplicate entries. The data team proposes collecting two more ye
- AI is unsuitable for an autonomous decision when its errors cannot be acceptably controlled
AI results are probabilistic, so suitability depends on the consequence of error and the available controls. For consequential work, consider a human-reviewed assistive role if it can deliver value within the required oversight. If the process requires absolute predictability or the necessary review and safeguards are not feasible, use an appropriate deterministic or human process instead of forcing autonomous AI into the role.
Trap Using strong average accuracy as authorization for unreviewed consequential decisions, which does not establish that the remaining errors are acceptable or controlled.
4 questions test this
- A logistics company plans AI for two tasks. One is basic routing of incoming shipping documents into folders, where occasional errors are easily corrected. The other is deciding customs compliance fil
- A lender's AI model reached 97 percent average accuracy in testing for small-business loan decisions. The chief executive proposes letting it approve and deny loans with no human review. Denials affec
- A pension administrator calculates monthly benefit payments by using formulas set in law. Every result must be exactly reproducible for audits, and the same inputs must always produce the same output.
- A small urgent-care clinic wants AI to decide in real time which arriving patients are seen first. Errors could delay treatment of critical patients. The clinic has no staff available to review the AI
- A staged AI transition limits business exposure while the replacement proves itself
When replacing an established process or platform, introduce the new capability to a limited population or workflow before expanding. Define service acceptance criteria, ownership of exceptions, and a usable fallback or return path for unacceptable results. Retain the current capability until the business has evidence that the replacement can support the required work without unacceptable disruption.
Trap Retiring the current process immediately after a successful demonstration, which removes continuity protection before representative operational validation.
4 questions test this
- A bank is piloting an AI assistant that prepares responses to routine account-servicing requests for one branch region. The pilot is going well, and regional managers want to expand it. No one has agr
- A property insurer is moving claim intake from its call center to an AI voice assistant in one state. The pilot plan defines acceptance criteria and an exceptions owner. It does not say what happens w
- A logistics company has run an AI scheduling tool alongside its legacy scheduling system for two months in one depot. Finance wants to stop paying for the legacy system now. The depot has not yet hand
- A regional grocery chain plans to replace its manual supplier-dispute process with an AI agent in all 120 stores on one weekend. The vendor demonstration went well, but finance staff have not used the
- A platform transition must preserve usable information and appropriate access
Before moving an AI-enabled process, inventory the information sources, formats, refresh needs, ownership, and access rules that make it work. Confirm that the destination can use the required information and preserve the relevant restrictions and freshness. Resolve these readiness gaps before business acceptance because moving the application does not itself establish that its knowledge and permissions remain correct.
Trap Approving the transition after copying the documents alone, which does not establish that the destination can use current content under the required access rules.
4 questions test this
- A utility plans to move an AI assistant for field technicians to a new platform next quarter. The assistant draws on equipment manuals, maintenance records, and safety bulletins kept by different depa
- A hospital moved its AI policy assistant to a new platform. The application works, but testing found that some nursing protocols are out of date and some restricted documents appear in answers to staf
- A law firm moved its document collection to a new AI knowledge platform. The migration team copied every document and asks for sign-off. In the old system, some matter files were visible only to the p
- A manufacturer is moving its technicians' AI assistant to a new platform. The old platform answered from text manuals, wiring diagrams stored as images, and parts spreadsheets. All files were copied,
- Moving a process to AI changes human work as well as software
An AI process transition must define who reviews outputs, handles exceptions, approves consequential actions, and supports users. Prepare affected employees for the new workflow before shifting the work, and include that effort in readiness and cost decisions. Buying or switching technology does not automatically establish a workable division of responsibility between the AI capability and people.
Trap Using tool access as the employee transition plan, which supplies software availability without establishing new responsibilities or exception handling.
4 questions test this
- A bank is moving loan pre-screening to an AI agent. The transition plan gives every loan officer a login to the new tool and a link to a video. It does not say who reviews the agent's recommendations,
- A telecom company is moving account adjustments to an AI agent. Most adjustments are small, but some exceed 500 dollars or close customer accounts. The current process has a supervisor sign-off for th
- An insurer's business case for moving claims triage to AI counts saved processing hours multiplied by labor cost, minus the software cost. It includes nothing for staff who will review AI output, hand
- A customer support organization will move refund handling to an AI-assisted workflow next month. Agents will shift from processing refunds to reviewing AI decisions and handling escalations. The agent
- AI continuity depends on every essential part of the business service
AI service continuity requires more than an available model: the process also depends on its information sources, connected business systems, support arrangements, and recovery capability. Identify essential dependencies and obtain evidence that the recovery arrangements can support the required business service. A provider availability commitment does not by itself demonstrate that the complete workflow can recover after a disruption.
Trap Using a model provider's uptime commitment as the complete continuity plan, which leaves the workflow's data and connected-system dependencies unassessed.
4 questions test this
- A bank documented a recovery plan for its AI fraud-review workflow, including a standby knowledge store and a secondary model endpoint. The plan has never been exercised. The bank's risk committee own
- An online retailer is setting continuity targets for its new AI product-question assistant. The IT team proposes zero downtime and zero data loss because the assistant is customer facing. Finance worr
- A manufacturer's AI maintenance assistant answers from a search index built from engineering documents in a separate document store. The continuity plan backs up the index nightly but not the document
- A travel company's AI booking assistant depends on a model provider, the company's fare database, a payment system, and a document store of travel policies. The continuity plan cites only the model pr
Measure and demonstrate AI business value
Read full chapterUnlock with Premium — includes all practice exams and the complete study guide.
Position AI for competitive advantage
Read full chapterUnlock with Premium — includes all practice exams and the complete study guide.
AI Governance and Responsible AI Leadership
Apply Responsible AI Principles to Business Decisions
Read full chapterCheat sheet
Sharp facts the exam loves — scan these before test day.
- Fairness goals must reflect the people and outcomes a use case affects
Fairness considers impacts on different stakeholder groups, not just whether an AI system treats every input identically. Similar treatment of comparable individuals and appropriate outcomes across groups answer different fairness concerns; choose the goal with affected stakeholders and document conflicts between goals. Access also matters: balanced prediction rates do not make a service fair if people with disabilities cannot use it.
Trap Using identical interaction requirements for every customer as the fairness standard, which overlooks accessibility and differences in stakeholder needs.
5 questions test this
- A delivery platform's AI assigns jobs with one formula that it applies identically to every courier. Leaders call the formula fair for that reason. Data show that couriers based in outer suburbs recei
- A bank is launching an AI voice assistant for account servicing. To be fair, the project team proposes that every customer must use the same voice-only interaction, with no exceptions. Some customers
- An insurer uses AI to set renewal premiums. Its group-level metrics show similar average premiums across demographic groups. However, two customers with nearly identical risk profiles, who differ only
- A retailer's AI hiring screener does not use applicants' race or gender as inputs. The vendor therefore claims that the screener is fair. However, the model uses postal codes and school names, which c
- A university's AI tool ranks scholarship applicants. Analysts confirm that its goal of equal selection rates across applicant groups conflicts with its goal of giving similar scores to applicants with
- An explanation of one decision serves a different need from an explanation of overall behavior
Explainability helps people understand why an AI system produced an output. When an applicant needs reasons for one denial, require an explanation of that individual decision; when an oversight team needs to understand general behavior, require information about the factors influencing predictions across cases. Set the required explanation level with business and compliance stakeholders before committing to a solution.
Trap Providing a portfolio-wide factor summary to explain an individual denial, which does not establish what influenced that applicant's decision.
3 questions test this
- A credit union must tell each declined loan applicant the main reasons for the decision. The vendor offers a report that lists the five factors that matter most across the whole loan portfolio. The cr
- An insurer's model risk committee must approve a claims-triage model. The committee wants to understand which factors drive the model's predictions across all claims and to check that the overall patt
- A pharmaceutical company deploys a generative AI assistant that answers researchers' questions from internal study reports. Researchers say that they cannot tell which reports shaped an answer, so the
- Privacy governs appropriate data use while security protects data and systems
Privacy concerns whether information is appropriately obtained, used, and managed; security concerns protection against unauthorized access, manipulation, and disclosure. An authorized employee can still use personal information for an inappropriate purpose, so a privacy review is needed even when access is secure. Conversely, a permitted business purpose still requires protection against theft and tampering.
Trap Using encryption approval as the authorization for a new personal-data purpose, which evaluates protection rather than whether that use is permitted.
4 questions test this
- A bank's customer chatbot can look up account details for authenticated users. Testers find that a logged-in user can type hidden instructions that make the chatbot reveal another customer's account s
- A hospital's privacy office has approved using de-identified patient records to train an AI model that predicts readmissions. The approved purpose is clear and documented. The project team now propose
- A retailer collected customer purchase histories for order fulfillment. A product team now wants to use the same data to train an AI model that predicts customers' health conditions for targeted ads.
- A call center records customer calls for quality and training purposes, as its notice states. Its analytics team, which has approved access, wants to use the transcripts to train an AI that flags call
- Factual correctness does not establish that an AI output is safe
Safety addresses harmful outputs, actions, and misuse, whereas veracity addresses factual correctness. Accurate instructions can still enable harm, so a business must assess permitted uses as well as truthfulness. A harmless-sounding answer can also be factually wrong, requiring factual validation even when it passes a harmful-content check.
Trap Using a factual-accuracy review as the safety approval, which does not assess harmful uses of accurate information.
4 questions test this
- A media company uses a generative AI tool to draft investment news summaries. Every draft passes the company's harmful-content filter. An editor later discovers that a published summary cited a market
- A law firm's AI assistant drafts answers to client questions based on the firm's policy library. Evaluators currently score drafts only for fluent writing and absence of offensive language. Partners w
- An education company is building an AI tutor for children aged 8 to 12. A test answer about historical battles was factually accurate but included graphic descriptions of violence. The product team ar
- A chemical supplier is launching an AI assistant that answers customers' product questions. The launch plan treats a factual-accuracy review by chemists as the assistant's safety approval. Testers fou
- Robustness means maintaining appropriate performance as inputs vary
Robustness concerns correct outputs for expected and unexpected inputs, including ordinary changes in language, image quality, or environmental conditions. Require evaluation on the variations actual users will encounter, not only clean examples. Distinguish changes that should affect the answer from irrelevant variations the system should tolerate, and assess combinations such as accented speech with background noise.
Trap Using higher service uptime as the remedy for dialect-dependent errors, which measures availability rather than performance under input variation.
4 questions test this
- A pharmacy chain is building an AI assistant that answers medication questions typed by customers. Customers use varied spellings, abbreviations, and casual wording, while some changes in wording, suc
- A utility's AI voice assistant misunderstands customers who speak with certain regional dialects, which causes wrong account actions. The operations team proposes buying a higher-availability hosting
- A call center's AI transcription tool passed separate tests for accented speech and for background noise. In production, accuracy drops sharply for accented callers in noisy environments such as train
- An insurer is testing an AI tool that reads photographed claim forms. All test images were high-resolution scans prepared by the vendor. In production, customers will submit phone photos taken in poor
- Controllability requires interventions that reliably change AI behavior
Controllability means having mechanisms to monitor and steer system behavior, including adjustment, override, or rollback where appropriate. Judge a control by whether it produces the intended change quickly enough for the business risk, and test whether it creates unintended effects elsewhere. A visible control is insufficient evidence unless users can operate it effectively and understand when it cannot work.
Trap Using a monitoring dashboard as the complete controllability mechanism, which provides visibility without an effective way to intervene.
4 questions test this
- A bank's AI fraud system can be paused by operations staff if it starts wrongly blocking legitimate card payments. A drill showed that the pause took six hours to take effect because of manual approva
- A marketing team's AI writing tool lets users set a tone control from formal to casual. User tests show that the setting changes the tone for short emails but is silently ignored for long reports, so
- A social platform lets moderators raise the strength of its AI content filter during breaking news events. After a recent increase, harmful posts dropped, but posts from news organizations reporting o
- A retailer's AI pricing system adjusts online prices every hour. The governance team approved a real-time dashboard that shows price changes as its controllability mechanism. During a test, the system
- Equal approval rates and equal recognition of qualified people are different fairness goals
Demographic parity asks whether groups receive positive decisions at comparable rates, such as similar shares of all applicants being recommended for a job. Equal opportunity asks whether genuinely qualified applicants are correctly recommended at comparable rates across groups. Because the first comparison includes all applicants and the second includes qualified applicants, meeting one goal does not establish the other; agree which concern matters in the business context before judging the results.
Trap Using similar overall accuracy across groups to assess equal recognition of qualified candidates, because overall correctness also counts negative cases and can conceal differences in identifying qualified people.
- AI disclosure should let people make informed choices before relying on the system
Transparency enables people to make informed choices about engaging with AI. Identify where users first encounter the system and where they act on its outputs, then disclose AI involvement and relevant limitations at those points. Match the prominence and frequency of disclosures to the context and risk rather than relying only on a distant policy page.
Trap Placing the only AI notice in an annual corporate report, which does not inform users at the points where they engage with or rely on AI.
4 questions test this
- A tax software company adds an AI feature that suggests deductions. The feature cannot handle some complex situations, such as income earned in several countries. Users currently learn about this limi
- A retailer added an AI shopping assistant with a short label that says it uses AI. Leaders believe the disclosure requirement is complete. A survey finds that half of the customers still think a human
- A bank uses AI in two places: a feature that recommends articles on its website, and a tool that pre-screens mortgage applications. The compliance team proposes the same small footnote for both. Mortg
- An insurer uses AI to draft responses in its customer chat service. The only statement that AI is involved appears in the company's annual corporate report. Customers act on chat answers to decide whe
- System cards communicate intended uses and limitations but do not replace local evaluation
An AI system card brings together intended uses, limitations, responsible AI design choices, and relevant performance information. Use it to judge whether a proposed application fits the documented boundaries and to prepare guidance for downstream users. The deploying organization must still evaluate performance on its own inputs and workflow; supplier documentation does not establish suitability for every customer context.
Trap Using a supplier's general service card as the go-live evidence for a new population, which omits evaluation of the deployer's specific inputs and conditions.
4 questions test this
- A healthcare provider plans to use a vendor's speech-to-text AI service to transcribe doctor visits. The vendor's service card reports strong accuracy on general business speech. The provider's visits
- A logistics company deploys a vendor's AI route-planning service for its dispatchers. The vendor's service card states that the service is not designed for hazardous-material shipments, which have spe
- A bank is considering an AI document-processing service for two uses: extracting fields from standard loan forms and reading handwritten notes from branch staff. The service card lists typed forms as
- A company built an internal AI model that drafts first responses to supplier contract questions. Several business units now want to use it. Leaders ask the team to create a system card. A team member
- Useful transparency is tailored to the audience and protects sensitive information
A transparency strategy should match each stakeholder's information needs, expertise, accessibility requirements, and communication channel. Customers may need plain-language capabilities and limitations, while authorized reviewers need more detailed evidence. Review public disclosures separately from restricted materials so that openness does not reveal personal data, proprietary information, or exploitable security details.
Trap Publishing raw internal evaluation records to maximize transparency, which can expose protected information without giving users an understandable explanation.
4 questions test this
- A card issuer's AI fraud system blocks suspicious transactions. Customers ask how it decides, and a product manager proposes publishing the exact score thresholds and detection rules online. The secur
- A lender wants to show that its AI credit model is transparent. A manager proposes publishing the full internal evaluation records online, including test datasets with customer details and notes on kn
- A tax authority's AI voice assistant answers callers on its phone helpline. Information about what the assistant can and cannot do appears only in a technical PDF on the authority's website. Most help
- A hotel chain is preparing a public explanation of its AI booking assistant. A draft includes example conversations copied from production logs, with guest names and room numbers, and a list of known
- AI user guidance must change when capabilities, limitations, or intended uses change
AI documentation needs an owner, review process, version history, and a way to reach affected users. Trigger review after system releases, new uses, significant stakeholder feedback, or relevant changes in expectations. Updating an internal document is only part of the task: publish the approved guidance through the channels users actually rely on and address outdated copies.
Trap Updating the engineering release notes alone, which does not revise or distribute the guidance business users need for responsible use.
4 questions test this
- An insurer revised the guidance for its AI claims-estimate tool after finding that the tool is unreliable for flood damage. The new guidance is published on the intranet. However, adjusters still use
- A retailer's AI pricing assistant was approved for setting prices on in-store items. Managers now want to use it for online flash sales, which it was not designed for. Store staff have also given feed
- A bank updated its AI expense-review assistant so that it no longer checks receipts in foreign currencies. Engineers recorded the change in their release notes. Finance staff, who rely on a user guide
- A manufacturer has published guidance for its AI quality-inspection tool, but no one is responsible for keeping it current. Different plants use different versions of the guidance, and nobody can tell
- Uncertainty information is useful only if users know when to rely on or question an output
AI guidance should explain uncertainty, known limitations, and the circumstances that call for additional verification. Evaluate whether different user groups actually recognize when an output deserves trust, further checking, or rejection. Measure comprehension and decision quality as well as reported trust, because a persuasive interface can increase confidence without improving judgment.
Trap Using higher user trust ratings as the sole proof of effective explanations, which does not show that users can identify unreliable recommendations.
4 questions test this
- A utility's AI tool gives call center agents repair-time estimates for power outages. The estimates are reliable for routine outages but often wrong during major storms, and agents pass the wrong time
- A legal research tool shows paralegals a confidence label with each AI-suggested case citation. After a redesign, average trust ratings rose from 3.1 to 4.4 out of 5. Partners will expand the tool to
- A bank redesigned the explanations in its AI fraud-alert tool, which flags about 2,000 card transactions each day for analyst review. After the redesign, analysts rated their trust in the tool much hi
- A clinic's AI tool suggests likely diagnoses with a confidence score for each one. Nurses say they do not know what to do with a score of 60 percent, so some ignore all suggestions while others follow
- Business performance improvements cannot substitute for mandatory obligations
When an AI use case has a mandatory explanation, privacy, or other compliance requirement, treat that requirement as a condition of proceeding. Accept a lower business-performance result if it is the feasible compliant option and still meets necessary operating standards. If no option meets the obligation, change the scope or defer the use rather than promising compliance after release.
Trap Selecting the highest-accuracy option with a later compliance retrofit, which leaves the mandatory obligation unmet when the system begins making decisions.
4 questions test this
- A fitness app must launch an AI coaching feature within six weeks under a signed partnership contract. Local law requires explicit consent before health data is used to train models. A model trained o
- An auto insurer must replace its retiring claims system within three months. Regulation requires a licensed adjuster to approve every claim denial before it takes effect. Option A denies claims automa
- A bank wants an AI assistant to recommend investment products to retail customers. Local rules require each recommendation to include an explanation of its suitability for the customer. After several
- A city housing agency wants AI to decide rental-assistance eligibility and to sort incoming applications by missing documents. Local law requires every eligibility denial notice to state the specific
- A responsible AI tradeoff considers who receives the benefit and who bears the harm
Evaluate benefits and harms separately for affected stakeholder groups rather than treating company-wide gains as a complete justification. Include people indirectly affected by decisions and groups that may experience disproportionately severe harm. Use explicit organizational principles and stakeholder needs to prioritize competing outcomes, and document the rationale so later decisions use the same standards.
Trap Using total company savings as the complete tradeoff criterion, which leaves the distribution and severity of stakeholder harms unexamined.
4 questions test this
- A delivery company's AI route optimizer cuts fuel costs by 12 percent company-wide. The new routes send heavy trucks past two primary schools during morning drop-off. The business case counts only com
- A bank's AI collections tool raised loan recoveries by 9 percent by contacting overdue customers more often. Complaints across the whole customer base stayed low, but customers in documented financial
- A property management company sells an AI tenant-screening tool to landlords. Its business case lists benefits for landlords, such as faster approvals and fewer defaults. Rental applicants never use t
- A utility plans to replace phone agents with an AI voice system, which would save USD 5 million each year. Analysis shows that many older customers and customers with hearing impairments cannot comple
- Privacy-enhancing choices must be evaluated alongside utility and fairness
Privacy protections such as minimizing, de-identifying, or aggregating information can reduce exposure but may also reduce useful detail. Evaluate the effect on the business task and on different groups, particularly where data is sparse. Choose the least sensitive information that can support the authorized purpose, and document acceptable utility tradeoffs without relaxing mandatory privacy obligations.
Trap Maximizing data aggregation as the only design goal, which can remove detail needed for useful or fair outcomes in smaller groups.
4 questions test this
- A clinic must de-identify patient notes before using them to train an AI documentation assistant. De-identification lowers the assistant's accuracy for medication names. The project lead proposes skip
- A telecom company wants to use an AI service to draft personalized retention offers. The marketing team proposes sending the service each customer's full record, including payment history, location hi
- A bank plans to protect privacy by aggregating customer data to the regional level before training a credit model. Testing shows that the model's accuracy drops sharply for a small rural customer grou
- A hiring platform removed age, gender, and ethnicity fields from its candidate data to protect privacy before training a screening model. The fairness team now reports that it cannot measure whether o
- Safety controls should be assessed for missed harms and unnecessary blocking
Protective filters can miss harmful material and can also block legitimate requests. Assess both kinds of error against the use case, user population, and harm severity before agreeing acceptance criteria. A stricter control may be justified for severe harms, while ambiguous legitimate requests may need review rather than automatic rejection; monitor useful service delivery alongside protection effectiveness.
Trap Using the percentage of blocked requests as the sole safety success metric, which rewards unnecessary blocking as well as genuine protection.
4 questions test this
- A bank deployed content filters on its customer chatbot. The safety team reports success because the filters block 18 percent of all customer requests. Customer complaints show that many blocked reque
- An insurer's AI assistant answers policyholder questions. Its filter automatically rejects any question that mentions injuries. As a result, legitimate questions about how to file an injury claim are
- A mental health support app uses AI to respond to user messages. Filters that detect messages about self-harm sometimes flag ordinary conversations about stress. Product managers want to relax the sel
- A travel company launched an AI booking assistant with content filters that met its acceptance criteria before launch. Six months later, the company tracks only the number of harmful responses that th
- Narrowing an AI use case can preserve value when broad deployment is not supportable
When a system cannot meet essential criteria across a broad application, restrict it to domains, inputs, locations, or users for which it can meet the required standards. Reassess benefits and risks and update the criteria for that narrower use. Scope reduction is a valid risk treatment when meaningful value remains; persistent inability to control harm is a reason to stop that use, not lower essential standards.
Trap Lowering the safety acceptance threshold to preserve the original rollout scope, which changes the protection standard instead of resolving the unsupported use.
4 questions test this
- A pharmacy narrowed its AI assistant from all medication questions to dosage reminders for one chronic condition. Even in this narrow scope, testing shows that the assistant sometimes gives wrong dosa
- A bank narrowed its AI advisor from all investors to retail investors who use standardized products, because the broader scope failed its release criteria. The team plans to launch the narrower versio
- A logistics company's AI support agent gives unreliable answers when customers type long free-text messages with several requests. It performs reliably when customers choose a request type and enter a
- A company built a general-purpose AI help assistant. Release testing shows that it consistently fails the safety criteria on medical questions, despite several redesigns, but it meets all criteria on
- Responsible AI belongs in initial planning and continues throughout the lifecycle
Governance by design incorporates responsible AI requirements when defining the use case, before data and solution commitments lock in avoidable risks. Carry those requirements into evaluation, user guidance, production monitoring, and retirement. Iterative pilots can still move quickly, but their boundaries and required approvals must be clear from the start rather than postponed until production.
Trap Scheduling responsible AI review only after the pilot is selected for production, which allows consequential data and workflow decisions to precede the relevant requirements.
5 questions test this
- A media company wants to pilot a generative AI tool that drafts article summaries within six weeks. Leaders want fast iteration during the pilot. The compliance lead worries that the pilot will quietl
- A hospital network wants an AI assistant for its staff. The innovation team proposes building one assistant that can answer any staff question and deciding later which uses are acceptable. The governa
- A bank plans to retire an AI model that scores small-business loan applications. Two internal workflows depend on the model's scores, and its training dataset contains applicant records. Leaders want
- A retail company will deploy an AI pricing assistant in three countries next year. The product team has drafted the use case and wants to sign a vendor contract next week. Nobody has yet checked which
- An insurance company launched an AI assistant that helps adjusters estimate repair costs. The launch passed every responsible AI release criterion. The project sponsor now proposes closing the respons
- AI planning must include upstream contributors and people indirectly affected by outputs
Stakeholder mapping should cover direct users, people whose information is processed, people affected without using the system, and upstream contributors such as suppliers. Seek perspectives outside the project team, including vulnerable populations where relevant, to reveal missed requirements and harm scenarios. Revisit the map when the workflow or audience changes rather than treating the initial list as permanent.
Trap Surveying only paying users to establish stakeholder requirements, which omits data subjects, indirect stakeholders, and upstream dependencies.
5 questions test this
- A retailer is planning an AI demand forecasting system. The forecasts will depend on weather data from an external supplier and on a model component licensed from a vendor. The project's stakeholder l
- A telecom company is planning an AI system that flags accounts for possible fraud by analyzing call records. The product team proposes surveying paying subscribers to collect all stakeholder requireme
- A bank deployed an AI assistant that answers employees' policy questions. The bank now plans to let customers use the same assistant through its mobile app. The stakeholder map was created two years a
- A pharmaceutical company's AI team built a stakeholder list for a drug-safety report summarizer. The list names data scientists, engineers, and the safety analysts who will use the tool. Late in testi
- A city utility is planning an AI chatbot that will help residents apply for bill-payment assistance. The project team consists of engineers and billing analysts. Some likely applicants are elderly or
- Responsible AI requirements need measurable pass-or-fail release criteria
Translate expected benefits and identified harms into clear release questions, measurements, and acceptance thresholds before building or buying around the use case. Use existing performance evidence, stakeholder requirements, and risk tolerance to set realistic targets, and identify who is affected by each failed criterion. A complex promise such as a good user experience may need separate criteria for correctness, response time, and usability; no single universal accuracy percentage establishes responsible readiness.
5 questions test this
- A recruiting platform has listed benefits and harms for an AI tool that ranks job applicants. Its draft release criteria measure only benefits, such as recruiter time saved per hire. The harm list inc
- A travel company is defining release criteria for an AI booking assistant. The business sponsor's only stated criterion is that the assistant must deliver a good user experience. The product team need
- An insurer is setting a release threshold for an AI model that triages incoming claims. A consultant recommends a 99 percent accuracy target because it sounds safe. The insurer's current manual triage
- A bank is comparing three vendor proposals for an AI tool that summarizes customer complaints. Each vendor reports its own accuracy figure from different test data. The bank has identified expected be
- A legal services firm plans to release an AI tool that drafts contract summaries. The draft release criterion reads: leadership is comfortable with summary quality after reviewing examples. Two review
- Risk mitigation can combine safer system behavior, protective filtering, and user guidance
Assign each identified harm to a mitigation strategy: improve the underlying behavior, screen inputs or outputs, guide users, or combine these approaches. Match the strategy to how the harm arises and how much users can reasonably control. Guidance is valuable for appropriate use, while risks requiring enforced boundaries need system safeguards as well; evaluate the resulting protection rather than assuming one strategy fits every harm.
Trap Using a user disclaimer as the sole control for a prohibited system action, which asks users to respect a boundary that the system still permits.
5 questions test this
- A bank's customer chatbot must never provide personalized investment advice, which bank policy prohibits. The project team proposes a banner telling customers not to ask for investment advice. Testing
- A software company's AI coding assistant keeps generating code that uses a deprecated internal library. The team has already added user warnings and an output filter, but the problem persists across m
- An insurer's AI assistant answers customers' coverage questions. Testing shows that it sometimes states coverage terms that do not appear in the policy documents. Most customers cannot tell whether an
- A bank's internal AI assistant summarizes customer emails for service agents. Agents sometimes paste full account numbers into their requests, and bank policy prohibits sending account numbers to the
- A healthcare provider assigned two mitigations for its patient chatbot. A guardrail blocks dosage instructions, and on-screen guidance tells patients to contact a clinician about medications. The proj
- Critical or subjective AI evaluations benefit from an independent second assessment
When safety, fairness, or another consequential judgment depends heavily on interpretation, obtain corroboration from qualified reviewers outside the delivery team's incentives. For higher-risk uses, an independent team can develop evaluation cases from the documented intended use rather than relying entirely on cases chosen by the supplier or project team. Investigate material disagreements before approval and keep both findings as decision evidence.
Trap Repeating the delivery team's own demonstration as an independent review, which retains the same case-selection assumptions and incentives.
5 questions test this
- An insurer ran parallel safety evaluations of its AI claims assistant. The delivery team found no serious issues. An independent quality assurance team, using the same criteria, found several cases of
- A retailer's AI team has a limited budget for independent evaluation of a new shopping assistant. Planned evaluations include response latency, uptime, a safety assessment of harmful advice, and an as
- A bank's AI assistant team and an independent research team will both evaluate whether the assistant's answers show unwanted bias. The research team plans to use its own criteria and a different datas
- A healthcare company is buying an AI triage system that the supplier will help deploy. The supplier offers its own test cases, which show excellent results. Because the use is high risk, leaders want
- A public agency must choose independent reviewers for a high-risk AI eligibility system. Candidates include the system's original developers, a vendor that hopes to win the maintenance contract, and a
- Evaluation evidence must identify the system and data versions actually being approved
System updates require renewed evaluation and traceable records of the version, evaluation data, conditions, methods, and results. Recheck that criteria still cover current capabilities and relevant expectations instead of carrying forward approval for a materially different system. Link evaluation evidence to the release decision so an auditor can reconstruct what was tested and why that version was accepted.
Trap Using the previous version's safety report for an expanded release, which provides evidence about a different system and possibly different risks.
5 questions test this
- A manufacturer keeps evaluation reports for its AI defect-detection model in a shared folder, separate from its model and dataset records. A regulator asks the company to show which model build and da
- A company's AI support agent was approved last year based on a safety report for version 1, which could only answer questions. Version 2 can now issue refunds and change customer orders. The product t
- A bank's governance board approved the release of an AI document assistant in a meeting, but no one recorded why it was approved, who signed off, or which conditions applied. Six months later, an inte
- A retailer is preparing to release version 3 of its AI recommendation assistant. Version 3 adds image search, which did not exist when the release criteria were written. The team plans to reuse the ex
- An insurer evaluates its AI claims model on a public benchmark that it downloads from an external site for each test run. The benchmark's maintainers sometimes update the data without notice. Results
- Automated evaluation scales assessment but its judgments still need validation
Choose automated evaluation for repeatable measurements it can capture reliably, and use qualified human assessment for context or experience the automated method misses. An AI system used to judge another AI system should be checked against human judgments before its scores become approval evidence. Human assessment also needs clear criteria and consistency checks; neither automation nor expert involvement makes an evaluation method automatically reliable.
Trap Using an AI evaluator's high scores as self-validating approval evidence, which omits whether its judgments match the intended quality standard.
5 questions test this
- A bank's AI assistant passes an automated accuracy metric for answers about account fees. However, customer feedback shows that some technically correct answers are confusing and do not resolve the cu
- A healthcare company asks three clinicians to rate an AI tool's patient-education answers for accuracy. Review of the ratings shows that one clinician scores almost every answer as excellent, while th
- A bank is planning evaluations for an AI mortgage assistant. Two release criteria are under discussion: whether responses arrive within two seconds, and whether first-time buyers find the application
- A company uses a large language model as a judge to score its customer service assistant's answers for helpfulness. The judge gives average scores of 4.8 out of 5, and the product team wants to use th
- An insurer's AI assistant writes summaries of claim files. Release criteria require that summaries be coherent and follow the insurer's style guidelines. The team must evaluate about 50,000 summaries
- Human oversight should be placed where expertise can prevent consequential errors
Map the business workflow to identify decisions whose errors are hard to judge or have serious consequences. Require human review before an effect occurs when later correction would be inadequate; lower-risk, reversible activity may support automated handling with supervisory review. Oversight placement depends on the actual consequence and the value of human expertise, not merely whether a system is customer-facing.
Trap Using a monthly retrospective sample as the only oversight for irreversible individual decisions, which places review after the point at which it could prevent harm.
5 questions test this
- A mortgage lender uses AI to extract income figures from scanned application documents. Most scans are clear, but some are low quality or handwritten, and extraction errors then become hard to spot la
- A manufacturer has two AI tools. A customer-facing chatbot answers questions about store hours and product colors. An internal tool recommends which employees to lay off during restructuring. The over
- A health insurer uses an AI system to recommend denying coverage for some specialist treatments. Denials take effect immediately and are hard to reverse once a patient cancels treatment. The current o
- A professional services firm uses AI to tag incoming internal documents by topic for its search portal. Wrong tags are easy to correct and cause only minor search delays. The firm's governance lead pr
- A bank uses an AI tool that summarizes customer complaint emails for internal trend reports, with no human review. The bank now wants to use the same summaries to decide which customers receive compen
- Human review is meaningful only when reviewers can assess and overturn AI recommendations
Human reviewers need relevant expertise, training, sufficient information, and authority to reject or correct an AI recommendation. Give them the input evidence, explanations, and limitations needed for independent judgment, not just an approve button. Evaluate the human-and-AI workflow as well as the system itself so that nominal review does not become routine acceptance of unreliable outputs.
Trap Adding mandatory approval clicks for staff who cannot inspect the evidence or overturn the recommendation, which creates nominal rather than effective oversight.
4 questions test this
- An insurer added a mandatory human approval step for AI-recommended claim rejections. Reviewers see only the recommendation and an Approve button. Rejecting a recommendation requires a manager's writt
- A retailer's fraud team reviews AI alerts before blocking customer orders. The vendor reports 96 percent model accuracy. However, the team's combined decisions have caused rising wrongful order blocks
- A tax agency's caseworkers review AI recommendations to audit specific returns. Caseworkers see only a risk score from 1 to 10. They report that they cannot tell why a return scored high, so they eith
- A hospital uses AI to flag chest scans that might show early lung disease. The current reviewers are administrative staff who confirm flags by checking that the scan file opened correctly. The hospita
- Escalation rules must specify both the trigger and the person authorized to act
Define escalation criteria using factors such as harm severity, uncertainty, policy violations, and business impact. Specify the reviewer, required response, decision authority, and documentation so that an exception reaches someone able to resolve it. High-impact policy violations may require escalation even when an output appears confident, while ambiguous lower-risk cases may follow a different review path.
Trap Routing every exception to a general inbox without decision rights, which supplies notification but no accountable resolution path.
4 questions test this
- A payments company escalates AI fraud alerts to analysts when certain criteria are met. Six months after launch, analysts report that many escalations are false alarms, while some real fraud patterns
- A social media company uses AI to moderate user posts. Some flagged posts are clearly violent threats, while others are ambiguous jokes with low risk. All flags currently follow one review path, and t
- A grocery chain's AI system raises about 150 food safety flags from supplier data each day, most of them minor. Flags are emailed to store managers, who cannot remove products from other stores. Last
- A telecom company's AI system flags billing anomalies that may overcharge customers. All flagged exceptions go to a shared mailbox that nobody owns. Exceptions often sit for weeks, and no one has auth
- People affected by AI decisions need a way to challenge and correct outcomes
A recourse process lets affected people report problems, challenge an outcome, and obtain review of incorrect information or decisions. Make the route understandable and accessible and give reviewers the ability to provide an appropriate remedy. Feed validated complaints into system evaluation as well, because resolving one case and identifying a recurring problem serve different needs.
Trap Offering only a satisfaction survey after an adverse decision, which gathers opinions without providing review or correction of that decision.
4 questions test this
- A government agency uses AI to help decide eligibility for housing benefits. Applicants who are denied receive only a notice that says they are ineligible. Advocacy groups say that applicants cannot c
- A utility company uses AI to decide which customers qualify for a discounted rate. Customers can dispute a decision only by mailing a form described on page 14 of the terms of service. Many affected c
- A credit card issuer's AI system lowers credit limits automatically. Customers who appeal speak to an agent who can explain the decision but cannot change it or fix incorrect data in the customer reco
- A rental marketplace uses an AI tool to screen tenant applications. Rejected applicants currently receive only a satisfaction survey about the application experience. Several applicants say that the r
- Agent autonomy requires explicit boundaries on actions taken for the organization
When AI can act rather than merely recommend, define which activities it may perform autonomously and which require human approval. Set approval boundaries before irreversible or consequential effects and extend oversight records to actions and external interactions, not just chat responses. An approved information assistant does not automatically have approval to make commitments or change business records on the company's behalf.
Trap Extending a chatbot's approval to autonomous business transactions, which treats permission to provide information as permission to act.
4 questions test this
- A retailer approved an AI chatbot last year to answer customer questions about orders and return policies. The product team has now connected the same chatbot to the payment system so that it can issu
- A sales manager connected an unvetted AI agent to the corporate mailbox so that it can reply to customers and promise delivery dates automatically. No one approved the agent's ability to make commitme
- A telecom company's AI agent can update customer plans, apply credits, and cancel contracts. Cancelling a contract ends service and releases the customer's phone number, which cannot always be recover
- A bank's AI agent schedules payments and sends emails to suppliers on the bank's behalf. Its oversight records contain only the chat conversations between staff and the agent. When a supplier disputes
- Guardrails turn AI policy boundaries into enforced checks
Guardrails are safeguards that enforce permitted boundaries on AI inputs, outputs, or actions, such as blocking prohibited content or preventing an action until a reviewer approves it. Recognize them as the enforcement layer that supports the organization's mitigation policy, rather than as another name for user guidance or human judgment. Guardrails complement human oversight: unfamiliar or consequential cases still need the appropriate review and escalation path.
Trap Using a system prompt as the sole policy-enforcement control, because instructions to the model do not independently verify compliance.
4 questions test this
- A bank's AI assistant must never give personalized investment advice. The team's only control is a system prompt that tells the model not to give such advice. Testers found that rephrased requests som
- A bank's policy prohibits storing full card numbers from its AI support chat. Customers sometimes type card numbers, and the assistant has repeated them back. Auditors require a control that checks ev
- A health insurer's policy prohibits its AI assistant from giving diagnoses. A project manager proposes meeting this policy by adding a sentence to the user guide that asks members not to request diagn
- A media company configured guardrails for its AI writing assistant to enforce a policy against defamatory claims about real people. The configuration was based on the policy's general wording, and no
Establish AI Governance and Regulatory Compliance
Read full chapterUnlock with Premium — includes all practice exams and the complete study guide.
Identify Enterprise AI Risks and Direct Mitigation
Read full chapterUnlock with Premium — includes all practice exams and the complete study guide.
Business Readiness, Leadership, and AI Transformation
Assess AI business readiness and maturity
Read full chapterCheat sheet
Sharp facts the exam loves — scan these before test day.
- AI readiness spans business capabilities as well as technology
AWS CAF-aligned AI adoption guidance assesses business, people, governance, platform, security, and operations together. Business addresses strategic outcomes, people addresses skills and collaboration, and governance addresses risk and decision rules; platform, security, and operations address enabling technology, protection, and reliable service delivery. Use these perspectives to locate the capability that needs attention rather than treating technical sophistication as a complete readiness assessment.
Trap Using a cloud infrastructure inventory as the enterprise readiness score because it omits organizational and oversight capabilities.
5 questions test this
- A bank's readiness review finds strong AI policies, clear ethical guidelines, and named risk owners. However, the review also finds that customer prompts could expose sensitive data and that access to
- A company already uses the AWS Cloud Adoption Framework (AWS CAF) for its cloud program. Its AI lead wants to write a readiness framework from scratch. The chief information officer wants to reuse exi
- A telecom company has skilled engineers, strong security controls, and a modern AI platform, and its AI demonstrations impress visitors. However, no one can show how any of this work supports corporat
- An insurer's production generative AI assistant gives accurate answers and has approved policies. However, nobody tracks its performance, no feedback process exists, and updates are released without l
- A manufacturing company calculates an enterprise AI readiness score from an inventory of its cloud infrastructure and data science tools. The score is high. However, the business has not agreed on tar
- Readiness must be assessed against a specific business use case
An AI readiness assessment should connect the intended business outcome to the data, organizational capabilities, and operational requirements needed to achieve it. A company with only broad ideas can use the assessment to refine them into measurable, feasible use cases. Being ready for a bounded internal assistant does not establish readiness for a consequential customer-facing decision process with different dependencies.
Trap Using a vendor demonstration's success as readiness evidence because it does not establish the company's own prerequisites.
5 questions test this
- A department store group's leaders say only that they want AI to improve the customer experience. They have no specific use case or measures, and each department reads the goal differently. An executi
- A bank successfully deployed an internal AI assistant that helps staff search policy documents. Leaders now want to use AI to decide customer loan applications. They argue that the earlier success sho
- A vendor demonstrates an AI assistant that answers policy questions flawlessly by using the vendor's sample documents. An insurer's executives conclude that the company is ready to deploy it to custom
- An energy company wants to assess whether it is ready for an AI assistant for field technicians. The assessment team plans to start by listing every dataset and system in the company. The sponsor fear
- A freight company finished a readiness assessment for an AI shipment-exception assistant. Two of six carrier data feeds are usable now, and the target workflow is clear. Leaders want early value witho
- Organizational readiness includes willingness and ability to change
An organizational readiness assessment examines people's desire and ability to adopt new ways of working, alongside cultural and organizational barriers. Surveys can establish a readiness baseline, while stakeholder interviews expose differences in leadership understanding and commitment. Compare responses across affected groups so enthusiasm at headquarters does not conceal a workforce readiness gap elsewhere.
Trap Using training attendance as evidence of cultural acceptance because participation does not establish willingness to change.
5 questions test this
- A hospital's organization readiness survey is complete. Scores are low for communications and moderate for leadership. The project manager plans to file the results and carry on. The chief nursing off
- A manufacturer's readiness survey gives a high average score for leadership support. However, plant managers describe mixed messages, and two executives appear to disagree about the purpose of the AI
- A bank's employees are skilled and well informed about an AI initiative. Yet few will try the new tools, because teams say errors are punished and nobody wants to be blamed for a failed experiment. Th
- An insurer reports that 96% of its claims staff have attended AI awareness training, so leaders declare the organization culturally ready. Supervisors report that many staff still avoid the new assist
- A retailer's readiness survey shows strong enthusiasm for AI at headquarters, and the overall score is high. Store managers make up a third of the respondents, and their comments mention distrust of t
- Awareness and structured experimentation represent different maturity levels
The AWS generative AI maturity model calls foundational awareness and use-case exploration Envision, and active, structured proofs of concept and pilots Experiment. Experiment includes assigned cross-functional teams, initial governance, and internal capability development. Classify an enterprise by demonstrated activities and supporting practices, not by executives' intentions or the availability of an AI tool.
Trap Using the number of purchased AI subscriptions to assign maturity because purchases do not establish structured adoption practices.
7 questions test this
- A logistics company bought 4,000 generative AI assistant subscriptions last quarter. Employees attended awareness workshops, and leaders listed possible use cases. No team has been assigned to AI work
- A manufacturer has reached the Envision level. Leaders understand generative AI basics, but no use case is linked to a business objective, and cultural readiness has not been examined. The chief infor
- A healthcare provider is at the Experiment level with two proofs of concept underway. Its leaders want to know whether the work is creating value, and they want employees to gain skills for later stag
- A retail company's chief executive announced that the company will be AI-first within two years. Managers must now complete a maturity self-assessment. Several managers plan to select a higher level b
- A bank assigned a cross-functional team of IT, business, and risk staff to two generative AI pilots. The team has a structured training plan, selected models, and an initial governance framework. Neit
- A consulting client rates itself at the Experiment level of the AWS generative AI maturity model. Leaders cite enthusiasm across departments and a vendor demonstration last month. The consultant must
- An insurance company runs three generative AI pilots. Volunteers work on them in their spare time, and each pilot sets its own rules for data use and risk review. The company believes it has reached t
- A few supported production applications do not yet constitute enterprise-scale maturity
In the AWS generative AI maturity model, Launch means a select set of production applications delivers measurable outcomes with governance, monitoring, and support. Scale adds widespread departmental adoption, reusable capabilities, self-service access, and an implemented enterprise operating model. A reliable standalone deployment demonstrates a different level of maturity from a repeatable capability that many business units can use.
Trap Using the size of a single deployment's audience as the maturity measure because it does not establish enterprise-wide repeatability.
6 questions test this
- Six departments at a retailer each built and run their own generative AI application in production. All six are monitored and deliver measured results. Each department wrote its own prompts, controls,
- A global insurer has generative AI in production across claims, underwriting, marketing, and finance. Teams draw on a shared library of reusable components through a self-service portal. An operating
- A financial services firm has production generative AI applications in several departments. Every new application waits in a queue for a manual review by one central board. Reviews repeat the same che
- A travel company moved a generative AI itinerary assistant from a successful proof of concept into production for customers. No one monitors response quality, no support team exists, and no responsibl
- A utilities company runs two generative AI applications in production. Both are governed, monitored, and supported, and both report measurable cost savings. No other department uses them, and no reusa
- A media company has three production generative AI applications, each with a first-version RACI matrix agreed for its own team. As more departments ask for AI, accountability questions keep reaching e
- AI maturity can differ across capabilities and business units
The AWS generative AI maturity model treats maturity as a continuum within each adoption aspect, rather than requiring every team and capability to share one label. Assess uneven strengths and gaps separately, and select the appropriate progression in the context of business goals and readiness. A more advanced level is not automatically the right immediate target for every use case.
Trap Using the most advanced department's maturity rating for investment planning because it hides weaker capabilities elsewhere.
6 questions test this
- A logistics company has a strong generative AI platform with automated pipelines and well-run environments. The platform team rates the company at Launch. An assessor finds that no risk assessment pro
- A software company's Business and Platform pillars are strong, with production applications and measured value. Its Governance pillar is far behind, with only informal guidance for responsible AI use.
- A regional hospital group uses a generative AI assistant in its back-office scheduling team. The assistant meets the team's goals, and the team has no plans for more AI work. A vendor says that every
- The chief executive of a consumer goods company directs every business unit to reach the Scale level by year end. Sales already runs production applications. Manufacturing is only exploring use cases,
- A strategy consultant tells a retailer that the AWS generative AI maturity model forbids any team from working at Launch until all teams have completed Experiment. The retailer's e-commerce team is re
- A bank's digital banking unit reached production with governed generative AI and reusable patterns. The lending and compliance units have not begun pilots. The chief financial officer proposes to budg
- An AI workforce gap can involve business expertise as well as technical skills
AI transformation needs both technical specialists and people who connect AI opportunities to products, processes, and customer value. Compare the roles and skills required by the strategy with the current workforce, then distinguish capabilities that can be developed internally from specialist gaps that require hiring or partners. Existing employees with deep business knowledge can be valuable candidates for transition into AI-related roles.
Trap Using a larger data science hiring target to solve weak business ownership because technical headcount does not supply product or domain accountability.
6 questions test this
- An insurer needs AI product owners who understand claims rules, customer needs, and daily operations. External candidates for these roles are scarce and often lack insurance knowledge. Several senior
- A pharmaceutical company is forming a scientifically ambitious program to apply AI to drug discovery. It plans to hire ten PhD researchers with years of experience. The research head wants an all-scie
- A telecommunications company employs 25 data scientists and ML engineers, and its models pass every technical review. Even so, no one manages the AI roadmap or identifies which customer needs the mode
- A bank's AI readiness report says only that the organization lacks AI talent. The human resources director cannot build a plan from this statement. The report must now support next year's budget, and
- A logistics company's AI strategy covers route planning, warehouse forecasting, and customer service assistants. The human resources team proposes buying AI courses for all 6,000 employees next quarte
- A manufacturer wants AI to support plant maintenance and supplier negotiations. Its IT director assessed AI skills in the IT department and reports no serious gap. Plant managers and procurement leads
- Gap analysis must distinguish what changes from what must continue
Compare the current and desired future states of AI-affected processes, roles, policies, skills, and technology. Record both the capabilities that must change and the responsibilities or service obligations that remain necessary. This establishes the actual transformation gap and identifies affected stakeholder groups, including those outside the immediate project team.
Trap Using the new application's feature backlog as the complete gap assessment because features do not capture unchanged obligations or organizational impacts.
5 questions test this
- A change impact assessment for an AI underwriting tool shows that underwriters need new review skills, that team leaders need new performance metrics, and that finance needs budget for training. The p
- A retailer's project team is mapping the effects of an AI demand-forecasting system. Its analysis covers only planners on the project. The system will also change how store managers order stock, how f
- A claims department plans to add an AI assistant. The project team offers its feature backlog as the complete gap assessment. The department head notes that the backlog does not mention adjuster roles
- A utility is planning to route field service work through an AI scheduling system. The gap analysis shows clear differences between the current and future process. Experienced crews have raised concer
- A bank will use AI to draft responses to customer complaints. The gap analysis lists every new task and system. An auditor points out that it says nothing about the regulatory response deadlines and h
- Missing approval authority is a governance capability gap
AI governance readiness requires defined authority, risk review expectations, and accountability for decisions. When teams cannot identify who may approve a use case or how ethical and regulatory concerns are reviewed, the missing capability is governance even if delivery activities are well coordinated. Assess whether the organization can make and evidence accountable decisions, separately from whether it can execute project tasks.
Trap Using a shared project tracker as the remedy for missing go-live authority because visibility does not confer decision rights.
6 questions test this
- A product manager at a large retailer has an approved budget for an AI pricing use case. Colleagues disagree about whether any committee must sign off, and nobody can say what documents approvers want
- An airline reviewed why its AI projects stall before release. Its data is reliable, its staff are trained, and its tools and schedules are sound. However, no one can say who accepts AI risk, who revie
- A bank is preparing an AI onboarding assistant for customers in six countries. The delivery plan is complete, but nobody has confirmed where customer data will be processed or which national and indus
- A bank's AI teams coordinate well and meet their delivery dates. Four finished applications have still waited for months because nobody knows who may authorize go-live. The program office proposes a s
- A consulting team is scoring a hospital group's AI governance readiness. The hospital's AI projects finish on time, and weekly reports are excellent. The team proposes a high governance score from tho
- A media company is moving three generative AI tools toward production. Project teams handle delivery well, yet each team applies its own view of acceptable content, and reviewers have no agreed standa
- AI development priorities should combine strategic impact and feasibility
When choosing which capabilities and use cases to develop first, compare expected business impact, strategic alignment, and feasibility at the current maturity level. A high-value opportunity with unmet prerequisites may need foundational work before a pilot is justified. A feasible project earns priority when it advances a meaningful business objective, not merely because it is easy to demonstrate.
Trap Using technical novelty to rank investments because novelty does not establish business impact or organizational feasibility.
6 questions test this
- A regional credit union has only finished its awareness work and has no AI project experience. An automated credit decision tool carries high regulatory risk and needs controls the credit union lacks.
- A manufacturer's marketing team scored its own AI ideas for business impact, and its IT team separately scored them for technical feasibility. The two lists conflict, and security staff saw neither. T
- A hospital group's most valuable AI idea is a clinical summary assistant. Its patient records are fragmented, and access rules are undefined, so a pilot cannot run responsibly yet. A scheduling assist
- A consumer bank's innovation group ranks next year's AI ideas by how recent each model technique is. The chief financial officer asks how that ranking reflects the bank's goals of lower servicing cost
- A diversified manufacturer's highest-scoring AI idea would automate quality reports in a product line that the board has agreed to sell next year. A second idea scores slightly lower on expected savin
- A consumer goods company's steering group ranks a demand-sensing idea first on strategic value. A planner then reports that the sales data it needs sits with two distributors and may be incomplete. Th
- Blocking capability gaps determine investment sequence
When a strategic AI initiative depends on unavailable data, unresolved ownership, or missing workforce capabilities, prioritize the prerequisite that prevents responsible progress. Coordinate investments across people, process, technology, and governance so one workstream does not advance beyond its dependencies. Other improvements can proceed in parallel when they do not depend on the blocked capability.
Trap Using additional compute funding to accelerate an initiative awaiting data-use approval because capacity does not resolve that dependency.
6 questions test this
- A shipping company's route optimization initiative is blocked while legal reviews a data-sharing agreement with carriers. The review will take three months. A planner literacy course and a dispatch pr
- A bank's AI platform team is two quarters ahead of plan, while its risk review process and frontline training have not started. The platform lead wants to continue at full speed. The next release cann
- An insurer's fraud-detection initiative is waiting for approval to use customer claims data. The project lead asks for extra compute funding so that work will move faster once the approval arrives. Co
- A retailer has approved data, funding, and a finished AI pricing tool for store managers. A readiness check shows that store managers lack the skills to interpret or challenge its recommendations, and
- A bank's AI assistant program has four workstreams: platform, data cleanup, staff skills, and approval policy. Each workstream lead plans alone, and the platform team is ahead while the others lag. Th
- An insurer's customer service and marketing units both claim control of customer interaction data. As a result, no one can authorize its use for an AI retention initiative that the board has prioritiz
- Shared capability needs can justify investments that benefit several initiatives
When several strategic use cases need the same data foundation, skills, or governed access capability, identify the overlap before funding separate solutions. A shared investment can remove repeated barriers and establish a reusable foundation for later initiatives. Keep its justification tied to the business initiatives it enables rather than building a platform without identified consumers.
Trap Using separate departmental funding requests to treat a common prerequisite as unrelated local work because this duplicates the enabling investment.
6 questions test this
- An energy company's four strategic AI use cases are under review. Three of them need governed access to the same asset and maintenance records. The fourth needs a specialized image model that no other
- A utility's platform team asks for a large budget to build an enterprise AI data platform. Its proposal details storage and performance but names no initiative that will use the platform. The chief fi
- Five departments at an insurer each plan to buy their own prompt engineering course for staff working on strategic AI use cases. The courses cover nearly identical skills, and none connects to the ins
- A financial services company has four production generative AI applications. Each department now wants a fifth and sixth application and requests separate funds for prompt management, evaluation, and
- Claims, underwriting, and customer service at an insurer each request funds to clean and connect the same customer records. Each request describes the work as local to one AI use case. All three use c
- A retailer built a shared customer data foundation for three AI initiatives. A central team of five handles every access request, and its headcount is frozen. New strategic initiatives now wait for we
- Partners can bridge skill gaps while internal leaders retain the business strategy
When an AI initiative needs expertise that the organization cannot supply immediately, use external specialists to accelerate bounded work and transfer knowledge to employees. Keep strategy leadership closely connected to the business and develop the internal capabilities needed to sustain the initiative. Partner support can complement hiring and retraining rather than replacing all internal ownership.
Trap Using a permanent external delivery contract as the workforce development plan because delivery alone does not transfer organizational capability.
6 questions test this
- A telecom company wants to test a generative AI assistant for field engineers this quarter. No employee has built one, and hiring will take at least nine months. The company wants its own engineers to
- A bank used a partner to build its first AI assistant. The contract ends in six months, and no bank employee can maintain or improve the solution. The partner proposes renewing on the same terms. The
- A retailer's board approved an AI customer service program, but the retailer has few AI specialists. The proposed workforce plan is a ten-year contract in which one consulting firm delivers and operat
- A bank's partner engagement for an AI fraud-alert tool ends next quarter. The partner's report counts features delivered and defects closed. The bank's stated reason for the engagement was to help sta
- A logistics company's AI agenda has stalled for a year while it searches for a lead AI engineer. No initiatives are running, and sponsors are losing interest. Recruiters expect no hire this year. The
- A mid-sized insurer plans to outsource its whole AI agenda to a consulting firm, including choosing which business problems to address. The firm's model engineers are strong, and the insurer lacks suc
- A progression pathway turns assessed gaps into accountable development work
An AI progression roadmap connects the current capability state to the desired business outcomes through prioritized improvements in talent, tools, data, and governance. Give the work owners, resources, dependencies, and measurable milestones so progress can be evaluated. A maturity label becomes useful when it guides specific development commitments rather than remaining an assessment result.
Trap Using a target maturity score as the entire transformation plan because a score does not assign or resource the improvements.
6 questions test this
- A logistics company's AI team built its progression roadmap from a vendor's release calendar, and each quarter adds that vendor's newest features. Last year's assessment gaps remain open. Business lea
- After a maturity assessment, a retailer's chief strategy officer sets one goal: reach the Launch level on every pillar within 18 months. The goal sits on a single slide. Department heads ask what they
- An airline's AI roadmap lists only purchases: a model platform, an agent builder, and a search tool. A recent assessment found gaps in staff skills, data quality, and risk review, and the airline's go
- A drug manufacturer approved an AI roadmap with owners, budgets, and milestones. Since then, delivery teams have reported progress only among themselves. Two milestones slipped, and the business spons
- A retailer's AI roadmap lists twelve well-chosen improvements in talent, tools, data, and governance. Two quarters later, little has happened. Each item was assigned to a department rather than a pers
- A utility's AI roadmap commits to better data quality and prompt skills this year. The steering committee tracks each workstream as green, yellow, or red based on its leader's opinion. The committee w
- Early-stage investment should fund bounded learning before enterprise commitments
For an organization moving beyond awareness, secure budget and people for manageable proofs of concept with explicit success criteria. This develops practical capabilities and tests value before committing to a broad production program. Production-grade and enterprise-wide capabilities become appropriate investments as evidence and maturity support the next step.
Trap Using an enterprise-wide license commitment to secure early stakeholder confidence because the investment precedes validation of the use cases.
6 questions test this
- An energy company approved money for a generative AI proof of concept that summarizes maintenance reports. The sponsor plans to judge success by how impressed executives feel at the final demonstratio
- A transport company secured a modest budget for its first generative AI work. Department heads submit nine ideas, and each asks for a full rollout to its own staff. The chief executive wants to show v
- A transport company finished a generative AI proof of concept for route notes. Users liked it, and the sponsor wants a large production budget at once. Finance will not fund production on opinion alon
- A mid-sized insurer has finished its awareness work and chosen two use cases to test. The infrastructure team proposes spending the whole AI budget this year on production-grade pipelines, tiered supp
- A department store group wants to start generative AI proofs of concept this year. The human resources director proposes hiring a 50-person AI department before any use case has been tested. Finance f
- A professional services firm is moving beyond AI awareness. A vendor offers a discount on a three-year, firm-wide license if the firm signs this quarter. The partners believe that the commitment will
- Readiness findings should change the roadmap when organizational conditions change
Reassess organizational readiness at important milestones and update the progression plan as scope, stakeholder participation, and capability needs evolve. Findings may justify additional sponsorship, a revised schedule, or different communications and training investments. Sharing assessment results with participants also makes the basis for changes visible instead of leaving the roadmap disconnected from current readiness.
Trap Using the original readiness assessment as a permanent approval because it does not reflect later changes in scope or staffing.
6 questions test this
- A hospital group's follow-up readiness survey shows that nurses believe they lack the skills to work with a new AI triage assistant. Leadership support and communication scores remain high. Go-live is
- An insurer's follow-up readiness assessment shows that scores on culture and willingness to change have fallen since a recent reorganization. Rollout of an AI underwriting assistant is three weeks awa
- A telecommunications company built its AI roadmap on a business case that assumes 40% of agents will adopt a new assistant within six months. A readiness assessment at the pilot milestone shows that m
- A retailer plans a year-long AI transformation. The change lead proposes one readiness survey at the start and no others, to avoid survey fatigue among store staff. The transformation director wants t
- An insurer completed an organization readiness assessment for an AI claims assistant in one region. A year later, leaders expand the project to five countries and add underwriting. The original sponso
- A bank's readiness survey shows strong staff interest in AI but low scores on whether senior leaders actively support the transformation. The current roadmap assumes that business units will release s
Establish data and infrastructure foundations for AI
Read full chapterCheat sheet
Sharp facts the exam loves — scan these before test day.
- Data quality dimensions identify different kinds of readiness defects
Data accuracy asks whether values are correct, completeness asks whether required information is present, and consistency asks whether representations agree across records and systems. Timeliness asks whether data is current enough for its intended use. Evaluate the dimensions that matter to the business process so a dataset's strength in one dimension does not conceal a different defect.
Trap Using a high record-count target as the data acceptance criterion because volume does not establish correctness or completeness.
4 questions test this
- A property insurer wants to train a risk model on building records. Every required field is filled in, and the fields use the same formats across systems. A sample inspection finds that clerks often m
- A grocery chain plans an AI tool that suggests same-day restocking. Its shelf data scores well in a quality review for accuracy and completeness. The data reaches the planning system three days after
- A transit agency is buying 20 million rider feedback records for an AI service analysis tool. The vendor proposes that delivery of the full volume should count as acceptance. Agency leaders do not wan
- A retailer's dashboard gives its product dataset an overall quality score of 91 out of 100, which averages six facets. The team building an AI product search notes that shoppers filter on product attr
- Reliable AI inputs require consistent collection and source-level quality controls
When sites collect the same business information differently, standardize collection practices, definitions, and quality checks before expanding an AI initiative. Assign responsibility for recurring defects and improve the collection process at their source. Consistent controls across locations make reliable inputs an ongoing capability instead of a one-time cleanup.
Trap Using centralized output correction to compensate for inconsistent store inputs because it leaves the recurring source defect in place.
4 questions test this
- A hotel group's data team spent a month cleaning guest data before an AI launch, and the first results were strong. Six months later, the same formatting and accuracy defects are back. The team propos
- A bank's regional offices each collect small-business loan data. Each region knows its customers well, but they use different definitions, and an AI credit-review tool gets uneven inputs. The bank wan
- A distributor's AI demand forecasts suffer because the same warehouses submit late and malformed files every month. The analytics team fixes the files itself. No one at the warehouses knows about the
- A restaurant chain's AI tool summarizes guest complaints. Stores record the same complaint in different ways, so a central team corrects the summaries each week. The correction work grows every month.
- Data must represent the people and situations the AI initiative will serve
Data readiness includes relevance to the use case and representation of the intended users, operating conditions, and business situations. A large, internally consistent dataset can still omit important populations or contexts. Evaluate coverage of the target setting and potential representational bias before treating existing records as an adequate foundation.
Trap Using clean historical records as the default representative dataset because cleanliness does not establish relevance to the target population.
4 questions test this
- A national bank tested an AI chat assistant on logs from its large city branches. The logs are clean and well labeled, and the test scores are strong. The bank will offer the assistant to all customer
- A telecom company wants an AI assistant for business customers. The only available conversation history comes from its consumer support team, and it is large and well organized. Business customers ask
- A hospital network has 15 years of records from its teaching hospitals. It wants an AI tool that flags readmission risk at its new community clinics, which serve older patients and more rural resident
- A state benefits agency holds ten years of spotless application records. Planners want to use them to train a tool that helps gig workers and recent migrants, because the records passed each quality t
- Required data freshness depends on how quickly the business information changes
Set data update expectations according to the use case's need for current information. A dynamic knowledge source or time-sensitive decision process needs a different refresh approach from a stable reference collection. Include update frequency and quality monitoring in readiness requirements so data that was suitable at the pilot's start remains usable later.
Trap Using the document archive's publication schedule for a live policy assistant because that schedule may not meet its currency requirement.
4 questions test this
- An airline's AI assistant helps gate agents rebook stranded passengers. It answers from seat availability that is loaded overnight, so it often offers seats that are already gone. Availability changes
- A company's HR assistant answers from a history handbook that rarely changes and from benefits rules that change every month. All sources now refresh each quarter, so benefits answers are sometimes ou
- A pharmacy chain's AI assistant for store staff performed well at launch, using the current drug coverage lists. The readiness checklist looked only at data quality on launch day. Coverage lists chang
- A manufacturer's AI assistant answers questions from engineering standards that change about once a year. The vendor offers continuous real-time synchronization at a premium, and the product manager a
- Uniqueness and validity reveal defects that correct-looking values can conceal
Uniqueness checks for unwanted duplicate records of the same business entity or event; duplicates can inflate counts and give repeated evidence undue weight even when their individual values are correct. Validity asks whether values meet the required formats and business rules, such as an accepted identifier format or a permitted category. Include duplicate rate and the share of values passing validity rules in the data-readiness scorecard, with acceptance criteria tied to the intended business use. Passing a format rule does not establish that a value is factually accurate, and an accurate value does not establish that its record is unique.
Trap Using a high required-field completion rate as sufficient evidence of readiness when duplicate customers and invalid identifiers remain.
4 questions test this
- A car dealer group is preparing vehicle records for an AI pricing tool. Reviewers find that many vehicle identifiers have the wrong number of characters. Some status entries use words that are not on
- A manufacturer's supplier list passed a field audit, and each address and tax number matches the real company. An AI spend analysis still shows more suppliers than the firm actually uses, because some
- A telecom firm will use customer records to personalize renewal offers with AI. Its data vendor reports 99 percent field completion. In a pilot, some customers received two offers, and some plan codes
- A lender's acceptance report for a customer dataset shows that 98 percent of required fields are filled in, so the data passes. Reviewers then find the same customers listed several times and account
- Fragmented enterprise data requires a unified access and governance strategy
When AI must use information across disconnected systems, establish a unified data strategy that connects and governs the required sources. Unified access can work across existing locations; it does not require every asset to reside in one physical repository. Resolve fragmentation in access, formats, and governance as an enterprise foundation rather than leaving each initiative to negotiate an isolated solution.
Trap Using additional processing capacity to expand a single-database pilot because capacity does not connect disconnected information sources.
5 questions test this
- A telecom company runs 25 customer data systems. Its first AI initiative, an agent-assist tool, needs records from four of them. Leaders want the tool live this year and want the next three initiative
- A manufacturer has three AI initiatives that each need supplier, quality, and shipment data. Each project team negotiates its own access with the same system owners and builds its own connections. App
- A logistics company piloted an AI document assistant against one contract database and reached 94% accuracy. The steering committee wants to approve a company-wide rollout across 11 business units, wh
- A retailer's AI merchandising assistant performed well with the sales database alone. After the pilot expanded, it now gives incomplete recommendations because inventory, promotion, and supplier infor
- A global insurer wants an AI claims assistant to use records held in four regional systems. Local regulations require some records to remain in their home regions. Leaders want the assistant live with
- Data democratization means easier authorized access, not unrestricted access
A data sharing framework should make relevant data discoverable and accessible to appropriate users while preserving privacy and use restrictions. Define clear access-request workflows and role-appropriate permissions so routine legitimate use does not depend on ad hoc negotiation. Excessive control can recreate silos, while excessive access exposes information beyond its permitted audience.
Trap Using an organization-wide read permission to accelerate collaboration because broad availability does not establish authorization for every user.
4 questions test this
- A retailer wants to speed up AI experiments. The operations director proposes giving every employee read permission on all company datasets, including payroll and customer identity records. Analysts n
- A logistics firm's AI teams ask individual managers for data, and each manager applies a different test. Similar requests get different answers, and legitimate work waits for weeks. Leaders want routi
- A pharmaceutical company requires a vice president's signature for every data request, including requests for low-risk operational reports. Project teams now keep private spreadsheet extracts that fal
- A media company let every employee copy any dataset into a personal workspace to speed up AI work. Six months later, several copies have gone stale, and some subscriber records sit in workspaces with
Shared business glossaries, data dictionaries, and reference definitions give teams a consistent interpretation of enterprise data. Reconcile meanings such as customer status, product identifiers, and reporting periods before using information together. Technical connectivity solves access, whereas shared semantics make the combined information interpretable and comparable.
Trap Using a common file format as proof of interoperability because matching formats can still carry conflicting business meanings.
5 questions test this
- Sales and service teams at an insurer use the term active policy differently, and an AI retention model needs one meaning. Data engineers can change either field. The service director and sales direct
- A global manufacturer's AI analysts keep asking which definition of on-time delivery is correct, because each plant uses its own. The company has now agreed on one definition. It wants analysts to see
- A distributor bought two companies and connected all three order systems. An AI assistant now reports each customer's total orders, but the same buyer appears under different account numbers in each s
- A bank's AI team joined card and loan data, and every job ran without errors. A review then found that one division reports revenue by booking date and the other by settlement date. The team proposes
- A hospital network combines discharge data from two regions for an AI readmission model. Both regions export the same file format with identical column names. Results differ widely, because one region
- An AI data strategy must account for useful unstructured information
Enterprise AI can need documents, messages, images, or other unstructured material as well as structured business records. Include the relevant formats in data discovery, classification, quality assessment, and authorized access planning. Plan how users and AI applications will find and understand this information's context while preserving appropriate controls.
Trap Using the existing reporting warehouse inventory as the complete data assessment because important business knowledge may reside outside tabular systems.
5 questions test this
- A construction firm is rating its readiness for an AI project assistant. The data team's assessment covers only the finance warehouse, and it reports high readiness. Project staff say that their knowl
- A law firm wants an AI assistant to search 80,000 client contracts and emails. Some items carry confidentiality terms, but no one has recorded which ones. Partners want the assistant available to asso
- A property insurer wants AI to help assess claims. Adjusters rely on policy records, phone call recordings, and damage photos. The data plan covers only text and tables, and leaders assume that other
- A utility's HR assistant will answer employee policy questions from 12,000 documents on an intranet. A sample shows that different departments have posted conflicting versions of the same policy, with
- A media company plans AI tools that search its video, audio, and document archive, which grows by terabytes each week. Its current foundation was sized for nightly reporting on structured tables. Busi
- Data silos restrict usable business context and create reconciliation work
Data silos are isolated systems or departmental collections whose information is difficult for other authorized users or applications to access or share. They limit the business context available to AI and can create duplicate or inconsistent records that require manual transfer and reconciliation. Diagnose whether the isolation comes from disconnected technology or from organizational barriers such as ownership disputes and unwillingness to share, because technical connectivity alone does not resolve the latter.
Trap Funding model retraining to compensate for business records that remain inaccessible to the AI application.
5 questions test this
- A bank's AI relationship assistant gives weak recommendations because account history sits in a departmental system that the assistant cannot access. A vendor proposes to fund model re-training to imp
- A new chief data officer suspects that data silos are slowing AI work but cannot say where they are. Business units describe different problems, and no overview of systems exists. The officer wants an
- A bank replaced its departmental data stores with one unified platform for an AI lending project. Six months later, the platform works as designed, yet most departments still keep key files on their o
- A manufacturer's sales and service systems already exchange data through modern APIs, yet its AI warranty model still lacks service history. The service vice president refuses to release the records,
- A manufacturer's quality data sits in a 25-year-old system with no modern interfaces, so staff export it by hand for an AI analysis tool. The exports are slow and error-prone. Leaders want a dependabl
- Data strategy should work backward from business initiatives
A data strategy should identify which domains, sources, quality levels, and access capabilities are needed for targeted business initiatives. Translate the business requirements into a data roadmap and measures of whether the required information is fit for its intended purpose. This makes the data foundation an enabler of business value rather than a collection program seeking a future purpose.
Trap Using total enterprise data collected as the data strategy's success measure because collection volume does not establish support for business priorities.
4 questions test this
- A manufacturer's data office plans to spend a year listing every dataset in the company before it writes a data strategy. The chief executive has funded three AI initiatives for this year. The executi
- A logistics firm has funded an AI route planning initiative, and the data team is drafting the strategy section for it. The first draft lists the firm's storage platforms and its total data volume. Th
- An insurer's leaders have agreed on four priority AI initiatives and the data that each one needs. Six months later, little has changed. Teams disagree on who acts first, when work should finish, and
- A retailer's data strategy reports success as the number of terabytes collected each quarter, and the figure rises every year. Yet AI teams say that they still lack the data that their priority projec
- Data ownership and stewardship provide different forms of accountability
Data owners define policies for their data, including access conditions and key business definitions, while data stewards apply governance in day-to-day work, identify issues, and escalate them. Technical teams provide the supporting mechanisms, but business responsibilities must still be explicit. Route policy decisions to the responsible owner and recurring quality or governance issues through the stewardship process.
Trap Using the storage administrator as the default business data owner because technical custody does not establish authority over permitted business use.
4 questions test this
- A marketing team asks to use customer service transcripts to train an AI model for campaign targeting. The current policy does not address this use. The storage team can grant access in minutes. The d
- A company is assigning ownership of its employee data ahead of an AI workforce planning project. The storage administrator manages the systems and suggests that this makes them the owner. HR leaders d
- A company's new AI program is confused about roles. Engineers are deciding who can see customer data, business staff think that IT handles all quality issues, and no one follows up on policy breaches.
- A call center's AI assistant keeps giving wrong answers because product codes in its knowledge data are often mislabeled. Support agents see the errors daily, but no one acts on them. The problem seem
When business units need local data expertise and autonomy, federated governance lets domains manage their data while a central function maintains shared policies and common capabilities. Preserve coherence across domains so information can still be linked and used responsibly. Centralized governance may fit common control needs, but local ownership need not mean independent, incompatible standards.
Trap Using separate departmental dictionaries and access rules as the definition of federation because incompatible rules undermine cross-domain use.
4 questions test this
- A consumer goods company's regional units understand local markets and data far better than its head office. All data requests now wait on one central team, and projects stall. Regulators expect consi
- A media group's brands each manage their own audience data, and their local knowledge is strong. Every brand now runs the same data quality checks with separately bought tools, so the group pays for t
- A company is moving to a federated data model. Domain teams will run their own data products, and the products must work together so that AI projects can combine them. Leaders are now deciding what th
- A retailer uses a federated data model. Its supply chain domain has built a delivery performance data product, and other domains want to combine it with their own data for an AI project. The domain wa
- A data sharing framework must define the conditions of use
Data sharing rules should specify who may use which data, for what purposes, and under what protection and compliance conditions. Assess privacy, confidentiality, consent, and applicable use restrictions alongside availability and quality; sensitive information may require protection or exclusion. Make ownership, approval, and ongoing responsibilities clear across organizational boundaries so authorized access is paired with an authorized AI purpose.
Trap Using a successful interdepartmental file transfer as evidence of an adequate sharing agreement because transport does not define permitted reuse.
4 questions test this
- A retailer's divisions want to share customer data with a central AI team. The data includes payment details that the model does not need, along with useful purchase history. The privacy office wants
- A company's HR team successfully sent an employee survey file to the analytics team, which wants to use it to train an AI retention model. The project lead says that the completed transfer shows that
- A retail chain will share sales data with an external supplier's AI team for a joint demand forecast. Both parties have agreed on the data fields. No one has stated who approves changes, who is accoun
- A sales team received a customer support dataset through a secure, approved file transfer. It now says that the data is cleared for an AI churn model. Customers gave this information to get help with
- A data catalog helps users find and understand candidate data
A data catalog inventories data assets and their metadata, including ownership, meaning, provenance, and quality information. It supports discovery and informed access requests instead of forcing each AI team to rediscover sources. Treat the catalog as part of governance: a listing alone does not supply the policies, stewardship, and quality processes that make data trustworthy.
Trap Using folder names as the enterprise data catalog because locations alone do not explain ownership, meaning, or fitness for use.
4 questions test this
- A bank's catalog holds complete technical metadata, such as tables, columns, and data types. Marketing analysts planning an AI project still cannot tell what a field means for the business or whether
- An airline's AI teams each keep private lists of useful datasets, and a list is lost when an analyst moves on. New teams rediscover the same sources and cannot tell who owns them or how reliable they
- A pharmaceutical company's catalog lists every research dataset, but AI scientists cannot tell how to obtain any of them. They email various contacts, and requests stall. The company wants scientists
- An insurer must show auditors which AI data stores contain regulated information. Its catalog already lists the stores. Compliance staff know the rules, and application teams know the data, but neithe
- Lineage and version records make AI data use traceable
Data lineage records where information originated and how it changed, while version records identify the particular dataset state used. Together they support impact assessment, investigation of quality problems, and reproduction of prior results. A dataset can be suitable for operational use without being sufficient evidence of what supported an earlier AI output.
Trap Using the source department's name as the lineage record because ownership metadata does not describe transformations or the dataset state used.
4 questions test this
- A bank's AI credit model uses a dataset that is labeled in its catalog with the source department's name. An examiner asks where the data originated and how it was changed before training. The team ca
- A food company learns that a recalled ingredient table fed several planning datasets for two months. Some AI forecasts and executive reports may have used those datasets. The quality team wants to cor
- A health insurer's AI model produced a coverage recommendation last year, and a regulator now asks for the exact data behind it. The current dataset is suitable for daily operations, but it has been u
- A logistics company's AI delivery-time model gave a disputed estimate to a major customer three months ago. The operations director says that the dataset passes every daily quality check, so it proves
- AI data needs lifecycle rules rather than indefinite accumulation
Data governance should define lifecycle policies for retaining, updating, and disposing of data in line with business needs and compliance obligations. Include AI-related data assets in those policies so old copies do not become unmanaged exceptions. Retention supports legitimate operational and evidence needs, while disposal controls data without a justified use.
5 questions test this
- A hospital network retires an AI triage pilot. The project space still holds training data with patient details, along with the evaluation results that regulators may ask about. No one owns the space
- An insurer's model risk team needs the data behind each approved claims model for seven years. Separately, data scientists hold many abandoned experiment copies with customer details. One executive wa
- A company's lifecycle policy covers its production systems. An audit finds that AI teams hold fine-tuning datasets, evaluation samples, and test copies in project spaces, and no rule applies to them.
- A bank's AI teams keep every extract from finished projects, because the data might be useful someday. Many extracts hold customer details, and no rule says when they should go. Legal wants exposure r
- A retailer applies one five-year retention rule to everything that AI teams use, from public product photos to customer chat logs. Security leaders say that the logs stay too long. Merchandising leade
- Infrastructure requirements must reflect workload demand and service expectations
Assess whether the foundation can handle the expected data volume, arrival rate, concurrent demand, and required response speed. A real-time interaction needs different responsiveness from a scheduled batch task, and a successful low-volume demonstration does not establish peak-load capacity. Ask for a scalable capacity approach that balances performance with cost rather than assuming the largest possible environment is necessary.
Trap Using average daily request volume alone to approve capacity because it can conceal short periods of high concurrent demand.
5 questions test this
- A retailer tested an AI shopping assistant with 40 employees, and responses were fast. The company plans to open it to 300,000 customers on a major sale weekend. The project sponsor says that the test
- A travel company's AI assistant sees heavy demand during holiday booking periods and light demand for most of the year. The finance team objects to paying year-round for peak capacity. The operations
- A card issuer wants an AI model to score each transaction for fraud within seconds, before approval. Transaction data currently reaches its analytics platform in a nightly file. The pilot used last mo
- A health plan will launch an AI enrollment assistant. Planners expect 50,000 requests per day on average, and a vendor recommends sizing capacity to that average. During the six-week open enrollment p
- A bank has two AI initiatives. One scores overnight batches of loan documents for review the next morning. The other helps contact center agents during live calls. A sponsor proposes one set of perfor
- An AI capability needs a workable connection to the business process
Assess how the AI solution will obtain necessary information and deliver results into the systems and workflows people already use. Compatibility, authorized data access, and operational handoffs are foundational requirements, not optional additions after a demonstration. A standalone tool can prove a capability without proving that the organization can use it effectively in its end-to-end process.
Trap Using a standalone application's demonstration score as integration acceptance because it does not establish usable connections to the business workflow.
5 questions test this
- A utility is evaluating an AI scheduling product for field crews. The demonstration impressed leaders. The utility's work order system is 20 years old and lacks the modern interfaces that the vendor's
- A sponsor's business case for an AI forecasting tool includes licenses and a pilot but no funding for connecting the tool to planning systems. The sponsor plans to address integration after the demons
- An insurer's AI tool drafts claim summaries in a separate web application. Adjusters copy each summary into the claims system by hand. Adoption has stalled, and errors have appeared during copying. Th
- A telecom company's AI assistant scored 96% in a standalone demonstration of answering billing questions. The sponsor wants to treat that score as integration acceptance and release it to agents. Agen
- A distributor wants an AI assistant to answer order status questions from customers. The demonstration used a static export of last month's orders. In production, the assistant must reflect current or
- Managed AI platforms can reduce the foundation an enterprise must build itself
When assessing foundational needs, distinguish access to managed foundation models from a broader requirement for custom machine learning capabilities. Amazon Bedrock provides managed access to foundation models for generative AI applications, while Amazon SageMaker AI supports building, training, and deploying machine learning models. Match the platform category and supporting skills to the initiative instead of assuming every AI project needs an internally built model environment.
Trap Using a custom model-development program for every generative AI use case because some initiatives need managed model access rather than new model construction.
5 questions test this
- A logistics company chose a managed foundation model service for a document assistant. The project plan had included budget for model hosting, data preparation, integration, and access governance. The
- A professional services firm wants a generative AI tool that drafts proposals from approved templates. An executive proposes hiring a team to build and train the firm's own foundation model on dedicat
- A bank's AI portfolio includes an employee assistant built on foundation models and a credit risk model that its quantitative team must build and validate itself. The CIO wants one platform mandated f
- A consultant is assessing foundational needs for a client's next AI initiative. The client assumes that every AI project needs an internal model environment. Before recommending a platform category an
- A regional retailer will launch a customer FAQ assistant on managed foundation models. The HR director has drafted a plan to hire five model-training specialists. The assistant will use existing model
- A usable AI foundation needs protection and observability as well as capacity
Foundational assessment should include the ability to protect sensitive data, govern access, record relevant activity, and observe service and AI quality. Computational capacity cannot establish these capabilities, and a managed service does not remove the need to assess them for the complete application. State the business requirements and obtain evidence from the responsible technical and risk teams without treating infrastructure configuration as the business strategist's task.
Trap Using a compute performance benchmark as the complete platform acceptance test because it says little about protection or operational visibility.
5 questions test this
- A business strategist is leading a foundation assessment for an AI claims tool. The strategist is not a security specialist and cannot judge technical controls. Leaders still expect an informed view o
- A retailer's AI assistant has met its uptime and latency targets for three months. Customer complaints about wrong answers are rising, yet operations dashboards stay green. The assessment team is revi
- A vendor offers a benchmark that shows its AI platform handling a bank's expected workload with ease. The steering group wants to use the result as the final acceptance test. The platform will handle
- An insurer reviews an AI platform that lets adjusters ask questions about claim files, which contain medical details. The review covers capacity and latency. Internal audit asks how the insurer will s
- A retailer selects a managed AI service for a customer assistant, which will read order history and loyalty data. The IT lead proposes dropping the data protection review, because the provider secures
Lead AI change and build workforce capabilities
Read full chapterUnlock with Premium — includes all practice exams and the complete study guide.
Scale AI from pilots to enterprise-wide deployments
Read full chapterUnlock with Premium — includes all practice exams and the complete study guide.