Validate data quality and manage bias

• DQDL checks require supported columns rather than nested or list-valued data
• Glue Data Quality rules turn data requirements into repeatable checks
• A valid primary key must be both non-null and unique
• Referential integrity checks validate keys against a reference dataset
• A schema-valid dataset can still be too stale for its intended use
• Aggregate reconciliation can detect missing value even when row counts match
• Data Wrangler insights identify anomalous samples and dataset quality issues
• A private Ground Truth work team assigns labeling to selected workers
• The labeling task must match the prediction's required spatial detail
• Named entity annotation labels spans rather than whole documents
• Automated labeling uses human input where automatic confidence is insufficient
• Label verification assesses annotations while adjustment corrects them
• Deduplicate source examples before assigning independent dataset splits
• Split by key keeps related observations from appearing in more than one partition
• Time-dependent evaluation should preserve the direction from past to future
• Stratified splitting preserves category representation across partitions
• Shuffling changes record order without repairing missing population coverage
• Keep augmented variants of an example out of the validation and test partitions
• Pretraining bias metrics examine the dataset before predictions exist
• The Class Imbalance bias metric measures representation of facet groups
• Balanced group counts do not guarantee equal positive-label proportions
• Conditional disparity analysis can reveal patterns hidden by aggregate rates
• Multimodal distribution checks need meaningful labels and group metadata
• Data bias remediation should correct the collection or annotation process that produced the disparity
• Oversampling increases minority exposure without removing majority examples
• Undersampling reduces class dominance by discarding majority examples
• SMOTE creates synthetic minority examples by interpolating feature vectors
• Class weighting changes the training penalty without resampling records
• Image augmentation is valid only when the transformation preserves the target meaning
• Median imputation limits the influence of extreme observed values
• Categorical imputation and Fill missing are different choices
• A missing-value indicator preserves whether a value was originally absent
• Outliers should be handled according to validity and the model's intended population
• Outlier detection methods differ in their sensitivity to extreme values
• Exact-match deduplication does not detect every near-duplicate document
• Prompt-response validation must check correctness as well as file syntax
• Training content needs its own safety screening before customization
• Training integrity includes coverage of realistic requests and edge cases

Unlock with Premium — includes all practice exams and the complete study guide.