Train and customize ML and AI models
Unlock the complete study guide.
Bundled into the existing AWS Certified Machine Learning Engineer - Associate premium course — no separate purchase.
14-day money-back guarantee — no questions asked.
Included in this chapter:
- Two ways to customize: weights or requests
- Script mode and SageMaker AI training storage paths
- Epochs, steps, batch size and learning rate
- Diagnosing fit: overfitting and underfitting
- Catastrophic forgetting in FM fine-tuning
- Automatic model tuning strategies
- Configuring a tuning job
- Managed warm pools: shorter startup between matching jobs
- Managed Spot Training: lower price, with checkpoints to resume
- Distributed training: data or model parallelism
- Using multiple models
- Prompt engineering for task-specific behavior
- Inference parameters: randomness, length, stopping
- Optimizing retrieval and embeddings
Automatic model tuning strategies: Bayesian vs Random vs Grid vs Hyperband
| Criterion | Bayesian | Random | Grid | Hyperband |
|---|---|---|---|---|
| How the search proceeds | Regression over the results of completed trials picks the next values | Random draw from the ranges, independent of earlier results | Works through each unrun combination of the categorical values | Gives promising trials more epochs and stops weak ones early |
| Results that steer the search | Results of completed trials | None | None | Intermediate and final results of trials |
| Concurrency guidance | Limited: its sequential nature cannot massively scale | Highest: trials are independent | No specific guidance; chosen for coverage and reproducibility | Scales to many parallel trials |
| Range types | Categorical, integer, continuous | Categorical, integer, continuous | Categorical only | Categorical, integer, continuous |
| Number of trials | Capped by the total-trial limit (MaxNumberOfTrainingJobs) | Capped by the total-trial limit (MaxNumberOfTrainingJobs) | Equals the number of distinct combinations | Capped by the total-trial limit (MaxNumberOfTrainingJobs) |
| Fits when | Each trial is costly and should build on earlier results | Trials must be independent or run at high concurrency | A small enumerated space must be covered completely or reproduced | Iterative training reports a metric per epoch or resource level |
Decision tree
Cheat sheet
Unlock with Premium — includes all practice exams and the complete study guide.
References
- https://docs.aws.amazon.com/machine-learning/latest/dg/model-fit-underfitting-vs-overfitting.html
- https://docs.aws.amazon.com/bedrock/latest/userguide/prompt-engineering-guidelines.html
- https://docs.aws.amazon.com/sagemaker/latest/dg/algorithms-choose.html
- https://docs.aws.amazon.com/nova/latest/userguide/nova-forge-sft.html
- https://docs.aws.amazon.com/aws-certification/latest/machine-learning-engineer-associate-02/machine-learning-engineer-associate-02-domain2.html
- https://docs.aws.amazon.com/sagemaker/latest/dg/model-checkpoints.html
- https://docs.aws.amazon.com/sagemaker/latest/dg/model-train-storage-env-var-summary.html
- https://docs.aws.amazon.com/sagemaker/latest/dg/distributed-training.html
- https://docs.aws.amazon.com/sagemaker/latest/dg/autopilot-llms-finetuning-hyperparameters.html
- https://aws.amazon.com/what-is/overfitting/
- https://docs.aws.amazon.com/sagemaker/latest/dg/automatic-model-tuning-how-it-works.html
- https://docs.aws.amazon.com/sagemaker/latest/dg/automatic-model-tuning-considerations.html
- https://docs.aws.amazon.com/sagemaker/latest/dg/automatic-model-tuning-define-metrics-variables.html
- https://docs.aws.amazon.com/sagemaker/latest/dg/automatic-model-tuning-define-ranges.html
- https://docs.aws.amazon.com/sagemaker/latest/dg/automatic-model-tuning-warm-start.html
- https://docs.aws.amazon.com/sagemaker/latest/dg/train-warm-pools.html
- https://docs.aws.amazon.com/sagemaker/latest/dg/model-managed-spot-training.html
- https://docs.aws.amazon.com/sagemaker/latest/dg/autopilot-metrics-validation.html
- https://docs.aws.amazon.com/bedrock/latest/userguide/prompt-routing.html
- https://docs.aws.amazon.com/bedrock/latest/userguide/prompt-management.html
- https://docs.aws.amazon.com/bedrock/latest/userguide/inference-parameters.html
- https://docs.aws.amazon.com/bedrock/latest/userguide/titan-embedding-models.html
- https://docs.aws.amazon.com/bedrock/latest/userguide/kb-test-config.html
- https://docs.aws.amazon.com/bedrock/latest/userguide/knowledge-base-build-graphs.html
- https://docs.aws.amazon.com/bedrock/latest/userguide/rerank.html