Domain 3 of 4 · Chapter 3 of 3

CI/CD Pipelines for ML

Unlock the complete study guide + 1,040 practice questions across 16 full exams.

Bundled into the existing AWS Certified Machine Learning Engineer - Associate premium course — no separate purchase.

Included in this chapter:

  • What CI/CD means once the artifact is a model
  • Choosing an orchestrator: Pipelines, Step Functions, MWAA
  • From registry approval to a guarded endpoint deploy
  • Standardizing with Projects and closing the retraining loop
  • Reading the exam stem: which pipeline piece is asked

Choosing the orchestration tool for an ML workflow

CriterionSageMaker PipelinesStep FunctionsAmazon MWAA (Airflow)CodePipeline
Primary roleML-native training DAGGeneral service state machineGeneral DAG schedulerSoftware release pipeline
Native ML stepsProcessing, training, tuning, model, condition, registerHas a SageMaker integration, but steps are genericOperators incl. SageMaker, but genericStages call builds/deploys, not ML steps
Infrastructure to manageServerless, noneServerless, noneManaged Airflow environment you size and pay forServerless, none
ML lineage trackingBuilt inNoNoNo
Best whenPure SageMaker ML lifecycleWorkflow spans many AWS servicesTeam already runs AirflowOrchestrating the end-to-end release

Decision tree

Orchestrating the releaseor sequencing the ML steps?The releaseCodePipelinestages call CodeBuild + deployThe ML stepsPure SageMaker ML DAG?train / tune / register onlyYesSageMaker Pipelinesserverless, lineage built inNoTeam already runs Airflow?existing DAGs / operatorsNo: spans servicesStep Functionsgeneral state machineYesAmazon MWAAmanaged Airflow, sized + paidAlways on the endpoint update: canary or linear guardrailCloudWatch alarms auto-roll-back to the blue fleet on regression

Cheat sheet

  • Treat a retrained model like code: ship it through a CI/CD pipeline
  • CodePipeline conducts, CodeBuild runs the work, CodeDeploy pushes the artifact
  • Use SageMaker Pipelines as the ML-native DAG for the model build
  • Pick Step Functions when the workflow spans many AWS services
  • Choose Amazon MWAA when the team already runs Apache Airflow
  • Model Registry approval is the gate between training and deployment
  • Let EventBridge turn a registry approval into a deploy-pipeline trigger
  • Update a live endpoint with a deployment guardrail, not a raw swap
  • Match the traffic-shifting mode to the risk: all-at-once, canary, or linear
  • A CloudWatch alarm during the baking period auto-rolls-back the deploy
  • Use a rolling deployment when a full second fleet is too much to provision
  • Stand up a standardized MLOps pipeline with a SageMaker Projects template
  • Drive automatic retraining with an EventBridge rule
  • Code repositories and pipelines are complementary, not the same thing
  • Gate model promotion on metrics with a ConditionStep, not just unit tests
  • Nest the tools: CodePipeline runs the release, SageMaker Pipelines runs the model build
  • Cache unchanged SageMaker Pipeline steps with CacheConfig and an ISO-8601 expire_after
  • EventBridge needs a role with sagemaker:StartPipelineExecution and an events.amazonaws.com trust to launch a pipeline
  • Pass event fields into a pipeline as parameters with SageMakerPipelineParameters JSONPath
  • Match commits to a branch with a CodeCommit Repository State Change event pattern
  • Notify or chain on job outcomes with SageMaker Job State Change events
  • Append .sync to a SageMaker Task ARN to make Step Functions wait for the job
  • Combine Retry and Catch in Step Functions for backoff plus fallback routing
  • A Lambda that starts a SageMaker job also needs iam:PassRole for the job's execution role

Unlock with Premium — includes all practice exams and the complete study guide.

Also tested in

References

  1. AWS CodePipeline User Guide — What is CodePipeline?
  2. AWS CodeBuild User Guide — What is CodeBuild?
  3. AWS CodeDeploy User Guide — What is CodeDeploy?
  4. Amazon SageMaker AI Developer Guide — Pipelines
  5. AWS Step Functions Developer Guide — What is Step Functions?
  6. Amazon MWAA User Guide — What is Amazon Managed Workflows for Apache Airflow?
  7. Amazon SageMaker AI Developer Guide — Update the approval status of a model
  8. Amazon SageMaker AI Developer Guide — Events that SageMaker AI sends to Amazon EventBridge
  9. Amazon SageMaker AI Developer Guide — Deployment guardrails
  10. Amazon SageMaker AI Developer Guide — Blue/green deployments
  11. Amazon SageMaker AI Developer Guide — MLOps automation with SageMaker Projects
  12. Amazon EventBridge User Guide — What is Amazon EventBridge?