Previous version: MLA-C01. The current version is MLA-C02.
Domain 4 of 4 · Chapter 2 of 3

Infrastructure & Cost Optimization

Which cost tool answers which question

QuestionCost ExplorerAWS BudgetsTrusted AdvisorCompute Optimizer
Primary jobAnalyze and forecast spendAlert on threshold breachRecommend best-practice fixesRecommend instance rightsizing
Time orientationPast + forecast (13 mo / 18 mo)Forward-looking guardrailCurrent-state findingsCurrent-state findings
Cost optimization checkTrend and driver analysisn/a (controls, not finds)Idle / underutilized resources, RI coverageOver- or under-provisioned EC2, ASG, Lambda, EBS
Acts automatically?No (analysis only)Yes (alerts + budget actions)No (recommendations only)No (recommendations only)
Plan requirementFree in consoleFree for cost/usage budgetsCost checks need Business+/EnterpriseFree; opt-in

Decision tree

What is the goal?infra or cost decisionPay less for capacity?Inference Recommenderrightsize SM endpointCompute Optimizerrightsize EC2/EBSSpot / Savings Planmatch usage shapeWatch or alert on spend?CostExplorerAWSBudgetsTrace latency? X-Ray + Logs Insightscapacity stuck? raise service quotaSize or buyMonitor costendpointEC2buyanalyzealertelse: operate
Cheat sheet

Cheat sheet

  • Pick the instance family from the binding resource
  • Inferentia is inference-only, Trainium is training-only
  • Inference Recommender rightsizes a SageMaker endpoint by load-testing it
  • Inference Recommender Default job runs ~45 min, Advanced ~2 hours
  • Compute Optimizer rightsizes EC2, ASG, EBS, and Lambda from history
  • SageMaker AI Savings Plans cut steady usage up to 64%
  • Run interruption-tolerant ML work on EC2 Spot
  • On-Demand is the default only until the baseline is known
  • A steady customer-facing endpoint commits, it does not run on Spot
  • Cost allocation tags must be applied and activated before they attribute spend
  • Cost Explorer analyzes and forecasts, Budgets alerts on thresholds
  • AWS Budgets alerts but does not stop resources by itself
  • Trusted Advisor and Compute Optimizer recommend; they do not act
  • Trusted Advisor cost checks need a Business or Enterprise plan
  • Use X-Ray to find which hop adds latency
  • CloudWatch Logs Insights queries logs to find the cause
  • CloudTrail logs who invoked re-training; EventBridge triggers it
  • Check service quotas first when an endpoint cannot scale
  • Graviton offers up to 40% better price-performance
  • Target-track on SageMakerVariantInvocationsPerInstance with scale-out cooldown shorter than scale-in
  • CostPerInference is the metric for price-performance at throughput
  • Benchmarking an existing endpoint requires one variant, no autoscaling, and a non-prod copy

Also tested in

References

  1. AWS Inferentia
  2. AWS Trainium
  3. Amazon SageMaker Inference Recommender
  4. What is AWS Compute Optimizer?
  5. Machine Learning Savings Plans pricing
  6. Spot Instance interruptions
  7. Analyzing log data with CloudWatch Logs Insights
  8. What is AWS X-Ray?
  9. AWS CloudTrail User Guide
  10. Analyzing your costs and usage with AWS Cost Explorer
  11. Managing your costs with AWS Budgets
  12. AWS Trusted Advisor