Domain 4 of 4 · Chapter 3 of 3

Secure ML and AI Workloads and Endpoints

Unlock the complete study guide.

Bundled into the existing AWS Certified Machine Learning Engineer - Associate premium course — no separate purchase.

14-day money-back guarantee — no questions asked.

Included in this chapter:

  • AWS security topics for ML and AI workloads
  • Execution roles: the permissions a job actually uses
  • Why a request stays denied after you add an allow
  • Amazon Bedrock inference actions for streaming and nonstreaming calls
  • Credentials in foundation model applications
  • Encryption at rest and between training instances
  • Private endpoints for inference and artifact traffic
  • Controlling what job and model containers can reach
  • Guardrails filters: what each catches and misses
  • Applying a guardrail: what is evaluated, and when
  • Screening code, container images and stored data
  • Audit evidence: CloudTrail, AWS Config and invocation logs
  • Common mistakes in securing ML and AI workloads

AWS security topics for ML and AI workloads compared

AspectIdentity and access managementData protectionInfrastructure securityAmazon Bedrock GuardrailsVulnerability management and sensitive data discoveryLogging and monitoring
Question it answersWho can be authenticated and authorized to act on resources, and with which role or credential?How is data protected at rest and in transit, and which KMS key and key permissions apply?Which network paths can traffic take?Which undesirable content in user inputs or model responses is detected and filtered?Which code and container images have software vulnerabilities, and which stored data is sensitive?What happened, and who did it?
Main AWS controlsExecution roles, iam:PassRole, permissions boundaries, Bedrock API keys, Secrets ManagerCustomer managed KMS keys for ML storage volumes and job output, SSE-KMS, inter-container traffic encryptionSubnets, security groups, interface and gateway VPC endpoints, network isolationAmazon Bedrock Guardrails filters, ApplyGuardrailAmazon Inspector Code Security and CI/CD scans, ECR enhanced scanning, MacieCloudTrail management and data events, AWS Config, Bedrock model invocation logging
What it cannot doOpen a network path or screen contentReplace data permissions: reading an SSE-KMS object needs S3 read and kms:DecryptAuthorize a request; policies still decideInspect tool arguments, tool results or tool definitionsStop a harmful request at inference timePrevent the action it records
Typical wrong substituteGranting the submitting user more S3 accessEncrypting the training volume to protect traffic between instancesVPC peering in place of a bucket policy grantA word list standing in for a whole denied topicA clean push-time scan treated as permanent proofA management-events-only trail for knowledge base retrievals

Decision tree

Must you prove who did what,or what a configuration was?Logging and monitoringCloudTrail, AWS Config, invocation logsMust model input or output be filtered?Amazon Bedrock Guardrailsfilters, ApplyGuardrailMust code, images or stored data be scanned?Vulnerability managementand sensitive data discoveryMust traffic be kept off certain network paths?Infrastructure securityVPC endpoints, network isolationMust data be encrypted at rest or in transit?Data protectionKMS keys, inter-container encryptionIdentity and access managementexecution roles, iam:PassRole, API keysYesYesYesYesYesNoNoNoNoNo
Cheat sheet

Cheat sheet

  • The execution role governs SageMaker AI access to workload artifacts
  • A role needs both service trust and permissions for the work it performs
  • Passing an execution role requires permission on the submitting identity
  • An explicit deny cannot be repaired by adding another allow
  • SageMaker AI full-access policy does not grant object access to every S3 bucket
  • Encrypted S3 artifacts require both data-access and key-use authorization
  • Streaming and nonstreaming Bedrock inference use different invocation permissions
  • Customer-managed encryption keys must be configured separately for ML storage and persisted outputs
  • Runtime interface endpoints provide private client access to inference APIs
  • An S3 gateway endpoint provides private artifact access from the VPC
  • Private DNS lets standard service names resolve to interface endpoints
  • SageMaker AI network isolation blocks container outbound calls while preserving managed artifact transfer
  • An endpoint policy restricts its network path rather than every access path
  • VPC-enabled model hosting needs permissions to manage network interfaces
  • S3 policy conditions must match the request context of VPC endpoint traffic
  • Private processing jobs need scoped traffic rules that preserve required peer communication
  • Distributed-training communications need their own encryption control
  • CloudTrail records who performed an AWS API action
  • AWS Config evaluates recorded resource configurations over time
  • Bedrock prompt and response investigation requires invocation logging
  • Agent and knowledge-base runtime auditing requires the appropriate CloudTrail data events
  • Encrypted invocation-log delivery needs permission on the destination and key
  • Role-passing evidence appears in the service operation that received the role
  • A Bedrock stream-completion audit event is not another invocation
  • ECR enhanced scanning adds language-package vulnerability coverage
  • Continuous image scanning can detect vulnerabilities disclosed after image publication
  • Inspector can assess container vulnerabilities before deployment through CI/CD integration
  • Source and infrastructure code need assessment beyond container-image scanning
  • Macie locates sensitive data stored in S3 rather than filtering live model conversations
  • CodeGuru Reviewer provides source-code recommendations only through existing repository associations
  • Bedrock API keys provide bearer authentication rather than signed AWS requests
  • Short-term Bedrock keys are the production choice for bearer-token integrations
  • Long-running bearer clients need a refresh-capable token generator
  • Long-term Bedrock API keys are scoped to their creation Region
  • Bedrock key compromise requires revoking the affected credential path
  • Secrets Manager separates application secrets from deployed code
  • Guardrail filter types address different application restrictions
  • Sensitive-information policies can reject content or preserve it with redaction
  • Sensitive-information filters do not automatically sanitize every tool field
  • Contextual grounding checks compare a response with both source evidence and the query
  • Guardrails can evaluate content without invoking a foundation model
  • Synchronous streaming guardrails scan chunks before releasing them
  • A published guardrail version stabilizes the policy used in production
  • Converse system prompts need explicit marking for guardrail assessment
  • Prompt-attack filtering must distinguish untrusted input from developer instructions

Unlock with Premium — includes all practice exams and the complete study guide.

Also tested in

References

  1. https://docs.aws.amazon.com/vpc/latest/privatelink/concepts.html
  2. https://docs.aws.amazon.com/vpc/latest/privatelink/vpc-endpoints-access.html
  3. https://docs.aws.amazon.com/aws-certification/latest/machine-learning-engineer-associate-02/machine-learning-engineer-associate-02-domain4.html
  4. https://docs.aws.amazon.com/sagemaker/latest/dg/security.html
  5. https://docs.aws.amazon.com/inspector/latest/user/what-is-inspector.html
  6. https://docs.aws.amazon.com/macie/latest/user/what-is-macie.html
  7. https://docs.aws.amazon.com/bedrock/latest/userguide/guardrails.html
  8. https://docs.aws.amazon.com/sagemaker/latest/dg/security-iam.html
  9. https://docs.aws.amazon.com/sagemaker/latest/dg/sagemaker-incident-response.html
  10. https://docs.aws.amazon.com/aws-certification/latest/machine-learning-engineer-associate-02/machine-learning-engineer-associate-02-domain3.html
  11. https://docs.aws.amazon.com/sagemaker/latest/dg/train-encrypt.html
  12. https://docs.aws.amazon.com/sagemaker/latest/dg/sagemaker-roles.html
  13. https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_use_passrole.html
  14. https://docs.aws.amazon.com/aws-managed-policy/latest/reference/AmazonSageMakerFullAccess.html
  15. https://docs.aws.amazon.com/IAM/latest/UserGuide/access_policies_boundaries.html
  16. https://docs.aws.amazon.com/organizations/latest/userguide/orgs_manage_policies_scps.html
  17. https://docs.aws.amazon.com/bedrock/latest/APIReference/API_runtime_ConverseStream.html
  18. https://docs.aws.amazon.com/bedrock/latest/userguide/api-keys.html
  19. https://docs.aws.amazon.com/secretsmanager/latest/userguide/intro.html
  20. https://docs.aws.amazon.com/AmazonS3/latest/userguide/UsingKMSEncryption.html
  21. https://docs.aws.amazon.com/kms/latest/developerguide/key-policies.html
  22. https://docs.aws.amazon.com/sagemaker/latest/dg/interface-vpc-endpoint.html
  23. https://docs.aws.amazon.com/bedrock/latest/userguide/vpc-interface-endpoints.html
  24. https://docs.aws.amazon.com/vpc/latest/privatelink/vpc-endpoints-s3.html
  25. https://docs.aws.amazon.com/IAM/latest/UserGuide/reference_policies_evaluation-logic_policy-eval-denyallow.html
  26. https://docs.aws.amazon.com/sagemaker/latest/dg/process-vpc.html
  27. https://docs.aws.amazon.com/sagemaker/latest/dg/mkt-algo-model-internet-free.html
  28. https://docs.aws.amazon.com/vpc/latest/userguide/security-group-rules.html
  29. https://docs.aws.amazon.com/bedrock/latest/userguide/guardrails-prompt-attack.html
  30. https://docs.aws.amazon.com/bedrock/latest/userguide/guardrails-tiers.html
  31. https://docs.aws.amazon.com/bedrock/latest/userguide/guardrails-use-converse-api.html
  32. https://docs.aws.amazon.com/bedrock/latest/userguide/guardrails-streaming.html
  33. https://docs.aws.amazon.com/inspector/latest/user/code-security-assessments.html
  34. https://docs.aws.amazon.com/codeguru/latest/reviewer-ug/welcome.html
  35. https://docs.aws.amazon.com/codeguru/latest/reviewer-ug/codeguru-reviewer-availability-change.html
  36. https://docs.aws.amazon.com/inspector/latest/user/scanning-cicd.html
  37. https://docs.aws.amazon.com/AmazonECR/latest/userguide/image-scanning.html
  38. https://docs.aws.amazon.com/macie/latest/user/data-classification.html
  39. https://docs.aws.amazon.com/bedrock/latest/userguide/logging-using-cloudtrail.html
  40. https://docs.aws.amazon.com/config/latest/developerguide/WhatIsConfig.html
  41. https://docs.aws.amazon.com/bedrock/latest/userguide/model-invocation-logging.html
  42. https://docs.aws.amazon.com/sagemaker/latest/dg/model-monitor-data-capture.html