Domain 2 of 4 · Chapter 3 of 4

Data Lifecycle Management

Unlock the complete study guide + 1,040 practice questions across 16 full exams.

Bundled into the existing AWS Certified Data Engineer - Associate premium course — no separate purchase.

Included in this chapter:

  • The S3 storage-class cost ladder
  • Lifecycle rules: transition and expiration
  • Versioning, MFA Delete, and DynamoDB TTL
  • Moving data with Redshift COPY and UNLOAD
  • Resiliency, availability, and recognizing the question

S3 storage classes: retrieval, durability, and lifecycle minimums

Storage classFirst-byte accessAvailability ZonesMin storage durationLifecycle role
S3 StandardMilliseconds≥ 3NoneDefault landing tier for hot data
S3 Standard-IAMilliseconds≥ 330 daysTransition target for infrequently read data
S3 One Zone-IAMilliseconds130 daysCheaper IA for reproducible data only
S3 Intelligent-TieringMilliseconds (archive tiers slower)≥ 3NoneAuto-tiering when access is unpredictable
S3 Glacier Instant RetrievalMilliseconds≥ 390 daysArchive needing instant access, rarely read
S3 Glacier Flexible RetrievalMinutes to hours (restore)≥ 390 daysArchive tolerating minutes-to-hours restore
S3 Glacier Deep ArchiveHours (restore)≥ 3180 daysLowest-cost long-term/compliance archive

Decision tree

Is the access patternpredictable?NoS3 Intelligent-Tieringauto-tiers; objects under 128 KB stay frequentYesNeed retrieval inmilliseconds?YesNoHow often is it read?frequent vs infrequent vs archive-but-instantHow long can youwait for a restore?FrequentS3 Standardhot data, no minimumA few times a quarterS3 Standard-IA30-day min; One Zone-IA if reproducibleRarely, but instantGlacier Instant Retrievalms GET, 90-day minMinutes to hoursGlacier Flexible RetrievalStandard 3–5 h, Bulk 5–12 h12 h+ is fineGlacier Deep Archive12 h Std / 48 h Bulk, 180-day minAlways: drive the chosen class with an S3 Lifecycle ruleIA transition needs 30+ days stored; only Glacier classes accept a 0-day transition trigger

Cheat sheet

  • Pick the S3 storage class by retrieval speed first, then access frequency
  • Use Standard-IA for data read a few times a quarter, not hot data
  • One Zone-IA stores one AZ copy, so only use it for reproducible data
  • Glacier Instant Retrieval gives millisecond reads for rarely-accessed archives
  • Match the Glacier Flexible restore tier to how long you can wait
  • Deep Archive is the cheapest class, with 12-to-48-hour restores
  • Use Intelligent-Tiering when the access pattern is unpredictable
  • All S3 classes give 11 nines durability except One Zone-IA
  • Lifecycle transition actions move objects to cheaper classes by age
  • A lifecycle rule cannot transition to IA before 30 days
  • Lifecycle expiration actions delete objects at a set age
  • Prune noncurrent versions with their own lifecycle actions
  • S3 Versioning keeps every version and turns deletes into markers
  • MFA Delete gates permanent version deletes and suspending versioning
  • DynamoDB TTL auto-expires items on an epoch timestamp at no write cost
  • Expired DynamoDB items still appear in reads until TTL removes them
  • Use Redshift COPY for bulk parallel loads, not many INSERTs
  • Use Redshift UNLOAD to export query results to S3 as Parquet
  • Use S3 Replication for DR, low-latency reads, or data residency
  • Use AWS Backup for centralized, policy-driven recovery across services
  • S3 Lifecycle skips objects smaller than 128 KB by default
  • When a Lifecycle object is eligible for both, expiration wins over transition
  • Kinesis Data Streams retention runs 24 hours to 365 days, billed in tiers above 7 days
  • Increasing Kinesis retention never brings back already-expired records
  • Tier OpenSearch indexes hot to UltraWarm to cold with ISM policies

Unlock with Premium — includes all practice exams and the complete study guide.

References

  1. Understanding and managing Amazon S3 storage classes
  2. Understanding archive retrieval options (S3 Glacier)
  3. Amazon S3 Intelligent-Tiering storage class
  4. Transitioning objects using Amazon S3 Lifecycle
  5. Expiring objects using Amazon S3 Lifecycle
  6. Retaining multiple versions of objects with S3 Versioning
  7. Configuring MFA delete
  8. Using time to live (TTL) in DynamoDB
  9. COPY (Amazon Redshift SQL command reference)
  10. UNLOAD (Amazon Redshift SQL command reference)
  11. Replicating objects within and across Regions
  12. What is AWS Backup?