High-Performing Databases

Aurora is RDS's cloud-native cousin. MySQL- and PostgreSQL-compatible at the wire level, but the storage engine and infrastructure are entirely different. The exam tests when Aurora's features matter and when RDS suffices.

Storage architecture:

• RDS: traditional engine with EBS storage. Multi-AZ = synchronous replication to a passive standby in another AZ (storage replication via the engine's binlog).
• Aurora: distributed storage layer. Each write commits to 6 storage nodes across 3 AZs before acknowledging. Storage auto-scales 10 GB → 128 TiB. Replicas read from the shared storage (no log shipping).

HA and failover:

• RDS Multi-AZ: 60-120 sec failover. Standby NOT readable.
• RDS Multi-AZ DB cluster (newer for MySQL/PostgreSQL): 1 writer + 2 readable standbys across 3 AZs. ~35 sec failover.
• Aurora: 1 writer + up to 15 readers. Reader can auto-promote on writer failure (~1 min). All readers are read-only but accessible.

Replication lag:

• RDS Read Replicas: async; lag from seconds to minutes under heavy writes.
• Aurora Replicas: shared storage means typical < 100 ms lag (often < 10 ms).

Storage features unique to Aurora:

• Backtrack (MySQL only): rewind cluster up to 72 hours in-place (no restore). For "oops, dropped a table" scenarios.
• Cloning: instant zero-copy clones using copy-on-write storage. Use for dev/test branches off production data.
• Aurora Global Database: cross-region replication < 1 s RPO; managed promote ~1 min RTO.
• Aurora Serverless v2: ACU-based autoscaling in 0.5-ACU increments (0.5 → 128 ACUs).
• Parallel Query (MySQL): pushes WHERE/aggregation down to the storage layer for analytical queries on transactional data.
• Aurora I/O-Optimized: switch billing modes — pay more for compute, zero per-I/O. Break-even at ~25% I/O of total Aurora cost.

Storage features unique to RDS:

• Other database engines: Oracle, SQL Server, MariaDB — only RDS supports these. Aurora is MySQL/PostgreSQL only.
• Stop/start instance (up to 7 days): pause RDS or Aurora — pay storage only.
• Database-specific tuning knobs: more direct parameter-group control over engine internals.

Cost:

• RDS: hourly instance + EBS storage + I/O charges.
• Aurora Standard: hourly instance + storage + per-I/O charges.
• Aurora I/O-Optimized: ~30% more compute cost; zero per-I/O.
• Aurora Serverless v2: per-ACU-hour pricing.

Decision pattern:

• New MySQL or PostgreSQL workload → Aurora (default).
• Need Oracle / SQL Server / MariaDB → RDS.
• Variable / unpredictable load → Aurora Serverless v2.
• Need < 100 ms read replica lag → Aurora.
• Need cross-region RPO < 1 s → Aurora Global Database.
• Need backtrack / cloning → Aurora.
• Tiny workload (< 1 vCPU) where Aurora minimum ($0.10/ACU-hr) is too high → RDS t3.micro etc.
• Legacy app with specific Oracle/SQL features → RDS for those engines.

DynamoDB's performance depends entirely on partition key design. A poorly-distributed key creates hot partitions that throttle writes regardless of total provisioned capacity.

Partition basics:

• Items physically distributed across partitions by hash(partition_key).
• Each partition supports up to 3 000 RCU + 1 000 WCU (in provisioned mode).
• A single hot key consuming > 1 000 WCU throttles even if the table has 100 000 WCU provisioned.

Adaptive capacity (since 2018):

• DynamoDB auto-shifts capacity TO hot partitions FROM cold ones in the same table.
• Works at the partition level, not the key level. Still bounded by per-partition limits.
• Doesn't eliminate hot-key problems for low-cardinality partition keys.

Key cardinality matters:

• High-cardinality partition key (UUIDs, customer IDs, order IDs): writes distribute evenly. No hot partition issues.
• Low-cardinality key (e.g. "tenantType" with values "premium" and "basic"): all writes hit 2 partitions max. Throttled at 1 000 WCU each → 2 000 WCU table-wide ceiling.

Composite keys (the common pattern):

Use partition_key + sort_key. The partition key shards; the sort key orders within a partition.

• Partition key = userId → all that user's items live in one partition.
• Sort key = timestamp → query "most recent N items for this user" via Query with ScanIndexForward=false, Limit=N.

Write sharding for very hot logical keys:

If one user generates 10 000 writes/sec (above 1 000 WCU partition limit), shard the key:

• Use userId#0..N as partition key (where N is a fixed bucket count).
• Writes randomly pick a bucket; reads query all N partitions.
• Trades write hot-partition for fan-out reads.

Global Secondary Indexes (GSI):

• Index with a DIFFERENT partition key + (optional) sort key than the base table.
• Eventually consistent reads (typically sub-second lag).
• Has its own RCU/WCU provisioning (or on-demand) — separate from the base table.
• Up to 20 GSIs per table.
• Use for: query patterns that aren't supported by the base table's keys (e.g. base table is by orderId, GSI is by customerId to look up all orders for a customer).

Local Secondary Indexes (LSI):

• Index with the SAME partition key as base table but different sort key.
• Strongly consistent reads supported.
• Must be created at table-creation time (can't add later).
• Counts against the 10 GB per-partition-key item collection limit.
• Rarely the right answer on the exam — usually GSI fits better.

DynamoDB Streams (change capture):

• 24h retention.
• Stream record options: KEYS_ONLY, NEW_IMAGE, OLD_IMAGE, NEW_AND_OLD_IMAGES.
• Consumer pattern: Lambda triggered by stream → react to changes (audit, propagate to OpenSearch, invalidate cache).
• Enables DynamoDB Global Tables under the hood (multi-region multi-active).

TTL (time-to-live):

• Auto-delete items after a timestamp (specified in an item attribute).
• Deletes are best-effort (typically within 48h of TTL).
• Free deletes (don't consume WCU).
• Use for: session tables, time-limited tokens, log cleanup.

DAX (DynamoDB Accelerator) is a write-through cache in front of DynamoDB. Reads through DAX = microseconds (vs single-digit ms direct). Sits in your VPC.

Why DAX over ElastiCache for DynamoDB caching:

• DynamoDB-native API: code uses the standard DynamoDB SDK with the DAX endpoint. No application changes beyond endpoint swap.
• Write-through built-in: writes go to DAX AND DynamoDB; cache stays consistent.
• No cache-invalidation logic: DAX handles it.
• vs ElastiCache: ElastiCache requires custom application code to fetch from DynamoDB on miss + populate cache + invalidate on writes. More flexible but more code.

Architecture:

• DAX cluster = 1 primary node + 0-9 read replicas across multiple AZs.
• Primary handles writes (write-through to DynamoDB); replicas serve reads.
• Up to 10 nodes per cluster. Sizing example: 1 primary + 2 replicas across 3 AZs.

Cache modes:

• Item cache: caches individual items by primary key. TTL configurable (default 5 min). LRU eviction.
• Query cache: caches Query and Scan results. TTL configurable (default 5 min).

Eventual consistency by default — DAX reads return potentially stale data (from cache). For strongly-consistent reads, configure the request to bypass DAX and go directly to DynamoDB.

Setup steps:

1. Create DAX cluster: pick node type (e.g. dax.r5.large), node count, VPC + subnets.
2. Create a subnet group spanning multiple AZs.
3. Create an IAM role for DAX nodes (allows access to underlying DynamoDB tables).
4. Set up security group: allow port 8111 (DAX cluster comm).
5. Update application: install DAX SDK; point DynamoDB client at DAX cluster endpoint.

Latency impact:

• Cached read: ~1 ms (within VPC).
• Cache miss: DAX fetches from DynamoDB (single-digit ms), populates cache, returns.
• Cached write: write-through to DynamoDB + cache update (~few ms).

Cost:

• DAX nodes priced like EC2 instances ($/hr).
• Per-node sizing matters: cache size = node RAM × node count × replication factor.
• Smaller cluster (3 × r5.large) ≈ $400/month; large cluster (10 × r5.4xlarge) ≈ $4 000/month.

When DAX is the right answer:

• Read-heavy DynamoDB workload (>80% reads).
• Hot keys generating GetItem traffic.
• Microsecond read latency required.
• Workload can tolerate eventual consistency.

When NOT:

• Write-heavy workload — DAX doesn't help; writes always go to DynamoDB.
• Strong-consistency reads required — DAX bypasses cache for those.
• Small workload — ElastiCache or direct DynamoDB is cheaper.
• Workload uses Transaction APIs heavily — TransactGetItems / TransactWriteItems bypass DAX.

Compared to DynamoDB built-in caching:

• DynamoDB has no built-in cache at the API layer (the storage engine has its own internal cache, but it's not exposed).
• DAX is the supported caching solution. Custom ElastiCache + cache-aside is an alternative for finer control.

Common exam scenarios:

• 'DynamoDB read latency too high' → DAX.
• 'Microsecond reads against DynamoDB' → DAX.
• 'Cache DynamoDB with minimal app changes' → DAX.
• 'Cache DynamoDB with custom invalidation logic' → ElastiCache + custom code.

ElastiCache offers managed Redis and Memcached. Each has very different scaling and HA models. Picking wrong means you can't shard, or you can't replicate, or you're paying for features you don't need.

Redis cluster modes:

Cluster Mode Disabled (single-shard):

• 1 primary node + 0-5 replicas (read-only).
• Single shard → all data on one node.
• Failover: replica auto-promotes (~1 min).
• Max RAM: limited by largest available node type.
• Use for: small workloads where total dataset fits on one node + need replicas for read scaling or HA.

Cluster Mode Enabled (sharded):

• 1-500 shards; each shard is 1 primary + 0-5 replicas.
• Data partitioned across shards by hash slot (16 384 slots distributed across shards).
• Total cluster capacity = shard count × per-shard RAM.
• Failover: per-shard, automatic.
• Use for: larger datasets, write-scaling workloads, very-high-throughput.

Redis Serverless (newer):

• Auto-scales ECPUs and storage independently.
• Multi-AZ HA built-in.
• No node management.
• Trade-off: less control over individual node placement.

Memcached:

• No replication; no persistence.
• Multi-threaded per node (one node uses all cores; Redis is single-threaded with side processes).
• Auto-discovery: clients auto-discover all nodes in the cluster.
• Use for: simple key-value caching where lost data is acceptable.
• Wrong for: anything needing data structures (Redis lists/sets/sorted sets), persistence, or replication.

Redis vs Memcached (the canonical exam question):

Feature	Redis	Memcached
Data structures	Strings, lists, sets, sorted sets, hashes, streams, geo, hyperloglog	Strings only
Persistence	Optional (snapshot + AOF)	None
Replication	Yes (read replicas, multi-AZ)	No
Cluster mode (sharding)	Yes	Manual via client-side hashing
Pub/sub	Yes	No
Transactions	Yes (MULTI/EXEC)	No
Multi-threading	Single-threaded (per node)	Multi-threaded
Snapshots / backups	Yes	No

Default answer: Redis unless the question explicitly says 'simple multi-threaded caching'.

Caching strategies (applies to both):

• Lazy loading: read miss → fetch from DB → cache. Cache only holds requested data. Stale data possible (until TTL).
• Write-through: every write goes to cache AND DB. Always fresh; slower writes; cache may hold rarely-read data.
• Write-around: writes go to DB only; cache populated on read miss. Reduces cache pollution from write-heavy workloads.
• TTL-based expiration: time-bounded staleness regardless of strategy.

Common patterns:

• Session store (Redis): cluster mode disabled with replication for HA; persistence on (RDB + AOF).
• Leaderboard (Redis): cluster mode enabled with sharding for write scaling; uses sorted sets.
• Database read cache (Redis or Memcached): cluster mode for scale; lazy-loading; short TTL.
• Geo-spatial queries (Redis only): GEOADD + GEORADIUS commands.

Reserved Nodes:

• 1- or 3-year commitment for ~30-55% discount (similar to EC2 RIs).
• All-Upfront / Partial-Upfront / No-Upfront payment options.
• Used after running on-demand for ~30 days to verify the right node type.