Domain 3 of 5 · Chapter 2 of 4

Data Warehouse

Unlock the complete study guide + 1,040 practice questions across 16 full exams.

Bundled into the existing Professional Data Engineer premium course — no separate purchase.

Included in this chapter:

  • Why warehouse design flips the OLTP rules
  • Shaping the data: denormalize with nested and repeated fields
  • Physical layout: partition, cluster, materialize
  • Mapping requirements to the model, and exam traps

How each design lever reduces cost or latency

Design leverWhat it doesCuts bytes scanned?Best for
Denormalize into wide tableFolds related data into one table so reads avoid joinsIndirectly (fewer joins, less shuffle)The default analytic table
Nested / repeated fieldsModels one-to-many in a single row via ARRAY of STRUCTYes (prunes to referenced sub-fields)Parent with child records read together
PartitioningSplits a table by date / timestamp / integer rangeYes (skips non-matching partitions)Queries that filter on the partition column
ClusteringSorts rows within a partition by up to 4 columnsYes (reads fewer blocks)Filter / group on high-cardinality columns
Materialized viewPrecomputes and incrementally refreshes an aggregationYes (scans the small precomputed result)The same heavy aggregation run repeatedly
Star schema (kept dimensions)Joins facts to separate dimension tablesNo (adds a join)Large dimensions that change often

Cheat sheet

  • Denormalize for BigQuery, because the join costs more than the storage
  • Model one-to-many with an ARRAY of STRUCT, not a child table
  • Nesting is not flattening, flattening duplicates the parent
  • Partition on one date, timestamp, or integer-range column
  • Cluster on up to four columns, ordered left to right
  • Partition to skip data, cluster to sort within it, and stack both
  • Cluster, do not partition, on a high-cardinality column
  • Set require_partition_filter to stop accidental full scans
  • A materialized view precomputes an aggregation and refreshes incrementally
  • BigQuery reroutes base-table queries to a matching materialized view
  • A logical view stores query text, a materialized view stores results
  • Keep an independently-changing dimension as a joined table
  • Set the table grain to the dominant query
  • Let the access pattern pick the partition and cluster columns
  • Layout changes fix a single slow query, more slots do not
  • Leave small lookup tables flat
  • A table's partition expiration overrides the dataset default
  • Load files into BigQuery on arrival with an event-driven transfer

Unlock with Premium — includes all practice exams and the complete study guide.

References

  1. BigQuery storage overview
  2. Control costs in BigQuery
  3. Use nested and repeated fields (query performance best practices) Well-Architected
  4. Introduction to partitioned tables
  5. Introduction to clustered tables
  6. Introduction to materialized views
  7. Introduction to reservations