Source of truth with active transactions
Remember CDC
Full name: Change Data Capture
Memory aid: Capture Data Changes in their change stream as a cue.
Retention, ordering, schema changes, and replay behavior affect reliable consumption.
Study area: Data & integration · Batch 9
Relevant study for: Azure Data Fundamentals DP-900 · CompTIA Cloud+
Practice with exam flashcards →Data / Knowledge library
Change Data Capture (CDC)
Emits inserts, updates, and deletes from a database change stream
Why learn Change Data Capture (CDC)?
It feeds replicas, analytics, caches, and integrations incrementally
The problem it solves
It moves data changes without repeatedly scanning whole tables
Where it came from
Change Data Capture (CDC) is a database concept that emits inserts, updates, and deletes from a database change stream. Its guarantees and tradeoffs depend on the concrete implementation and operating context.
Build your understanding
Work through these exercises in order. Try the task before opening a reference.
- Define Change Data Capture (CDC) and distinguish it from a neighboring concept.
- Choose a realistic task where Change Data Capture (CDC) is appropriate.
- Identify a failure mode caused by misusing Change Data Capture (CDC).
- Interpret a small design or result involving Change Data Capture (CDC).
- Explain one tradeoff or limitation of Change Data Capture (CDC).
Documentation and sources
- PostgreSQL Documentation (www.postgresql.org)
Connect the ideas
Explore next
Real World Example
Real-time Inventory Synchronization for Global Retail
Global retail enterprises managing distributed warehouse systems and e-commerce storefronts.
The business problem
Traditional polling-based synchronization creates significant latency and excessive load on the primary transaction database, leading to stale inventory counts during high-traffic sales events.
How this topic solves it
Change Data Capture (CDC) utilizes the PostgreSQL logical decoding mechanism to stream row-level changes (inserts, updates, and deletes) directly from the Write-Ahead Log (WAL) to downstream inventory services without querying the application tables.
- Configure the PostgreSQL server with wal_level set to logical to enable the generation of information required for decoding.
- Create a replication slot to track the position of the data stream and ensure the database retains necessary WAL files.
- Initialize a logical decoding output plugin to transform internal WAL records into a consumable format like JSON.
- Implement a consumer service that reads from the replication slot and updates the secondary inventory cache in real-time.
This diagram illustrates the flow of data from the primary database to the inventory cache using logical decoding.
Write-Ahead Log containing change records
Plugin parsing WAL into events
Downstream system receiving updates
- Primary DB → Writes changes → WAL
- WAL → Streams logs → Logical Decoder
- Logical Decoder → Emits events → Inventory Cache
Expected business outcome
Achieving near-instantaneous inventory updates across distributed systems, ensuring that storefronts reflect accurate stock levels without impacting primary database performance.
Limitations and checks
Logical replication slots can cause disk space exhaustion if the consumer service fails or disconnects for an extended period, as the database must retain all WAL segments since the last confirmed read. Administrators must monitor the pg_replication_slots view to ensure the restart_lsn is advancing.