What is change data capture?
Change data capture (CDC) takes a database's own record of what changed and turns it into a stream other systems can read. In Postgres, that record is the write-ahead log and the mechanism is logical replication.
The write-ahead log
Before Postgres changes a page on disk, it writes a description of the change to its write-ahead log, the WAL. The WAL is how Postgres recovers from a crash and how replicas keep up. It is also a complete, ordered record of every committed insert, update and delete.
Logical replication
Physical replication copies raw WAL bytes to an identical standby. Logical replication instead decodes the WAL into row-level changes — this table, this primary key, these old and new values — that any client can read over a replication connection. Three things make it work:
wal_level = logicaltells Postgres to write enough detail into the WAL to decode rows.- A publication names the tables whose changes to expose.
- A replication slot is a durable bookmark. Postgres keeps WAL from that point until the reader acknowledges it, so nothing is lost while the reader is away.
Every position in the log has an address, an LSN. Acknowledging an LSN tells Postgres that everything before it has been handled and the WAL can be recycled.
What a hosted service adds
Reading the stream is the easy part. Running it well is not: the reader has to keep the connection alive, acknowledge quickly so WAL does not pile up on your database, survive its own restarts without losing or duplicating anything, cope with schema changes mid-stream, and deliver to the systems that actually need the data — with retries and in order.
That is the job Waltail does. It reads the stream, stores every change before acknowledging it, and delivers to queues, streams, caches, search indexes and webhooks with per-row ordering and an idempotency key on every message.
How it compares with the alternatives
| Approach | How it works | Trade-off |
|---|---|---|
| Triggers | A function runs inside the transaction and calls out, or writes an audit table | Slows every write; a failing trigger fails or loses the change; couples the database to other systems |
| Polling | A job looks for rows where updated_at > last_run | Misses deletes and rows without the column; adds load and delay |
| Dual writes | The application writes the database and then the queue | The two can disagree — the classic distributed-systems bug |
| Outbox table | Events go in a table in the same transaction; a worker drains it | Correct, but a table and a worker to own in every service |
| Change data capture | Read committed changes from the WAL | Correct and complete, off the write path — it just needs a reader that is run well |
What to expect from the guarantees
Honest CDC is at-least-once: every change arrives once or, rarely, twice. Each message carries an idempotency key so your consumer can drop the duplicate. "Exactly-once delivery" across a network is not achievable in practice; exactly-once processing is, with that key. Ordering is per row: changes to the same primary key arrive in the order they were committed, which is what most consumers need.
Ready to try it? Start with the quickstart →