Delivery, retries and ordering.
What happens between a change being captured and your destination accepting it — and what happens when the destination does not.
The guarantee: at-least-once
A change is acknowledged to your database only after Waltail has stored it, and it is retried until your destination accepts it. So every change arrives at least once.
In rare cases it can arrive twice — for example if a worker crashes after the destination accepted a message but before the success was recorded. Every message carries a unique idempotency_key so your consumer can spot and drop the duplicate.
Ordering
Changes to the same row — same schema.table:pk — are delivered in commit order and are never in flight twice at once. Changes to different rows flow in parallel. If a message is retrying or parked, later messages for the same row wait behind it, so your consumer never sees a row's history out of order.
Batch size and rate limit
Each destination has two settings you can leave at the defaults. Batch size (default 100) is how many messages Waltail sends per request — one webhook POST, one Kafka produce, one S3 object. Rate limit (default unlimited) caps messages per second. Throttled messages stay pending rather than being picked up early, so the backlog numbers in the console stay honest.
When a delivery fails
Waltail retries with exponential backoff from a 30-second base, plus a little jitter: about 30 s, 1 min, 2 min, 4 min. After the fifth failed attempt the message is parked as undelivered and is not retried on its own.
The console shows the undelivered count per destination, and for each message its payload and the last error. When the destination is healthy again, replay one message or all of them with one click — see replay.
For HTTP destinations, any 2xx counts as success. A 429 pauses the destination for the Retry-After period (30 s if the header is missing) without using up a retry or counting as egress.
If a worker crashes
A message stuck in delivering for more than two minutes — a worker that died mid-batch — goes back to pending and another worker picks it up. This is the one place a duplicate can come from.
If you pause a pipeline
Pausing stops capture; changes already captured keep delivering. The replication slot stays in place, so WAL builds up on your database while paused. The console shows slot lag per pipeline so you can see how much.
What counts as egress
Every delivery attempt is metered, successful or not, into the hourly usage the console shows. Pauses caused by a 429 are not metered.
Next: replay and retention →