Postgres changes into Amazon SQS.
The default queue for AWS-hosted backends, fed straight from your database's write-ahead log. FIFO queues get per-row message groups and deduplication for free.
The SQS sink sends each change as a message whose body is the change JSON. On a FIFO queue the row's ordering key becomes the message group and the idempotency key the deduplication id, so SQS itself enforces per-row order and drops redeliveries.
Delivery to SQS in the same region is close to free for us, which is one reason sinks are never charged — you pay for egress only.
How delivery works
| Aspect | Behaviour for Amazon SQS |
|---|---|
| Message | Body = the change JSON; message attributes idempotency_key and ordering_key. |
| Batching | SendMessageBatch in chunks of 10 (the SQS limit); a batch of 100 is ten calls. |
| Ordering | FIFO queues: MessageGroupId = ordering key, so changes to one row are consumed in order. Standard queues: best effort, use the attribute to re-sequence. |
| Idempotency | FIFO queues: MessageDeduplicationId = idempotency key. Standard queues: dedupe on the attribute. |
| On failure | Any rejected entry fails the batch and the batch is retried. A failed request is retried up to 5 times with exponential backoff (30 s, 1, 2, 4 min, with jitter). After the fifth failure the message is parked as undelivered, the console shows it with the last error, and one click replays it — later changes to the same row wait behind it so order is preserved. |
| Connection check | GetQueueAttributes on the queue URL verifies the URL, region and credentials. |
Set up in three steps
-
Connect your database
Paste a connection string and press Test. Waltail checks the version (12+), that
wal_levelislogical, that the user may replicate, and that a slot is free — and shows the fix for anything that fails. Then it creates its own publication and replication slot. Connection guide → -
Add Amazon SQS as a destination
Pick the tables to stream, choose Amazon SQS, and fill in Queue URL, Region, Access key ID, Secret access key. Press Test — Waltail checks it can reach and write to Amazon SQS before anything is saved.
-
Change a row
Insert or update a row. The console shows the first event as it is captured and delivered; from then on, every committed change follows within about a second. Have rows that already exist? Run a backfill — confirm the estimate and they arrive through Amazon SQS as
readevents.
Configuration
Fields as the console asks for them. Secrets are encrypted at rest and never shown again.
| Field | Console label | Required | Notes |
|---|---|---|---|
queue_url | Queue URL | Yes | A .fifo URL switches on group and deduplication ids. |
region | Region | Yes | AWS region of the resource, e.g. us-east-1. |
access_key_id | Access key ID | Yes | Static IAM credentials; Waltail does not assume roles. |
secret_access_key | Secret access key | Yes | Stored encrypted, never shown again. |
endpoint | Custom endpoint | Optional | Overrides the AWS endpoint for VPC endpoints or LocalStack. |
Every sink also has two tuning settings: batch size (messages per request, default 100) and rate limit (messages per second, default unlimited). Changes to one row are never in flight twice at once.
Example
Every change is this JSON envelope — data is the row after the change, old the row before it when replica identity provides it, null otherwise:
{
"op": "update",
"table": "public.orders",
"data": { "id": 42, "status": "shipped", "total": 19.99 },
"old": { "id": 42, "status": "paid", "total": 19.99 },
"seq": 137,
"idempotency_key": "9f1c3d2e-6b4a-4c7e-9d0f-2a1b3c4d5e6f",
"commit_lsn": "0/1A2B3C8"
}Questions
Standard or FIFO?
FIFO if your consumer cares about per-row order or exactly-once processing; SQS enforces both from the ids Waltail sets. Standard if you want maximum throughput and handle order yourself.
What IAM permissions does the key need?
sqs:SendMessage, sqs:SendMessageBatch and sqs:GetQueueAttributes on the queue.
What about the 256 KB message limit?
A single change larger than the SQS limit is rejected by SQS and parked as undelivered with the error. Wide JSON or bytea columns are the usual cause.