Keep OpenSearch in sync with Postgres.
Same bulk upsert-and-delete path as Elasticsearch, pointed at OpenSearch or Amazon OpenSearch Service.
The OpenSearch sink shares its bulk code path with Elasticsearch: inserts and updates index the row's data under a document id derived from the primary key, deletes remove it, and each batch is one _bulk request.
How delivery works
| Aspect | Behaviour for OpenSearch |
|---|---|
| Operation | Insert/update → index by _id with the row's data; delete → delete by _id. |
| Batching | One _bulk NDJSON request per batch. |
| Ordering | Per-row order preserved. |
| Idempotency | Indexing by _id; retries overwrite. |
| On failure | Any item error other than a 404 on delete fails the batch. Any 2xx is accepted. A 429 pauses the sink for the Retry-After period (30 s if absent) without spending a retry or metering egress. A failed request is retried up to 5 times with exponential backoff (30 s, 1, 2, 4 min, with jitter). After the fifth failure the message is parked as undelivered, the console shows it with the last error, and one click replays it — later changes to the same row wait behind it so order is preserved. |
| Connection check | GET / on the cluster. |
Set up in three steps
-
Connect your database
Paste a connection string and press Test. Waltail checks the version (12+), that
wal_levelislogical, that the user may replicate, and that a slot is free — and shows the fix for anything that fails. Then it creates its own publication and replication slot. Connection guide → -
Add OpenSearch as a destination
Pick the tables to stream, choose OpenSearch, and fill in Cluster URL, Index. Press Test — Waltail checks it can reach and write to OpenSearch before anything is saved.
-
Change a row
Insert or update a row. The console shows the first event as it is captured and delivered; from then on, every committed change follows within about a second. Have rows that already exist? Run a backfill — confirm the estimate and they arrive through OpenSearch as
readevents.
Configuration
Fields as the console asks for them. Secrets are encrypted at rest and never shown again.
| Field | Console label | Required | Notes |
|---|---|---|---|
endpoint | Cluster URL | Yes | |
index | Index | Yes | Created by the first write if it does not exist. |
username | Username | Optional | Basic auth. |
password | Password | Optional |
Every sink also has two tuning settings: batch size (messages per request, default 100) and rate limit (messages per second, default unlimited). Changes to one row are never in flight twice at once.
Example
Every change is this JSON envelope — data is the row after the change, old the row before it when replica identity provides it, null otherwise:
{
"op": "update",
"table": "public.orders",
"data": { "id": 42, "status": "shipped", "total": 19.99 },
"old": { "id": 42, "status": "paid", "total": 19.99 },
"seq": 137,
"idempotency_key": "9f1c3d2e-6b4a-4c7e-9d0f-2a1b3c4d5e6f",
"commit_lsn": "0/1A2B3C8"
}Questions
Does it work with Amazon OpenSearch Service?
Yes, with fine-grained access control and a master user, or a domain open to the Waltail egress IPs. Use the domain endpoint as the cluster URL.
Elasticsearch or OpenSearch sink?
Pick the one that matches your cluster; the write path is the same.