Keep Typesense in sync with Postgres.
Row changes become document upserts and deletes in your collection, in commit order per row.
The Typesense sink imports each batch's inserts and updates with action=upsert and deletes the rest by id. Runs of upserts and deletes are flushed in order so a delete never overtakes the update before it. The document id is derived from the row's primary key.
How delivery works
| Aspect | Behaviour for Typesense |
|---|---|
| Operation | Insert/update → document = row data + id, imported with action=upsert; delete → DELETE /documents/<id> (404 counts as done). |
| Batching | Consecutive upserts go in one import request; deletes are sent per id, in order. |
| Ordering | Per-row order preserved across upsert and delete runs. |
| Idempotency | Upsert by id; retries overwrite. |
| On failure | Any failed import line fails the batch. Any 2xx is accepted. A 429 pauses the sink for the Retry-After period (30 s if absent) without spending a retry or metering egress. A failed request is retried up to 5 times with exponential backoff (30 s, 1, 2, 4 min, with jitter). After the fifth failure the message is parked as undelivered, the console shows it with the last error, and one click replays it — later changes to the same row wait behind it so order is preserved. |
| Connection check | GET /collections/<collection> — reachability, key and collection. |
Set up in three steps
-
Connect your database
Paste a connection string and press Test. Waltail checks the version (12+), that
wal_levelislogical, that the user may replicate, and that a slot is free — and shows the fix for anything that fails. Then it creates its own publication and replication slot. Connection guide → -
Add Typesense as a destination
Pick the tables to stream, choose Typesense, and fill in Server URL, API key, Collection. Press Test — Waltail checks it can reach and write to Typesense before anything is saved.
-
Change a row
Insert or update a row. The console shows the first event as it is captured and delivered; from then on, every committed change follows within about a second. Have rows that already exist? Run a backfill — confirm the estimate and they arrive through Typesense as
readevents.
Configuration
Fields as the console asks for them. Secrets are encrypted at rest and never shown again.
| Field | Console label | Required | Notes |
|---|---|---|---|
endpoint | Server URL | Yes | |
api_key | API key | Yes | |
collection | Collection | Yes | Create it first (auto-schema works). |
Every sink also has two tuning settings: batch size (messages per request, default 100) and rate limit (messages per second, default unlimited). Changes to one row are never in flight twice at once.
Example
Every change is this JSON envelope — data is the row after the change, old the row before it when replica identity provides it, null otherwise:
{
"op": "update",
"table": "public.orders",
"data": { "id": 42, "status": "shipped", "total": 19.99 },
"old": { "id": 42, "status": "paid", "total": 19.99 },
"seq": 137,
"idempotency_key": "9f1c3d2e-6b4a-4c7e-9d0f-2a1b3c4d5e6f",
"commit_lsn": "0/1A2B3C8"
}Questions
Do I need a schema?
Create the collection first; a schema with .* auto fields is enough to start and you can tighten it later.
What is the document id?
The ordering key with characters outside [A-Za-z0-9_-] replaced by _, e.g. public_products_17.