Google Cloud SQL.
One flag, one user attribute, one authorized network. Pub/Sub makes a natural first destination.
Waltail needs three things from Cloud SQL: wal_level = logical, a user that is allowed to replicate, and a way to reach the database over the network. Set them up in the steps below. When you paste the connection string in Waltail, the console runs six checks and tells you if anything is still missing.
1. Set the flag
Set the instance flag cloudsql.logical_decoding to on. The instance restarts and wal_level becomes logical.
2. User
CREATE ROLE waltail WITH LOGIN PASSWORD '…';
ALTER ROLE waltail WITH REPLICATION;
GRANT CONNECT ON DATABASE app TO waltail;
Run these as the postgres user (a cloudsqlsuperuser). The user also needs to own the tables you stream, or belong to the role that owns them.
3. Network
Add Waltail's address to the instance's authorized networks, or connect over the public IP with SSL. The Cloud SQL Auth Proxy is not needed for replication.
Then, in Waltail
Paste the connection string
Use the direct database host. A transaction-mode pooler (PgBouncer and similar) cannot carry a replication connection, so the
reachablecheck would fail there.Pick tables
Choose the tables to stream. A table needs a primary key — or
REPLICA IDENTITY FULLor an index identity — for updates and deletes to replicate. The console marks any table that does not qualify and says why.Add a destination
Waltail creates a publication and a replication slot on your database (both named
pgcdc_<pipeline>) and starts streaming. The quickstart walks through the rest.