PostgreSQL CDC configuration options

Reference for all configuration parameters for connecting to PostgreSQL CDC.

This page provides a complete reference for all configuration parameters used when connecting RisingWave to a PostgreSQL database for Change Data Capture (CDC) via the WITH clause of a CREATE SOURCE or CREATE TABLE ... FROM source statement. For step-by-step instructions on setting up PostgreSQL for CDC and connecting to RisingWave, see the Connect to PostgreSQL CDC guide.

Syntax

Syntax for creating a CDC source.

CREATE SOURCE [ IF NOT EXISTS ] source_name WITH (
   connector='postgres-cdc',
   <field>=<value>, ...
);

Syntax for creating a CDC table on top of this CDC Source. Note that a primary key is required and must be consistent with the upstream table. We must also specify the Postgres table name (pg_table_name) which we are selecting from.

CREATE TABLE [ IF NOT EXISTS ] table_name (
   column_name data_type PRIMARY KEY , ...
   PRIMARY KEY ( column_name, ... )
)
[ INCLUDE timestamp AS column_name ]
WITH (
    snapshot='true'
)
FROM source TABLE pg_table_name;

Connector parameters

Parameter	Description	Required
connector	Must be set to 'postgres-cdc'.	Yes
hostname	The hostname or IP address of your PostgreSQL database server.	Yes
port	The port number of your PostgreSQL database server. The default PostgreSQL port is 5432.	Yes
username	The username for connecting to your PostgreSQL database. This user must have the necessary privileges for CDC (see the Connect to PostgreSQL CDC guide).	Yes
password	The password for the PostgreSQL user.	Yes
database.name	The name of the PostgreSQL database to connect to.	Yes
schema.name	The name of the PostgreSQL schema containing the table you want to ingest data from. If not specified, defaults to 'public'.	No
table.name	The name of the PostgreSQL table to ingest data from. Note: When creating a table from a shared source, this parameter is not specified in the WITH clause. Instead, it’s specified in the FROM source TABLE pg_table_name clause.	Yes
slot.name	The name of the PostgreSQL replication slot to use. If not specified, RisingWave will generate a unique, random slot name. Each source should have its own unique slot name. Valid characters are lowercase letters, numbers, and underscore, with length no more than 63 characters.	No
auto.schema.change	Specify whether you want to enable replicating Postgres table schema change	No
ssl.mode	The ssl.mode parameter determines the level of SSL/TLS encryption for secure communication with Postgres. Accepted values are disabled, preferred, required, verify-ca, and verify-full. The default value is disabled. When set to required, it enforces TLS for establishing a connection; When set to verify-ca, it verifies that the server is trustworthy by checking the certificate chain up to the root certificate stored on the client; When set to verify-full, it verifies the certificate and also ensures the server hostname matches the name in the certificate.	No
ssl.root.cert	Specify the root certificate secret. You must create secret first and then use it here.	No
publication.name	The name of the PostgreSQL publication to use. If not specified, defaults to 'rw_publication'.	No
publication.create.enable	Whether to automatically create the publication if it doesn’t exist. Defaults to true. If set to false and the publication doesn’t exist, an error will occur.	No
transactional	Whether to enable transactions for the CDC table. Defaults to true for shared sources and false otherwise. Note that transactions involving changes to more than 4096 rows cannot be guaranteed due to performance considerations. Also supported for shared CDC sources for multi-table transactions.	No

Parameters for CREATE TABLE ... FROM source

These parameters are used only when creating a table from a shared CDC source (CREATE TABLE ... FROM source).

Parameter	Description	Required	Default
snapshot	If false, disables CDC backfill (initial snapshot). Only upstream events after table creation will be consumed. This option only applies to tables created from a shared source.	No	true
snapshot.interval	Specifies the barrier interval for buffering upstream events.	No	1
snapshot.batch_size	Specifies the batch size of a snapshot read query from the upstream table.	No	1000

Debezium parameters

You can also specify any valid Debezium PostgreSQL connector configuration property in the WITH clause. Prefix the Debezium parameter name with debezium..

For example, to skip unknown DDL statements, use:

debezium.schema.history.internal.skip.unparseable.ddl = 'true'

Data format

RisingWave uses the Debezium JSON format for PostgreSQL CDC data. This format does not need to be explicitly specified in the CREATE SOURCE or CREATE TABLE statement when using the postgres-cdc connector. For details on the structure of Debezium JSON, see Data formats and encoding options.

Time travel

Time travel is not supported.

Metadata options

Field	Notes
database_name	Name of the database.
schema_name	Name of the schema.
table_name	Name of the table.