Ingest data from webhook

Describes how to ingest data from webhook to RisingWave.

A webhook is a mechanism that enables real-time data transfer between applications by sending immediate notifications when specific events occur. Instead of continuously polling for updates, applications can receive data automatically, making it an efficient way to integrate with third-party services.

RisingWave can serve as a webhook destination, directly accepting HTTP requests from external services and storing the incoming data in its tables. When a webhook is triggered, RisingWave processes and ingests the data in real-time.

This direct integration eliminates the need for an intermediary message broker like Kafka. Instead of setting up and maintaining an extra Kafka cluster, you can directly send data to RisingWave to process it in real-time, which enables efficient data ingestion and stream processing without extra infrastructure.

PUBLIC PREVIEW

This feature is currently in public preview, meaning it is nearing the final product but may not yet be fully stable. If you encounter any issues or have feedback, please reach out to us via our Slack channel. Your input is valuable in helping us improve this feature. For more details, see our Public Preview Feature List.

Creating a webhook table in RisingWave

To utilize webhook sources in RisingWave, you need to create a table configured to accept webhook requests. Below is a basic example of how to set up such a table:

CREATE SECRET test_secret WITH (backend = 'meta') AS 'secret_value';

CREATE TABLE wbhtable (
  data JSONB
) WITH (
  connector = 'webhook'
) VALIDATE SECRET test_secret AS secure_compare(
  headers->>'{header of signature}',
  {signature generation expressions}
);

Parameter or clause	Description
CREATE SECRET	Securely stores a secret value in RisingWave for request validation.
CREATE TABLE	Defines a table with a JSONB column to store webhook payload data.
connector	Configures the table to accept incoming HTTP webhook requests
VALIDATE SECRET...AS...	Authenticates requests using the stored secret and signature comparison.
secure_compare()	Validates requests by matching the header signature against the computed signature, ensuring only authenticated requests are processed. Note secure_compare(...) is the only supported validation function for webhook tables.
header_of_signature	Specifies which HTTP header contains the incoming signature.
signature_generation_expressions	Expression to compute the expected signature using the secret and payload.

Currently, secret management is a premium edition feature. You can also use raw string for verification. The following example use raw string 'secret_value' to compute signature.

create table wbhtable (
  data JSONB
) WITH (
  connector = 'webhook',
) VALIDATE AS secure_compare(
  headers->>'x-hub-signature',
  'sha1=' || encode(hmac('secret_value', data, 'sha1'), 'hex')
);

Supported webhook sources and authentication methods

RisingWave has been verified to work with the following webhook sources and authentication methods:

webhook source	Authentication methods
GitHub	SHA-1 HMAC, SHA-256 HMAC
Segment	SHA-1 HMAC
HubSpot	API Key, Signature V2
AWS EventBridge	Bearer Token
Rudderstack	Bearer Token

While only the above sources have been thoroughly tested, RisingWave can support additional webhook sources and authentication methods. You can integrate other services using similar configurations.

See also

Step-by-step guide to help you set up and configure your webhook sources:

• Ingest from Github webhook
• Ingest from Segment webhook
• Ingest from HubSpot webhook
• Ingest from Amazon EventBridge webhook
• Ingest from RudderStack webhook