Sink data from RisingWave to Redis

This guide describes how to sink data from RisingWave to Redis.

Redis is an open-source, in-memory data structure store, often referred to as a data structure server. RisingWave sinks data to Redis in the form of strings storing key-value pairs in the specified format (JSON or TEMPLATE), so a primary key must always be provided.

You can test out this process on your own device by using the redis-sink demo in the integration_test directory of the RisingWave repository.

Prerequisites

Before sinking data from RisingWave to Redis, please ensure the following:

• The Redis database you want to sink to is accessible from RisingWave.
• Ensure you have an upstream materialized view or source in RisingWave that you can sink data from.

syntax

CREATE SINK [ IF NOT EXISTS ] sink_name
[FROM sink_from | AS select_query]
WITH (
   connector='redis',
   connector_parameter = 'value', ...
)
FORMAT data_format ENCODE data_encode [ (
    key = 'value' ) ]
[KEY ENCODE key_encode [(...)]]
;

Parameters

Name	Description
redis.url	Required. Choose either the Redis cluster address or a non-cluster Redis address. If the address is a cluster address, it should be in the form of a JSON array, like redis.url= '["redis://redis-server:6379/"]'. If the address is a non-cluster address, it should be in the form of a string, like redis.url= 'redis://redis-server:6379/'.
primary_key	Required. The primary keys of the sink. If necessary, use , to delimit the primary key columns.

FORMAT and ENCODE options

These options should be set in FORMAT data_format ENCODE data_encode (key = 'value'), instead of the WITH clause

Field	Notes
data_format	Data format. Allowed formats: PLAIN: Output data with insert operations. UPSERT: Output data as a changelog stream.
data_encode	Data encoding. Supported encodings: JSON: date: number of days since the Common Era (CE). interval: P<years>Y<months>M<days>DT<hours>H<minutes>M<seconds>S format string. time without time zone: number of milliseconds past the last midnight. timestamp: number of milliseconds since the Epoch. TEMPLATE: converts data to the string specified by key_format/value_format.
force_append_only	If true, forces the sink to be PLAIN (also known as append-only), even if it cannot be.
key_format	Required if data_encode is TEMPLATE. Specify the format for the key as a string.
value_format	Required if data_encode is TEMPLATE. Specify the format for the value as a string.
key_encode	Optional. When specified, the key encode can only be TEXT, and the primary key should be one and only one of the following types: varchar, bool, smallint, int, and bigint; When absent, both key and value will use the same setting of ENCODE data_encode ( ... ).

Example

Assume we create a materialized view, bhv_mv, from a source.

CREATE MATERIALIZED VIEW bhv_mv AS
SELECT
    user_id,
    target_id,
    event_timestamp
FROM
    source_1;

We can sink data from bhv_mv to Redis by creating a sink. Here, data_encode is JSON.

CREATE SINK redis_sink
FROM bhv_mv WITH (
    connector = 'redis',
    primary_key = 'user_id',
    redis.url= 'redis://127.0.0.1:6379/'
) FORMAT PLAIN ENCODE JSON (
    force_append_only='true'
);

We can sink data from bhv_mv to Redis by creating a sink. Here, data_encode is TEMPLATE, so key_format and value_format must be defined.

CREATE SINK redis_sink_2
FROM bhv_mv WITH (
    primary_key = 'user_id',
    connector = 'redis',
    redis.url= 'redis://127.0.0.1:6379/',
) FORMAT PLAIN ENCODE TEMPLATE (
    force_append_only='true',
    key_format = 'UserID:{user_id}',
    value_format = 'TargetID:{target_id},EventTimestamp{event_timestamp}'
);