Supported protobuf types - RisingWave

Conversion

RisingWave converts well-known types from the protobuf library to specific types in RisingWave. The conversion is as follows:

Protobuf type	RisingWave type
any	`jsonb`
double	`double precision`
float	`real`
int32	`int`
int64	`bigint`
uint32	`bigint`
uint64	`decimal`
sint32	`int`
sint64	`bigint`
fixed32	`bigint`
fixed64	`decimal`
sfixed32	`int`
sfixed64	`bigint`
bool	`boolean`
string	`varchar`
bytes	`bytea`
enum	`varchar`
message	`struct`. See details in Nested messages.
messages_as_jsonb	`jsonb`. See details in Handle recursive definitions.
repeated	`array`
map	`map`. See details in Map.
google.protobuf.Struct	Not supported
google.protobuf.Timestamp	`struct<seconds bigint, nanos int>`
google.protobuf.Duration	`struct<seconds bigint, nanos int>`
google.protobuf.Any	`struct<type_url varchar, value bytea>`
google.protobuf.Int32Value	`struct<value int>`
google.protobuf.StringValue	`struct<value varchar>`

Nested messages

The nested fields are transformed into columns within a struct type. For example, a Protobuf message with the following structure:

message NestedMessage {
  int32 id = 1;
  string name = 2;
}

Will be converted to struct<id int, name varchar> in RisingWave.

Handle recursive definitions

When detecting a recursive definition in the protobuf, RisingWave will reject the statement and show the circular dependency. Adding dependency items to messages_as_jsonb with full type name separated by comma can solve the case. For example:

CREATE TABLE opentelemetry_test 
WITH ( 
    ${RISEDEV_KAFKA_WITH_OPTIONS_COMMON}, 
    topic = 'opentelemetry_test' 
) 
FORMAT PLAIN 
ENCODE PROTOBUF ( 
    schema.registry = '${RISEDEV_SCHEMA_REGISTRY_URL}', 
    message = 'opentelemetry_test.OTLPTestMessage', 
    messages_as_jsonb = 'opentelemetry.proto.common.v1.ArrayValue,opentelemetry.proto.common.v1.KeyValueList,opentelemetry.proto.common.v1.AnyValue'
);

CREATE SOURCE