Export metrics

This article describes how to use metrics API to export metrics from a RisingWave Cloud project to various monitoring systems like Prometheus, DataDog, and InfluxDB. The metrics include all major components such as etcd, frontend, compute, compactor, and meta.

By default, a newly created service account isn’t given any permissions. To export metrics, you need to grant your service account the ProjectAdmin role. See RBAC for more details.

Step 1: Generate API key

Generate the API key ID and API key secret in the Cloud Portal. See Generate an API key for details.

Step 2: Get Cloud_HOST

Get the corresponding CLOUD_HOST for your region and Cloud provider from the table below:

Region/CloudProvider	CLOUD_HOST
us-east-2/aws	canary-useast2-mgmt.risingwave.cloud
us-east-1/aws	prod-aws-usea1-mgmt.risingwave.cloud
us-west-2/aws	prod-aws-uswe2-mgmt.risingwave.cloud
eu-west-2/aws	prod-aws-euwe2-mgmt.risingwave.cloud
eu-north-1/aws	prod-aws-euno1-mgmt.risingwave.cloud
ap-southeast-1/aws	prod-aws-apse1-mgmt.risingwave.cloud
europe-west3/gcp	prod-gcp-euwe3-mgmt.risingwave.cloud
europe-north1/gcp	prod-gcp-euno1-mgmt.risingwave.cloud

Step 3: Configure monitoring systems

Choose one of the following methods to configure monitoring systems.

The metrics are formatted according to Prometheus standards. If your monitoring collection mode is compatible with the Prometheus format, refer to the Prometheus section below to configure the collection.

Prometheus

To import the metrics to Prometheus, edit the scrape_configs section in the prometheus.yaml file as follows:

scrape_configs:
- job_name: risingwave-remote
  static_configs:
  - targets:
    - {CLOUD_HOST}
  metrics_path: /api/v1/tenant/{CLUSTER_ID}/metrics
  basic_auth:
    username: {API_KEY}
    password: {API_SECRET}
  scrape_interval: 60s
  scrape_timeout: 60s
  scheme: https

Notes

• Update the Cloud_HOST according to the region and Cloud provider of your RisingWave cluster.
• Update the CLUSTER_ID with the specific cluster ID in your RisingWave environment. For example, if your cluster URL is risingwave-cloud.com/cluster/168/us-central1/overview/, replace CLUSTER_ID with the value 168.

DataDog

To import the metrics to DataDog, use DataDog Agent and DataDog integration. If you have not installed the DataDog agent, see installation guide.

Next, edit the openmetrics.d/conf.yaml file at the root of your Agent configuration directory as follows, and then restart your agents.

instances:
- openmetrics_endpoint: http://{CLOUD_HOST}/api/v1/tenant/{CLUSTER_ID}/metrics
  namespace: risingwave
  metrics:
  - .*
  auth_type: basic
  username: {API_KEY}
  password: {API_SECRET}

Notes

• Update the Cloud_HOST according to the region and Cloud provider of your RisingWave cluster.
• Update the CLUSTER_ID with the specific cluster ID in your RisingWave environment. For example, if your cluster URL is risingwave-cloud.com/cluster/168/us-central1/overview/, replace CLUSTER_ID with the value 168.
• The limit for the agent is 2000 metrics per instance. To increase the limit, please contact DataDog support.

InfluxDB

To import the metrics to InfluxDB, you need to configure Telegraf first. See instructions on how to use Telegraf to scrape Prometheus metrics and the Prometheus input plugin.

In the Telegraf configuration, you need to fill in the Prometheus input section, see Prometheus for details.

Step 4: Set up dashboards

To set up dashboards based on key metrics, see RisingWave open source user-dashboard.