Monitoring Metrics Collection

Enabling Metrics

  • The Master metric listening port is the service port, and the metric monitoring module is enabled by default.

  • The Blobstore metric listening port is the service port, and the service custom monitoring metric items are enabled by default. Public metric items need to be opened by modifying the configuration file. Refer to:

  • Other modules need to configure the metric listening port, which is disabled by default.

    • exporterPort: Metric listening port.
    • consulAddr: Consul registration server address. If set, the automatic discovery service of the CubeFS node exporter can be realized in conjunction with the Prometheus automatic discovery mechanism. If not set, the Consul automatic registration service will not be enabled.
    • consulMeta: Consul metadata configuration. Non-required item, set metadata information when registering with Consul.
    • ipFilter: Regular expression-based filter. Non-required item, default is empty. Exposed to Consul, used when the machine has multiple IPs. Supports forward and reverse filtering.
    • enablePid: Whether to report partition id, default is false; if you want to display the information of dp or mp in the cluster, you can configure it as true.
{
  "exporterPort": 9505,
  "consulAddr": "http://consul.prometheus-cfs.local",
  "consulMeta": "k1=v1;k2=v2",
  "ipFilter": "10.17.*",
  "enablePid": "false"
}

Requesting the corresponding metric listening interface of the service can obtain monitoring metrics, such as curl localhost:port/metrics

Collecting Metrics

For Master, MetaNode, DataNode, and ObjectNode, there are two ways to implement metric collection:

  • Configure the Consul address of Prometheus (or the Consul address that supports the Prometheus standard syntax). After the configuration takes effect, Prometheus will actively pull monitoring metrics.
  • If the Consul address is not configured, the following example is used:

Modify the Prometheus YAML configuration file and add the metric collection source

# prometheus.yml
- job_name: 'cubefs01'
file_sd_configs:
  - files: [ '/home/service/app/prometheus/cubefs01/*.yml' ]
  refresh_interval: 10s

Access the exporter and create an exporter file under the above configuration directory. Taking Master as an example, create the master_exporter.yaml file

# master_exporter.yaml
- targets: [ 'master1_ip:17010' ]
  labels:
    cluster: cubefs01

After the configuration is complete, start Prometheus.

The relevant services of the Erasure Coding Subsystem (Blobstore) currently only support the second method of collecting metrics.

Edit on GitHub
Enabling Metrics
Collecting Metrics