Skip to content

Switch the MINIO scrape pipeline to Metrics V3 and configured bucket metrics #767

Description

@Vonng

Parent: #763

Problem

The public Pigsty scrape configuration still defaults to /minio/v2/metrics/cluster. That endpoint does not cover all node/drive/resource data expected by modern SILO Console widgets, while SILO standardizes on Metrics V3. Bucket metrics also need an explicit per-bucket path.

Proposed baseline

Use the simple, complete topology first:

  • scrape /minio/metrics/v3 for every SILO instance;
  • for every bucket declared in minio_buckets, also scrape /minio/metrics/v3/bucket/<bucket>;
  • keep one logical job="minio" and preserve cls, ins, ip, instance, and flavor;
  • select the per-target bucket path through file-SD/relabel metadata;
  • keep existing RustFS health-probe behavior intact;
  • make legacy MinIO behavior explicit instead of sending unsupported V3 paths to it accidentally.

This intentionally favors complete coverage and a small configuration surface. For N instances and B managed buckets, the expected scrape-target count is N × (1 + B). Measure the actual series count and duplicated cluster/bucket-wide series before adding a more complex designated-target optimization.

Required validation

  • Inventory the V3 metric families and series produced by single-node and distributed SILO.
  • Quantify per-bucket series growth and verify that only Pigsty-managed buckets are added by default.
  • Identify V3 cluster-wide gauges repeated by each instance; retain the raw data but document the correct dashboard aggregation.
  • Verify target uniqueness. Do not manufacture uniqueness by putting a bucket name only in the display-form instance label if another stable label is required for querying.
  • Verify scrape authentication, TLS, reload, and target health.

Acceptance criteria

  • Every SILO instance exposes a healthy V3 root target.
  • Every configured bucket exposes a healthy bucket target; no undeclared bucket is scraped by default.
  • Series retain stable job, cls, ins, and bucket identity labels.
  • Target and series counts are documented for at least 1-node and multi-node test clusters.
  • RustFS and any retained legacy MinIO profile do not regress.
  • VictoriaMetrics reload succeeds and post-reload queries prove root, node/drive/resource, and bucket families are present.

Metadata

Metadata

Assignees

No one assigned

    Labels

    ENHANCEMENTNew feature or requestMINIOMINIO RelatedMONITORMonitoring related

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions