Proposal: Monitoring Metrics about Chaos Mesh

[STRRL]

Feature Request

Is your feature request related to a problem? Please describe:
No.

Describe the feature you'd like:
Monitoring components of Chaos Mesh by exposing /metrics HTTP endpoints with https://github.com/prometheus/client_golang

We already have several metrics in chaos-controller-manager and chaos-daemon, but it's not enough.

We want to describe our logic pattern and performance data by metrics, which includes:

chaos-controller-manager:

• time histogram for each Reconcile() in Reconciler
• count of Chaos Experiments, Schedule, and Workflow
• count of the emitted event by chaos-controller-manager
• common metrics of grpc-client
• metrics for kubernetes webhook

chaos-daemon:

• common metrics of grpc-server (already provided)
• count of processes controlled by bpm (background process manager)
• count of iptables/ipset/tc rules

chaos-dashboard:

• time histogram for each HTTP query
• count for archived object
• time histogram for archive reconciler

At last, we also want to provide a Grafana Dashboard to show these metrics.

Describe alternatives you've considered:
No alternatives yet.

Teachability, Documentation, Adoption, Migration Strategy:
Related skills: golang, prometheus, grafana

[CIPHERTron]

Heyy @STRRL I would love to work on this issue. 🚀

[STRRL]

Hi @CIPHERTron , this proposal will be available on LFX mentorship 2021 Q3 2021, so I could not just assign this for you right now. :P

Please take a look at the LFX mentorship! Looking forward to your participation!

[JuanPabloGarciaMonzon]

I have worked with Chaos Mesh in the last semester it's a really interesting technologie, it would be such a great experience to be able to work in this issue and to colaborate with Chaos Mesh.

[anonymousr007]

Hi @STRRL ,I want to work in this issue .

[STRRL]

closed with #2397