Rough architecture outline

[zpratt]

Event types

• deployments
• alert manager triggers
  • these can come from all sorts of rules, such as:
    • production issues: app in a crash loop, dependency unavailable, etc
    • security issues, using prisma's prometheus integration
• using a GitHub webhook to observe pushes to the designated deployment branch of the repo being tracked (starts the lead time clock)
  * this will allow customers to track MTTU for security issues

Aggregation

Prometheus would then be deployed to each cluster and used as the central hub for all metrics. There will need to be some kind of event correlation to be used as well.

Thanos can then be used to aggregate metrics across clusters/environments.

Grafana can then be used to visualize the data.