About prometheus deploy large-scale deployment problems

[xiaoqge]

I can not understand prometheus pull this way, I have nearly 2,000 servers, some single server running the application 2-3, if I use open source prometheus CLIENT EXPORTERS need to start 2-3 on a single server a client exportes. There is no other way? What if I also need to monitor CPU memory disk on the server, and also requires DJANGO on the Monitoring Server and MYSQL I'll need to download the client exports three applications, so deploying it is not too much trouble?

[brian-brazil]

A single beefy Prometheus server can likely handle that load though splitting into several Prometheus servers would be better (presuming that's 2k servers in one datacenter). See http://www.robustperception.io/scaling-and-federating-prometheus/ Assuming you have good infrastructure, none of this should present a major challenge.

[xiaoqge]

@brian-brazil Thank you, I mean, the server needs to be monitored simultaneously monitor mysql redis and also need to monitor CPU, memory, disk, if I use an existing (https://prometheus.io/docs/instrumenting/exporters/) EXPORTERS means I need to start being a lot of listening ports on this server, unless I have to write an integrated mysql and redis, disk, cpu, memory exportes. But if one day my server is monitored deleted MYSQL, do I also need to edit your client exportes code? Is there a better way? please help me

[brian-brazil]

This is what service discovery is for (https://prometheus.io/blog/2015/06/01/advanced-service-discovery/), when a given host no longer has a given service then Prometheus would no longer monitor it. Your configuration management can add/remove exporters as part of adding/removing the associated service.

[xiaoqge]

@brian-brazil Thank you very much, there is a problem, why prometheus the alert rule are not stored inside the DB, similar ZABBIX can set the alarm go rules WEB UI. What put this document is designed to do?

[brian-brazil]

Our paradigm is that all configuration is provided via files. If there was some form of WebUI we'd need to be able to sync with replicas, integrate with configuration management, offer an API etc. - not trivial problems. Instead we leave it up to the user how they manage the configuration.

[xiaoqge]

@brian-brazil thank you! Problems Close

[lock]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.