[Proposal]: collect kubernetes events, and provide related alert rules

[knightXun]

For now, promethues can use events to discover node, pod ,endpoints,ingress, then add the target into the monitor list.I plan to make prometheus walk further that collect kubernetes events then add the related rules to alert.
I notice heapster can collect these events,but it should config some place to sink, for example elasticsearch, then use some trick to alert them for users. I don't think this kind of way is good. I think prometheus can collect them and sink them to the tsdb,send alert to the alertmanager. In this way, we not only can effectively monitor some bad situations such as NodeNotReady and NodeReboots, but also let this events be stored in a reliable places.

I made it in my local repository, If needed,I can finish this feature as soon as possible.

[knightXun]

/cc @gouthamve @beorn7 @brian-brazil

[gouthamve]

Hi, have you taken a look at https://github.com/kubernetes/kube-state-metrics? We essentially alert on those situations using the above.

Having said that what you need is an exporter like kube-state-metrics to do the metric exposing which prometheus can scrape to ingest data.

I'm closing this as a usage question. To make your question, and all replies, easier to find, we suggest you move this over to our user mailing list, which you can also search. If you prefer more interactive help, join or our IRC channel, #prometheus on irc.freenode.net. Please be aware that our IRC channel has no logs, is not searchable, and that people might not answer quickly if they are busy or asleep. If in doubt, you should choose the mailing list.

Once your questions have been answered, please add a short line pointing to relevant replies in case anyone stumbles here via a search engine in the future.

[lock]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.