Prometheus Exporter

[davidnewhall]

Help Wanted (Updated 11/25/2019)

Okay, so I've added a /metrics endpoint to the poller, and I've exposed all of the metrics except UDM (I'll get there). I need help. I need people to test this code and let me know if the data you're looking for is formatted in a way that works for you. I've never used Prometheus, and I'm probably not about to start using it. I need those of you who do use it to try this out and provide feedback!

The prometheus-enabled version is most easily available through docker hub by pulling the prometheus label. Your other option is to check-out the dn2_prometheus branch and compile from source. If you need a package, find the beta release on the Releases page.

To enable prometheus set the mode in the config file to prometheus and point prometheus at port 9130. You can run this side-by-side with another copy doing influx; that's how I've been testing it locally.

You may provide feedback here or in the pull request. There are currently two dashboards available:

• Client Insights
• UAP Insights

Thank you!

Original Post Follows

Gauging interest.

If I added a /metrics endpoint to UniFi Poller for a Prometheus exporter, would anyone use it?

Looks like a large chunk of work, but shouldn't be any worse to maintain than the influx support once it's written.

[rknightion]

I definitely would!
At the moment the unifi poller is the only reason I have to run influx

[davidnewhall]

@rssfed23 Lets chat. Your use case is something I want to learn more about, so I know how to make the most impact. Searching github for "prometheus unifi" has quite a lot of hits and it seems other projects exist that do this already. My number one question to you is: Why not use one of those?

Can you jump into the Discord some time and hit me up? Thanks!

[Kramins]

I would also use it as a endpoint for influx v2 scraper, I'm sad to see the wontfix label

[davidnewhall]

What is an "endpoint for influx v2 scraper" ?

[Kramins]

Infuxdb v2 has the ability to remotely scrap an http endpoint for metric (Prometheus metric) data.

https://v2.docs.influxdata.com/v2.0/write-data/scrape-data/scrapable-endpoints/

[SuperQ]

When it comes to these kinds of tools, I ask a couple questions.

• Which one is the most feature complete.
• Which one has good active maintenance.
• Which one follows best practices most closely.

Sadly, the unifi_exporter was archived as the author wasn't able to keep up maintaining it.

That leaves this project as one of the few nice Go implementations for getting data out of Unifi. I also haven't been able to convince anyone at Ubiquiti to directly support the Prometheus API format, which would make things a whole lot easier.

So, while it's not best practice for Prometheus exporters to have their own internal loops, it might be worthwhile for this tool to have Prometheus format as an option.

[jmb12686]

Gauging interest.

If I added a /metrics endpoint to UniFi Poller for a Prometheus exporter, would anyone use it?

Looks like a large chunk of work, but shouldn't be any worse to maintain than the influx support once it's written.

I just discovered your Dashboards on Grafana.com, as I have setup an entire Prometheus monitoring stack for my container clusters. I would absolutely use the prometheus exporter, as I am hesitant to setup yet another component into my system (InfluxDB). Also, all my monitoring rules and alerting logic is within the Prometheus stack.

The prometheus stack is quite nice, with integrated solutions for alerting pipelines and first class Grafana support.

I would be willing to help test, but can't dedicate time for submitting any pull requests at the moment unfortunately.

[davidnewhall]

Thank you for the feedback everyone! There is a considerable amount of work required to support another backend schema, but I'll definitely look into it more. I may be able to do something to universally identify fields and tags, that can be translated into other time series databases. We shall see, but please do not hold your breath. This will take a while..

[davidnewhall]

I also haven't been able to convince anyone at Ubiquiti to directly support the Prometheus API format

After what I've been through trying to instrument prometheus in this app, I do not blame them.

I added preliminary support for Prometheus, and I'd like some volunteers to kick the tires. Jump on the discord and lets chat. https://discord.gg/KnyKYt2

[SuperQ]

Instrumenting for Prometheus is extremely easy.

[davidnewhall]

Instrumenting for Prometheus is extremely easy.

In Golang? The library has so many entry points, it's nearly impossible to know which ones to use without having prior understanding of the prometheus data model. It took me a minute to untangle. I still don't have any idea how it would work in other languages or as raw data output. I assume the UniFi controller is written in java, maybe it's easier there.

As someone who finds this easy, I'd really appreciate your feedback, review and ideals on the linked pull request. If you can test the updated code, it's available on docker hub with the tag prometheus. It currently contains enough code to export clients and USW data. More work is needed for UAP and USG. I may have USG implemented later today. UAP is a lot of work I'll get to eventually.

The pull request contains a lot of other cleanup stuff, but you can find the prometheus-specific code in the new promunifi package.

[SuperQ]

Yes, understanding a bit of the Prometheus data model is necessary. Most of the problems you're running into are much more easily handled when instrumenting the source code by the original authors. Writing exporters is more complicated because you're having to write a data translator, rather than just exposing the data correctly in the first place.

Looking at the PR, mostly promunifi/collector.go, it looks like it's mostly fine. There are a bunch of issues with the metric naming, because we have a specific best practices for these to make them more easily human readable. For example instead of Drops, it should probably be drops_total, as it's probably a counter. (This value is missing from return []*metricExports.

Also things like TxRate is marked as a counter, but is probably a gauge.

Another issue is you should drop the NewMetricWithTimestamp. It's unnecessary and problematic to expose timestamps from exporters. Everyone thinks they need this when first looking at Prometheus, but in reality you don't want to. Timestamps are only used for transferring data between time-series databases, not from exporters.

[davidnewhall]

Thanks for the feedback! The names that were there was just copy/paste from the struct to speed up my dev process. Thanks for pointing out how wrong they were.

TxRate was missed as a gauge, fixed now. Please let me know if you see any other counter/gauge mixups.

I removed the time stamp code entirely. I also removed IDS since there is no metric data there to put into prometheus.

Lastly, I did not understand what you put in parenthesis. The value is missing? Did I miss adding something? Please elaborate.

This commit addresses your feedback: cc66677 - and I've added 3 or more commits since this one adjusting the names and descriptions even further.

[SuperQ]

No worries, the code is mostly reasonable. I'm a bit busy with a conference this week, but I'll try and keep working on the review.

[davidnewhall]

Prometheus Support is here. I've ported some dashboards: https://grafana.com/grafana/dashboards/11315
https://grafana.com/grafana/dashboards/11314
looking for feedback!

[davidnewhall]

I'm done with the code for now. Prometheus support is working and two of five dashboards are available. I'll make revisions to them, probably with support from @SuperQ as well as get the other 3 created. This will take some time, but I'll get there. The dashboards will be followed by many documentation and wiki updates.

[davidnewhall]

Closing the loop on this. The majority of the bugs were worked out and v1.6 released with Prometheus support and all five dashboards.

https://github.com/davidnewhall/unifi-poller/releases/tag/v1.6.0