Prom2: Protobuf Exposition Format Support?

[snarlysodboxer]

There seems to be some question as to whether the Protobuf Exposition Format will be supported in Prometheus 2.0. If it's not going to be supported, we assume we should stop developing Prometheus 1.X Protobuf code in all client libraries.

Thanks!

[fabxc]

The current tendency (but no final call) is that text format works really well and allows us to do some fancy ingestion optimisations.
Currently Prometheus 2.0 cannot ingest protobuf anymore. Do you have a an application or exporter that can only export the protobuf format?

@beorn7 @brian-brazil @juliusv

[brian-brazil]

I am aware of one (internal) application that's proto only, but once client_golang supports timestamps I presume it'll switch over.

[beorn7]

I have a number of concerns about dropping protobuf support, beyond the question if client libraries support the text format or not. I'm not sure if a GH issue is the right place to make the final call. Perhaps open a shared Google doc and develop that into a design-doc/RFC? @fabxc It would be great if you could document the ingestion optimizations there.

[kongchen]

Some of our service's metrics size reaches 10MB in text format and we're currently moving to protocol buffer format, cannot imaging pb is not supported in future.

[fabxc]

@kongchen from what we have gathered so far, the wire size does not substantially differ between protobuf and text format+gzip. Have you compared the two in terms of size and latency?

[beorn7]

In fact, gzip'd text format is smaller than uncompressed proto format. However, usually both are compressed, and then the size is really not much different.

Things changes in scenarios where compression is not an option. With the dramatically increased ingestion efficiency of Prom2, I can imagine that the cost of compression becomes relevant even on the side of the Prometheus server. (On the side of the monitored target, it's even more likely that compression is a relevant cost.)

[kongchen]

@fabxc @beorn7 Thanks, we can use gzip for compression by enable server side gzip(tomcat actually).

However, now the only concern is that will this lead Prometheus server to a high cpu usage? After all we have thousands of nodes for the server to pull.

[beorn7]

Prometheus 2 uses dramatically less CPU for ingestion anyway. The CPU you need for compression is very small compared to that.

(What I meant above is that the CPU cycles needed for compression only become relevant because Prom2 uses so little CPU for the rest of the ingestion.)

[kongchen]

Thanks, look forward to it!

[SimenB]

Speaking of prometheus v2, is there an issue/document with what it'll contain/change? I see the milestone (https://github.com/prometheus/prometheus/milestone/4) but no general issue 🙂

[juliusv]

@SimenB Let's keep this issue on topic (please ask general questions on the mailing lists or chat), but https://prometheus.io/blog/2017/06/21/prometheus-20-alpha3-new-rule-format/ covers some of that.

[SimenB]

Now that Prometheus 2 is in RC, should we conclude that protobuf support is out?

[beorn7]

Not sure if there was a final call, but at another occasion, I wrote something down about text vs. protobuf: https://github.com/RichiH/OpenMetrics/blob/master/protobuf_vs_text.md

[zegelin]

Can the documentation be updated to reflect the decision to remove protobuf support?

Currently https://prometheus.io/docs/instrumenting/exposition_formats/ lists protobuf as "Supported in | Prometheus version >=0.4.0", yet version 2.1.0 doesn't support protobuf.

A custom exporter application I wrote used the protobuf format. After I upgraded to 2.1.0 I was wondering why I was getting errors err="no token found" in the logs. I had to do a bit of digging before I realised the documentation is out of date.

[juliusv]

@zegelin Good point, see prometheus/docs#972

[krasi-georgiev]

@brian-brazil I belive this can also be closed

[lock]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.