Prometheus 2 yields "no token found" when parsing Quobyte exports

[quolix]

Quobyte has its own metric system with a Prometheus export in every service. We use the Protocol Buffer format, which worked well with Prometheus <2.0.

All tested Prometheus 2 builds (beta+rc) yield a "no token found" error however and the following log output (prometheus-2.0.0-rc.2.linux-amd64):

level=warn ts=2017-11-02T09:27:49.860892044Z caller=scrape.go:673 component="target manager" scrape_pool=quobyte_consul target=http://10.10.2.83:55001/prometheus msg="append failed" err="no token found"

Here are the HTTP headers:

$ wget -S http://10.10.1.21:21001/prometheus
--2017-11-02 10:29:04--  http://10.10.1.21:21001/prometheus
Connecting to 10.10.1.21:21001... connected.
HTTP request sent, awaiting response... 
  HTTP/1.1 200 OK
  Date: Thu, 02 Nov 2017 09:29:04 GMT
  Content-type: application/vnd.google.protobuf; proto=io.prometheus.client.MetricFamily; encoding="delimited"
  Content-length: 209377
Length: 209377 (204K) [application/vnd.google.protobuf]

I tried both chunked and non-chunked encoding, adding and removing " in the values of the Content-type, none of that helped.

Output ist written with:

CodedOutputStream outputstream = CodedOutputStream.newInstance(bos);
... loop over Metrics and do:
outputstream.writeInt32NoTag(data.getSerializedSize());
metricFamily.writeTo(outputstream);
outputstream.flush();

Can you see if this is a response header problem or a problem in the response body? Is there any way to get better error information from Prometheus?

[gouthamve]

Hi, we are tracking protobuf support for Prom 2.0 here: #2788
Any chance you can move over to the text format? If you have good reasons to stick to protobuf, could you detail them in the issue linked, it will help us evaluate if we should be supporting protobuf in 2.0.

[quolix]

Ok, we'll add text format.

[lock]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.