unexpected end of data error with metrics in v2.6.0

[PsychoSid]

Got an issue with our custom exporter in Prometheus 2.6 (worked in 2.5 and below).

We are now getting-
msg="append failed" err="unexpected end of data, got "EOF""

Errors - it's not a timeout as it's well below the duration/timeout values.

Curl'ing the data looks good and piping through "promtool check metrics" complains about the lack of help text but that is all.

The only thing I can possible see it that some parts have spaces in (some don't) for example:-

msg_queues_is_durable{server="myserver",vpn="#config-sync",queue="#CFGSYNC/OWNER/ONE/VPN/TWO VPNNAME/CFG"} 1.0

But this has never been an issue before, this from the openmetricsparse_test.go:-

func TestOpenMetricsParseErrors(t *testing.T) {

  | cases := []struct {
  | input string
  | err string
  | }{
  | {
  | input: "",
  | err: "unexpected end of data, got "EOF"",
  | },

[PsychoSid]

It was the encoding. It's not gzip and the type wasn't coming in as text/plain but text/html.

I think this commit was the one that changed the behaviour.

https://github.com/prometheus/prometheus/pull/4912/files

The error message was a little odd. But makes sense ultimately.

Thanks again to Brian for pointing me in the right direction on this. Using https and the endpoint on k8s and going through a few layers made it fun :)