scraping node-exporter: "proto: can't skip unknown wire type 6 for io_prometheus_client.MetricFamily"

[discordianfish]

Hi,

I'm running prometheus with those dependencies/versions:

• prometheus: 034f6bb
• client_golang: 23e5e5fefd61ab1fce08080eb640956a70786856
• client_model: 775af9440e07a658e13f699448c745367bc12115

And 6 node exporters with those:

• node-exporter: 053db591224505c18b5b3cd996bb90ed40eaee4c
• client_golang: 457bc47eacdc954882e72bc81968061b5ca750a3
• client_model: 775af9440e07a658e13f699448c745367bc12115

For one node-exporter I get this error in prometheus after node-exporter runs some time: proto: can't skip unknown wire type 6 for io_prometheus_client.MetricFamily' from node-exporter.
This happens on only this one so far, even though they are docker containers and therefor the environment is the same for all of them. I'll update node-exporter so they all run the same client_golang version but if there is something relevent different I would expect all exporters to fail.

[beorn7]

Current theory is to blame
dfb9c2a

[discordianfish]

So I've downgraded prometheus to 0.6.0 (6b325bf) and after starting it again the error is gone. Let see if it reoccurs

[beorn7]

matttproud/golang_protobuf_extensions#4 should explain it all.
Once it is merged, please try again (or pull from my fork if you can't wait... :).

[discordianfish]

FYI: I ran 0.6.0 over the weekend and didn't reoccur.

[beorn7]

Cool. That was expected after matttproud/golang_protobuf_extensions#4 .
Just need to wait for Matt to merge the PR.

[matttproud]

If you can give those imperative tests in that pull req. another go per the
comments, I'll be in a good position to re-review the fundamental change.

2014-07-14 15:42 GMT+02:00 Björn Rabenstein notifications@github.com:

Closed #397 #397.

—
Reply to this email directly or view it on GitHub
#397 (comment).

Key: 0xC42234343FDE43A8
Fingerprint: 6945 9248 4362 68CB EA83 F1EA C422 3434 3FDE 43A8

[beorn7]

Sorry, we are in an offsite now, not much time.

And not really much time to add tests. Will comment on that PR later.

[garbamoussa]

Hi all,

I have integrated an ARIMA forecast into kapacitor by udf.

It starts and works fine, calculates forecast for each batch and writes back to influx. But always after 6 batches it stops writing, throwing error:

read error: proto: can’t skip unknown wire type 7 for agent.Response

Looking at the series that it should write, its not any different in batch 7. So i really have no idea.

Has anyone seen a similar error? Any idea how to start debugging?

Thanks a lot

[lock]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.