Skip to content

draft_protocol multifetch

dipohl edited this page Mar 30, 2018 · 2 revisions

RFC multifetch protocol

Content from Munin Trac Wiki. Was written by snide between 2010-06-16 and 2010-06-23.

This text follows RFC-like language. Refer to http://rfc.net/rfc2119.html.

'''Design goals:'''

  • Simple ability for plugins to send sub-5min sampled values
  • Storing protocol for [wiki:spec-spooldir spoolfetch repository]
  • Simplicity & transparency: storing by append-only

Existing model

Actually each plugin gives only one value.

Advantages

  • Lightweight plugin
  • Ability to have a streaming plugin

Disadvantages

  • For very high sampling rates (<< 1m) the overhead is noticable.

Proposed new model

A plugin MAY just send many timed-values at once. It MUST then prefix the values with an epoch. This format is mostly used for an efficient spoolfetch storage : the format is plain text and append-only. The different values MAY be separated with a blank line for easier parsing.

The epoch MUST be only increasing.

Streaming plugins

This enables having a streaming plugin : it never ends when asked for values.

It would be very useful for the cpu plugin that would open a streaming vmstat, massage its output, and restream the output. The advantage would be to have a very little overhead and then avoiding regular monitoring spikes.

Protocol example

field1.value 1234567890:1
field2.value 1234567890:2
field3.value 1234567890:3

field1.value 1234567891:10
field2.value 1234567891:20
field3.value 1234567891:30

field1.value 1234567900:10
field2.value 1234567900:60
field3.value 1234567900:7.3423