Design review for "SR-Level RRDs"

[robhoes]

The design is here: http://xapi-project.github.io/rrdd/futures/sr-level-rrds.html

[djs55]

Re: size of the VDI: if you request the space you need (e.g. 20 bytes) then the SR should round this up to the nearest possible unit. On LVM this might be 4MiB, with raw files this might be 20 bytes. So I think you just need to ask for what you need.

Could you define the SR capability? It would be good to have a name written down.

[robhoes]

Thanks djs55. I'll also make a note about the binary format for the VDI being tar, as you suggested.

[jonludlam]

It might be tricky to request the space up front if we want to support non-default data sources - enabling them would increase the size.

[jonludlam]

also, thinking about it, since we're storing xml with floats in, the size will vary anyway.

[robhoes]

I've updated the doc now.

Regarding the space, I wrote: "The VDI will be 4MB in size. This is a little more space than we would need for the RRDs we have in mind at the moment, but will give us enough headroom for the foreseeable future." Does that work for both of you?

[robhoes]

Jon says that tar is not needed, because there is only ever going to be one RRD archive file, which we can copy straight onto the VDI. I'll fix that in v3.

[johnelse]

What happens if the host dies or the connection to the storage is broken while the archive is being written? Do we need some kind of double-buffering like we have with the redo log?

[jonludlam]

We might actually want the tar back - the thing is we'll need some sort of framing for the data as it won't be 4MiB precisely and we don't want to copy in/out 4MiB speculatively. Tar might work quite nicely as a framing format.

[johnelse]

After discussion with @robhoes I've done some rework of the semantics of the rrd_updates handler (#161). Rob, could you check that I haven't missed anything?

[robhoes]

@johnelse Looks good; I've merged the PR.

[johnelse]

Made another rework of the rrd_updates API after discussion with the XenCenter team (#162)