New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
rbd: introduce v2 format for rbd export/import. #10487
Changes from all commits
636d7cd
e10b0c0
4658cf1
12f50c4
262c8d3
f6192de
21be324
de671d7
bc80b14
58c8e8c
5e03460
26b858e
55edd81
8679521
912ff4d
d7479a0
4ee764f
f16899a
08d0c2a
435104a
c2356be
8d2a613
aa913ce
fedf772
5b14582
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -60,6 +60,83 @@ Zero data | |
- le64: length | ||
|
||
|
||
Final Record | ||
~~~~~~~~~~~~ | ||
|
||
End | ||
--- | ||
|
||
- u8: 'e' | ||
|
||
|
||
Header | ||
~~~~~~ | ||
|
||
"rbd diff v2\\n" | ||
|
||
Metadata records | ||
~~~~~~~~~~~~~~~~ | ||
|
||
Every record has a one byte "tag" that identifies the record type, | ||
followed by length of data, and then some other data. | ||
|
||
Metadata records come in the first part of the image. Order is not | ||
important, as long as all the metadata records come before the data | ||
records. | ||
|
||
In v2, we have the following metadata in each section: | ||
(1 Bytes) tag. | ||
(8 Bytes) length. | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I think it's safe to use a 32bit length -- the only thing that requires 64bit is the write/zero tag, but since the maximum object size is 32M, they technically only require 32bit lengths as well (and should be updated to reflect that). There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Yes, but what about keeping 8Bytes here in case of we want to break the limit of 32M someday, and 4Bytes for each tag sounds not a big waste for an output file of a rbd export-diff. |
||
(n Bytes) data. | ||
|
||
In this way, we can skip the unrecognized tag. | ||
|
||
From snap | ||
--------- | ||
|
||
- u8: 'f' | ||
- le64: length of appending data (4 + length) | ||
- le32: snap name length | ||
- snap name | ||
|
||
To snap | ||
------- | ||
|
||
- u8: 't' | ||
- le64: length of appending data (4 + length) | ||
- le32: snap name length | ||
- snap name | ||
|
||
Size | ||
---- | ||
|
||
- u8: 's' | ||
- le64: length of appending data (8) | ||
- le64: (ending) image size | ||
|
||
Data Records | ||
~~~~~~~~~~~~ | ||
|
||
These records come in the second part of the sequence. | ||
|
||
Updated data | ||
------------ | ||
|
||
- u8: 'w' | ||
- le64: length of appending data (8 + 8 + length) | ||
- le64: offset | ||
- le64: length | ||
- length bytes of actual data | ||
|
||
Zero data | ||
--------- | ||
|
||
- u8: 'z' | ||
- le64: length of appending data (8 + 8) | ||
- le64: offset | ||
- le64: length | ||
|
||
|
||
Final Record | ||
~~~~~~~~~~~~ | ||
|
||
|
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,84 @@ | ||
RBD Export & Import | ||
=================== | ||
|
||
This is a file format of an RBD image or snapshot. It's a sparse format | ||
for the full image. There are three recording sections in the file. | ||
|
||
(1) Header. | ||
(2) Metadata. | ||
(3) Diffs. | ||
|
||
Header | ||
~~~~~~ | ||
|
||
"rbd image v2\\n" | ||
|
||
Metadata records | ||
~~~~~~~~~~~~~~~~ | ||
|
||
Every record has a one byte "tag" that identifies the record type, | ||
followed by length of data, and then some other data. | ||
|
||
Metadata records come in the first part of the image. Order is not | ||
important, as long as all the metadata records come before the data | ||
records. | ||
|
||
In v2, we have the following metadata in each section: | ||
(1 Bytes) tag. | ||
(8 Bytes) length. | ||
(n Bytes) data. | ||
|
||
In this way, we can skip the unrecognized tag. | ||
|
||
Image order | ||
----------- | ||
|
||
- u8: 'O' | ||
- le64: length of appending data (8) | ||
- le64: image order | ||
|
||
Image format | ||
------------ | ||
|
||
- u8: 'F' | ||
- le64: length of appending data (8) | ||
- le64: image format | ||
|
||
Image Features | ||
-------------- | ||
|
||
- u8: 'T' | ||
- le64: length of appending data (8) | ||
- le64: image features | ||
|
||
Image Stripe unit | ||
----------------- | ||
|
||
- u8: 'U' | ||
- le64: length of appending data (8) | ||
- le64: image striping unit | ||
|
||
Image Stripe count | ||
------------------ | ||
|
||
- u8: 'C' | ||
- le64: length of appending data (8) | ||
- le64: image striping count | ||
|
||
Final Record | ||
~~~~~~~~~~~~ | ||
|
||
End | ||
--- | ||
|
||
- u8: 'E' | ||
|
||
|
||
Diffs records | ||
~~~~~~~~~~~~~~~~~ | ||
Record the all snapshots and the HEAD in this section. | ||
|
||
- le64: number of diffs | ||
- Diffs ... | ||
|
||
Detail please refer to rbd-diff.rst |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It also would be good to add a 1-byte tag struct version and 1-byte tag struct minimum compatibility version with the new length field (i.e. we might want to add support for recording snapshot timestamps in the "to snap tag" or CRCs to the data tag, but we shouldn't need to create a new tags to handle it).
Basically, you can re-use the existing ENCODE_START/ENCODE_FINISH methods
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I would like to make every tag as simple as possible. Then I think we can introduce a new tags for snap_timestamp and CRC in diffs. This way, we can also keep the backward compatibility, we only need to set the timestamp when we found a tag for it.