-
Notifications
You must be signed in to change notification settings - Fork 283
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
CA-396751: write updated RRDD data before headers #5915
Conversation
Ensure that the updated data and metadata are written before the headers are updated otherwise xcp-rrdd might start reading the data block before all the data is populated and thus run off the end of the data. Signed-off-by: Mark Syms <mark.syms@cloud.com>
# First write the updated data and metadata | ||
encoded_datasource_header = 'DATASOURCES'.encode() | ||
# DATASOURCES + 20 for 32 + 32 + 32 + 64 | ||
self.dest.seek(len(encoded_datasource_header) + 20) | ||
for val in data_values: | ||
# This is already big endian encoded | ||
self.dest.write(val) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The reader can do its reading at any time? Is it reasonable to assume that this action of writing the the data/metadata is atomic?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
the reader should not read any more than the header unless and until the timestamp
in the header changes from the last data read. Once that changes the data can be read.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Header and data are protected by checksums. So if you read garbage, the checksum should tell you. You can't avoid reading garbage in the general case because there is always a race between the data and its checksum being written as no locking is implemented.
The total amount of data can change, and in particular shrink. I believe the memory-mapped file is of constant size though.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ok, I think at least it is better to write the data before writing the header, that sounds like a safer appraoch than the other way round
Ensure that the updated data and metadata are written before the headers are updated otherwise xcp-rrdd might start reading the data block before all the data is populated and thus run off the end of the data.