CA-396751: write updated RRDD data before headers #5915
Conversation
Ensure that the updated data and metadata are written before the headers are updated otherwise xcp-rrdd might start reading the data block before all the data is populated and thus run off the end of the data. Signed-off-by: Mark Syms <mark.syms@cloud.com>
| # First write the updated data and metadata | ||
| encoded_datasource_header = 'DATASOURCES'.encode() | ||
| # DATASOURCES + 20 for 32 + 32 + 32 + 64 | ||
| self.dest.seek(len(encoded_datasource_header) + 20) | ||
| for val in data_values: | ||
| # This is already big endian encoded | ||
| self.dest.write(val) |
There was a problem hiding this comment.
The reader can do its reading at any time? Is it reasonable to assume that this action of writing the the data/metadata is atomic?
There was a problem hiding this comment.
the reader should not read any more than the header unless and until the timestamp in the header changes from the last data read. Once that changes the data can be read.
There was a problem hiding this comment.
Header and data are protected by checksums. So if you read garbage, the checksum should tell you. You can't avoid reading garbage in the general case because there is always a race between the data and its checksum being written as no locking is implemented.
The total amount of data can change, and in particular shrink. I believe the memory-mapped file is of constant size though.
There was a problem hiding this comment.
Ok, I think at least it is better to write the data before writing the header, that sounds like a safer appraoch than the other way round
Ensure that the updated data and metadata are written before the headers are updated otherwise xcp-rrdd might start reading the data block before all the data is populated and thus run off the end of the data.