feat(storage): add option to disable checksums and improve robustness of full_object_checksum validation#17665
Conversation
|
Warning Gemini encountered an error creating the review. You can try again by commenting |
|
review / submit after #17658 |
…le-checksum-robustness with refactor
No region tags are edited in this PR.This comment is generated by snippet-bot.
|
|
/gemini review |
There was a problem hiding this comment.
Code Review
This pull request introduces an enable_checksum parameter to control CRC32C checksum calculations during async object writes, adds validation for full_object_checksum in finalize(), and ensures proper stream cleanup. Key feedback includes removing undocumented parameters from the download_ranges docstring, wrapping both send() and recv() in the try...finally block for robust cleanup, and explicitly rejecting boolean values in the full_object_checksum type check along with adding a corresponding unit test.
This PR introduces performance options and robustness improvements to the asynchronous append operations in
AsyncAppendableObjectWriter.Key Changes:
Disable Checksums for Performance:
enable_checksumoptional parameter (default:True) toAsyncAppendableObjectWriter.append().False, the writer skips calculation of chunk-level CRC32C checksums in_WriteResumptionStrategy, improving append throughput.Robustness of Finalization Checksum Validation:
isinstance(int)) and range checking ([0, 2**32-1]) forfull_object_checksuminfinalize().try...finallyblock. This guarantees that the underlying stream is closed and local writer state variables are reset on mismatch exceptions/errors, preventing stream leaks.if-elif-elseformat.