Discard 0-sized writes to files #7638
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
pchickey
approved these changes
Dec 5, 2023
github-merge-queue
bot
removed this pull request from the merge queue due to failed status checks
github-merge-queue
bot
removed this pull request from the merge queue due to failed status checks
alexcrichton
added a commit
to alexcrichton/system-interface
that referenced
this pull request
Jan 28, 2024
This commit it borne out of CI failures on bytecodealliance/wasmtime#7638. Investigating this failure has revealed a number of aspects here which I've attempted to address in this PR. The notable changes here are: * The current code in this crate was handling the case where `FileExt::seek_write` on Windows was leaving intermediate bytes as undefined when a write happened beyond the end of a file. I believe that this is due to an error in the documentation of the Rust standard library which I've submitted rust-lang/rust#120452 to fix. * Removing handling of "always write zeros" handles the primary failure of the PR bytecodealliance/wasmtime#7638 which is that `write_vectored_at` was always returning 0 on Windows for writes past the end of the file. This is because Windows doesn't have a vectored file write so the vector chosen was the first nonempty vector which was the one containing zeros to extend the file. That meant that the method always returned zero. * Previously the methods here used file locking which appeared to handle the case where the file was calculated and then the write happened. Given that this no longer happens I've removed the locking here. * The `write_all_at` method had a loop around `reopen_write` handling the `Interrupted` error but no other methods did, so I opted to remove the loop and leave that to the internals of `reopen_write` if necessary. * Other methods related to this are all simplified to directly use `seek_write` and avoid handling the case where writes past the end need to write zeros (as zeros are guaranteed by Windows). Overall my hope is to use this to unblock bytecodealliance/wasmtime#7638 to get more platform-agnostic behavior for writing beyond the end of a file.
|
Took me quite some time to set aside time to debug this failure on Windows, and it turned up bytecodealliance/system-interface#43 |
sunfishcode
pushed a commit
to bytecodealliance/system-interface
that referenced
this pull request
Feb 13, 2024
This commit it borne out of CI failures on bytecodealliance/wasmtime#7638. Investigating this failure has revealed a number of aspects here which I've attempted to address in this PR. The notable changes here are: * The current code in this crate was handling the case where `FileExt::seek_write` on Windows was leaving intermediate bytes as undefined when a write happened beyond the end of a file. I believe that this is due to an error in the documentation of the Rust standard library which I've submitted rust-lang/rust#120452 to fix. * Removing handling of "always write zeros" handles the primary failure of the PR bytecodealliance/wasmtime#7638 which is that `write_vectored_at` was always returning 0 on Windows for writes past the end of the file. This is because Windows doesn't have a vectored file write so the vector chosen was the first nonempty vector which was the one containing zeros to extend the file. That meant that the method always returned zero. * Previously the methods here used file locking which appeared to handle the case where the file was calculated and then the write happened. Given that this no longer happens I've removed the locking here. * The `write_all_at` method had a loop around `reopen_write` handling the `Interrupted` error but no other methods did, so I opted to remove the loop and leave that to the internals of `reopen_write` if necessary. * Other methods related to this are all simplified to directly use `seek_write` and avoid handling the case where writes past the end need to write zeros (as zeros are guaranteed by Windows). Overall my hope is to use this to unblock bytecodealliance/wasmtime#7638 to get more platform-agnostic behavior for writing beyond the end of a file.
alexcrichton
added a commit
to alexcrichton/wasmtime
that referenced
this pull request
Feb 13, 2024
This is to get bytecodealliance#7638 passing CI, but I wanted to separate this out from that to have it go through the queue in isolation in case any issues arise.
github-merge-queue bot
pushed a commit
that referenced
this pull request
Feb 13, 2024
This is to get #7638 passing CI, but I wanted to separate this out from that to have it go through the queue in isolation in case any issues arise.
This commit comes from bytecodealliance#7633 where Windows and Unix would behave differently when writing at a particular file offset. Notably Unix semantics [indicate]: > Before any action described below is taken, and if nbyte is zero > and the file is a regular file, the write() function may detect > and return errors as described below. In the absence of errors, > or if error detection is not performed, the write() function > shall return zero and have no other results. If nbyte is zero and > the file is not a regular file, the results are unspecified. These semantics are a bit easier to emulate on Windows so the host implementation now discards any attempt to perform I/O if a zero-sized write is detected. [indicate]: https://man7.org/linux/man-pages/man3/write.3p.html Closes bytecodealliance#7633
alexcrichton
force-pushed
the
fix-pwrite-behavior
branch
from
da98aa5
to
302a0ed
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This commit comes from #7633 where Windows and Unix would behave differently when writing at a particular file offset. Notably Unix semantics indicate:
These semantics are a bit easier to emulate on Windows so the host implementation now discards any attempt to perform I/O if a zero-sized write is detected.
Closes #7633