New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fix CLI Handling of Permissions and Ownership (Again) #3432
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
One fewer `stat()` call to make per operation!
Avoids a race condition in which we unintentionally open up permissions to the wrong group.
Cyan4973
reviewed
Jan 18, 2023
@@ -185,7 +185,7 @@ int UTIL_isRegularFileStat(const stat_t* statbuf) | |||
int UTIL_chmod(char const* filename, const stat_t* statbuf, mode_t permissions) | |||
{ | |||
stat_t localStatBuf; | |||
UTIL_TRACE_CALL("UTIL_chmod(%s, %u)", filename, (unsigned)permissions); | |||
UTIL_TRACE_CALL("UTIL_chmod(%s, %#4o)", filename, (unsigned)permissions); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
First time I see this formating instruction...
Cyan4973
approved these changes
Jan 18, 2023
Merged
dongsupark
added a commit
to flatcar/scripts
that referenced
this pull request
Aug 11, 2023
Since #950 was merged, tarball files `flatcar-{packages,sdk}-*.tar.zst` have been created with mode 0600 instead of 0644. As a result, the files with mode 0600 were uploaded to bincache, but afterwards `copy-to-origin.sh` that in turn runs rsync from bincache to the origin server could not read the tarballs. To fix that, it is necessary to chmod from 0600 to 0644 to make it readable by rsync during the release process. All of that happens because zstd sets the mode of the output file to 0600 in case of temporary files to avoid race condition. See also facebook/zstd#1644, facebook/zstd#3432.
dongsupark
added a commit
to flatcar/scripts
that referenced
this pull request
Aug 11, 2023
Since #950 was merged, tarball files `flatcar-{packages,sdk}-*.tar.zst` have been created with mode 0600 instead of 0644. As a result, the files with mode 0600 were uploaded to bincache, but afterwards `copy-to-origin.sh` that in turn runs rsync from bincache to the origin server could not read the tarballs. To fix that, it is necessary to chmod from 0600 to 0644 to make it readable by rsync during the release process. All of that happens because zstd sets the mode of the output file to 0600 in case of temporary files to avoid race condition. See also facebook/zstd#1644, facebook/zstd#3432.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR switches the
zstd
CLI's handling of output files' ownership and permissions back (again). This PR bringszstd
's behavior back into alignment withgzip
's, without introducing any gaps where output files have incorrectly loose permissions.This PR addresses #2946. Previous issues and PRs on this topic include #2739, #2525, #2491 and #2495, and #1630 and #1644.
When the input is not a regular file, we create the output file with default
0666
permissions (modulated by theumask
), and then don't change anything at the end of the operation.When the input is a regular file, we create it with
0600
(u+rw) permissions, and then copy the input's ownership, permissions, and timestamps to the output file at the end of the operation. E.g.:We also leverage a trick found in
gzip
to avoid a different permissions/ownership race by setting the correct group, then permissions, then user. Otherwise there would be a window of time in which incorrectly lax permissions might be exposed to the wrong user or group. (It's not meaningful for there to be incorrectly lax user permissions exposed to the user that zstd is running as / the file is created as, since an attacker running as that user would already be able tochmod()
the file to make it user r/w/x-able. But without this 3-step process, it would be possible for incorrectly lax permissions to be applied to the group the file is created as, or the user the file ends up being owned by.)Finally, this PR refactors how we stat files slightly, to reduce the count of
stat()
calls we make. A further improvement might be to switch to doingfstat()
calls.Adding testing for this PR is a challenge. We have reasonably good coverage of checking that timestamps and mode bits are copied over. But it's difficult to extend that to testing that the user and group are copied over correctly. (1) We can't rely on particular other users/groups existing across all the platforms on which we want to test. (2) Zstd will mostly fail to
chown()
the output file anyways:I have locally verified that it does correctly propagate ownership when invoked with
sudo
.