-
-
Notifications
You must be signed in to change notification settings - Fork 183
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Normalize unicode internally using NFD
Previously, the path reservation system, which defends against unicode path name collisions (the subject of a handful of past CVE issues), was using NFKD normalization internally to determine of two paths would be likely to reference the same file on disk. This has the weird effect of normalizing things like `℀` into simple decomposed character strings, for example `a/c`. These can contain slashes and double-dot sections, which means that the path reservations may end up reserving more (or different) paths than intended. Thankfully, tar was already *extracting* properly, even if the path reservations collided, and these collisions resulted in tar being *more* aggressive than it should be in restricting parallel extraction, rather than less. That's a good direction to err in, for security, but also, made tar less efficient than it could be in some edge cases. Using NFD normalization, unicode characters are not decomposed in compatibility mode, but still result in matching path reservation keys as intended. This does not cause any change in observed behavior, other than allowing some files to be extracted in parallel where it is provably safe to do so. Credit: discovered by @Sim4n6. This did not result in a juicy security vulnerability, but it sure looked like one at first. They were extremely patient, thorough, and persistent in trying to pin this down to a POC and CVE. There is very little reward or visibility when a security researcher finds a bug that doesn't result in a security disclosure, but the attempt often results in improvements to the project.
- Loading branch information
Showing
5 changed files
with
60 additions
and
3 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,30 @@ | ||
/* IMPORTANT | ||
* This snapshot file is auto-generated, but designed for humans. | ||
* It should be checked into source control and tracked carefully. | ||
* Re-generate by setting TAP_SNAPSHOT=1 and running tests. | ||
* Make sure to inspect the output below. Do not ignore changes! | ||
*/ | ||
'use strict' | ||
exports[`test/normalize-unicode.js TAP normalize with strip slashes "1/4foo.txt" > normalized 1`] = ` | ||
1/4foo.txt | ||
` | ||
|
||
exports[`test/normalize-unicode.js TAP normalize with strip slashes "\\\\a\\\\b\\\\c\\\\d\\\\" > normalized 1`] = ` | ||
/a/b/c/d | ||
` | ||
|
||
exports[`test/normalize-unicode.js TAP normalize with strip slashes "¼foo.txt" > normalized 1`] = ` | ||
¼foo.txt | ||
` | ||
|
||
exports[`test/normalize-unicode.js TAP normalize with strip slashes "﹨aaaa﹨dddd﹨" > normalized 1`] = ` | ||
﹨aaaa﹨dddd﹨ | ||
` | ||
|
||
exports[`test/normalize-unicode.js TAP normalize with strip slashes "\bbb\eee\" > normalized 1`] = ` | ||
\bbb\eee\ | ||
` | ||
|
||
exports[`test/normalize-unicode.js TAP normalize with strip slashes "\\\\\eee\\\\\\" > normalized 1`] = ` | ||
\\\\\eee\\\\\\ | ||
` |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters