Release v0.27.8 #4942
Merged
Release v0.27.8 #4942
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Previously, an error in `git_config_next` would be mistaken as a successful load, because the previous call would have succeeded. Coverity saw the subsequent check for a completed iteration as dead, so let's make it useful again. CID 1391374
CID 1386176, 1386177, 1388219
CID 1395011
We have no need to take a non-const reference. This does involve some other work to make sure we don't mix const and non-const variables, but by splitting what we want each variable to do we can also simplify the logic for when we do want to free a new reference we might have allocated.
We want to allow the creation of annotated commits out of annotated tags and for that we have to peel the reference all the way to the commit instead of stopping at the first id it provides.
When git_filter_apply_fn callback returns a error while smudging proxy_stream_close ends up returning without closing the stream. This is turn makes blob_content_to_file crash as it asserts the stream being closed whether there are errors or not. Closing the target stream on error fixes this problem.
If the routine exits with error before stream or hash_ctx is initialized, the program will segfault when trying to free them.
When we try to add a configuration file with `git_config_add_file_ondisk`, we treat nonexisting files as empty. We do this by performing a stat call, ignoring ENOENT errors. This works just fine in case the file or any of its parents simply does not exist, but there is also the case where any of the parent directories is not a directory, but a file. So e.g. trying to add a configuration file "/dev/null/.gitconfig" will fail, as `errno` will be ENOTDIR instead of ENOENT. Catch ENOTDIR in addition to ENOENT to fix the issue. Add a test that verifies we are able to add configuration files with such an invalid path file just fine.
While commit objects usually should have only one author field, our commit parser actually handles the case where a commit has multiple author fields because some tools that exist in the wild actually write them. Detection of those additional author fields is done by using a simple `git__prefixcmp`, checking whether the current line starts with the string "author ". In case where we are handed a non-NUL-terminated string that ends directly after the space, though, we may have an out-of-bounds read of one byte when trying to compare the expected final NUL byte. Fix the issue by using `git__prefixncmp` instead of `git_prefixcmp`. Unfortunately, a test cannot be easily written to catch this case. While we could test the last error message and verify that it didn't in fact fail parsing a signature (because that would indicate that it has in fact tried to parse the additional "author " field, which it shouldn't be able to detect in the first place), this doesn't work as the next line needs to be the "committer" field, which would error out with the same error message even if we hadn't done an out-of-bounds read. As objects read from the object database are always NUL terminated, this issue cannot be triggered in normal code and thus it's not security critical.
Handle two null argument cases that occur in the unit tests. One is in library code, the other is in test code. Detected by running unit tests with undefined behavior sanitizer: ```bash # build mkdir build && cd build cmake -DBUILD_CLAR=ON -DCMAKE_C_FLAGS="-fsanitize=address \ -fsanitize=undefined -fstack-usage -static-libasan" .. cmake --build . # run with asan ASAN_OPTIONS="allocator_may_return_null=1" ./libgit2_clar ... ............../libgit2/src/apply.c:316:3: runtime error: null pointer \ passed as argument 1, which is declared to never be null ...................../libgit2/tests/apply/fromfile.c:46:3: runtime \ error: null pointer passed as argument 1, which is declared to never be null ```
When parsing a signature's timezone offset, we first check whether there is a timezone at all by verifying that there are still bytes left to read following the time itself. The check thus looks like `time_end + 1 < buffer_end`, which is actually correct in this case. After setting the timezone's start pointer to that location, we compute the remaining bytes by using the formula `buffer_end - tz_start + 1`, re-using the previous `time_end + 1`. But this is in fact missing the braces around `(tz_start + 1)`, thus leading to an overestimation of the remaining bytes by a length of two. In case of a non-NUL terminated buffer, this will result in an overflow. The function `git_signature__parse` is only used in two locations. First is `git_signature_from_buffer`, which only accepts a string without a length. The string thus necessarily has to be NUL terminated and cannot trigger the issue. The other function is `git_commit__parse_raw`, which can in fact trigger the error as it may receive non-NUL terminated commit data. But as objects read from the ODB are always NUL-terminated by us as a cautionary measure, it cannot trigger the issue either. In other words, this error does not have any impact on security.
|
I've added the backport-0.27.8 label to all the PRs that you have proposed in here. |
|
Additional proposals:
|
|
And #4670: Fix negative gitignore rules with leading directories |
ethomson
force-pushed
the
ethomson/v0.27.8
branch
2 times, most recently
from
23ebbc7
to
5646095
Compare
|
Oh, yeah. I didn't remember #4871 depending on the raw functions of #4374, but in fact they are only needed for the tests. I'm not feeling too comfortable with cherry-picking them from #4374, so I'd propose to skip d4ad658 and the newly introduced tree tests. I've pushed a branch "pks/v0.27.8-without-odb-raw" to my fork. Diff: |
ethomson
force-pushed
the
ethomson/v0.27.8
branch
from
5646095
to
e1f5813
Compare
The `git__strntol` family of functions accepts leading spaces and will simply skip them. The skipping will not honor the provided buffer's length, though, which may lead it to read outside of the provided buffer's bounds if it is not a simple NUL-terminated string. Furthermore, if leading space is trimmed, the function will further advance the pointer but not update the number of remaining bytes, which may also lead to out-of-bounds reads. Fix the issue by properly paying attention to the buffer length and updating it when stripping leading whitespace characters. Add a test that verifies that we won't read past the provided buffer length.
When parsing a tree entry's mode, we will eagerly parse until we hit a character that is not in the accepted set of octal digits '0' - '7'. If the provided buffer is not a NUL terminated one, we may thus read out-of-bounds. Fix the issue by passing the buffer length to `parse_mode` and paying attention to it. Note that this is not a vulnerability in our usual code paths, as all object data read from the ODB is NUL terminated.
The `git__strntol` family of functions has the ability to auto-detect a number's base if the string has either the common '0x' prefix for hexadecimal numbers or '0' prefix for octal numbers. The detection of such prefixes and following handling has two major issues though that are being fixed in one go now. - We do not do any bounds checking previous to verifying the '0x' base. While we do verify that there is at least one digit available previously, we fail to verify that there are two digits available and thus may do an out-of-bounds read when parsing this two-character-prefix. - When skipping the prefix of such numbers, we only update the pointer length without also updating the number of remaining bytes. Thus if we try to parse a number '0x1' of total length 3, we will first skip the first two bytes and then try to read 3 bytes starting at '1'. Fix both issues by disentangling the logic. Instead of doing the detection and skipping of such prefixes in one go, we will now first try to detect the base while also honoring how many bytes are left. Only if we have a valid base that is either 8 or 16 and have one of the known prefixes, we will now advance the pointer and update the remaining bytes in one step. Add some tests that verify that no out-of-bounds parsing happens and that autodetection works as advertised.
The `parse_mode` option uses an open-coded octal number parser. The parser is quite naive in that it simply parses until hitting a character that is not in the accepted range of '0' - '7', completely ignoring the fact that we can at most accept a 16 bit unsigned integer as filemode. If the filemode is bigger than UINT16_MAX, it will thus overflow and provide an invalid filemode for the object entry. Fix the issue by using `git__strntol32` instead and doing a bounds check. As this function already handles overflows, it neatly solves the problem. Note that previously, `parse_mode` was also skipping the character immediately after the filemode. In proper trees, this should be a simple space, but in fact the parser accepted any character and simply skipped over it. As a consequence of using `git__strntol32`, we now need to an explicit check for a trailing whitespace after having parsed the filemode. Because of the newly introduced error message, the test object::tree::parse::mode_doesnt_cause_oob_read needs adjustment to its error message check, which in fact is a good thing as it demonstrates that we now fail looking for the whitespace immediately following the filemode. Add a test that shows that we will fail to parse such invalid filemodes now.
When parsing a number, we accept a leading plus or minus sign to return a positive or negative number. When the parsed string has such a leading sign, we set up a flag indicating that the number is negative and advance the pointer to the next character in that string. This misses updating the number of bytes in the string, though, which is why the parser may later on do an out-of-bounds read. Fix the issue by correctly updating both the pointer and the number of remaining bytes. Furthermore, we need to check whether we actually have any bytes left after having advanced the pointer, as otherwise the auto-detection of the base may do an out-of-bonuds access. Add a test that detects the out-of-bound read. Note that this is not actually security critical. While there are a lot of places where the function is called, all of these places are guarded or irrelevant: - commit list: this operates on objects from the ODB, which are always NUL terminated any may thus not trigger the off-by-one OOB read. - config: the configuration is NUL terminated. - curl stream: user input is being parsed that is always NUL terminated - index: the index is read via `git_futils_readbuffer`, which always NUL terminates it. - loose objects: used to parse the length from the object's header. As we check previously that the buffer contains a NUL byte, this is safe. - rebase: this parses numbers from the rebase instruction sheet. As the rebase code uses `git_futils_readbuffer`, the buffer is always NUL terminated. - revparse: this parses a user provided buffer that is NUL terminated. - signature: this parser the header information of objects. As objects read from the ODB are always NUL terminated, this is a non-issue. The constructor `git_signature_from_buffer` does not accept a length parameter for the buffer, so the buffer needs to be NUL terminated, as well. - smart transport: the buffer that is parsed is NUL terminated - tree cache: this parses the tree cache from the index extension. The index itself is read via `git_futils_readbuffer`, which always NUL terminates it. - winhttp transport: user input is being parsed that is always NUL terminated
The function `parse_number` was replaced by `git_parse_advance_digit` which is provided by the parser interface in commit 252f2ee (parse: implement and use `git_parse_advance_digit`, 2017-07-14). As there are no remaining callers, remove it.
…files
When computing whether a file is ignored, we simply search for the first
matching rule and return whether it is a positive ignore rule (the file
is really ignored) or whether it is a negative ignore rule (the file is
being unignored). Each rule has a set of flags which are being passed to
`fnmatch`, depending on what kind of rule it is. E.g. in case it is a
negative ignore we add a flag `GIT_ATTR_FNMATCH_NEGATIVE`, in case it
contains a glob we set the `GIT_ATTR_FNMATCH_HASGLOB` flag.
One of these flags is the `GIT_ATTR_FNMATCH_LEADINGDIR` flag, which is
always set in case the pattern has a trailing "/*" or in case the
pattern is negative. The flag causes the `fnmatch` function to return a
match in case a string is a leading directory of another, e.g. "dir/"
matches "dir/foo/bar.c". In case of negative patterns, this is wrong in
certain cases.
Take the following simple example of a gitignore:
dir/
!dir/
The `LEADINGDIR` flag causes "!dir/" to match "dir/foo/bar.c", and we
correctly unignore the directory. But take this example:
*.test
!dir/*
We expect everything in "dir/" to be unignored, but e.g. a file in a
subdirectory of dir should be ignored, as the "*" does not cross
directory hierarchies. With `LEADINGDIR`, though, we would just see that
"dir/" matches and return that the file is unignored, even if it is
contained in a subdirectory. Instead, we want to ignore leading
directories here and check "*.test". Afterwards, we have to iterate up
to the parent directory and do the same checks.
To fix the issue, disallow matching against leading directories in
gitignore files. This can be trivially done by just adding the
`GIT_ATTR_FNMATCH_NOLEADINGDIR` to the spec passed to
`git_attr_fnmatch__parse`. Due to a bug in that function, though, this
flag is being ignored for negative patterns, which is fixed in this
commit, as well. As a last fix, we need to ignore rules that are
supposed to match a directory when our path itself is a file.
All together, these changes fix the described error case.
When checking whether a rule negates another rule, we were checking whether a rule had the `GIT_ATTR_FNMATCH_LEADINGDIR` flag set and, if so, added a "/*" to its end before passing it to `fnmatch`. Our code now sets `GIT_ATTR_FNMATCH_NOLEADINGDIR`, thus the `LEADINGDIR` flag shall never be set. Furthermore, due to the `NOLEADINGDIR` flag, trailing globs do not get consumed by our ignore parser anymore. Clean up code by just dropping this now useless logic.
ethomson
force-pushed
the
ethomson/v0.27.8
branch
from
e1f5813
to
703885a
Compare
|
Hmm, I don't see pks/v0.27.8-without-odb-raw, but I've redone this without d4ad658 and the tests. LMK if I missed something. |
|
On Fri, Jan 25, 2019 at 09:49:43AM -0800, Edward Thomson wrote:
Hmm, I don't see [pks/v0.27.8-without-odb-raw](https://github.com/pks-t/libgit2/tree/pks/v0.27.8-without-odb-raw), but I've redone this without d4ad658 and the tests. LMK if I missed something.
Weird, I definitely remember pushing it.
Anyway, except for one typo fix you did in "CHANGELOG.md",
703885a (version: bump to v0.27.8, 2019-01-18) looks exactly
the same as my local branch does.
|
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Let's release v0.27.8! I've backported a bunch of fixes.