libgit2/libgit2

This is a security release fixing out-of-bounds reads when
processing smart-protocol "ng" packets.

When parsing an "ng" packet, we keep track of both the current position
as well as the remaining length of the packet itself. But instead of
taking care not to exceed the length, we pass the current pointer's
position to strchr, which will search for a certain character until
hitting NUL. It is thus possible to create a crafted packet which
doesn't contain a NUL byte to trigger an out-of-bounds read.

The issue was discovered by the oss-fuzz project, issue 9406.

This is a security release fixing out-of-bounds reads when reading objects from a packfile. This corresponds to CVE-2018-10887 and CVE-2018-10888, which were both reported by Riccardo Schirone.

When packing objects into a single so-called packfile, objects may not get stored as complete copies but instead as deltas against another object "base". A specially crafted delta object could trigger an integer overflow and thus bypass our input validation, which may result in copying memory before or after the base object into the final deflated object. This may lead to objects containing copies of system memory being written into the object database. As the hash of those objects cannot be easily controlled by the attacker, it is unlikely that any of those objects will be valid and referenced by the commit graph.

Note that the error could also be triggered by the function git_apply__patch. But as this function is not in use outside of our test suite, it is not a possible attack vector.

This is a security release fixing out-of-bounds reads when reading objects from a packfile. This corresponds to CVE-2018-10887 and CVE-2018-10888, which were both reported by Riccardo Schirone.

When packing objects into a single so-called packfile, objects may not get stored as complete copies but instead as deltas against another object "base". A specially crafted delta object could trigger an integer overflow and thus bypass our input validation, which may result in copying memory before or after the base object into the final deflated object. This may lead to objects containing copies of system memory being written into the object database. As the hash of those objects cannot be easily controlled by the attacker, it is unlikely that any of those objects will be valid and referenced by the commit graph.

Note that the error could also be triggered by the function git_apply__patch. But as this function is not in use outside of our test suite, it is not a possible attack vector.

This is a bugfix release. It includes the following non-exclusive list of
improvements, which have been backported from the master branch:

• Fix builds with LibreSSL 2.7.

• Fix for git_diff_status_char() not returning the correct mapping for
  GIT_DELTA_TYPECHANGE.

• Fix for the submodules API not reporting errors when parsing the ".gitmodules"
  file.

• Fix for accepting a ".gitmodules" file where two submodules have the same
  path.

• Fix for hiding references in a graph walk not always limiting the graph
  correctly.

• Fix for directory patterns with trailing spaces in attribute files not being
  handled correctly.

• Fix SSH transports not properly disconnecting from the server.

• Fix reading HEAD reflog in worktrees.

• Update our copy of SHA1DC to fix errors with endianess on some platforms.

A list of commits since the last release follows:

b2e7d8c22 transports: ssh: disconnect session before freeing it
b89988c7f transports: ssh: replace deprecated function `libssh2_session_startup`
4d4a7dbf5 sha1dc: update to fix errors with endianess
59012bf41 odb: mempack: fix leaking objects when freeing mempacks
a714e836d transports: local: fix assert when fetching into repo with symrefs
b260fdc84 attr_file: fix handling of directory patterns with trailing spaces
e9ee7bd0a fixed stack smashing due to wrong size of struct stat on the stack on 32-bit systems with 64-bit file descriptor offsets enabled (added -D_FILE_OFFSET_BITS=64 when compiling the test suite)
e2a80124d refs: preserve the owning refdb when duping reference
b6623be06 tests: ensure worktrees' head have owners too
0f88adb65 Submodule API should report .gitmodules parse errors
07011e60c revwalk: fix uninteresting revs sometimes not limiting graphwalk
16b62dd4c diff: Add missing GIT_DELTA_TYPECHANGE -> 'T' mapping.
2569056d1 typo: Fixed a trivial typo in test function.
0f09d9f55 Fix build with LibreSSL 2.7
7fa6c8ce5 util: fix missing headers for MinGW environments
1cc6cc990 appveyor: disable DHE to avoid spurious failures
dad649871 appveyor: fix typo in registry key to disable DHE
a137cdbd9 refspec: check for valid parameters in git_refspec__dwim_one
96329606d worktree: Read worktree specific reflog for HEAD
2fe887e6f remote: repo is optional here
8fa0b34bd local: fix a leaking reference when iterating over a symref
b2f3ff567 worktree: fix calloc of the wrong object type
0c8ff50fe cmake: resolve libraries found by pkg-config
f2e5c092e cmake: remove now-useless LIBGIT2_LIBDIRS handling
7392799dd submodule: detect duplicated submodule paths
358651170 tests: submodule: do not rely on config iteration order
0818adece CHANGELOG.md: update for release v0.27.2
853ef86ac version: bump soversion to v0.27.2

This is a security release fixing insufficient validation of submodule names (CVE-2018-11235, reported by Etienne Stalmans).

While submodule names come from the untrusted ".gitmodules" file, we blindly append the name to $GIT_DIR/modules to construct the final path of the submodule repository. In case the name contains e.g. ../, an adversary would be able to escape your repository and write data at arbitrary paths. In accordance with git, we now enforce some rules for submodule names which will cause libgit2 to ignore these malicious names.

libgit2 is not susceptible to CVE-2018-11233.

This is a security release fixing insufficient validation of submodule names (CVE-2018-11235, reported by Etienne Stalmans).

While submodule names come from the untrusted ".gitmodules" file, we blindly append the name to $GIT_DIR/modules to construct the final path of the submodule repository. In case the name contains e.g. ../, an adversary would be able to escape your repository and write data at arbitrary paths. In accordance with git, we now enforce some rules for submodule names which will cause libgit2 to ignore these malicious names.

libgit2 is not susceptible to CVE-2018-11233.

This is the first release of the v0.27 series, "Stadtrandsiedlung". The changelog follows.

Changes or improvements

• Improved p_unlink in posix_w32.c to try and make a file writable
  before sleeping in the retry loop to prevent unnecessary calls to sleep.

• The CMake build infrastructure has been improved to speed up building time.

• A new CMake option "-DUSE_HTTPS=" makes it possible to explicitly
  choose an HTTP backend.

• A new CMake option "-DSHA1_BACKEND=" makes it possible to explicitly
  choose an SHA1 backend. The collision-detecting backend is now the default.

• A new CMake option "-DUSE_BUNDLED_ZLIB" makes it possible to explicitly use
  the bundled zlib library.

• A new CMake option "-DENABLE_REPRODUCIBLE_BUILDS" makes it possible to
  generate a reproducible static archive. This requires support from your
  toolchain.

• The minimum required CMake version has been bumped to 2.8.11.

• Writing to a configuration file now preserves the case of the key given by the
  caller for the case-insensitive portions of the key (existing sections are
  used even if they don't match).

• We now support conditional includes in configuration files.

• Fix for handling re-reading of configuration files with includes.

• Fix for reading patches which contain exact renames only.

• Fix for reading patches with whitespace in the compared files' paths.

• We will now fill FETCH_HEAD from all passed refspecs instead of overwriting
  with the last one.

• There is a new diff option, GIT_DIFF_INDENT_HEURISTIC which activates a
  heuristic which takes into account whitespace and indentation in order to
  produce better diffs when dealing with ambiguous diff hunks.

• Fix for pattern-based ignore rules where files ignored by a rule cannot be
  un-ignored by another rule.

• Sockets opened by libgit2 are now being closed on exec(3) if the platform
  supports it.

• Fix for peeling annotated tags from packed-refs files.

• Fix reading huge loose objects from the object database.

• Fix files not being treated as modified when only the file mode has changed.

• We now explicitly reject adding submodules to the index via
  git_index_add_frombuffer.

• Fix handling of GIT_DIFF_FIND_RENAMES_FROM_REWRITES raising SIGABRT when
  one file has been deleted and another file has been rewritten.

• Fix for WinHTTP not properly handling NTLM and Negotiate challenges.

• When using SSH-based transports, we now repeatedly ask for the passphrase to
  decrypt the private key in case a wrong passphrase is being provided.

• When generating conflict markers, they will now use the same line endings as
  the rest of the file.

API additions

• The git_merge_file_options structure now contains a new setting,
  marker_size. This allows users to set the size of markers that
  delineate the sides of merged files in the output conflict file.
  By default this is 7 (GIT_MERGE_CONFLICT_MARKER_SIZE), which
  produces output markers like <<<<<<< and >>>>>>>.

• git_remote_create_detached() creates a remote that is not associated
  to any repository (and does not apply configuration like 'insteadof' rules).
  This is mostly useful for e.g. emulating git ls-remote behavior.

• git_diff_patchid() lets you generate patch IDs for diffs.

• git_status_options now has an additional field baseline to allow creating
  status lists against different trees.

• New family of functions to allow creating notes for a specific notes commit
  instead of for a notes reference.

• New family of functions to allow parsing message trailers. This API is still
  experimental and may change in future releases.

Breaking API changes

• Signatures now distinguish between +0000 and -0000 UTC offsets.

• The certificate check callback in the WinHTTP transport will now receive the
  message_cb_payload instead of the cred_acquire_payload.

• We are now reading symlinked directories under .git/refs.

• We now refuse creating branches named "HEAD".

• We now refuse reading and writing all-zero object IDs into the
  object database.

• We now read the effective user's configuration file instead of the real user's
  configuration in case libgit2 runs as part of a setuid binary.

• The git_odb_open_rstream function and its readstream callback in the
  git_odb_backend interface have changed their signatures to allow providing
  the object's size and type to the caller.

This is a bugfix release. It includes the following non-exclusive list of
improvements, which have been backported from the master branch:

• Fix cloning of the libgit2 project with git clone --recursive by removing an
  invalid submodule from our testing data.

• Fix endianness of the port in p_getaddrinfo().

• Fix handling of negative gitignore rules with wildcards.

• Fix handling of case-insensitive negative gitignore rules.

• Fix resolving references to a tag if the reference is stored with its fully
  resolved OID in the packed-refs file.

• Fix checkout not treating worktree files as modified when only their mode has
  changed.

• Fix rename detection with GIT_DIFF_FIND_RENAMES_FROM_REWRITES.

• Enable Windows 7 and earlier to use TLS 1.2.

A list of commits since the last release follows:

cc9b0b6c5 tests: try to init with empty template path
82bb59b47 repository: do not initialize templates if dir is an empty string
e4517af3e repository: remove trailing whitespace
54d4e5de8 Remove invalid submodule
e7c24ea20 tests: fix the rebase-submodule test
f908bb8ea Convert port with htons() in p_getaddrinfo()
58197758b ignore: fix indentation of comment block
8d86cdd46 ignore: return early to avoid useless indentation
5c15cd949 ignore: keep negative rules containing wildcards
32cc5edce tests: status: additional test for negative ignores with pattern
4296a36b7 ignore: honor case insensitivity for negative ignores
21f77af92 signature: don't leave a dangling pointer to the strings on parse failure
3ca2bb390 sha1_position: convert do-while to while
c3fbf9058 Clear the remote_ref_name buffer in git_push_update_tips()
e29ab6feb proxy: add a free function for the options's pointers
8d7dcb10f curl: free the proxy options
27a8092bb curl: free the user-provided proxy credentials
93ecb61ae proxy: rename the options freeing function
c27022355 Use SOCK_CLOEXEC when creating sockets
cda18f9ba refs: do not use peeled OID if peeling to a tag
f41e86d62 transports: smart: fix memory leak when skipping symbolic refs
243881795 checkout: treat files as modified if mode differs
8631357ea checkout: do not test file mode on Windows
e66bc08c3 checkout: test force checkout when mode changes
68842cbb5 Ignore trailing whitespace in .gitignore files (as git itself does)
5c3a42ad4 Include git2/worktree.h in git2.h
05a753d4d diff: remove unused macros `DIFF_FLAG_*`
3c4e0ceef diff_generate: fix unsetting diff flags
049e1de59 openssl: fix thread-safety on non-glibc POSIX systems
5cc3971a6 libFuzzer: Fix a git_packfile_stream leak
a3cd5e941 libFuzzer: Fix missing trailer crash
a42e11aea libFuzzer: Prevent a potential shift overflow
feb00daff Using unsigned instead
8f189cbfe Simplified overflow condition
7ad0cee6c hash: openssl: check return values of SHA1_* functions
7cc805467 streams: openssl: fix thread-safety for OpenSSL error messages
34f1ded97 stransport: provide error message on trust failures
a521f5b16 diff_file: properly refcount blobs when initializing file contents
e83efde45 Fix unpack double free
5e97bdaf3 odb: export mempack backend
08ab59020 Introduce additional criss-cross merge branches
457a81bb0 oidarray: introduce git_oidarray__reverse
b2b370773 merge: reverse merge bases for recursive merge
dc51d7740 merge::trees::recursive: test for virtual base building
3619e0f09 Add failing test case for virtual commit merge base issue
b3c0d43ce merge: virtual commit should be last argument to merge-base
be205dfa4 tests: diff::rename: use defines for commit OIDs
e229e90d7 tests: add rename-rewrite scenarios to "renames" repository
e74e05ed1 diff_tform: fix rename detection with rewrite/delete pair
3983fc1dc checkout: take mode into account when comparing index to baseline
736159003 checkout test: ensure workdir perms are updated
1b853c483 checkout test: further ensure workdir perms are updated
9bdc00b1b mingw: update TLS option flags
aa0127c0f winhttp: include constants for TLS 1.1/1.2 support
9ab8d1532 winhttp: enable TLS 1.2 on Windows 7 and earlier
c24b15c36 win32: strncmp -> git__strncmp
9e98f49d4 tree: initialize the id we use for testing submodule insertions
b35c30986 curl: explicitly initialize and cleanup global curl state
7c8ddef0c CHANGELOG.md: update for v0.26.3