Add https support #656

wants to merge 155 commits into


None yet

carlosmn commented Apr 30, 2012

Add support for GnuTLS and OpenSSL.

No validation is done of the certificate. First I want to get feedback on the implementation of the basic functionality.

rwilcox and others added some commits Feb 29, 2012

@rwilcox rwilcox implement support for username@host:path URLs in transport_find_fn() 5844891
@rwilcox rwilcox fix up previous SSH path parsing commit based on @carlosmn feedback 253d6df
@rwilcox rwilcox introduced new function: git_remote_supported_url() <-- returns true …
…if this version of libgit2 supports the correct transport mechanism for a URL or path
@tuxdna tuxdna Add specfile and packaging instruction for creating Fedora RPM ef5c4ee
@vmg vmg Merge pull request #583 from tuxdna/development
Add specfile and packaging instruction for creating Fedora RPM
@arrbee arrbee Fix usage of "new" for fieldname in public header
This should restore the ability to include libgit2 headers
in C++ projects.
@arrbee arrbee Copy values to avoid strict aliasing warning
To make this code more resilient to future changes, we'll
explicitly translate the libgit2 structure to the libxdiff
@arrbee arrbee Convert from strnlen to git_text_is_binary
Since strnlen is not supported on all platforms and since we
now have the shiny new git_text_is_binary in the filtering
code, let's convert diff binary detection to use the new stuff.
@carlosmn carlosmn Make git_remote_supported_url() public and shorten error string 4f8efc9
@carlosmn carlosmn Merge branch 'ssh-urls' into development 864ac49
Authmillenon Rename git_oid_to_string to git_oid_tostr
To conform the naming scheme of git_oid_fromstr we should change the
name of git_oid_to_string to git_oid_tostr.
@vmg vmg Merge pull request #588 from authmillenon/development
Rename git_oid_to_string to git_oid_tostr
@petdance petdance Add links to the README and alphabetize the languages to make them
easier for the casual reader to find.
@vmg vmg Merge pull request #589 from petdance/b8d08292c934b7e21738c61531b340d…

Cleanups to the README
@aroben aroben Fix the build on Windows 7bed25a
@carlosmn carlosmn config: write out section headers with subsections correctly
write_section() mistakenly treated is input as the whole variable name
instead of simply the section (and possibly subsection) and would
confuse "section.subsection" as a section plus variable name and
produce a wrong section header.

Fix this and include a test for writing "section.subsection.var" and
reading it from the file.
@vmg vmg Merge pull request #593 from aroben/windows-build-fix
Fix the build on Windows
Evan Hanson add chicken-git bindings to readme 0e8144f
@vmg vmg Merge pull request #594 from evhan/development
add chicken-git bindings to readme
schu examples/diff: update example code
Signed-off-by: schu <>

arrbee commented on 288c8a2 Mar 13, 2012


ben and others added some commits Mar 21, 2012

@ben ben Adding multi-cpu compile option when generating MSVC projects. 06c081e
@arrbee arrbee Merge pull request #605 from benstraub/win-multi-cpu-compile
Adding multi-cpu compile option when generating MSVC projects.
@ben ben Ported t04_commit.c to Clar.
Created a copy of tests/resources/testrepo.git that is compatible
with the Clar sandboxing helpers.

Restructured commit test suites to use Clar sandbox helpers.

Now using typed data arrays rather than lots of macros to define test
schu diff_output: remove unused parameter
Signed-off-by: schu <>
@arrbee arrbee Merge pull request #608 from schu/remove-unused-parameter
diff_output: remove unused parameter
@vmg vmg tree: Fix homing entry search 181bbf1
@ben ben t03_objwrite.c ported. bcbabe6
@ben ben t06_index.c ported. 6c106ee
@ben ben t07_hashtable.c ported. 2ef582b
@ben ben t08_tag.c ported.
Also cleaned up some names for things that used to be macros.
@ben ben Cleaned up build issues under Linux. Had to disable a file-mode check
in tag/write.c.
@ben ben Added Clar build-flag note to readme. 23cb35f
@ben ben Turning on runtime checks when building debug under MSVC. 9f75a9c
@ben ben t09-tree.c ported. 9a39a36
@ben ben Moved tag tests to object suite. 6bb7499
@ben ben Fixed linux build/test issues. dde61de
@ben ben t10-refs.c ported. 00a4893
@ben ben t12-repo.c ported, although kind of messy.
It'd be nice to be rid of all the #define's, but converting to const
char*'s would be even messier, and less declarative.
@ben ben t13-threads.c ported. 1cb9b31
@ben ben Ref normalization test helpers now doing internal asserts. 8e82600
@ben ben Removing test suites that have been ported to Clar:
* t00 / 6e86fb3 (mentioned in #406)
* t01 / fc60c4a (mentioned in #406)
* t03 / bcbabe6
* t04 / PR #606
* t05 / d96f2c3
* t06 / 6c106ee
* t07 / 2ef582b
* t08 / b482c42
* t09 / 9a39a36
* t10 / 00a4893
* t12 / 7c3a4a7
* t13 / 1cb9b31
* t17 / cdaa6ff (mentioned in #406)
* t18 / efabc08
@ben ben Moved more assertions inside Clar test helpers. 270303c
@ben ben Moved testing resources to clar, and removed old tests directory.
Removed the BUILD_CLAR CMake flag, and updated the readme.
@ben ben Discovered cl_git_strequal! Mounted a crusade! 20ec426
@ben ben Simple readability fixes. b173121
@drahosp drahosp Removed my duplicate entry 3433a69
@drahosp drahosp Added Travis CI build status 3702931
@drahosp drahosp Initial Travis CI build f9abcbd
@vmg vmg Merge pull request #616 from drahosp/development
Travis-CI integration
@vmg vmg travis: Change notify email 5afe95d
@vmg vmg Merge pull request #615 from benstraub/port-all-tests-to-clar
Port all tests to clar
@vmg vmg Merge pull request #606 from benstraub/t04_commit_to_clar
Ported t04_commit.c to Clar.
@vmg vmg tests: Cleanup & fix test suite 471bb8b
schu tests-clar/commit: fix memory leaks
Signed-off-by: schu <>
schu tests-clar/object: remove unused helper print_tree()
Signed-off-by: schu <>
schu tests-clar/index: actually assert result
Signed-off-by: schu <>
@vmg vmg Merge pull request #617 from schu/tests-fixup
tests-clar: cleanup
@vmg vmg clar: Properly create file in helper bbb3723
@carlosmn carlosmn mwindow: make sure the whole range is contained inside the same window
Looking through the open windows to check whether we can re-use an
open window should take into account whether both `offset` and `offset
+ extra` are contained within the same window. Failure to do so can
lead to invalid memory accesses. This closes #614.

While we're in the area remove an outdated assert.
@csware csware WIN32 is not always defined, use GIT_WIN32 instead
Signed-off-by: Sven Strickroth <>
@csware csware GetFileAttributes does not work for utf-8 encoded paths
Signed-off-by: Sven Strickroth <>
@vmg vmg Merge pull request #635 from csware/win32-utf8
GetFileAttributes does not work for utf-8 encoded paths
@vmg vmg Merge pull request #634 from csware/fix-win64-build
WIN32 is not always defined, use GIT_WIN32 instead
@csware csware Check for _WIN32 is sufficient, even for x64 compilers
There is no need to check for _WIN32 and _WIN64. x64 compiler also set _WIN32 (compare

Signed-off-by: Sven Strickroth <>
@csware csware Check for _WIN32 instead of GIT_WIN32 or WIN32 to detect windows buil…
…d environments

This fixes a possible compilation issue (when GIT_WIN32 was not set) which was introduced in revision 69a4bc1.

Signed-off-by: Sven Strickroth <>
@vmg vmg Merge pull request #636 from csware/WIN32-fixes
Win32 fixes
@carlosmn @arrbee carlosmn tree-cache: don't error out on a childless invalidated entry
The code used to assume that there had to be data after the newline in
a tree cache extension entry. This isn't true for a childless
invalidated entry if it's the last one, as there won't be any children
nor a hash to take up space.

Adapt the off-by-one comparison to also work in this case. Fixes #633.

This pull request passes (merged b76b49ac into 960d778).

@vmg vmg commented on an outdated diff May 2, 2012

@@ -81,6 +82,21 @@ IF (NOT CMAKE_BUILD_TYPE)
SET(CMAKE_BUILD_TYPE "Debug" CACHE STRING "Choose the type of build, options are: Debug Release RelWithDebInfo MinSizeRel." FORCE)

vmg May 2, 2012


We need GIT_ prefixes for these macros. GIT_SSL or the likes.

@vmg vmg commented on an outdated diff May 2, 2012

@@ -46,6 +51,22 @@ static void net_set_error(const char *str)
+#ifdef USE_GNUTLS

vmg May 2, 2012


Lots of macros here. Can you move the macros inside the function body to avoid duplicating the function header?

@vmg vmg and 1 other commented on an outdated diff May 2, 2012

@@ -90,7 +92,14 @@ static int do_connect(transport_http *t, const char *host, const char *port)
if (t->parent.connected && http_should_keep_alive(&t->parser))
return 0;
- if ((s = gitno_connect(host, port)) < 0)
+#ifdef USE_SSL

vmg May 2, 2012


This can be abstracted.

int gitno_connect(git_transport *t, const char *host, int port)

carlosmn May 2, 2012


Whether we use SSL isn't currently in git_transport but transport_http because that's the only one that can do SSL. I can put it in the main class if you prefer, but it's just going to be set to 0 on every other transport. Maybe gitno_connect() should get a extra parameter to tell it whether it should init ssl?

@vmg vmg commented on an outdated diff May 2, 2012

@@ -99,6 +108,16 @@ static int do_connect(transport_http *t, const char *host, const char *port)
return 0;
+static int do_send(transport_http *t, const void *buf, size_t len)
+#ifdef USE_SSL

vmg May 2, 2012



@vmg vmg commented on an outdated diff May 2, 2012

@@ -231,7 +250,12 @@ static int store_refs(transport_http *t)
settings.on_body = on_body_store_refs;
settings.on_message_complete = on_message_complete;
- gitno_buffer_setup(&buf, buffer, sizeof(buffer), t->socket);
+#ifdef USE_SSL

vmg May 2, 2012


And this.

@vmg vmg commented on an outdated diff May 2, 2012

@@ -680,3 +728,25 @@ int git_transport_http(git_transport **out)
*out = (git_transport *) t;
return 0;
+#ifdef USE_SSL

vmg May 2, 2012


Share function header! Move macros inside!


vmg commented May 2, 2012

Regarding the abstractions: I'm not too sure about how to accomplish these, but I think it would be nice to keep the macros inside the actual socket functions instead of having ssl and non-ssl functions and deciding which ones to call through macros.

This pull request fails (merged 30ce5016 into b880214).


carlosmn commented May 2, 2012

This should cover the largest concerns.

There's a couple of functions where I didn't make them share the header because that would have needed an extra #ifdef GIT_SSL around all of it, and that seemed to be going too far.

This pull request fails (merged 2a9df36d into b880214).


carlosmn commented May 2, 2012

Ignore the travis errors. They come from the error handling revamp.

@vmg vmg and 1 other commented on an outdated diff May 2, 2012

@@ -26,6 +26,7 @@ INCLUDE_DIRECTORIES(src include deps/http-parser)
FILE(GLOB SRC_HTTP deps/http-parser/*.c)

vmg May 2, 2012


Why is this so separated from the rest of the SSL config? Also: why os GnuTLS the first option? AFAIK OpenSSL should be more widely available.


carlosmn May 2, 2012


It used to be next to the Zlib finding. It's earlier because I started earlier and I'd rather use that one. I'll change the order.

This pull request fails (merged bcc2f5a into 3fd99be).

vmg and others added some commits May 2, 2012

@vmg vmg Merge branch 'new-error-handling' into development
@vmg vmg Update test suite 946a6dc
@vmg vmg Move test resources 8c83fea
@vmg vmg Backport more test data a1d0802
@arrbee arrbee chmod for writability when writing test files 17f1c9f
@vmg vmg Boom b02bcd9
@vmg vmg Remove old and unused error codes 3fbcac8
@vmg vmg buffer: Add `git_buf_vprintf` baaf1c4
@vmg vmg errors: Use a git_buf for building error strings 2277216

This pull request fails (merged aa7fad5 into 3fd99be).

vmg and others added some commits May 3, 2012

@vmg vmg Silence return value warning 76873c0
@schu schu notes: honor core.notesRef
Setting core.notesRef allows to change the default notes reference used
by Git. Check if set before using GIT_NOTES_DEFAULT_REF. Fixes #649.
@schu schu notes: add git_note_default_ref()
Add git_note_default_ref to allow easy retrieval of the currently set
default notes reference.
@arrbee arrbee Support reading attributes from index
Depending on the operation, we need to consider gitattributes
in both the work dir and the index.  This adds a parameter to
all of the gitattributes related functions that allows user
control of attribute reading behavior (i.e. prefer workdir,
prefer index, only use index).

This fix also covers allowing us to check attributes (and
hence do diff and status) on bare repositories.

This was a somewhat larger change that I hoped because it had
to change the cache key used for gitattributes files.
@arrbee arrbee Fix memory leaks and use after free b709e95

nulltoken commented on src/buffer.c in baaf1c4 May 4, 2012

@tanoku 🔥 va_copy doesn't look valid on Windows

2>....\libgit2\src\buffer.c(157): warning C4013: 'va_copy' undefined; assuming extern returning int


peff replied May 4, 2012

We had to deal with this in git.git. You can find our portability setup here, which has been working for a few years (and certainly works on windows):


nulltoken replied May 5, 2012

@peff I think @tanoku already found the source-of-all-wonders (cf. 1adf8c6) :)

vmg and others added some commits May 4, 2012

@vmg vmg compat: va_copy on Win32 systems 1adf8c6
@arrbee arrbee Fixing issue with test data 3ec1fa5
@vmg vmg clar: Properly create files in helper 674a198
@arrbee arrbee Fix valgrind issues
There are three changes here:
- correctly propogate error code from failed object lookups
- make zlib inflate use our allocators
- add OID to notfound error in ODB lookups
@scottjg scottjg Fix clar generated code to compile on MINGW32
MINGW32 does not define _mktemp_s, so we can just use _mktemp instead. See
the non-compressed/non-base64-encoded version of the patch here:
@scottjg scottjg Fix gitno_connect() error handling on Windows
gitno_connect() can return an error or socket, which is fine on most
platforms where sockets are file descriptors (signed int), but on Windows,
SOCKET is an unsigned type, which is problematic when we are trying to
test if the socket was actually a negative error code.

This fix seperates the error code and socket in gitno_connect(), and fixes
the error handling in do_connect() functions to compensate. It appears
that git_connect() and the git-transport do_connect() functions had bugs
in the non-windows cases too (leaking sockets, and not properly reporting
connection error, respectively) so I went ahead and fixed those too.
@scottjg scottjg Fix missing prototype warning in utf-conv.c b47e0a7
@scottjg scottjg Fix unsigned/signed comparison on Windows in commitstagedfile.c 35cdd26
@vmg vmg Merge pull request #663 from schu/notes-honor-config
Honor core.notesRef config option
@vmg vmg notes: Cleanup error handling f95e8cc
@vmg vmg Merge pull request #664 from arrbee/attrs-from-index
Support git attrs from index (and bare repo)
@vmg vmg Merge pull request #659 from libgit2/development-merge
@vmg vmg Merge remote-tracking branch 'scottjg/fix-mingw32' into development
@vmg vmg compat: Add `stdarg.h` include 3972ca4
@carlosmn carlosmn transport git: don't use 'error' uninitialized 49ac5ac
@scottjg scottjg Update clar to latest version
Fixes the mingw32 build issues.
@scottjg scottjg Define explicit _WIN32_WINNT version in makefile
Previously, it was defined in netops.c, but it's also needed in one of the
clar tests, so I figured we might as well just make it global for the
whole project.

Without it, the mingw32 linker won't resolve GetProcessId() (called from
the core/errors.c clar test) because of some conditionals in windows.h.
@carlosmn carlosmn clar helper: don't dereference giterr_last() if it's NULL
It can cause segfaults if the call didn't set an error
@nulltoken nulltoken commit/tag: ensure the message is cleaned up
'git commit' and 'git tag -a' enforce some conventions, like cleaning up excess whitespace and making sure that the last line ends with a '\n'. This Fix replicates this behavior.

Fix libgit2/libgit2sharp#117
@nulltoken nulltoken diff: remove unused parameter 8d0f467
@nulltoken nulltoken diff: When diffing two blobs, ensure the delta callback parameter is …
…filled with relevant information
@nulltoken nulltoken diff: fix the diffing of a concrete blob against a null one 4f80676
@nulltoken nulltoken tests: add two binary blobs to attr test repository
 - edf3dce ->
 - de863bf ->
@nulltoken nulltoken diff: make git_diff_blobs() able to detect binary blobs 28ef7f9
@nulltoken nulltoken diff: fix the diffing of two identical blobs 9a29f8d
@nulltoken nulltoken diff: improve git_diff_blobs() documentation d1c4312
@nulltoken nulltoken compat: make p_open able to accept optional mode when passing the O_C…
…REAT flag

This has the nice side effect of making test_attr_repo__staging_properly_normalizes_line_endings_according_to_gitattributes_directives() test pass again on Windows. This test started to fail after commit 674a198 was applied.
@carlosmn carlosmn travis: run the tests verbosely c2d82a6
@arrbee arrbee Merge pull request #667 from nulltoken/fix/p_open
compat: make p_open able to accept optional mode when passing the O_CREAT flag
@arrbee arrbee Fix directory finding for attrs
The fix to support attrs on bare repos went a little too far
in trying to avoid using the working directory and ended up
not processing the input path quite correctly.
@vmg vmg Merge pull request #672 from scottjg/more-mingw32-fixes
More mingw32 compilation fixes.

juzna commented on 3fbcac8 May 8, 2012

This breaks php-git build :(


vmg replied May 8, 2012

This will break the error handling in many of the libgit2 wrappers. The fix should be rather straightforward, though, and the results are worth it.


carlosmn replied May 8, 2012

Breaks how? Does it cause runtime problems? Can't it build? If it's just a matter of being out of sync, then it's a matter of updating php-git.

juzna replied May 8, 2012

Yep, php-git won't build because it still has references to the removed constants. I'll add an issue there to update.
(sorry for such stupid report, I was already too tired yesterday).

nulltoken and others added some commits May 7, 2012

@nulltoken nulltoken remote: ensure the allocated remote is freed when an error occurs dur…
…ing its loading
@nulltoken nulltoken remote: make git_remote_load() return GIT_ENOTFOUND when the remote u…
…rl cannot be retrieved from the config file
@nulltoken nulltoken object: make git_object_lookup() return GIT_ENOTFOUND when searching …
…for an existing object by specifying an incorrect type
@nulltoken nulltoken path: Make git_path_prettify() properly handle ENOTDIR errno value 4681156
@nulltoken nulltoken compat: make p_realpath Windows implementation be a bit more POSIX co…
…mpliant and fail if the provided path does not lead to an existing entry
@nulltoken nulltoken repository: ensure git_repository_open() returns ENOTFOUND when being…
… passed a path leading to no repository
@nulltoken nulltoken fileops: replace integer usage with more explicit enum in some git_fu…
…tils_rmdir_r() calls
@nulltoken nulltoken repository: ensure git_repository_discover() returns ENOTFOUND when u…
…nable to find a repository given the constraints
@nulltoken nulltoken status: Prevent git_status_file() from returning ENOTFOUND when not a…
@carlosmn carlosmn Minor error fixes
Clear the error in pkt when we notice that the remote is starting to
send the packfile.

Fix the format string for Windows networking errors.
@arrbee arrbee Merge pull request #674 from nulltoken/topic/GIT_ENOTFOUND
Improve the interop with bindings
@carlosmn carlosmn config: don't use freed memory on error
Change the order and set a NULL so we don't try to access freed memory
in case of an error.
@carlosmn carlosmn remote: add git_remote_add()
Helper function to create a remote with the default settings
@carlosmn carlosmn remotes: change git_remote_new's signature
Add a fetch refspec arguemnt and make the arguments (name, url,
refspec), as that order makes more sense.
@arrbee arrbee Clean up warnings and tests 1957984
@vmg vmg Merge pull request #676 from carlosmn/remotes
Add git_remote_add() and change signature for _new()
@arrbee arrbee Merge pull request #668 from nulltoken/topic/binary-blobs
Enhancing the blob diffing experience
@vmg vmg Merge pull request #670 from nulltoken/ntk/topic/clean-commit_message
Clean commit and tag messages
@vmg vmg message: Proper OOM handling b1e2ba2
@vmg vmg message: Cleanup fd5faae
@arrbee arrbee Add support for diffing index with no HEAD
When a repo is first created, there is no HEAD yet and attempting
to diff files in the index was showing nothing because a tree
iterator could not be constructed.  This adds an "empty" iterator
and falls back on that when the head cannot be looked up.
@vmg vmg Merge pull request #677 from arrbee/status-without-head
Add support for diffing index with no HEAD
@vmg vmg msvc: Do not use `isspace`
Locale-aware bullshit bitting my ass again yo
@carlosmn carlosmn indexer: close the pack's fd before renaming it
Windows gets upset if we rename a file with an open descriptor.
@nulltoken nulltoken util: Fix git__isspace() implementation
The characters <space>, <form-feed>, <newline>, <carriage-return>, <tab>, and <vertical-tab> are part of the "space" definition.

@carlosmn carlosmn remote: don't try to create tag annotations as refs/tags/v0.1.0^{}
Skip them for now. Eventually we might want to filter these out
@carlosmn carlosmn fetch: filter tag annotation pseudo-refs while generating wants
These objects aren't considered as being advertised, so asking for
them will cause the remote end to close the connection. This makes the
checking in update_tips() unnecessary, because they don't get inserted
in the list.
@schu schu tests-clar/diff: fix missing-prototype warning b470019
@arrbee arrbee Merge pull request #679 from nulltoken/fix/git__isspace
util: Fix git__isspace() implementation
@arrbee arrbee Fix 64-bit build warning 2aa1e94
@arrbee arrbee Improve repo initialization to be more like git
This adds a bunch of template files to the initialization for
hooks, info/exclude, and description.  This makes our initialized
repo look more like core gits.
@nulltoken nulltoken Fix MSVC compilation issue
    exp() is already defined in math.h. This leads to LMSVC complaining

    ..\..\libgit2\tests-clar\diff\blob.c(5): error C2365: 'exp' : redefinition; previous definition was 'function'

    Renaming the variable fixes this issue.
@vmg vmg Merge pull request #678 from nulltoken/fix/msvc-weird-error
Fix MSVC compilation issue
@vmg vmg clar: Update from upstream a9d9965
@carlosmn carlosmn examples: update network examples error handling
Use giterr_last() and make sure it's not NULL.
@arrbee arrbee Fixed leaks and added tests db62807
@carlosmn carlosmn examples: fix an oopsie 41178b4
@nulltoken nulltoken object: make git_object_lookup() return GIT_ENOTFOUND when searching …
…for an existing object by specifying an incorrect type

This fix complements cb0ce16 and cover the following additional use cases

 - retrieving an object which has been previously searched, found and cached
 - retrieving an object through an non ambiguous abbreviated id
@vmg vmg Merge pull request #687 from nulltoken/fix/object-lookup-take-2
object: make git_object_lookup() return GIT_ENOTFOUND - The sequel
@vmg vmg Merge pull request #683 from arrbee/better-repo-init
Improve repo initialization to be more like git
@carlosmn carlosmn http: add https support when GnuTLS is available
If it's not available, an error saying so will be returned when trying
to use a https:// URL.

This also unifies a lot of the network code to use git_transport in
many places instead of an socket descriptor.
@carlosmn carlosmn https: make it work with OpenSSL as well
Add specific functions that use OpenSSL instead of GnuTLS
@carlosmn carlosmn ssl: teardown the connection on close
This should help us free some resources, though the libraries do keep
some buffers allocated regardless.

carlosmn closed this May 14, 2012

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment