Proxy settings ignored while cloning with libgit2 1.0.1

[civodul]

Reproduction steps

Compile this code with libgit2 1.0.1:

#include <git2.h>

int
main ()
{
  git_libgit2_init ();

  git_clone_options opts;
  git_clone_options_init (&opts, GIT_CLONE_OPTIONS_VERSION);

  opts.fetch_opts.proxy_opts.type = GIT_PROXY_SPECIFIED;
  opts.fetch_opts.proxy_opts.url = "http://localhost:8118";

  git_repository *repo;
  git_clone (&repo, "http://example.org/whatever.git", "/tmp/example",
	     &opts);

  return 0;
}

Run it under strace and notice that it connects to example.org directly instead of connecting to the specified proxy:

socket(AF_INET6, SOCK_STREAM|SOCK_CLOEXEC, IPPROTO_TCP) = 3
connect(3, {sa_family=AF_INET6, sin6_port=htons(80), sin6_flowinfo=htonl(0), inet_pton(AF_INET6, "2606:2800:220:1:248:1893:25c8:1946", &sin6_addr), sin6_scope_id=0}, 28) = 0
sendto(3, "GET http://example.org/whatever."..., 151, 0, NULL, 0) = 151
recvfrom(3, "HTTP/1.1 404 Not Found\r\nAccept-R"..., 16384, 0, NULL, NULL) = 1595

Expected behavior

Should connect to localhost:8118 and send a GET request there.

Actual behavior

Connects directly to example.org.

Version of libgit2 (release number or SHA1)

1.0.1

Operating system(s) tested

GNU/Linux (GNU Guix), x86_64.

[ethomson]

I'd like a few more details - we have several tests that exercise proxies.

Did this work before and stopped working in 1.0.1?

Have you found any workarounds?

Does it work for fetch but not clone, etc?

Do the proxy environment variables work?

[civodul]

Hi,

It looks like a regression compared to 0.28.5.

Here's a slightly edited version that can be used both with 0.28 and 1.0.1:

#include <git2.h>
#include <assert.h>
#include <stdio.h>

int
main ()
{
  int err;
  err = git_libgit2_init ();
  assert (err == 1);

  git_clone_options opts;
  err = git_clone_init_options (&opts, GIT_CLONE_OPTIONS_VERSION);
  assert (err == 0);

  opts.fetch_opts.proxy_opts.type = GIT_PROXY_SPECIFIED;
  opts.fetch_opts.proxy_opts.url = "http://localhost:8118/";

  git_repository *repo;
  err = git_clone (&repo, "http://example.org/whatever.git", "/tmp/example",
		   &opts);
  assert (err != 0);

  const git_error *gerr;
  gerr = giterr_last ();
  fprintf (stderr, "last error: %s\n", gerr->message);

  return 0;
}

With 0.28.5, the strace output shows that the program connects to the proxy:

4535  connect(3, {sa_family=AF_INET, sin_port=htons(8118), sin_addr=inet_addr("127.0.0.1")}, 16) = 0
4535  sendto(3, "GET http://example.org:80/whatever.git/info/refs?service=git-upload-pack HTTP/1.1\r\nUser-Agent: git/2.0 (libgit2 0.28.5)\r\nHost: example.org\r\nAccept: */*\r\n\r\n", 155, 0, NULL, 0) = 155

This is in contrast with 1.0.1:

4674  connect(3, {sa_family=AF_INET6, sin6_port=htons(80), sin6_flowinfo=htonl(0), inet_pton(AF_INET6, "2606:2800:220:1:248:1893:25c8:1946", &sin6_addr), sin6_scope_id=0}, 28) = 0
4674  sendto(3, "GET http://example.org/whatever.git/info/refs?service=git-upload-pack HTTP/1.1\r\nUser-Agent: git/2.0 (libgit2 1.0.1)\r\nHost: example.org\r\nAccept: */*\r\n\r\n", 151, 0, NULL, 0) = 151

I haven't tried with fetch and haven't found any workarounds.

Environment variables http_proxy and https_proxy are not honored, but AFAIK they were not honored in 0.28 either and I didn't expect it to be the case.

Thanks,
Ludo'.

[ethomson]

I'm just seeing now that you're using http with your git server. I believe that this is the problem. If you use https, you should be able to use your proxy?

I think that this is the culprit: https://github.com/libgit2/libgit2/blob/master/src/transports/httpclient.c#L1007

@civodul if you change this to omit the https check, does this work for you?

There's probably a broader question about whether we should really support non-ssl http connections through proxies without enabling some sort of setting - much like we don't allow http basic authentication over non-ssl http.

[civodul]

@ethomson Oh you're right, changing the URL to https does the trick:

22576 connect(3, {sa_family=AF_INET, sin_port=htons(8118), sin_addr=inet_addr("127.0.0.1")}, 16) = 0
22576 sendto(3, "CONNECT example.org:443 HTTP/1.1\r\nUser-Agent: git/2.0 (libgit2 1.0.1)\r\nHost: localhost\r\n\r\n", 90, 0, NULL, 0) = 90

FWIW I found about this regression via a unit test of Guile-Git: https://gitlab.com/guile-git/guile-git/-/blob/master/tests/proxy.scm. From that respect, I would think it's still a regression.

WDYT?

Thank you!

[ethomson]

I agree, does the code change that I suggested resolve it for you or are you not able to recompile libgit2 from source?

[civodul]

The change you suggest allows libgit2 to connect to the proxy for an http URL. However, it sill makes a CONNECT request when it should be a GET:

connect(3, {sa_family=AF_INET, sin_port=htons(8118), sin_addr=inet_addr("127.0.0.1")}, 16) 
= 0
sendto(3, "CONNECT example.org:80 HTTP/1.1\r\nUser-Agent: git/2.0 (libgit2 1.0.0)\r\nHost: 
localhost\r\n\r\n", 89, 0, NULL, 0) = 89
recvfrom(3, "HTTP/1.1 200 Connection established\r\n\r\n", 16384, 0, NULL, NULL) = 39
[...]
write(2, "last error: SSL error: error:1408F10B:SSL routines:ssl3_get_record:wrong version number\n", 88) = 88

[bbain]

I have also run into this issue. This is indeed a problem when using a HTTP Proxy to talk to a HTTP remote. When the git client is configured in the same way it correctly makes a connection to the HTTP proxy with the full URL in the request so the proxy knows which server to forward it to.

This code in libgit2 seems to generate the correct request for a HTTP Proxy -> HTTP Upstream (i.e. it has the full URL of the remote in the request) its just that libgit2 does not connect to the proxy to make the request. It looks like libgit2 only supports CONNECT style HTTP proxy, that is, a HTTP proxy -> HTTPS Upstream.