No support for authenticated proxies

[byjrack]

Describe the bug

Desktop fails a clone with a HTTP/407

Version & OS

macOS 10.15.2
Tried GA and 2.5.3-beta1

Steps to reproduce the behavior

1. Clone repo

Expected behavior

Repo is cloned

Actual behavior

Error noting the 407 because it couldn't handle the authentication.

In the log I could see
2020-06-16T19:15:10.777Z - info: [ui] proxy url not resolved, https_proxy already set
Unclear if this is from gitconfig or envvar. Tried to make sure they were in sync, but always get the 407 even w user:pass@proxyurl in the envvar.

Additional context

Be nice to offer a fallback to use the git client as it handles the 407 with existing proxy statements in gitconfig including prompting for password via helper if needed. Rust's cargo has a flag (git-fetch-with-cli = true) to do this similar operation as the mix of configs, envvars, and system prefs can sometimes be challenging to figure out.

[tierninho]

@byjrack Thank you for the issue and apology for the troubles.

Please see a workaround here:
#4872 (comment)

The --unset flag can fix the issue where Git will attempt to establish a connection through an invalid proxy configuration (i.e. the user is actually not behind a proxy), which causes Git operations to error out. If you are using a proxy then you will want to keep that proxy configuration line in order to ensure Git operations work successfully.

Let us know how it goes.

[byjrack]

So I am behind a proxy and currently gitconfig works well to clone using cli, but just not with Desktop. What is the logic for Desktop when it comes to electing a proxy and does it support the targeted option?

[outofambit]

@byjrack i'll let @niik give a high level summary for the proxy logic, but yes, it does support the target option you linked to there.

i believe the error you are seeing in the log (proxy url not resolved, https_proxy already set) is because desktop is skips proxy resolution when it is already set in the gitconfig. if you can try @tierninho's suggestion above and let us know if it makes a difference, that would be helpful for further troubleshooting. thank you!

[byjrack]

So can it handle requesting the credentials like the git client?

Not sure I follow removing the thing that works either. What should be the result of removing the git config? I would assume it will just get no route because it wouldn't have a proxy config any longer?

[byjrack]

And to just confirm running git config --global --unset https.https://github.com.proxy didn't change the experience. It still seems that it was sourcing from env and not gitconfig as I could see "2020-06-17T16:45:17.977Z - info: [ui] proxy url not resolved, https_proxy already set" in the log.

Still stuck at fatal: unable to access 'https://github.com/.../': Received HTTP code 407 from proxy after CONNECT

[tierninho]

@byjrack

Still stuck at fatal: unable to access 'https://github.com/.../': Received HTTP code 407 from proxy after CONNECT

Based on that error, I did spot one potential solution which would force git to send the credentials and authentication method to the proxy:

git config --global http.proxyAuthMethod 'basic'

Source: https://git-scm.com/docs/git-config#git-config-httpproxyAuthMethod

@niik for thoughts on this

In the meantime, if you are interested in our setup and logic, I'd start with this pull request: #9154

[byjrack]

Yup it is an authenticated proxy so I will need to send an auth so not sure that PR would cover my issue. We also use WPAD so looking at that PR could it be that Desktop is using the system config (doesn't allow for using keychain) and because it wasn't configured to handle the auth request it is just failing? I also have Slack going and it will prompt for creds on a 407 so it's not a limit in Electron itself.

In any case (system, env, or gitconfig) unless Desktop can handle the 407 I will need to hardcode the secret in the proxyURL that desktop uses.

[byjrack]

And tracked down this page trying to set through the code which was pretty helpful.
https://github.com/desktop/desktop/blob/development/docs/technical/proxies.md

Reading that it seems like if the envvar is set than Desktop doesn't check w the OS. And in that case the envvar has to have the user:pass@proxy structure for an authd proxy or have some mechanism. Toying with my zshrc to see if I can make it work well ideally w/o a secret hard coded. Be nice if I could control the overridden credentialhelper as that could help with keychain access.

[niik]

Hi @byjrack, sorry for taking so long to get back to you.

Reading that it seems like if the envvar is set than Desktop doesn't check w the OS

Yes, you're entirely correct, if we see http_proxy or ALL_PROXY in the environment we will not consult the operating system. Even if that wasn't the case we currently don't support authenticating proxies when automatically resolving the proxy.

And in that case the envvar has to have the user:pass@proxy structure for an authd proxy

Correct, GitHub Desktop explicitly unsets the default credential helper for any network operation in order to coax Git to ask our ASKPASS helper through which we can then deliver the stored username and token from your authenticated session in GitHub Desktop.

desktop/app/src/lib/git/core.ts

Lines 403 to 408 in d6f7518

	const baseArgs = [
	// Explicitly unset any defined credential helper, we rely on our
	// own askpass for authentication.
	'-c',
	'credential.helper=',
	]

We don't unset the credential.<url>.helper as of right now but we're considering doing so since we've seen some users not being able to use GitHub Desktop due to having url-specific helpers configured. It's possible you could get it to work by adding url-specific helpers to your global config but then you'd be responsible for managing the credentials yourself.

One thing that I don't understand here is how you were able to sign in to the app itself. I was under the (perhaps faulty) assumption that GitHub Desktop itself (or rather Electron) would fail to make requests when located behind an authenticating proxy.

[byjrack]

No worries this is low priority stuff.

Can I assume the clone is always done with https (from the log it appears so) and not git://? I have a .insteadof in my gitconfig to handle that, but that was one of the questions I had when the API worked while the clone operation didn't. My proxy does cache auth for sessions so could be a timing issue mixed in.

I follow using the secret from Desktop for the Hub endpoints, but that wouldn't work for my proxy. In my case if I have https.proxy I will get a prompt in my shell based around the HTTP/407 and it will be cached in keychain as well using the osxhelper. And my GitHub User ID password is different than the network identity used for the proxy so there would be two independent challenges assuming the Hub action was authenticated. Electron/GHD would need to detect the 407 and prompt for credentials. Now I am on 2.5.3-b2 for Desktop which targets electron 7.1.8 which looked to have electron/electron#21135 merged so electron itself I think should support the 407. The ask_pass looks to be specifically focused on a hub endpoint challenge as well so not sure that would cover a proxy auth need.

Going to test out the URL'd credhelper and see if that can fix things. I need to stay off the browser though for a bit so that cached auth clears.

[niik]

which targets electron 7.1.8 which looked to have electron/electron#21135 merged so electron itself I think should support the 407.

I think we'd still need to subscribe to that event and fill in the proxy credentials which we currently don't do.

Can I assume the clone is always done with https (from the log it appears so) and not git://?

For GitHub.com I believe we respect your preference from the website so yes, as long as that's your default clone protocol we'll always use that. For GitHub Enterprise Server the administrator can override the default for all users.

The ask_pass looks to be specifically focused on a hub endpoint challenge as well so not sure that would cover a proxy auth need.

You're right, it wouldn't, I was using it as an example to explain why we're unsetting the credential helper.

[byjrack]

Is there any way to up the log level especially with env and gitconfig data? I am trying to confirm if the URL'd helper is meeting the proxy auth need, but want to be sure Desktop has the config I expect it to. I just see the https_proxy already set event, but that's not super helpful.

[niik]

Is there any way to up the log level especially with env and gitconfig data?

@byjrack I'm afraid the only thing I can think of would be opening the DevTools console and making sure you've got all levels enabled but I don't think we do much debug logging (which is the only category that doesn't get written to disk IIRC) in the proxy path.

[vinh-duongquoc]

@byjrack I also tried to fix this issue with my colleagues in team
By following many solutions, one of these work

This is my situation:

• The popup error message is same as your case
  fatal: unable to access 'https://xxx/.../': Received HTTP code 407 from proxy after CONNECT
• The log file also has this line
  info: [ui] proxy url not resolved, https_proxy already set
• My .gitconfig file is stored at U:\.gitconfig directory

My solution:

• Step 1: move .gitconfig file to another disk that your user account has full permission, e.g: C:\
• Step 2: add/update environment variables "HOME" value to folder directory of step 1
• Step 3: (Optional)
  Try http.proxy value of one of these value in .gitconfig file
  proxy = http://<username>:<pass>@<proxy host name>:<port>
proxy = http://<proxy host name>:<port>
• Step 4: Close and start Github Desktop and try again

@tierninho I have a question that my solution seems doesn't relate to the popup error message and log file content
That takes us a lot of time to find out the real solution, because I have to try all possible solutions that we can find in the internet

Hope this solution can help you

[SadPencil]

Please re-open issue #13975 as it is not relevant with issue #10026.