[JENKINS-37934] Add support for JGit's Apache's HTTP client and preemptive authentication

[olivierdagenais]

Situation

A few users have noticed that the Git plugin was ignoring the configured credentials (and instead tried to use the process's identity) under the following conditions:

• A job uses the Git SCM, configured to use the JGit "installation"
• The Git repository is hosted in Team Foundation Server
• The Jenkins build executes on Windows

Because JGit, by default, uses the JDKHttpConnectionFactory (which means URL#openConnection()), it turns out the JVM, on Windows, will attempt NTLM authentication using the identity of the current process and JGit's TransportHttp does not support NTLM as an authentication method when the attempt comes back with HTTP 401.

Action

1. Add org.eclipse.jgit:org.eclipse.jsit.http.apache as a dependency.
2. Introduce the JGit with Apache HTTP client "installation" option, illustrated in the screenshot below:
   
3. When the new "installation" above is used, enable the new PreemptiveAuthHttpClientConnection[Factory], which will attempt to authenticate with the provided credentials, bypassing JGit's authentication mechanism.

Result

By using the JGit with Apache HTTP client "installation", I was able to clone from a variety of repositories, without having to launch Jenkins using a domain identity that had TFS permissions on Windows. In fact, I tried to get Windows integrated authentication working and it mostly worked, except when I hit a TFS server that had Negotiate (Kerberos) enabled; JGit's TransportHttp noticed that Negotiate was used and then tried to "configure the connection" the second time around, but this failed because JGit expected explicit credentials to be provided. If integrated authentication from Windows nodes is needed, there's still the original JGit "installation" that can be configured on a per-job basis.

Please let me know if you would like me to add or change anything.

Manual testing

1. Deployed git-client.hpi from this branch to a Jenkins server.
2. Added the JGit with Apache HTTP client "installation" and clicked [Save].
3. Created the following Multi-configuration jobs, each with a single operating_system Slaves axis to run the same job on Linux (Ubuntu 14.6), Mac OS X (10.10.5) and Windows (2012 R2), using the jgitapache Git executable:
   1. GitHub HTTPS
      1. Repository URL: https://github.com/jenkinsci/tfs-plugin.git
      2. Credentials: - none -, because the repository is publicly-available
   2. GitHub SSH
      1. Repository URL: git@github.com:jenkinsci/tfs-plugin.git
      2. Credentials: Created a key pair and configured Jenkins with it.
   3. GitLab HTTP
      1. Repository URL: A Git repo in a default GitLab CE instance on the LAN
      2. Credentials: Username & password of the GitLab root user.
   4. Team Services HTTPS
      1. Repository URL: A Git repo in a Visual Studio Team Services account
      2. Credentials: E-mail address and personal access token (PAT)
   5. TFS HTTP domain
      1. Repository URL: A Git repo in a Team Foundation Server on the same domain
      2. Credentials: Domain username & password, with the username portion in the form user
   6. TFS HTTP off-domain, domain slash user
      1. Repository URL: A Git repo in a Team Foundation Server on the same domain
      2. Credentials: Domain username & password, with the username portion in the form DOMAIN\user
   7. TFS HTTP off-domain, user
      1. Repository URL: A Git repo in a Team Foundation Server on a different domain
      2. Credentials: Domain username & password, with the username portion in the form user
   8. TFS HTTP off-domain, user at domain
      1. Repository URL: A Git repo in a Team Foundation Server on a different domain
      2. Credentials: Domain username & password, with the username portion in the form user@domain
   9. TFS HTTP off-domain, user at fully-qualified domain
      1. Repository URL: A Git repo in a Team Foundation Server on a different domain
      2. Credentials: Domain username & password, with the username portion in the form user@fully.qualified.domain.example.com
4. All the jobs succeeded, except the last one. It seems to be a limitation of the NTCredentials class. Not a big deal as three other forms of the user name were accepted.

Mission accomplished!

[MarkEWaite]

@olivierdagenais could you review my evaluation branch? I have run various interactive tests and automated tests and believe your pull request is ready to merge, but would appreciate it very much if you would review the changes I made to resolve the conflicts from the master branch, and to set the line endings on 2 files to Unix style line endings.

[olivierdagenais]

@MarkEWaite I eventually figured out how to compare your branch against mine and they appear to be functionally identical. If you're happy with the changes, I'm happy. 👍

Cheers,

• Oli