Authentication does not work with for NTLM-authorized git repositories

[ccpp]

My composer.json:

{
	"repositories": [
		{ "type": "composer", "url": "https://composer.typo3.org/" },
		{ "type": "git", "url": "http://***:8080/tfs/DefaultCollection/***" }
	],
	(...)
}

Output of php56 ~/bin/composer.phar diagnose:

Checking composer.json: WARNING
License "GPL-2.0+" is a deprecated SPDX license identifier, use "GPL-2.0-or-later" instead
require.*** : exact version constraints (1.1.7) should be avoided if the package follows semantic versioning
require.*** : unbound version constraints (dev-master as 1.1.0) should be avoided
Checking platform settings: OK
Checking git settings: OK
Checking http connectivity to packagist: OK
Checking https connectivity to packagist: OK
Checking github.com rate limit: OK
Checking disk free space: OK
Checking pubkeys: 
Tags Public Key Fingerprint: 57815BA2 7E54DC31 7ECC7CC5 573090D0  87719BA6 8F3BB723 4E5D42D0 84A14642
Dev Public Key Fingerprint: 4AC45767 E5EC2265 2F0C1167 CBBB8A2B  0C708369 153E328C AD90147D AFE50952
OK
Checking composer version: OK
Composer version: 1.6.3
PHP version: 5.6.32
PHP binary path: /opt/remi/php56/root/usr/bin/php

When I run this command:

GIT_CURL_VERBOSE=1 php56 ~/bin/composer.phar install
# or also
GIT_CURL_VERBOSE=1 php56 /usr/bin/composer install

I get the following output: (having configured auth.json correctly, but also when entering the credentials by hand)

  [RuntimeException]                                                                                                                                                                         
  Failed to execute git clone --no-checkout 'http://***:***@***:8080/tfs/DefaultCollection/*** '/home/c  
  pl/src/***' && cd '/home/cpl/src/***' && git remote add composer 'http://***:***@***:8080/tfs/DefaultCollection/***' && git fetch composer                                         

(...)

  < HTTP/1.1 401 Unauthorized                                                                                                                                                                
  < Content-Type: text/html; charset=utf-8                                                                                                                                                   
  < Server: Microsoft-IIS/8.5                                                                                                                                                                
  < X-TFS-ProcessId: ***                                                                                                                                    
  < ActivityId: ***
  < X-TFS-Session: ***
  < X-VSS-E2EID: ***
  < X-FRAME-OPTIONS: SAMEORIGIN                                                                                                                                                              
  < X-TFS-SoapException: %3c%3fxml+version%3d%221.0%22+encoding%3d%22utf-8%22%3f%3e%3csoap%3aEnvelope+xmlns%3asoap%3d%22http%3a%2f%2fwww.w3.org%2f2003%2f05%2fsoap-envelope%22%3e%3csoap%3a  
  Body%3e%3csoap%3aFault%3e%3csoap%3aCode%3e%3csoap%3aValue%3esoap%3aReceiver%3c%2fsoap%3aValue%3e%3csoap%3aSubcode%3e%3csoap%3aValue%3eUnauthorizedRequestException%3c%2fsoap%3aValue%3e%3  
  c%2fsoap%3aSubcode%3e%3c%2fsoap%3aCode%3e%3csoap%3aReason%3e%3csoap%3aText+xml%3alang%3d%22en%22%3eTF400813%3a+Resource+not+available+for+anonymous+access.+Client+authentication+require  
  d.%3c%2fsoap%3aText%3e%3c%2fsoap%3aReason%3e%3c%2fsoap%3aFault%3e%3c%2fsoap%3aBody%3e%3c%2fsoap%3aEnvelope%3e                                                                              
  < X-TFS-ServiceError: TF400813%3a+Resource+not+available+for+anonymous+access.+Client+authentication+required.                                                                             
  < WWW-Authenticate: Bearer                                                                                                                                                                 
  < WWW-Authenticate: Basic realm="http://***:8080/tfs"                                                                                                                      
  * gss_init_sec_context() failed: : No Kerberos credentials available (default cache: KEYRING:persistent:1002)                                                                              
  < WWW-Authenticate: Negotiate                                                                                                                                                              
  < WWW-Authenticate: NTLM                                                                                                                                                                   
  < X-Powered-By: ASP.NET                                                                                                                                                                    
  < P3P: CP="CAO DSP COR ADMa DEV CONo TELo CUR PSA PSD TAI IVDo OUR SAMi BUS DEM NAV STA UNI COM INT PHY ONL FIN PUR LOC CNT"                                                               
  < Lfs-Authenticate: NTLM                                                                                                                                                                   
  < X-Content-Type-Options: nosniff                                                                                                                                                          
  < Date: Tue, 06 Feb 2018 15:08:57 GMT                                                                                                                                                      
  < Content-Length: 20200                                                                                                                                                                    
  <                                                                                                                                                                                          
  * Connection #0 to host *** left intact                                                                                                                                    
  fatal:***@***:8080/tfs/DefaultCollection/***'                                                                         
                                                                                                                                                                                             

install [--prefer-source] [--prefer-dist] [--dry-run] [--dev] [--no-dev] [--no-custom-installers] [--no-autoloader] [--no-scripts] [--no-progress] [--no-suggest] [-v|vv|vvv|--verbose] [-o|--optimize-autoloader] [-a|--classmap-authoritative] [--apcu-autoloader] [--ignore-platform-reqs] [--] [<packages>]...

And I expected this to happen:
a successful clone of the dependency :)

Further notes:

Without composer, a clone is possible only when user and password are not specified inside the URL.
However, it does work when

• using either kerberos (kinit) and specifying the username only in the repository URL, or
• entering the credentials manually using GIT_ASKPASS.

However: composer does everything that these options do not work:

• There is no way around writing the credentials into the remote URL
• GIT_ASKPASS is being unset, and
• Even if a kerberos ticket exists, composer asks for username/password.

IMO there are these possible solutions:

• (optionally) keep GIT_ASKPASS environment variable
• (optionally) remove all that wrapping around credentials input including rewriting the origin url and let git manage/ask the credentials,
• wrap with a proper GIT_ASKPASS (as Jenkins is doing it, for example) instead of putting username and password into the URL.

[ccpp]

... and I did read #4524 and #4697, but they are completely unrelated: This here is not about a Server error, but a valid "401 Unauthorized".

[Seldaek]

What do you set GIT_ASKPASS to to get it to work? Just curious because I have no idea how Kerberos/NTLM works. I think letting GIT_ASKPASS go through if it looks like a sane input might be a good option. Right now we always override it because it lead to issues where git prompts but it's not going through to the user so we end up blocking with a prompt that will never get any input. GIT_ASKPASS=echo fixes that.

[ccpp]

On the one hand, when not using Composer, it just works with any setting because git tries to use GIT_ASKPASS, and if that does not work (e.g. because of a missing DISPLAY), then git asks for the credentials on the console. Setting GIT_ASKPASS=echo does not work in this case because echo does not actually fail, it just echoes empty credentials. In this case, GIT_ASKPASS=false would be a better because false is designed to fail :)

But answering your question while looking at the example from Jenkins CI: they just built a temporary command such as /tmp/pass4619352834734512307.sh which just echoes username/password forward to git. Composer could either do the same thing or keep the askpass setting.

An alternative solution might be to have a configurable whitelist (with entries like, for example /tmp/pass*, to make it work below Jenkins, or **/ssh-askpass to make it work with manual input below ssh -X)

[Seldaek]

Setting GIT_ASKPASS=echo does not work in this case because echo does not actually fail, it just echoes empty credentials. In this case, GIT_ASKPASS=false would be a better because false is designed to fail :)

The problem is, we don't want git to prompt the user on the console, because that won't go through the composer process executor and it'll block there endlessly AFAIK. That's why we provide GIT_ASKPASS echo to make sure that if git prompts it gets "credentials" and then fails hard with invalid credentials so that we can then ask the user ourselves for credentials.

I'm still not sure how to best proceed here.

[Seldaek]

Closing as nobody cared to fix it in these many years, if still applicable do tell and I will reopen.