Using a private source repository with login and no verify_ssl the SSL verfication fails

[Gijom]

When using a private repository with login and setting verify_ssl = false the resulting pip command is not correct and the installation of a package fails.

$ python -m pipenv.help output

Pipenv version: '2018.6.25'

Pipenv location: '/usr/lib/python3.6/site-packages/pipenv'

Python location: '/usr/bin/python'

Other Python installations in PATH:

• 3.6: /usr/bin/python3.6m

• 3.6: /usr/bin/python3.6

• 3.6: /bin/python3.6

• 3.6.5: /usr/bin/python

• 3.6.5: /bin/python

• 3.6.5: /usr/bin/python3

• 3.6.5: /bin/python3

PEP 508 Information:

{'implementation_name': 'cpython',
 'implementation_version': '3.6.5',
 'os_name': 'posix',
 'platform_machine': 'x86_64',
 'platform_python_implementation': 'CPython',
 'platform_release': '2.10.0(0.325/5/3)',
 'platform_system': 'MSYS_NT-10.0',
 'platform_version': '2018-02-09 15:25',
 'python_full_version': '3.6.5',
 'python_version': '3.6',
 'sys_platform': 'msys'}

System environment variables:

• USERDOMAIN
• OS
• COMMONPROGRAMFILES
• PROCESSOR_LEVEL
• PSModulePath
• CommonProgramW6432
• CommonProgramFiles(x86)
• NetHome
• MSYSCON
• LANG
• TZ
• MSYSTEM_CARCH
• HOSTNAME
• PUBLIC
• OLDPWD
• CONFIG_SITE
• WD
• CONTITLE
• MSYSTEM_CHOST
• UATDATA
• LOGINSHELL
• USERNAME
• LOGONSERVER
• PROCESSOR_ARCHITECTURE
• tmp
• LOCALAPPDATA
• COMPUTERNAME
• USER
• !::
• DEFLOGDIR
• SYSTEMDRIVE
• USERPROFILE
• PATHEXT
• SYSTEMROOT
• USERDOMAIN_ROAMINGPROFILE
• PROCESSOR_IDENTIFIER
• PWD
• HOME
• TMP
• MSYSTEM_PREFIX
• https_proxy
• http_proxy
• LocalShare
• !C:
• PROCESSOR_REVISION
• USERDNSDOMAIN
• PROMPT
• NUMBER_OF_PROCESSORS
• ProgramW6432
• COMSPEC
• APPDATA
• SHELL
• TERM
• WINDIR
• ProgramData
• SHLVL
• PRINTER
• PROGRAMFILES
• MANPATH
• ORIGINAL_TEMP
• ORIGINAL_TMP
• ALLUSERSPROFILE
• TEMP
• temp
• MSYSTEM
• SESSIONNAME
• ProgramFiles(x86)
• PATH
• PS1
• HOMEDRIVE
• PKG_CONFIG_PATH
• INFOPATH
• HOMEPATH
• ORIGINAL_PATH
• LocalHome
• VS140COMNTOOLS
• _
• PYTHONDONTWRITEBYTECODE
• PIP_PYTHON_PATH

Pipenv–specific environment variables:

Debug–specific environment variables:

• PATH: /usr/local/bin:/usr/bin:/bin:/opt/bin:/c/Windows/System32:/c/Windows:/c/Windows/System32/Wbem:/c/Windows/System32/WindowsPowerShell/v1.0/
• SHELL: /usr/bin/bash
• LANG: en_US.UTF-8
• PWD: /home/USER

---

Contents of Pipfile ('/home/USER/Pipfile'):

[[source]]
url = "https://$USER:${PASS}@myserver.corp/pypi/simple"
verify_ssl = false
name = "artifactory"

[packages]

[dev-packages]

[requires]
python_version = "3.6"

Contents of Pipfile.lock ('/home/USER/Pipfile.lock'):

{
    "_meta": {
        "hash": {
            "sha256": "415dfdcb118dd9bdfef17671cb7dcd78dbd69b6ae7d4f39e8b44e71d60ca72e7"
        },
        "pipfile-spec": 6,
        "requires": {
            "python_version": "3.6"
        },
        "sources": [
            {
                "name": "pypi",
                "url": "https://pypi.org/simple",
                "verify_ssl": true
            }
        ]
    },
    "default": {},
    "develop": {}
}

---

Expected result

When using USER="name" PASS="test" pipenv install numpy -v I am expecting to have the numpy package installed.

Note that the USER and PASS variables are set in my Pipfile (see details above)

Actual result

When using the command the SSL authentication fails. And this is not surprising given the corresponding generated pip command obtained using the -v option:
/home/USER/.local/share/virtualenvs/USER-_5wChY8u/bin/pip" install --verbose "numpy" -i https://name:test@myserver.corp/pypi/simple --trusted-host name --exists-action w

As can be seen the trusted-host parameter is not the server name but the user name...

Additional comments and questions

• I am not sure this is a very good idea to show the login and password when using a verbose output (as it appears in the resulting pip command after the -i option)
• I am using verify_ssl = false because I could not find any certificate option (I tried cert = path/to/cert/pem/format without success - it does not appear as a --cert pip option)

[Gijom]

I think I found the culprit:
vendor/requirementslib/utils.py line 162

I think the line:
["--trusted-host", urlparse(sources[0]["url"]).netloc.split(":")[0]]
might/should be:
["--trusted-host", urlparse(sources[0]["url"]).hostname]

As for the other list of sources. Is there a specifc reason why this is different then for other sources ? If yes than an alternative would be to configure a second source as a private repo and keep the first pypi source in the Pipfile.

[techalchemy]

Thanks for tracking this down, I'm not totally sure -- @uranusjr do you remember?

[uranusjr]

Ah… I think I was trying to be too cute with urlparse’s SplitResult. hostname is not a namedtuple attribute, and I feared (I think) our _replace calls might ruin it. I’ll need to dig into the docs (and maybe urlparse source) to make sure if we can safely use hostname. Or we need to implement some better parsing logic here.

I’ll look into this later tonight.

[Gijom]

I do not know about the _replace calls but just in case it was unclear: The few lines below line 162 applies to other hosts than the first. these lines are using hostname. So whatever solution you choose it should probably be applied to all sources in the same way ?

[techalchemy]

It’s a bit complicated because of how the parser works. But the split is there for ports I assume? You might want netloc for that?

[uranusjr]

Checked the API, hostname should work here. PR opened at sarugaku/requirementslib#22.

[techalchemy]

I wonder if this is related to #2389 also