Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

pip will hang asking for the password of an index server even if the output is redirected #2920

Open
RonnyPfannschmidt opened this issue Jun 21, 2015 · 8 comments
Labels
help wanted For requesting inputs from other members of the community state: awaiting PR Feature discussed, PR is needed type: bug A confirmed bug or unintended behavior

Comments

@RonnyPfannschmidt
Copy link
Contributor

No description provided.

@rbtcollins
Copy link

pip can't tell the difference between | tee my.log (where you can see the prompt and enter it) and > log.txt.

If you want to run it headless, be sure to redirect input < /dev/null as well. That should make it error rather than hang. If it doesn't, then thats a bug, but on the information so far, I don't see the bug.

@untitaker
Copy link

pip should be able to tell the difference. sys.stdout.isatty can be used for this.

@pradyunsg pradyunsg added the type: bug A confirmed bug or unintended behavior label Oct 17, 2017
@pradyunsg
Copy link
Member

pradyunsg commented Oct 17, 2017

@pypa/pip-committers

Between "redirect input < /dev/null" vs "tell the difference sys.stdout.isatty"; which way do you guys think we should go?

/note-to-self mark as awaiting PR once decided.

@chrahunt
Copy link
Member

chrahunt commented Jul 6, 2019

I can confirm the situation has not changed with pip 19.1.1:

Given server.py:

from http.server import BaseHTTPRequestHandler, HTTPServer


class PromptForPassword(BaseHTTPRequestHandler):
    def do_GET(self):
        self.send_response(401)
        self.send_header('WWW-Authenticate', 'Basic')
        self.end_headers()


if __name__ == '__main__':
    server = HTTPServer(('', 8888), PromptForPassword)
    server.serve_forever()

Run python server.py >/dev/null 2>&1 &.

Then the following is pip's current behavior:

No redirection, both "User" and "Password" prompts are visible:

$ pip install --index-url http://localhost:8888/simple requests
Looking in indexes: http://localhost:8888/simple
Collecting requests
User for localhost:8888: guest
Password:
  WARNING: 401 Error, Credentials not correct for http://localhost:8888/simple/requests/
  ERROR: Could not find a version that satisfies the requirement requests (from versions: none)
ERROR: No matching distribution found for requests

With redirection, the "User" prompt is not visible and the "Password" prompt is visible:

$ pip install --index-url http://localhost:8888/simple requests > /dev/null
guest
Password:
  WARNING: 401 Error, Credentials not correct for http://localhost:8888/simple/requests/
  ERROR: Could not find a version that satisfies the requirement requests (from versions: none)
ERROR: No matching distribution found for requests

A quick test confirms that isatty cannot distinguish between a user-visible prompt and one that is redirected to a file:

$ alias example='python -c '"'"'import sys, pathlib; pathlib.Path("o").write_text(f"{sys.stdin.isatty()}; {sys.stdout.isatty()}\n")'"'"
$ example
True;True
$ example >/dev/null && cat o
True;False
$ example | tee /dev/null && cat o
True;False
$ example </dev/null && cat o
False;True

I think it is very common to behave differently for TTYs, with an option to force the behavior (see e.g. diff, git, ls and color output), so we should not worry about trying to prompt in the edge case where a user may be able to see the prompt but isatty is False.


I would propose the following approach: only prompt if sys.stdout.isatty() and sys.stdin.isatty() and then add a command-line flag (--no-prompt-auth, --prompt-auth) to force one way or another for users that want to ensure correct behavior.

This would give us an opportunity to give a descriptive error message instead of relying on the fact that </dev/null causes an exception:

$ pip install --index-url http://localhost:8888/simple requests < /dev/null
Looking in indexes: http://localhost:8888/simple
Collecting requests
User for localhost:8888: ERROR: Exception:
Traceback (most recent call last):
  File "/tmp/pip/2920/.venv/lib/python3.7/site-packages/pip/_internal/cli/base_command.py", line 178, in main
    status = self.run(options, args)
  File "/tmp/pip/2920/.venv/lib/python3.7/site-packages/pip/_internal/commands/install.py", line 352, in run
    resolver.resolve(requirement_set)
  File "/tmp/pip/2920/.venv/lib/python3.7/site-packages/pip/_internal/resolve.py", line 131, in resolve
    self._resolve_one(requirement_set, req)
  File "/tmp/pip/2920/.venv/lib/python3.7/site-packages/pip/_internal/resolve.py", line 294, in _resolve_one
    abstract_dist = self._get_abstract_dist_for(req_to_install)
  File "/tmp/pip/2920/.venv/lib/python3.7/site-packages/pip/_internal/resolve.py", line 242, in _get_abstract_dist_for
    self.require_hashes
  File "/tmp/pip/2920/.venv/lib/python3.7/site-packages/pip/_internal/operations/prepare.py", line 282, in prepare_linked_requirement
    req.populate_link(finder, upgrade_allowed, require_hashes)
  File "/tmp/pip/2920/.venv/lib/python3.7/site-packages/pip/_internal/req/req_install.py", line 198, in populate_link
    self.link = finder.find_requirement(self, upgrade)
  File "/tmp/pip/2920/.venv/lib/python3.7/site-packages/pip/_internal/index.py", line 766, in find_requirement
    candidates = self.find_candidates(req.name, req.specifier)
  File "/tmp/pip/2920/.venv/lib/python3.7/site-packages/pip/_internal/index.py", line 752, in find_candidates
    self.find_all_candidates(project_name),
  File "/tmp/pip/2920/.venv/lib/python3.7/site-packages/pip/_internal/index.py", line 716, in find_all_candidates
    for page in self._get_pages(url_locations, project_name):
  File "/tmp/pip/2920/.venv/lib/python3.7/site-packages/pip/_internal/index.py", line 846, in _get_pages
    page = _get_html_page(location, session=self.session)
  File "/tmp/pip/2920/.venv/lib/python3.7/site-packages/pip/_internal/index.py", line 231, in _get_html_page
    resp = _get_html_response(url, session=session)
  File "/tmp/pip/2920/.venv/lib/python3.7/site-packages/pip/_internal/index.py", line 179, in _get_html_response
    "Cache-Control": "max-age=0",
  File "/tmp/pip/2920/.venv/lib/python3.7/site-packages/pip/_vendor/requests/sessions.py", line 546, in get
    return self.request('GET', url, **kwargs)
  File "/tmp/pip/2920/.venv/lib/python3.7/site-packages/pip/_internal/download.py", line 439, in request
    return super(PipSession, self).request(method, url, *args, **kwargs)
  File "/tmp/pip/2920/.venv/lib/python3.7/site-packages/pip/_vendor/requests/sessions.py", line 533, in request
    resp = self.send(prep, **send_kwargs)
  File "/tmp/pip/2920/.venv/lib/python3.7/site-packages/pip/_vendor/requests/sessions.py", line 653, in send
    r = dispatch_hook('response', hooks, r, **kwargs)
  File "/tmp/pip/2920/.venv/lib/python3.7/site-packages/pip/_vendor/requests/hooks.py", line 31, in dispatch_hook
    _hook_data = hook(hook_data, **kwargs)
  File "/tmp/pip/2920/.venv/lib/python3.7/site-packages/pip/_internal/download.py", line 233, in handle_401
    username = six.moves.input("User for %s: " % parsed.netloc)
EOFError: EOF when reading a line

It's probably worth considering that the person that would set up scripts/CI to use this approach would not be the same one debugging it later faced with the exception above. Having an explicit option will be cross-platform, easier to google, and easier to test.

@pradyunsg
Copy link
Member

Thanks for the investigation here @chrahunt!

Hmm, seems like requests is showing that prompt. Maybe we can tell it not to do that and raise an exception instead?

@cs01
Copy link

cs01 commented Jan 2, 2020

Just a note that pipx is impacted by this behavior and would benefit from this change, pypa/pipx#219.

Ideally we would be able to pass a --non-interactive or --ci flag, but we could also work with pip being tty-aware and just call pip with output redirected.

@chrahunt
Copy link
Member

chrahunt commented Jan 2, 2020

Have you tried --no-input? It's not documented for some reason (see #2429 (comment)) but should work to avoid prompting for HTTP-based URLs.

@cs01
Copy link

cs01 commented Jan 2, 2020

I haven't, but that looks like exactly what we're looking for. Thank you!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
help wanted For requesting inputs from other members of the community state: awaiting PR Feature discussed, PR is needed type: bug A confirmed bug or unintended behavior
Projects
None yet
Development

No branches or pull requests

6 participants