pipenv shell fails to find /usr/local/bin/bash in pexpect #2496

keegancsmith · 2018-07-03T09:58:51Z

Issue description

I can't run pipenv shell. This is different to the other issues related to this issue, since my SHELL variable is correctly configured. It seems to be something pexpect is doing wrong.

Expected result

Successfully running pipenv shell

Actual result

$ pipenv --version
pipenv, version 2018.7.1
$ echo $SHELL
/usr/local/bin/bash
$ stat $SHELL
16777220 8618578105 lrwxr-xr-x 1 keegan admin 0 30 "Jun 15 08:03:49 2018" "Jun 15 08:03:49 2018" "Jun 15 08:03:49 2018" "Jun 15 08:03:49 2018" 4194304 0 0 /usr/local/bin/bash
$ pipenv shell --verbose
Launching subshell in virtual environment…
Traceback (most recent call last):
  File "/usr/local/bin/pipenv", line 11, in <module>
    sys.exit(cli())
  File "/usr/local/lib/python3.7/site-packages/pipenv/vendor/click/core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/pipenv/vendor/click/core.py", line 697, in main
    rv = self.invoke(ctx)
  File "/usr/local/lib/python3.7/site-packages/pipenv/vendor/click/core.py", line 1066, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/local/lib/python3.7/site-packages/pipenv/vendor/click/core.py", line 895, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/lib/python3.7/site-packages/pipenv/vendor/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/pipenv/cli.py", line 664, in shell
    three=three, python=python, fancy=fancy, shell_args=shell_args, pypi_mirror=pypi_mirror
  File "/usr/local/lib/python3.7/site-packages/pipenv/core.py", line 2163, in do_shell
    shell.fork_compat(*fork_args)
  File "/usr/local/lib/python3.7/site-packages/pipenv/shells.py", line 107, in fork_compat
    self.cmd, ['-i'], dimensions=(dims.lines, dims.columns),
  File "/usr/local/lib/python3.7/site-packages/pipenv/vendor/pexpect/pty_spawn.py", line 204, in __init__
    self._spawn(command, args, preexec_fn, dimensions)
  File "/usr/local/lib/python3.7/site-packages/pipenv/vendor/pexpect/pty_spawn.py", line 276, in _spawn
    'executable: %s.' % self.command)
pipenv.vendor.pexpect.exceptions.ExceptionPexpect: The command was not found or was not executable: -/usr/local/bin/bash.

Note: /usr/local/bin/bash is a symlink to /usr/local/Cellar/bash/4.4.23/bin/bash (managed by brew). Setting SHELL to that value or /bin/bash both fail.

Steps to replicate

Provide the steps to replicate (which usually at least includes the commands and the Pipfile).

$ pipenv --support

Pipenv version: '2018.7.1'

Pipenv location: '/usr/local/lib/python3.7/site-packages/pipenv'

Python location: '/usr/local/opt/python/bin/python3.7'

Other Python installations in PATH:

2.7: /usr/local/bin/python2.7
2.7: /usr/local/bin/python2.7
2.7: /usr/bin/python2.7
3.7: /usr/local/bin/python3.7m
3.7: /usr/local/bin/python3.7
2.7.15: /usr/local/bin/python
2.7.10: /usr/bin/python
2.7.15: /usr/local/bin/python2
3.7.0: /usr/local/bin/python3

PEP 508 Information:

{'implementation_name': 'cpython',
 'implementation_version': '3.7.0',
 'os_name': 'posix',
 'platform_machine': 'x86_64',
 'platform_python_implementation': 'CPython',
 'platform_release': '17.6.0',
 'platform_system': 'Darwin',
 'platform_version': 'Darwin Kernel Version 17.6.0: Tue May  8 15:22:16 PDT '
                     '2018; root:xnu-4570.61.1~1/RELEASE_X86_64',
 'python_full_version': '3.7.0',
 'python_version': '3.7',
 'sys_platform': 'darwin'}

System environment variables:

STY
SRCPATH
MAILCHECK
LANG
XPC_FLAGS
HISTCONTROL
OLDPWD
TERM_SESSION_ID
EDITOR
COLORTERM
PERL_MB_OPT
__CF_USER_TEXT_ENCODING
LOCKPRG
ITERM_SESSION_ID
SSH_AUTH_SOCK
USER
GOPATH
PWD
HOME
WINDOW
TERM_PROGRAM
TERM_PROGRAM_VERSION
Apple_PubSub_Socket_Render
ITERM_PROFILE
PERL_MM_OPT
TMPDIR
PERL_LOCAL_LIB_ROOT
XPC_SERVICE_NAME
SHELL
TERM
PERL5LIB
COLORFGBG
SHLVL
LOGNAME
GOBIN
PATH
HISTSIZE
HISTFILESIZE
TERMCAP
_
PYTHONDONTWRITEBYTECODE
PIP_PYTHON_PATH

Pipenv–specific environment variables:

Debug–specific environment variables:

PATH: /Users/keegan/.cargo/bin:/usr/local/opt/go/libexec/bin:/Users/keegan/bin:/Users/keegan/google-cloud-sdk/bin:/Users/keegan/.yarn/bin:/Users/keegan/perl5/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/Users/keegan/.cargo/bin:/usr/local/opt/go/libexec/bin:/Users/keegan/bin:/Users/keegan/google-cloud-sdk/bin:/Users/keegan/.yarn/bin:/Users/keegan/perl5/bin:/Users/keegan/go/bin:/Users/keegan/go/bin
SHELL: /usr/local/bin/bash
EDITOR: vim
LANG: en_GB.UTF-8
PWD: /Users/keegan/go/src/github.com/weaveworks/kubediff

Contents of Pipfile ('/Users/keegan/go/src/github.com/weaveworks/kubediff/Pipfile'):

[[source]]
url = "https://pypi.org/simple"
verify_ssl = true
name = "pypi"

[packages]
attrs = "*"
tabulate = "*"
PyYAML = "*"

[dev-packages]

[requires]
python_version = "3.7"

The text was updated successfully, but these errors were encountered:

techalchemy · 2018-07-03T13:04:19Z

Hmmm interesting. @uranusjr does this not follow symlinks currently?

uranusjr · 2018-07-03T13:05:53Z

It shouldn’t need to. I use Hombrew Bash myself on Mac as well, and it works correctly. Given /bin/bash also fails I’m inclided to think this is caused by something else…

@keegancsmith Did pipenv shell frun correctly before this version?

keegancsmith · 2018-07-03T13:11:52Z

I only occasionally do python development these days, so the last time I would of used pipenv shell would of been a few months ago. I believe I used it back then, but I'm not 100% certain. I'm happy to try another version if you provide instructions. I have tried pipenv installed from both pip and brew.

uranusjr · 2018-07-03T13:14:32Z

It’s fine. We switched to a new shell implementation recently, and if it worked previously, it is probably related to the new implementation.

I will try to debug it when I have access to my main Mac (which has the same Homebrew setup as yours).

Whisprin · 2018-07-04T09:26:51Z

I'm encountering the same issue with zsh (5.5.1) on Arch Linux. So it's neither bash nor Mac specific. It was working just a few days ago.

% pipenv --version
pipenv, version 2018.7.1
% echo $SHELL
/usr/bin/zsh
% stat $SHELL
  File: /usr/bin/zsh
  Size: 844656    	Blocks: 1656       IO Block: 4096   regular file

It is still working on another machine with zsh + pipenv version 2018.6.25

uranusjr · 2018-07-04T10:32:01Z

@keegancsmith @Whisprin Can you run this with the Python you install Pipenv for me?

(For Homebrew, this would be /usr/local/Cellar/pipenv/2018.7.1/libexec/bin/python by default.)

from pipenv.shells import _detect_emulator, detect_info
print(_detect_emulator())
print(detect_info())

And if detect_info returns a tuple, could you check if the second command is valid?

Example (from my Mac):

$ python
>>> from pipenv.shells import _detect_emulator, detect_info
>>> print(_detect_emulator())

>>> print(detect_info())
('bash', '/usr/local/bin/bash')
>>>

$ /usr/local/bin/bash -c help
GNU bash, version 4.4.23(1)-release (x86_64-apple-darwin17.5.0)
These shell commands are defined internally.  Type `help' to see this list.
Type `help name' to find out more about the function `name'.
Use `info bash' to find out more about the shell in general.
Use `man -k' or `info' to find out more about commands not in this list.

[tons of output… OK this command works.]

keegancsmith · 2018-07-04T11:51:56Z

I see it is detecting a stray - prefix for the shell abs path.

$ /usr/local/Cellar/pipenv/2018.7.1/libexec/bin/python
Python 3.7.0 (default, Jun 29 2018, 20:13:13)
[Clang 9.1.0 (clang-902.0.39.2)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from pipenv.shells import _detect_emulator, detect_info
>>> print(_detect_emulator())

>>> print(detect_info())
('bash', '-/usr/local/bin/bash')
>>>
$ /usr/local/bin/bash -c help | head
GNU bash, version 4.4.23(1)-release (x86_64-apple-darwin17.5.0)
These shell commands are defined internally.  Type `help' to see this list.
Type `help name' to find out more about the function `name'.
Use `info bash' to find out more about the shell in general.
Use `man -k' or `info' to find out more about commands not in this list.

A star (*) next to a name means that the command is disabled.

 job_spec [&]                            history [-c] [-d offset] [n] or hist>
 (( expression ))                        if COMMANDS; then COMMANDS; [ elif C>

keegancsmith · 2018-07-04T12:02:14Z

So it seems shellingham parses the output of a ps command. Here is mine:

$ ps -ww -o pid= -o ppid= -o args=
 1480  1477 /Applications/iTerm.app/Contents/MacOS/iTerm2 --server login -fp keegan
 1482  1481 -bash
 1556  1482 screen
 1558  1557 -/usr/local/bin/bash
 1706  1558 /Applications/Emacs.app/Contents/MacOS/Emacs-x86_64-10_10 -nw
77061  1706 /usr/local/bin/aspell -a -m -B --encoding=utf-8
 1562  1557 -/usr/local/bin/bash
87033  1557 -/usr/local/bin/bash
84732  1557 -/usr/local/bin/bash
89065  1557 -/usr/local/bin/bash
80216  1557 -/usr/local/bin/bash

note I use GNU screen, if that makes a difference.

I can see in the vendored sourcecode if it finds a shell in a known list of shells, it doesn't strip - from the process path. However, it does in other cases. So you just need to handle that. I see the upstream has changed quite a bit in master, so maybe this problem is no longer present in the latest version of shellingham.

uranusjr · 2018-07-04T13:39:13Z

Ah, I see the problem now. I was handling the shell path incorrectly if you chsh the login shell to point to an absolute path. I will use this information to fix it. Thanks so much for the help!

Whisprin · 2018-07-17T20:19:42Z

Will there be a new release containing the bug fix?

mixmastamyk · 2018-10-24T18:37:51Z

Is this not fixed yet in the latest on PyPI? Pretty egregious bug for a high-profile project:

⏵ pipenv --version
    pipenv, version 2018.10.13
⏵ pipenv shell
...
pipenv.vendor.pexpect.exceptions.ExceptionPexpect: The command was not found or was not executable: fish.

⏵ python3
>>> from pipenv.shells import _detect_emulator, detect_info
>>> detect_info()
('fish', 'fish')
^D

⏵ env PIPENV_SHELL="$SHELL" pipenv shell
Loading .env environment variables…
Launching subshell in virtual environment…

⏵  source .../dev-TrP3E4Ut/bin/activate.fish
(dev-TrP3E4Ut) ... ⏵

# worked.

uranusjr · 2018-10-24T21:10:48Z

Your problem seems to be different from the OP. She’ll detection works correctly, but fish is not found. Is it in PATH? What is your $SHELL?

mixmastamyk · 2018-10-25T02:45:45Z

Hi, it is in the path, but the code appears to be looking for an absolute path:

⏵ echo $SHELL
/usr/bin/fish

uranusjr · 2018-10-25T04:35:49Z

Is fish in your PATH?

mixmastamyk · 2018-10-25T15:46:54Z

Yes it is in /usr/bin which is in the PATH. pipenv shell used to work until I updated it yesterday.

uranusjr added the Type: Possible Bug This issue describes a possible bug in pipenv. label Jul 3, 2018

uranusjr self-assigned this Jul 3, 2018

uranusjr closed this as completed in sarugaku/shellingham@24acf49 Jul 4, 2018

uranusjr mentioned this issue Jul 4, 2018

Does not parse login shell output correctly on POSIX sarugaku/shellingham#6

Closed

uranusjr added a commit to sarugaku/shellingham that referenced this issue Jul 4, 2018

Add test based on pypa/pipenv#2496

3326a09

potens1 mentioned this issue Aug 27, 2018

Pipenv fails to start zsh #2743

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

pipenv shell fails to find /usr/local/bin/bash in pexpect #2496

pipenv shell fails to find /usr/local/bin/bash in pexpect #2496

keegancsmith commented Jul 3, 2018

techalchemy commented Jul 3, 2018

uranusjr commented Jul 3, 2018

keegancsmith commented Jul 3, 2018

uranusjr commented Jul 3, 2018

Whisprin commented Jul 4, 2018

uranusjr commented Jul 4, 2018

keegancsmith commented Jul 4, 2018

keegancsmith commented Jul 4, 2018

uranusjr commented Jul 4, 2018

Whisprin commented Jul 17, 2018

mixmastamyk commented Oct 24, 2018 •

edited

Loading

uranusjr commented Oct 24, 2018

mixmastamyk commented Oct 25, 2018

uranusjr commented Oct 25, 2018

mixmastamyk commented Oct 25, 2018

pipenv shell fails to find /usr/local/bin/bash in pexpect #2496

pipenv shell fails to find /usr/local/bin/bash in pexpect #2496

Comments

keegancsmith commented Jul 3, 2018

Issue description

Expected result

Actual result

Steps to replicate

techalchemy commented Jul 3, 2018

uranusjr commented Jul 3, 2018

keegancsmith commented Jul 3, 2018

uranusjr commented Jul 3, 2018

Whisprin commented Jul 4, 2018

uranusjr commented Jul 4, 2018

keegancsmith commented Jul 4, 2018

keegancsmith commented Jul 4, 2018

uranusjr commented Jul 4, 2018

Whisprin commented Jul 17, 2018

mixmastamyk commented Oct 24, 2018 • edited Loading

uranusjr commented Oct 24, 2018

mixmastamyk commented Oct 25, 2018

uranusjr commented Oct 25, 2018

mixmastamyk commented Oct 25, 2018

mixmastamyk commented Oct 24, 2018 •

edited

Loading