Fish shell support #32

Closed
fjarri opened this Issue Jul 15, 2013 · 31 comments

Comments

Projects
None yet
6 participants
@fjarri

fjarri commented Jul 15, 2013

Is the support of fish shell planned? There are some mentions of fish on wiki, but currently pyenv does not work in it.

Namely, the problem is pyenv init, which tries to perform a bash eval inside fish, which, of course, fails. Installation/uninstallation of Python versions seems to work normally.

@yyuu

This comment has been minimized.

Show comment
Hide comment
@yyuu

yyuu Jul 15, 2013

Collaborator

I'd like to support even for fish if it is available, but it seems somewhat difficult with current impl :(

Many of the code of pyenv is written in bash, but there are some of interactions between working shell. They might be the problem to support fish.

  1. pyenv init
    • need to prepare shell-specific init script
  2. pyenv shell
    • export is not workable on fish
  3. auto-completion
    • need shell-specific code
Collaborator

yyuu commented Jul 15, 2013

I'd like to support even for fish if it is available, but it seems somewhat difficult with current impl :(

Many of the code of pyenv is written in bash, but there are some of interactions between working shell. They might be the problem to support fish.

  1. pyenv init
    • need to prepare shell-specific init script
  2. pyenv shell
    • export is not workable on fish
  3. auto-completion
    • need shell-specific code
@fjarri

This comment has been minimized.

Show comment
Hide comment
@fjarri

fjarri Jul 15, 2013

I'd personally be happy if init worked, I don't really use 2 or 3. Do I understand it correctly that the primary function of init is to call rehash and add stuff to PATH, and that's what makes global/local work?

fjarri commented Jul 15, 2013

I'd personally be happy if init worked, I don't really use 2 or 3. Do I understand it correctly that the primary function of init is to call rehash and add stuff to PATH, and that's what makes global/local work?

@yyuu

This comment has been minimized.

Show comment
Hide comment
@yyuu

yyuu Jul 16, 2013

Collaborator

Yep, you're right. The global and local can work without init.

Collaborator

yyuu commented Jul 16, 2013

Yep, you're right. The global and local can work without init.

@ghost ghost assigned yyuu Jul 16, 2013

@yyuu

This comment has been minimized.

Show comment
Hide comment
@yyuu

yyuu Jul 16, 2013

Collaborator

Adding following snippet in your shell initialization script (something like ~/.profile of sh for fish) might help you.

setenv PYENV_ROOT "${HOME}/.pyenv"
setenv PATH "${PYENV_ROOT}/shims:${PYENV_ROOT}/bin:${PATH}"
pyenv rehash
Collaborator

yyuu commented Jul 16, 2013

Adding following snippet in your shell initialization script (something like ~/.profile of sh for fish) might help you.

setenv PYENV_ROOT "${HOME}/.pyenv"
setenv PATH "${PYENV_ROOT}/shims:${PYENV_ROOT}/bin:${PATH}"
pyenv rehash
@fjarri

This comment has been minimized.

Show comment
Hide comment
@fjarri

fjarri Jul 16, 2013

Thank you for the suggestion! In fish syntax it seems to be

set PYENV_ROOT $HOME/.pyenv
set -x PATH $PYENV_ROOT/shims $PYENV_ROOT/bin $PATH
pyenv rehash

The only problem is that modules that install stuff in bin do not work properly. For example, if I install yolk (a tool that generates a list of available modules), pip correctly installs it in ~/.pyenv/versions/2.7.5/lib/python2.7/site-packages/, and puts yolk executable in ~/.pyenv/versions/2.7.5/bin/, but when I call yolk from the command line, it results in the call to /usr/local/bin/yolk (the system one).

fjarri commented Jul 16, 2013

Thank you for the suggestion! In fish syntax it seems to be

set PYENV_ROOT $HOME/.pyenv
set -x PATH $PYENV_ROOT/shims $PYENV_ROOT/bin $PATH
pyenv rehash

The only problem is that modules that install stuff in bin do not work properly. For example, if I install yolk (a tool that generates a list of available modules), pip correctly installs it in ~/.pyenv/versions/2.7.5/lib/python2.7/site-packages/, and puts yolk executable in ~/.pyenv/versions/2.7.5/bin/, but when I call yolk from the command line, it results in the call to /usr/local/bin/yolk (the system one).

@yyuu

This comment has been minimized.

Show comment
Hide comment
@yyuu

yyuu Jul 16, 2013

Collaborator

You might need pyenv rehash to generate shim for yolk after installing pip install yolk. Plus, you might need to invoke something like rehash (zsh) or hash -r (bash) to update command table of the current shell.

Collaborator

yyuu commented Jul 16, 2013

You might need pyenv rehash to generate shim for yolk after installing pip install yolk. Plus, you might need to invoke something like rehash (zsh) or hash -r (bash) to update command table of the current shell.

@fjarri

This comment has been minimized.

Show comment
Hide comment
@fjarri

fjarri Jul 16, 2013

Oh, it seems that I did not quite understand what pyenv rehash does. Thank you, that did help, no additional shell commands were necessary.

fjarri commented Jul 16, 2013

Oh, it seems that I did not quite understand what pyenv rehash does. Thank you, that did help, no additional shell commands were necessary.

@yyuu

This comment has been minimized.

Show comment
Hide comment
@yyuu

yyuu Jul 19, 2013

Collaborator

Is there any other problem? Or, I wanna close this.

Updating Wiki about tips with fish is welcome :)

Collaborator

yyuu commented Jul 19, 2013

Is there any other problem? Or, I wanna close this.

Updating Wiki about tips with fish is welcome :)

@fjarri

This comment has been minimized.

Show comment
Hide comment
@fjarri

fjarri Jul 19, 2013

Well, there's still an issue of shell and auto-completion, which may be important for some people. Plus there are some weird problems of pip running 2to3 when I install packages for Py2 and not running it when I install packages for Py3, which does not happen when I install them from their source directory (I cannot pinpoint the cause of this yet).

Basically, what I'm saying is, the issue of me using pyenv with fish is somewhat resolved, but the issue of pyenv supporting fish is not, so use your best judgement :) I will update the wiki as soon as I resolve the pip problem.

fjarri commented Jul 19, 2013

Well, there's still an issue of shell and auto-completion, which may be important for some people. Plus there are some weird problems of pip running 2to3 when I install packages for Py2 and not running it when I install packages for Py3, which does not happen when I install them from their source directory (I cannot pinpoint the cause of this yet).

Basically, what I'm saying is, the issue of me using pyenv with fish is somewhat resolved, but the issue of pyenv supporting fish is not, so use your best judgement :) I will update the wiki as soon as I resolve the pip problem.

@yyuu

This comment has been minimized.

Show comment
Hide comment
@yyuu

yyuu Jul 19, 2013

Collaborator

My decision on this is, pyenv should even work on non-sh-style shells (like fish and csh), but init, shell and auto-completion are limited on these shells.

Supporting full features on non-sh-style shells is very hard and will not be implemented. This does not mean it is impossible; it is possible if I wrote scripts like libexec/pyenv-init.fish and completions/pyenv.fish and some more. But I will not implement them because I am not an expert of these shells. (patches are welcome!)

Even if the support is limited, most of the features (like local, global and install) are still available. I think this means pyenv is still useful for the users of non-sh-style shells.

Collaborator

yyuu commented Jul 19, 2013

My decision on this is, pyenv should even work on non-sh-style shells (like fish and csh), but init, shell and auto-completion are limited on these shells.

Supporting full features on non-sh-style shells is very hard and will not be implemented. This does not mean it is impossible; it is possible if I wrote scripts like libexec/pyenv-init.fish and completions/pyenv.fish and some more. But I will not implement them because I am not an expert of these shells. (patches are welcome!)

Even if the support is limited, most of the features (like local, global and install) are still available. I think this means pyenv is still useful for the users of non-sh-style shells.

@yyuu

This comment has been minimized.

Show comment
Hide comment
@yyuu

yyuu Jul 19, 2013

Collaborator

Please describe more about the weird problems of pip running 2to3.

  1. Which python version did you use with?
  2. What is the package did you try to pip install?
  3. Please paste the error output from pip
Collaborator

yyuu commented Jul 19, 2013

Please describe more about the weird problems of pip running 2to3.

  1. Which python version did you use with?
  2. What is the package did you try to pip install?
  3. Please paste the error output from pip
@fjarri

This comment has been minimized.

Show comment
Hide comment
@fjarri

fjarri Jul 20, 2013

For example:

~> pyenv global 2.7.5
~> pyenv rehash
~> pip install decorator -v
Downloading/unpacking decorator
  Running setup.py egg_info for package decorator
    running egg_info
    writing pip-egg-info/decorator.egg-info/PKG-INFO
    writing top-level names to pip-egg-info/decorator.egg-info/top_level.txt
    writing dependency_links to pip-egg-info/decorator.egg-info/dependency_links.txt
    warning: manifest_maker: standard file '-c' not found

    reading manifest file 'pip-egg-info/decorator.egg-info/SOURCES.txt'
    reading manifest template 'MANIFEST.in'
    warning: no files found matching 'documentation.py'
    warning: no files found matching 'documentation3.py'
    warning: no previously-included files found matching 'Makefile'
    writing manifest file 'pip-egg-info/decorator.egg-info/SOURCES.txt'
Installing collected packages: decorator
  Running setup.py install for decorator
    running install
    running build
    running build_py
    running install_lib
    copying build/lib/decorator.py -> /Users/bogdan/.pyenv/versions/2.7.5/lib/python2.7/site-packages
    byte-compiling /Users/bogdan/.pyenv/versions/2.7.5/lib/python2.7/site-packages/decorator.py to decorator.pyc
      File "/Users/bogdan/.pyenv/versions/2.7.5/lib/python2.7/site-packages/decorator.py", line 158
        print('Error in generated code:', file=sys.stderr)
                                              ^
    SyntaxError: invalid syntax

    running install_egg_info
    running egg_info
    writing src/decorator.egg-info/PKG-INFO
    writing top-level names to src/decorator.egg-info/top_level.txt
    writing dependency_links to src/decorator.egg-info/dependency_links.txt
    warning: manifest_maker: standard file '-c' not found

    reading manifest file 'src/decorator.egg-info/SOURCES.txt'
    reading manifest template 'MANIFEST.in'
    warning: no files found matching 'documentation.py'
    warning: no files found matching 'documentation3.py'
    warning: no previously-included files found matching 'Makefile'
    writing manifest file 'src/decorator.egg-info/SOURCES.txt'
    Copying src/decorator.egg-info to /Users/bogdan/.pyenv/versions/2.7.5/lib/python2.7/site-packages/decorator-3.4.0-py2.7.egg-info
    running install_scripts
    writing list of installed files to '/var/folders/hq/kzp38x892h7g5nj6rr5p6d7c0000gn/T/pip-tDOBwU-record/install-record.txt'
Successfully installed decorator
Cleaning up...

Note the syntax error. If you look into https://pypi.python.org/packages/source/d/decorator/decorator-3.4.0.tar.gz you'll see a Py2-style print statement there. If I download the sources, cd there and run pip install ., it installs correctly.

fjarri commented Jul 20, 2013

For example:

~> pyenv global 2.7.5
~> pyenv rehash
~> pip install decorator -v
Downloading/unpacking decorator
  Running setup.py egg_info for package decorator
    running egg_info
    writing pip-egg-info/decorator.egg-info/PKG-INFO
    writing top-level names to pip-egg-info/decorator.egg-info/top_level.txt
    writing dependency_links to pip-egg-info/decorator.egg-info/dependency_links.txt
    warning: manifest_maker: standard file '-c' not found

    reading manifest file 'pip-egg-info/decorator.egg-info/SOURCES.txt'
    reading manifest template 'MANIFEST.in'
    warning: no files found matching 'documentation.py'
    warning: no files found matching 'documentation3.py'
    warning: no previously-included files found matching 'Makefile'
    writing manifest file 'pip-egg-info/decorator.egg-info/SOURCES.txt'
Installing collected packages: decorator
  Running setup.py install for decorator
    running install
    running build
    running build_py
    running install_lib
    copying build/lib/decorator.py -> /Users/bogdan/.pyenv/versions/2.7.5/lib/python2.7/site-packages
    byte-compiling /Users/bogdan/.pyenv/versions/2.7.5/lib/python2.7/site-packages/decorator.py to decorator.pyc
      File "/Users/bogdan/.pyenv/versions/2.7.5/lib/python2.7/site-packages/decorator.py", line 158
        print('Error in generated code:', file=sys.stderr)
                                              ^
    SyntaxError: invalid syntax

    running install_egg_info
    running egg_info
    writing src/decorator.egg-info/PKG-INFO
    writing top-level names to src/decorator.egg-info/top_level.txt
    writing dependency_links to src/decorator.egg-info/dependency_links.txt
    warning: manifest_maker: standard file '-c' not found

    reading manifest file 'src/decorator.egg-info/SOURCES.txt'
    reading manifest template 'MANIFEST.in'
    warning: no files found matching 'documentation.py'
    warning: no files found matching 'documentation3.py'
    warning: no previously-included files found matching 'Makefile'
    writing manifest file 'src/decorator.egg-info/SOURCES.txt'
    Copying src/decorator.egg-info to /Users/bogdan/.pyenv/versions/2.7.5/lib/python2.7/site-packages/decorator-3.4.0-py2.7.egg-info
    running install_scripts
    writing list of installed files to '/var/folders/hq/kzp38x892h7g5nj6rr5p6d7c0000gn/T/pip-tDOBwU-record/install-record.txt'
Successfully installed decorator
Cleaning up...

Note the syntax error. If you look into https://pypi.python.org/packages/source/d/decorator/decorator-3.4.0.tar.gz you'll see a Py2-style print statement there. If I download the sources, cd there and run pip install ., it installs correctly.

@yyuu

This comment has been minimized.

Show comment
Hide comment
@yyuu

yyuu Jul 20, 2013

Collaborator

Couldn't confirm this weird behaviour.

I think that there is something wrong in distutils or setuptools. The setup.py running on 2.7.5 should not invoke 2to3 because the distutils of 2.7.5 lacks distutils.util.Mixin2to3. Setting the Python's environment variables of PYTHONHOME or PYTHONPATH might introduce this kind of confusion. But I'm not sure what is happening there :(

Collaborator

yyuu commented Jul 20, 2013

Couldn't confirm this weird behaviour.

I think that there is something wrong in distutils or setuptools. The setup.py running on 2.7.5 should not invoke 2to3 because the distutils of 2.7.5 lacks distutils.util.Mixin2to3. Setting the Python's environment variables of PYTHONHOME or PYTHONPATH might introduce this kind of confusion. But I'm not sure what is happening there :(

@yyuu

This comment has been minimized.

Show comment
Hide comment
@yyuu

yyuu Jul 23, 2013

Collaborator

I heard similar weird behaviour of 2to3 in #38. In that case, upgrading setuptools from 0.7.2 to 0.9.7 (latest) solves the problem.

Collaborator

yyuu commented Jul 23, 2013

I heard similar weird behaviour of 2to3 in #38. In that case, upgrading setuptools from 0.7.2 to 0.9.7 (latest) solves the problem.

@fjarri

This comment has been minimized.

Show comment
Hide comment
@fjarri

fjarri Jul 23, 2013

Upgrading setuptools did not work for me, unfortunately. I have also found some other mentions of this problem (e.g. http://code.google.com/p/sympy/issues/detail?id=3437), and it seems that it is probably something related to pip and not pyenv. I'll close this for now, since this issue is quite elusive.

fjarri commented Jul 23, 2013

Upgrading setuptools did not work for me, unfortunately. I have also found some other mentions of this problem (e.g. http://code.google.com/p/sympy/issues/detail?id=3437), and it seems that it is probably something related to pip and not pyenv. I'll close this for now, since this issue is quite elusive.

@fjarri fjarri closed this Jul 23, 2013

@fjarri

This comment has been minimized.

Show comment
Hide comment
@fjarri

fjarri Jul 24, 2013

Just for the sake of closure: the problem has been resolved by updating pip from 1.3.1 to 1.4. Sorry for blaming pyenv :)

fjarri commented Jul 24, 2013

Just for the sake of closure: the problem has been resolved by updating pip from 1.3.1 to 1.4. Sorry for blaming pyenv :)

@yyuu

This comment has been minimized.

Show comment
Hide comment
@yyuu

yyuu Jul 24, 2013

Collaborator

I noticed the update of pip by your mention here. Thanks for your comment :)

Collaborator

yyuu commented Jul 24, 2013

I noticed the update of pip by your mention here. Thanks for your comment :)

@yyuu

This comment has been minimized.

Show comment
Hide comment
@yyuu

yyuu Aug 15, 2013

Collaborator

@Manticore
I wrote some additional scripts to improve fish support of pyenv. The init, shell and completions are now working on fish. Please try fish-support branch at pyenv repo.

https://github.com/yyuu/pyenv/tree/fish-support

Initialization code for fish.

setenv PATH "$HOME/.pyenv/bin" $PATH
eval (pyenv init - fish)
Collaborator

yyuu commented Aug 15, 2013

@Manticore
I wrote some additional scripts to improve fish support of pyenv. The init, shell and completions are now working on fish. Please try fish-support branch at pyenv repo.

https://github.com/yyuu/pyenv/tree/fish-support

Initialization code for fish.

setenv PATH "$HOME/.pyenv/bin" $PATH
eval (pyenv init - fish)
@fjarri

This comment has been minimized.

Show comment
Hide comment
@fjarri

fjarri Aug 16, 2013

Yep, everything seems to work, thank you!

fjarri commented Aug 16, 2013

Yep, everything seems to work, thank you!

@yyuu

This comment has been minimized.

Show comment
Hide comment
@yyuu

yyuu Aug 16, 2013

Collaborator

ok. I'll merge it into master :)

Collaborator

yyuu commented Aug 16, 2013

ok. I'll merge it into master :)

yyuu pushed a commit that referenced this issue Aug 16, 2013

@tiagofernandez

This comment has been minimized.

Show comment
Hide comment
@tiagofernandez

tiagofernandez Oct 30, 2013

I guess the initialization code for Fish is currently . (pyenv init -|psub), right?
Since eval (pyenv init - fish) returns pyenv: no such command sh-'.

I guess the initialization code for Fish is currently . (pyenv init -|psub), right?
Since eval (pyenv init - fish) returns pyenv: no such command sh-'.

@yyuu

This comment has been minimized.

Show comment
Hide comment
@yyuu

yyuu Oct 31, 2013

Collaborator

@tiagofernandez
Oops. You're right. The initialization code has been changed during exporting this feature to rbenv (rbenv/rbenv#434).

Now I'm using following initialization code.

set -x PATH "$HOME/.pyenv/bin" $PATH
. (pyenv init - | psub)
Collaborator

yyuu commented Oct 31, 2013

@tiagofernandez
Oops. You're right. The initialization code has been changed during exporting this feature to rbenv (rbenv/rbenv#434).

Now I'm using following initialization code.

set -x PATH "$HOME/.pyenv/bin" $PATH
. (pyenv init - | psub)
@lvh

This comment has been minimized.

Show comment
Hide comment
@lvh

lvh Jan 2, 2015

For posterity: pyenv now supports fish! Try pyenv init in a shell and it will tell you:

# Load pyenv automatically by adding
# the following to ~/.config/fish/config.fish:

status --is-interactive; and . (pyenv init -|psub)

lvh commented Jan 2, 2015

For posterity: pyenv now supports fish! Try pyenv init in a shell and it will tell you:

# Load pyenv automatically by adding
# the following to ~/.config/fish/config.fish:

status --is-interactive; and . (pyenv init -|psub)
@ghost

This comment has been minimized.

Show comment
Hide comment
@ghost

ghost Feb 22, 2015

@yyuu pyenv is awesome, thanks.

Also for posterity, I thought I'd write up a small guide about what's happening here:

status --is-interactive; and . (pyenv init - | psub)

First, status --is-interactive sets the exit $status code to 0 (true) if the terminal is interactive.

As for the -, at least in Unix parlance, - means standard input, but pyenv uses it instead as a silent flag to skip printing a help message that explains how to add pyenv to your profile AFIK.

Now, psub is used for process substitution. Let me explain this via an example, say you want to diff two files:

diff file1 file2

but rather use a string like:

echo $string | diff file1 -

and with two strings:

diff (echo $s1 | psub) (echo $s2 | psub)

and translated to bash:

diff <($s1) <($s2)

In . (pyenv init - | psub), ``.' (a wrapper around thesource` builtin) will receive whatever comes out of `pyenv init -` as input.

. evaluates (parses) the output produced by pyenv init..., which is different from spawning a new process and running the output.

⚠️ . is deprecated, please use source from now :)

This, among other things, creates the directories at $PYENV_ROOT/shims, $PYENV_ROOT/versions and sets up pyenv environment.

ghost commented Feb 22, 2015

@yyuu pyenv is awesome, thanks.

Also for posterity, I thought I'd write up a small guide about what's happening here:

status --is-interactive; and . (pyenv init - | psub)

First, status --is-interactive sets the exit $status code to 0 (true) if the terminal is interactive.

As for the -, at least in Unix parlance, - means standard input, but pyenv uses it instead as a silent flag to skip printing a help message that explains how to add pyenv to your profile AFIK.

Now, psub is used for process substitution. Let me explain this via an example, say you want to diff two files:

diff file1 file2

but rather use a string like:

echo $string | diff file1 -

and with two strings:

diff (echo $s1 | psub) (echo $s2 | psub)

and translated to bash:

diff <($s1) <($s2)

In . (pyenv init - | psub), ``.' (a wrapper around thesource` builtin) will receive whatever comes out of `pyenv init -` as input.

. evaluates (parses) the output produced by pyenv init..., which is different from spawning a new process and running the output.

⚠️ . is deprecated, please use source from now :)

This, among other things, creates the directories at $PYENV_ROOT/shims, $PYENV_ROOT/versions and sets up pyenv environment.

@aviddiviner

This comment has been minimized.

Show comment
Hide comment
@aviddiviner

aviddiviner Jul 29, 2015

@bucaran: Thanks for that awesome explanation! Taught me about how psub works.. cool!

@bucaran: Thanks for that awesome explanation! Taught me about how psub works.. cool!

rajatvig added a commit to rajatvig/fish-pyenv that referenced this issue Dec 22, 2015

yjpark added a commit to yjpark/dotfiles that referenced this issue May 11, 2016

@sooheon

This comment has been minimized.

Show comment
Hide comment
@sooheon

sooheon May 30, 2016

@BRJ with a recent version of fish, I'm getting source: '.' command is deprecated, and doesn't work with STDIN anymore. Did you mean 'source' or './'? What is the form of this incantation without using ., but source directly?

sooheon commented May 30, 2016

@BRJ with a recent version of fish, I'm getting source: '.' command is deprecated, and doesn't work with STDIN anymore. Did you mean 'source' or './'? What is the form of this incantation without using ., but source directly?

@ghost

This comment has been minimized.

Show comment
Hide comment
@ghost

ghost May 30, 2016

@sooheon Yes, use source please.

. is history now :)

ghost commented May 30, 2016

@sooheon Yes, use source please.

. is history now :)

@sooheon

This comment has been minimized.

Show comment
Hide comment
@sooheon

sooheon May 30, 2016

Sorry but my question was just about the syntax of how to do that--directly replacing . with source gives an error.

sooheon commented May 30, 2016

Sorry but my question was just about the syntax of how to do that--directly replacing . with source gives an error.

@ghost

This comment has been minimized.

Show comment
Hide comment
@ghost

ghost May 30, 2016

@sooheon Hmm. Can you post the error?

You are using the fisherman plugin right?

ghost commented May 30, 2016

@sooheon Hmm. Can you post the error?

You are using the fisherman plugin right?

@sooheon

This comment has been minimized.

Show comment
Hide comment
@sooheon

sooheon May 30, 2016

I am now :). I had just copy pasted the snippet above in my fish.config before. Thanks.

sooheon commented May 30, 2016

I am now :). I had just copy pasted the snippet above in my fish.config before. Thanks.

@ghost

This comment has been minimized.

Show comment
Hide comment
@ghost

ghost May 30, 2016

If you run into other issues, please create an issue in the plugin's repo, fisherman's issue tracker or just contact me directly. Happy fishing! :)

ghost commented May 30, 2016

If you run into other issues, please create an issue in the plugin's repo, fisherman's issue tracker or just contact me directly. Happy fishing! :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment