Update install instructions for Bash and Zsh

[native-api]

• See https://github.com/native-api/pyenv/tree/bash_zsh_instructions#basic-github-checkout for the rendered README
• Clone https://github.com/native-api/pyenv/tree/bash_zsh_instructions and run pyenv init [<shell>] to see text instructions for <shell>:

pushd "$(pyenv root)"
git remote add pr https://github.com/native-api/pyenv
git fetch pr
git status    # see what branch/tag/commit you're currently at
git checkout bash_zsh_instructions

pyenv init

#undo the changes
git checkout <old branch/tag/commit from `git status` above>
git branch -D bash_zsh_instructions
git remote remove pr
popd

---

Make sure you have checked all steps below.

Prerequisite

• Please consider implementing the feature as a hook script or plugin as a first step.
  • pyenv has some powerful support for plugins and hook scripts. Please refer to Authoring plugins for details and try to implement it as a plugin if possible.
• Please consider contributing the patch upstream to rbenv, since we have borrowed most of the code from that project.
  • We occasionally import the changes from rbenv. In general, you can expect changes made in rbenv will be imported to pyenv too, eventually.
  • Generally speaking, we prefer not to make changes in the core in order to keep compatibility with rbenv.
• My PR addresses the following pyenv issue (if any)
  • Closes path manipulations need to be placed in ~/.zprofile for zsh user #1913
  • Closes WARNING: pyenv init - no longer sets PATH. #1915

Description

• Here are some details about my PR

Updated README and instructions for Bash and Zsh based on user feedback.
Going to ask for reviews before merging.

Tests

• [N/A] My PR adds the following unit tests (if any)

[agoose77]

@native-api thanks for creating a PR to try and tackle this.

The Issue 🐛

I feel that with all of this work on documenting exactly what to do for each shell, we're trying to solve the wrong problem; shell RC files are confusing, and not always consistent between shells and environments.

All of the issues that have been raised about this have stemmed from the fact that pyenv has several locations that need to end up in PATH, and users either don't know of all of them, or don't understand why they need to go in different RC files. This is not a criticism of these users; I have to reference what each shell does when I reply to these issues.

New Init API 🔩

I think that we should consider dropping the shell-specific approach entirely, and instead pyenv should just expose two commands:

pyenv init --path

and

pyenv init --shell

These are the two kinds of configuration that pyenv needs to set AFAICT. The pyenv init --shell command is an attempt to remove the visual precedence that pyenv init - has over pyenv init --path, and to make it clear that one is for shell functions, the other is for environment.

My rationale for dropping the per-shell documentation in pyenv itself is that it quickly explodes with the number of shells, OS default configurations (e.g. .profile sources .bashrc on Ubuntu), and login shells (e.g. I'm using GDM which explicitly sources ~/.profile despite SHELL=/usr/bin/zsh). This gets more confusing if users have non-standard configuration files already in use.

I think this would be easier to deal with through a wiki entry.

Old Init API 🕸️

Of course, this means that things will change for users. I don't know what the policy for API change is for this project, but if we don't want to break pyenv init - outright, then the safest approach is just to add a new --shell flag and retain the existing pyenv init - behaviour (but mark as deprecated). However, I would perhaps go further, and just make pyenv init - return cat <(echo "warning ...") <(pyenv init --path) <(pyenv init --shell). This way, users who haven't modified their RC files will be notified of the upcoming deprecation, but will not be confused by breakages.

Shell-Specific Help ℹ️

I still think that the shell-specific information is useful, but perhaps it doesn't belong in the tool. If we repurpose pyenv init - as an old-style (deprecated) configuration command, then we can add some information about what the new-style looks like, e.g.

cat <<- 'EOF' 
Warning, `pyenv init -` will no longer work in future releases of pyenv.
To avoid breakages when this occurs, please move to the new configuration mechanism.

The old `pyenv init -` command has been split into two new commands:
`pyenv init --path` and `pyenv init --shell`. 

You should remove the existing `pyenv init -` code, and instead 
1. Add `pyenv init --path` to your `~/.profile` (or equivalent) file
2. Add `pyenv init --shell` to your `~/.bashrc` (or equivalent) file

In some cases you might find that this does not work. This is likely because shell configuration is hard! 
Please visit .... for more information with shell-specific guidance.
EOF

source pyenv init --path
source pyenv init --shell 

What do you think?

[The-Judge]

No offense, but reading all the replies in #1915 to my initial request, it looks a bit as the thread got "hijacked" for "all-things-shell-issues" ;-)
Initially, I reported an issue for "bash" shell. Here's what I did:

1. I cloned Update install instructions for Bash and Zsh #1920 to /usr/src/pyenv and cd'ed into it
2. src/configure && make -C src (successfully)
3. export PYENV_ROOT="/usr/src/pyenv"
4. export PATH="${PYENV_ROOT}/bin:${PATH}"
5. which pyenv (provides /usr/src/pyenv/bin/pyenv)
6. Both, pyenv init and pyenv init bash provide:

# Load pyenv automatically by appending
# the following to ~/.bash_profile:

eval "$(pyenv init -)"

Nothing more; nothing about setting the PATH or anything; which seems wrong already, isn't it?

Anyways: I removed everything pyenv related from my .bashrc and .profile already before running this. I started with 1:1 doing what pyenv init suggests, knowing this will fail without the PATH manipulation and - to very few surprises - it did:

-bash: pyenv: command not found

So, I also added this before that line in .bash_profile, making it look like this:

root@judgepi:~# cat .bash_profile
#source ${HOME}/.bash_pyenv
export PYENV_ROOT="/usr/src/pyenv"
export PATH="$PYENV_ROOT/bin:$PATH"
eval "$(pyenv init -)"

root@judgepi:~#

Actually, it looks like this did the trick! I can use pyenv and do no longer receive the WARNING when SSH'ing into it:

~ $ ssh pi /bin/true
~ $ echo $?
0
~ $

[The-Judge]

PS: I like @agoose77 's idea! Debugging the global RC-landscape is out of scope for pyenv; at least "core". Supporting all shells and advice on them seems to be more related to sidekick-projects like pyenv/pyenv-installer.
As long as it is clear and transparent to the user what's the goal he/she has to take care about, I feel like pyenv's duty is done.

[native-api]

1. I cloned #1920 to `/usr/src/pyenv` and cd'ed into it

This is wrong. You need to switch your installed Pyenv to it. This is why I gave the exact commands needed right at the top.

[The-Judge]

1. I cloned #1920 to `/usr/src/pyenv` and cd'ed into it

This is wrong. You need to switch your installed Pyenv to it. This is why I gave the exact commands needed right at the top.

I did by modifying the PATH to that new location - didn't I?

[native-api]

I did by modifying the PATH to that new location - didn't I?

Oh, sorry, I missed that part (I honestly thought that no-one would go for the trouble :-) ). Then it indeed should work.

How exactly did you "clone #1920"? I suspect that you didn't check out the right branch and ended up at master from my fork -- which is stale and unused.

[aiguofer]

I'm wondering if pyenv init --path is even needed at all? the installation is already asking users to add export PATH="$PYENV_ROOT/bin:$PATH, why not just ask them to add export PATH="$PYENV_ROOT/shims:$PYENV_ROOT/bin:$PATH" (or the equivalent in each shell) instead?

I feel like the extra command to set the PATH is what's tripping people up as they think they need to replace the old command with that one.

[native-api]

I'm wondering if pyenv init --path is even needed at all? the installation is already asking users to add export PATH="$PYENV_ROOT/bin:$PATH, why not just ask them to add export PATH="$PYENV_ROOT/shims:$PYENV_ROOT/bin:$PATH" (or the equivalent in each shell) instead?

The idea is to make the configuration more future-proof. In the future, we may add/change steps in --path if something else needs to be done in the login shell.

I feel like the extra command to set the PATH is what's tripping people up as they think they need to replace the old command with that one.

I think that it's rather the fact that I failed to include the PYENV_ROOT and PATH= steps into the pyenv init instructions at all! (They were only included into the pyenv-installer postinstall message.) So the users who saw the warning didn't readily see that they need to move these lines to ~/.profile!
That was a blunder on my part.

[julianfortune]

source pyenv init --path
source pyenv init --shell 

What do you think?

@agoose77 This solution would work great for all of the use cases in my world! Just my 2 cents.

[native-api]

@agoose77 @julianfortune @The-Judge

I feel that with all of this work on documenting exactly what to do for each shell, we're trying to solve the wrong problem; shell RC files are confusing, and not always consistent between shells and environments.
<...>
My rationale for dropping the per-shell documentation in pyenv itself is that it quickly explodes with the number of shells, OS default configurations (e.g. .profile sources .bashrc on Ubuntu), and login shells (e.g. I'm using GDM which explicitly sources ~/.profile despite SHELL=/usr/bin/zsh). This gets more confusing if users have non-standard configuration files already in use.

• Not including ready-to-use invocations would erect a high barrier to entry. Even if they can (which is also not the case quite often), users don't want to have to think and waste time on it, they assume that if this is a production-ready software product, someone has already done the thinking for them!
  And rightfully so: if they have a common configuration, there must be someone who already thought of a good, turn-key solution for it, so there's absolutely no reason to reinvent the (square) wheel!
• These "documenting exactly what to do for each shell" have been in the README for years and have been working fine so far.

I think we can get the best of both worlds here if we include both an explanation of what the effect should be, and the ready-to-use examples for common setups.
This is what I just did in the update to this PR -- go see if it's to your satisfaction! 😉

I think that we should consider dropping the shell-specific approach entirely, and instead pyenv should just expose two commands:

<..>
pyenv init --path
<...>
pyenv init --shell

This is exactly what the interface is now (just with keeping the the old name for "--shell", see below).

These are the two kinds of configuration that pyenv needs to set AFAICT. The pyenv init --shell command is an attempt to remove the visual precedence that pyenv init - has over pyenv init --path, and to make it clear that one is for shell functions, the other is for environment.

I thought about renaming - to --rc -- both when writing #1898 and now -- and ultimately decided against it.

It's true that it'll make things marginally clearer. But the drawbacks are:

• Should we force the users to rewrite their configurations yet again, just after we've already frustrated them with this change and the broken and convoluted instructions?
  • That'll be very bad PR for the project. If we ever do that, we should rather first let the userbase calm down and forget the incident.
• Users don't deal with pyenv init at all outside of the initial configuration step. So as long as it's clear what to put where, they don't really care how these incantations are spelled.
  • So all the frustration and the extra work that we'll be forcing onto all our users would be for nothing, really.
  • See also below for another reason why I had to retain -. Concluding from it, the moment to rename it was during Split startup logic into PATH and everything else #1898, and now the opportunity is gone.

So my best idea for this -- if this is even worth doing, as per above -- is to introduce a new name as an alias to -, and wait for a long, long time before deprecating the original.

Of course, this means that things will change for users. I don't know what the policy for API change is for this project, but if we don't want to break pyenv init - outright <...>

The change was made to fix #1649. We needed to make a breaking change to force the users to change their configurations -- otherwise, they would keep falling victim to #1649 and would keep reporting it. And we needed to retain - so that we could use the existing shell configuration to prominently notify them about what they need to do.

Supporting all shells and advice on them seems to be more related to sidekick-projects like pyenv/pyenv-installer.

Actually, pyenv init (without arguments) is intended for Pyenv-installer. Since the instructions are specific to Pyenv's version rather than Pyenv-Installer's, it's natuiral to keep them in Pyenv's codebase to avoid duplication.

[agoose77]

@native-api thanks for the clarification. I agree on all points; I'm not pushing for any breaking changes any time soon.

I like the --path --shell flags because they make it explicit that this is a two-phase initialisation. I agree that this shouldn't break anything; it should just be an alias with a long-term deprecation policy. Given that we've now made public changes to the pyenv initialisation APIs, I also concur that pyenv init - needs to remain stable.

[aiguofer]

The updated instructions seem pretty complete to me. The only issue I can see is for anyone using a shell framework like prezto or oh-my-zsh. They generally have some plugin for pyenv which may do some, or all, of the initialization. For example, prezto runs eval "$(pyenv init - --no-rehash zsh)", while oh-my-zsh attempts to do it all.

I imagine you don't need to go into detail about this, but maybe just a mention if you're using one of those frameworks you might only need the PATH changes (seems like that's all that would be needed for both of those frameworks).

[shireenrao]

Thank you @aiguofer - even after following the changes mentioned, I was seeing issues. After seeing your comment on oh-my-zsh plugins I was able to resolve this by disabling the plugin.

[native-api]

For example, prezto runs eval "$(pyenv init - --no-rehash zsh)", while oh-my-zsh attempts to do it all.

Those projects will need to react to Pyenv changes.

[native-api]

I imagine you don't need to go into detail about this, but maybe just a mention if you're using one of those frameworks you might only need the PATH changes (seems like that's all that would be needed for both of those frameworks).

I feel that it's rather the responsibility of those projects to inform their users which configuration steps they're saving them from.

[aiguofer]

I imagine you don't need to go into detail about this, but maybe just a mention if you're using one of those frameworks you might only need the PATH changes (seems like that's all that would be needed for both of those frameworks).

I feel that it's rather the responsibility of those projects to inform their users which configuration steps they're saving them from.

Yeah that makes sense... Maybe just a mention to check how their framework handles pyenv then. I feel like a lot of users might unfortunately not be aware that their framework might already be doing something and thus think it's an issue with pyenv itself.