A shell-agnostic "activate" #247

Closed
datagrok opened this Issue Mar 30, 2012 · 22 comments

Comments

Projects
None yet
@datagrok

I wrote a brief diatribe, "Virtualenv's bin/activate is Doing It Wrong," documenting an approach to activating virtual environments that avoids sourcing, does not require multiple scripts to be maintained for every possible (unix) shell, and escapes several other annoyances I encounter with virtualenv's bin/activate script.

The document includes a proof-of-concept script that works well for me. If this is something that the community would be interested in, I'll fork the repo and offer a patch.

@carljm

This comment has been minimized.

Show comment
Hide comment
@carljm

carljm Mar 31, 2012

Contributor

Seems pretty sensible to me. Perhaps you could post this to the python-virtualenv google group mailing list and see if there's other feedback there?

The only quibble I have with the rant is the assertion that bin/activate is necessary to use a virtualenv. It is not necessary, it's merely a convenience. The only functional thing it does is modify your shell PATH to put ENV/bin first on it. You can just as well run ENV/bin/python (or ENV/bin/anything) directly with no "activate", and it will run "within" the virtualenv.

I think there's a sufficient number of wrapper scripts out there depending on a sourced activate (and the deactivate command), not to mention people accustomed to them, that replacing bin/activate directly is (unfortunately) a non-starter. This would have to be a new feature under a different name, and then perhaps we could consider beginning a (slow) deprecation process for the existing activate script(s).

Thanks for the comment!

Contributor

carljm commented Mar 31, 2012

Seems pretty sensible to me. Perhaps you could post this to the python-virtualenv google group mailing list and see if there's other feedback there?

The only quibble I have with the rant is the assertion that bin/activate is necessary to use a virtualenv. It is not necessary, it's merely a convenience. The only functional thing it does is modify your shell PATH to put ENV/bin first on it. You can just as well run ENV/bin/python (or ENV/bin/anything) directly with no "activate", and it will run "within" the virtualenv.

I think there's a sufficient number of wrapper scripts out there depending on a sourced activate (and the deactivate command), not to mention people accustomed to them, that replacing bin/activate directly is (unfortunately) a non-starter. This would have to be a new feature under a different name, and then perhaps we could consider beginning a (slow) deprecation process for the existing activate script(s).

Thanks for the comment!

@datagrok

This comment has been minimized.

Show comment
Hide comment
@datagrok

datagrok Mar 31, 2012

Thanks very much for the reply. You're absolutely right about my incorrect assertion. I'll adjust the language in my gist, and follow up with a query to the mailing list this weekend.

That existing scripts, etc., depend on ENV/bin/activate is completely understandable. I too would not want to yank the rug out from under any users. Since mine is a "system-level" executable, it need not conflict with or immediately replace the existing activate scripts. I could also imagine it implemented in Python, and invoked like virtualenv activate, or something, to avoid installing the extra executable on users' systems.

Thanks very much for the reply. You're absolutely right about my incorrect assertion. I'll adjust the language in my gist, and follow up with a query to the mailing list this weekend.

That existing scripts, etc., depend on ENV/bin/activate is completely understandable. I too would not want to yank the rug out from under any users. Since mine is a "system-level" executable, it need not conflict with or immediately replace the existing activate scripts. I could also imagine it implemented in Python, and invoked like virtualenv activate, or something, to avoid installing the extra executable on users' systems.

@carljm

This comment has been minimized.

Show comment
Hide comment
@carljm

carljm Mar 31, 2012

Contributor

Sounds good! I wouldn't want the use of this new feature to depend on the user still having virtualenv.py itself around; it's a useful feature of virtualenvs that they are self-sufficient once created. So I think this should be an additional script in ENV/bin, the only puzzle is figuring out what to name it (one of the two hard problems in computer science?)

Contributor

carljm commented Mar 31, 2012

Sounds good! I wouldn't want the use of this new feature to depend on the user still having virtualenv.py itself around; it's a useful feature of virtualenvs that they are self-sufficient once created. So I think this should be an additional script in ENV/bin, the only puzzle is figuring out what to name it (one of the two hard problems in computer science?)

@kennethreitz

This comment has been minimized.

Show comment
Hide comment
@kennethreitz

kennethreitz Mar 31, 2012

Source activation is one of my favorite of virtualenv today. Ideally this new version would still have that ability, just not require it, correct?

Source activation is one of my favorite of virtualenv today. Ideally this new version would still have that ability, just not require it, correct?

@carljm

This comment has been minimized.

Show comment
Hide comment
@carljm

carljm Mar 31, 2012

Contributor

Well, you'd use either the subshell activation script or the source activation script. I was thinking we might deprecate the latter over time - can you say what advantages you think it has over subshell activation?

Contributor

carljm commented Mar 31, 2012

Well, you'd use either the subshell activation script or the source activation script. I was thinking we might deprecate the latter over time - can you say what advantages you think it has over subshell activation?

@kennethreitz

This comment has been minimized.

Show comment
Hide comment
@kennethreitz

kennethreitz Mar 31, 2012

Can you use a subshell within, say, a bash script?

Can you use a subshell within, say, a bash script?

@carljm

This comment has been minimized.

Show comment
Hide comment
@carljm

carljm Mar 31, 2012

Contributor

I suppose you'd either preface each command in the script with the subshell-activate script, or run your entire bash script via the subshell-activate script. If you were clever you could probably even do it all in one file using a here-doc or something. None of those are quite the same as sourcing activate in a script - do you do that a lot? I prefer to use full paths to the env's bin directory instead.

Contributor

carljm commented Mar 31, 2012

I suppose you'd either preface each command in the script with the subshell-activate script, or run your entire bash script via the subshell-activate script. If you were clever you could probably even do it all in one file using a here-doc or something. None of those are quite the same as sourcing activate in a script - do you do that a lot? I prefer to use full paths to the env's bin directory instead.

@kennethreitz

This comment has been minimized.

Show comment
Hide comment
@kennethreitz

kennethreitz Mar 31, 2012

I do currently, yes. It feels really clean to me.

Would it be difficult to hack a sourceable version of the subshell perhaps?

I do currently, yes. It feels really clean to me.

Would it be difficult to hack a sourceable version of the subshell perhaps?

@datagrok

This comment has been minimized.

Show comment
Hide comment
@datagrok

datagrok Mar 31, 2012

@kennethreitz it sounds like you're currently building scripts that begin:

#!/bin/sh
source bin/activate
... (stuff) ...

It would be easy to support the pattern ssh-agent uses, so those scripts might look like this instead:

#/bin/sh
eval `bin/activate -s`
... (stuff) ...

But that assumes that you don't use the deactivate feature. (We can't use command substitution to define shell functions like deactivate.) That might be fine for a script like the above, where simply exiting the script accomplishes the same thing. Do you use deactivate inside your scripts?

All of the above ignores for the moment, for the sake of clarity, the point @carljm raised about the subshell-style activate script needing to be called something other than activate. (Which I agree with.)

@kennethreitz it sounds like you're currently building scripts that begin:

#!/bin/sh
source bin/activate
... (stuff) ...

It would be easy to support the pattern ssh-agent uses, so those scripts might look like this instead:

#/bin/sh
eval `bin/activate -s`
... (stuff) ...

But that assumes that you don't use the deactivate feature. (We can't use command substitution to define shell functions like deactivate.) That might be fine for a script like the above, where simply exiting the script accomplishes the same thing. Do you use deactivate inside your scripts?

All of the above ignores for the moment, for the sake of clarity, the point @carljm raised about the subshell-style activate script needing to be called something other than activate. (Which I agree with.)

@kennethreitz

This comment has been minimized.

Show comment
Hide comment
@kennethreitz

kennethreitz Mar 31, 2012

@datagrok thanks for the clarification. That will suit my personal needs well, actually.

@datagrok thanks for the clarification. That will suit my personal needs well, actually.

@podados

This comment has been minimized.

Show comment
Hide comment
@podados

podados Apr 2, 2012

Something like the "inve" script - https://github.com/podados/python-orb/ (easy_install -Z orb) .

podados commented Apr 2, 2012

Something like the "inve" script - https://github.com/podados/python-orb/ (easy_install -Z orb) .

@datagrok

This comment has been minimized.

Show comment
Hide comment
@datagrok

datagrok Apr 2, 2012

@podados re: python-orb

This creates a virtualenv called 'myenv' and creates an empty '.orb' file within the virtualenv directory. Subsequent orb commands first search for the '.orb' file and from this file's location the virtualenv's bin directory is known.

This approach very much resembles that of another tool I wrote, datagrok/subcommander. I'll give orb a try too. Great work!

datagrok commented Apr 2, 2012

@podados re: python-orb

This creates a virtualenv called 'myenv' and creates an empty '.orb' file within the virtualenv directory. Subsequent orb commands first search for the '.orb' file and from this file's location the virtualenv's bin directory is known.

This approach very much resembles that of another tool I wrote, datagrok/subcommander. I'll give orb a try too. Great work!

@westurner

This comment has been minimized.

Show comment
Hide comment
@westurner

westurner Feb 27, 2013

I don't know how portable this is, but in bash it's possible to use a file descriptor:

declare -gx _ACTIVATE='/usr/bin/activate'
source <($_ACTIVATE -s)
bash <($_ACTIVATE -s)

There are (not-so-) subtle differences between eval and source: http://askubuntu.com/q/174721

I don't know how portable this is, but in bash it's possible to use a file descriptor:

declare -gx _ACTIVATE='/usr/bin/activate'
source <($_ACTIVATE -s)
bash <($_ACTIVATE -s)

There are (not-so-) subtle differences between eval and source: http://askubuntu.com/q/174721

@wyuenho

This comment has been minimized.

Show comment
Hide comment
@wyuenho

wyuenho May 11, 2013

Hi, I just switched my shell to zsh and virtualenv is totally messing with my zsh prompt, I was just wondering if there was ever a patch produced that would fix this problem?

wyuenho commented May 11, 2013

Hi, I just switched my shell to zsh and virtualenv is totally messing with my zsh prompt, I was just wondering if there was ever a patch produced that would fix this problem?

@dstufft

This comment has been minimized.

Show comment
Hide comment
@dstufft

dstufft Sep 16, 2014

Member

So I poked at this briefly, and I don't think this actually works in practice, at least not in a way that I can make it happen.

The problem is that shells mess with $PATH on initialization either directly or as a result of the rc scripts people have. For instance, PATH=~/.virtualenvs/foo zsh on my machine gets me a $PATH without ~/.virtualenvs/foo in it at all because my rc scripts reset the $PATH, even if I take out the reset, I end up with them several items deeper into the $PATH because it does things like add ~/.local/bin and the like to my $PATH as well.

Member

dstufft commented Sep 16, 2014

So I poked at this briefly, and I don't think this actually works in practice, at least not in a way that I can make it happen.

The problem is that shells mess with $PATH on initialization either directly or as a result of the rc scripts people have. For instance, PATH=~/.virtualenvs/foo zsh on my machine gets me a $PATH without ~/.virtualenvs/foo in it at all because my rc scripts reset the $PATH, even if I take out the reset, I end up with them several items deeper into the $PATH because it does things like add ~/.local/bin and the like to my $PATH as well.

@asheidan

This comment has been minimized.

Show comment
Hide comment
@asheidan

asheidan Sep 16, 2014

That sounds like there is initialization in your rc that should be in say profile or similar.

I have run in to the same problem but after moving environment changes to my profile and forcing my shell not to source profile by default on startup it works.

That sounds like there is initialization in your rc that should be in say profile or similar.

I have run in to the same problem but after moving environment changes to my profile and forcing my shell not to source profile by default on startup it works.

@dstufft

This comment has been minimized.

Show comment
Hide comment
@dstufft

dstufft Sep 16, 2014

Member

Possibly, but here's the thing, the current activate scripts work just fine. Moving to something like inve $SHELL would be a massive regression for people whose rc scripts do that and telling them that it's their fault because they put something in their shell in the wrong place isn't particularly user friendly.

Member

dstufft commented Sep 16, 2014

Possibly, but here's the thing, the current activate scripts work just fine. Moving to something like inve $SHELL would be a massive regression for people whose rc scripts do that and telling them that it's their fault because they put something in their shell in the wrong place isn't particularly user friendly.

@asheidan

This comment has been minimized.

Show comment
Hide comment
@asheidan

asheidan Sep 16, 2014

The eternal problem, everything would be much easier without users...

You're correct that this would be a large problem and possible hurdle, especially for new users to virtualenv which might not be that "terminal"-savvy.

I'm not for removing the current functionality with source activate. But using the previously mentioned trick ssh-agent uses should work as well as the current system (depending on the functionality of the shell in question).

eval $(bin/activate -s)

Or

source <(bin/activate -s)

Is there a commonly used shell which doesn't support either source or eval?

The eternal problem, everything would be much easier without users...

You're correct that this would be a large problem and possible hurdle, especially for new users to virtualenv which might not be that "terminal"-savvy.

I'm not for removing the current functionality with source activate. But using the previously mentioned trick ssh-agent uses should work as well as the current system (depending on the functionality of the shell in question).

eval $(bin/activate -s)

Or

source <(bin/activate -s)

Is there a commonly used shell which doesn't support either source or eval?

@dstufft dstufft closed this in #581 Dec 24, 2014

@qwiglydee

This comment has been minimized.

Show comment
Hide comment
@qwiglydee

qwiglydee Nov 13, 2015

Ehm, and what is the resolution of the issue, other then discussions in #581 ?

Ehm, and what is the resolution of the issue, other then discussions in #581 ?

@hellerbarde

This comment has been minimized.

Show comment
Hide comment
@hellerbarde

hellerbarde Jul 2, 2017

in reference to @datagrok's #247 (comment), it should in fact be possible to define functions even in the eval construct.

Save this in a file called foo.sh:

#!/bin/bash
cat <<EOF
function foo() {
        echo "foo"
}
EOF

and then eval `./foo.sh`

in reference to @datagrok's #247 (comment), it should in fact be possible to define functions even in the eval construct.

Save this in a file called foo.sh:

#!/bin/bash
cat <<EOF
function foo() {
        echo "foo"
}
EOF

and then eval `./foo.sh`

@datagrok

This comment has been minimized.

Show comment
Hide comment
@datagrok

datagrok Jul 3, 2017

Wow, thank you @hellerbarde, you are absolutely right. I don't know why I thought that was not the case. I wish me-from-five-years-ago would have documented that.

I think maybe I conflated that with not being able to export a function from one shell to a subshell, the way you can with environment variables.

datagrok commented Jul 3, 2017

Wow, thank you @hellerbarde, you are absolutely right. I don't know why I thought that was not the case. I wish me-from-five-years-ago would have documented that.

I think maybe I conflated that with not being able to export a function from one shell to a subshell, the way you can with environment variables.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment