Default home not writable, %HOME% does not help (windows) #970

Closed
kubaraczkowski opened this Issue Nov 3, 2011 · 10 comments

Comments

Projects
None yet
4 participants
@kubaraczkowski

Hello, I have trouble with the new Ipython (I think I did not upgrade since 0.10) on windows. My default home directory is not writable (thanks admins!) so previously I had to setup HOME variable pointing to a better location. This, however, does not work with newer ipythons! Any idea why?

@takluyver

This comment has been minimized.

Show comment
Hide comment
@takluyver

takluyver Nov 3, 2011

Member

It looks like we've changed the order of checks slightly. The code for
Windows currently checks

  • HOMESHARE
  • HOMEDRIVE and HOMEPATH
  • USERPROFILE
  • Registry value
    HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\Explorer\Shell
    Folders\Personal
  • HOME

You can also override the location of the .ipython folder if you start it
with "ipython --InteractiveShell.ipython_dir=C:\writable_path.ipython".

Member

takluyver commented Nov 3, 2011

It looks like we've changed the order of checks slightly. The code for
Windows currently checks

  • HOMESHARE
  • HOMEDRIVE and HOMEPATH
  • USERPROFILE
  • Registry value
    HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\Explorer\Shell
    Folders\Personal
  • HOME

You can also override the location of the .ipython folder if you start it
with "ipython --InteractiveShell.ipython_dir=C:\writable_path.ipython".

@fperez

This comment has been minimized.

Show comment
Hide comment
@fperez

fperez Nov 3, 2011

Member

You can also set the environment variable IPYTHON_DIR and then put your ipython directory wherever you want. That's equivalent to Thomas' last suggestion, but is something you can set up once system-wide and be done with it. Let us know if that helps so we can close this issue.

Member

fperez commented Nov 3, 2011

You can also set the environment variable IPYTHON_DIR and then put your ipython directory wherever you want. That's equivalent to Thomas' last suggestion, but is something you can set up once system-wide and be done with it. Let us know if that helps so we can close this issue.

@kubaraczkowski

This comment has been minimized.

Show comment
Hide comment
@kubaraczkowski

kubaraczkowski Nov 7, 2011

Yes, IPYTHON_DIR does help. Could it be added to the windows installation documentation on the website
(http://ipython.org/ipython-doc/stable/install/install.html#windows) ?

Thanks.
Also, the "normal" windows variables (HOMEDRIVE, HOMEPATH) will probably be more often setup by windows administrators, whereas the HOME stays free for the user. What I mean is that that's the variable that is more safe to be changed in enterprise environment, hence more useful for using with custom python/ipython installations...

Yes, IPYTHON_DIR does help. Could it be added to the windows installation documentation on the website
(http://ipython.org/ipython-doc/stable/install/install.html#windows) ?

Thanks.
Also, the "normal" windows variables (HOMEDRIVE, HOMEPATH) will probably be more often setup by windows administrators, whereas the HOME stays free for the user. What I mean is that that's the variable that is more safe to be changed in enterprise environment, hence more useful for using with custom python/ipython installations...

@takluyver

This comment has been minimized.

Show comment
Hide comment
@takluyver

takluyver Nov 7, 2011

Member

I don't think it's a Windows-specific issue, although it would be bizarre to have a non-writable home directory on a Unix-y system. It should be in the docs somewhere, though.

Member

takluyver commented Nov 7, 2011

I don't think it's a Windows-specific issue, although it would be bizarre to have a non-writable home directory on a Unix-y system. It should be in the docs somewhere, though.

@fperez

This comment has been minimized.

Show comment
Hide comment
@fperez

fperez Nov 7, 2011

Member

On Mon, Nov 7, 2011 at 3:39 AM, Kuba Raczkowski
reply@reply.github.com
wrote:

Also,  the "normal" windows variables (HOMEDRIVE, HOMEPATH) will probably be more often setup by windows administrators, whereas the HOME stays free for the user. What I mean is that that's the variable that is more safe to be changed in enterprise environment, hence more useful for using with custom python/ipython installations...

This is good feedback, thanks. What do you guys think, should we
reorder things and move %HOME% to the top? We can still honor the
other variables, just allow users to override control without admin
support.

I'm +1 on that idea, but would be happy to reconsider if there's
feedback to the contrary.

Cheers,

f

Member

fperez commented Nov 7, 2011

On Mon, Nov 7, 2011 at 3:39 AM, Kuba Raczkowski
reply@reply.github.com
wrote:

Also,  the "normal" windows variables (HOMEDRIVE, HOMEPATH) will probably be more often setup by windows administrators, whereas the HOME stays free for the user. What I mean is that that's the variable that is more safe to be changed in enterprise environment, hence more useful for using with custom python/ipython installations...

This is good feedback, thanks. What do you guys think, should we
reorder things and move %HOME% to the top? We can still honor the
other variables, just allow users to override control without admin
support.

I'm +1 on that idea, but would be happy to reconsider if there's
feedback to the contrary.

Cheers,

f

@takluyver

This comment has been minimized.

Show comment
Hide comment
@takluyver

takluyver Nov 7, 2011

Member

Fine by me, but I'm not familiar with how environment variables are typically used on Windows. Let's take it to the mailing list - maybe there was a reason it got changed in the first place.

Member

takluyver commented Nov 7, 2011

Fine by me, but I'm not familiar with how environment variables are typically used on Windows. Let's take it to the mailing list - maybe there was a reason it got changed in the first place.

@minrk

This comment has been minimized.

Show comment
Hide comment
@minrk

minrk Nov 9, 2011

Member

Yes, this has been brought up in #747 as well - I think HOME should be first priority when setting the homedir.

Can I ask, is there a reason that we don't just use os.path.expanduser('~'), and leave it up to Python? Under what circumstances does Python not get the right answer?

Member

minrk commented Nov 9, 2011

Yes, this has been brought up in #747 as well - I think HOME should be first priority when setting the homedir.

Can I ask, is there a reason that we don't just use os.path.expanduser('~'), and leave it up to Python? Under what circumstances does Python not get the right answer?

@fperez

This comment has been minimized.

Show comment
Hide comment
@fperez

fperez Nov 9, 2011

Member

On Tue, Nov 8, 2011 at 8:20 PM, Min RK
reply@reply.github.com
wrote:

Yes, this has been brought up in #747 as well - I think HOME should be first priority when setting the homedir.

OK, HOME first then. Agreed.

Can I ask, is there a reason that we don't just use os.path.expanduser('~'), and leave it up to Python?  Under what circumstances does Python not get the right answer?

I don't really recall what os.path.expanduser('~') gives under
windows. Is it a sensible answer? I'm pretty sure the rather weird
contortions we have now are the product of fixing progressively
weirder crash reports, not that we wanted something that complicated
out of the gate.

Member

fperez commented Nov 9, 2011

On Tue, Nov 8, 2011 at 8:20 PM, Min RK
reply@reply.github.com
wrote:

Yes, this has been brought up in #747 as well - I think HOME should be first priority when setting the homedir.

OK, HOME first then. Agreed.

Can I ask, is there a reason that we don't just use os.path.expanduser('~'), and leave it up to Python?  Under what circumstances does Python not get the right answer?

I don't really recall what os.path.expanduser('~') gives under
windows. Is it a sensible answer? I'm pretty sure the rather weird
contortions we have now are the product of fixing progressively
weirder crash reports, not that we wanted something that complicated
out of the gate.

@minrk

This comment has been minimized.

Show comment
Hide comment
@minrk

minrk Nov 9, 2011

Member

It does almost exactly what we do, but in a different order, and notably excluding HOMESHARE. Unfortunately, based on Googling (many results being Brian trying to figure this out in the first place), it looks like cluster-users need to use HOMESHARE, because UNC paths should always be used there (with the wonderful contradiction that UNC paths must never be used for subprocesses).

I think @ellisonbg put HOMESHARE first, because that's the only way to get some parallel things to work on clusters. The problem is, it's the wrong answer approximately everywhere else.

Cluster users are necessarily more advanced than local users, so maybe they should not be prioritized at the expense of everyone else. If we fall back on os.path.expanduser('~'), then cluster users can trivially override the default with: os.environ['HOME'] = os.environ['HOMESHARE'] (or equivalent in the shell, before starting Python). The difference is, HOME is left to the user, whereas HOMESHARE/HOMEDRIVE/HOMEPATH are set by the system, so it makes sense that the easiest one to change should be the first priority, especially since it is already the first priority in Python itself.

I should also note that on first glance, I think the AvoidUNC shenanigans we have to do should be able to wrap os.system, just like they are used to wrap the subprocess. I do not believe that the two are coupled.

Member

minrk commented Nov 9, 2011

It does almost exactly what we do, but in a different order, and notably excluding HOMESHARE. Unfortunately, based on Googling (many results being Brian trying to figure this out in the first place), it looks like cluster-users need to use HOMESHARE, because UNC paths should always be used there (with the wonderful contradiction that UNC paths must never be used for subprocesses).

I think @ellisonbg put HOMESHARE first, because that's the only way to get some parallel things to work on clusters. The problem is, it's the wrong answer approximately everywhere else.

Cluster users are necessarily more advanced than local users, so maybe they should not be prioritized at the expense of everyone else. If we fall back on os.path.expanduser('~'), then cluster users can trivially override the default with: os.environ['HOME'] = os.environ['HOMESHARE'] (or equivalent in the shell, before starting Python). The difference is, HOME is left to the user, whereas HOMESHARE/HOMEDRIVE/HOMEPATH are set by the system, so it makes sense that the easiest one to change should be the first priority, especially since it is already the first priority in Python itself.

I should also note that on first glance, I think the AvoidUNC shenanigans we have to do should be able to wrap os.system, just like they are used to wrap the subprocess. I do not believe that the two are coupled.

@fperez

This comment has been minimized.

Show comment
Hide comment
@fperez

fperez Nov 9, 2011

Member

I agree, esp. if it means we can simplify some of our code over there and fall back on more stdlib code for this...

Member

fperez commented Nov 9, 2011

I agree, esp. if it means we can simplify some of our code over there and fall back on more stdlib code for this...

@minrk minrk closed this in 351c8fc Nov 24, 2011

stefanv pushed a commit to stefanv/ipython that referenced this issue Nov 30, 2011

defer to stdlib for path.get_home_dir()
We have elaborate and fragile logic for determining
home dir, and it is ultimately less reliable than the stdlib behavior
used for `os.path.expanduser('~')`.  This commit defers to that in
all cases other than a bundled Python in py2exe/py2app environments.

The one case where the default guess will *not* be correct, based on
inline comments, is on WinHPC, where all paths must be UNC (`\\foo`), and
thus HOMESHARE is the logical first choice.  However, HOMESHARE is
the wrong answer in approximately all other cases where it is defined,
and the fix for WinHPC users is the trivial `HOME=%HOMESHARE%`.

This removes the various tests of our Windows path resolution logic,
which are no longer relevant. Further, $HOME is used by the stdlib
as first priority on *all* platforms, so tests for this behavior are
no longer posix-specific.

closes gh-970
closes gh-747

mattvonrocketstein pushed a commit to mattvonrocketstein/ipython that referenced this issue Nov 3, 2014

defer to stdlib for path.get_home_dir()
We have elaborate and fragile logic for determining
home dir, and it is ultimately less reliable than the stdlib behavior
used for `os.path.expanduser('~')`.  This commit defers to that in
all cases other than a bundled Python in py2exe/py2app environments.

The one case where the default guess will *not* be correct, based on
inline comments, is on WinHPC, where all paths must be UNC (`\\foo`), and
thus HOMESHARE is the logical first choice.  However, HOMESHARE is
the wrong answer in approximately all other cases where it is defined,
and the fix for WinHPC users is the trivial `HOME=%HOMESHARE%`.

This removes the various tests of our Windows path resolution logic,
which are no longer relevant. Further, $HOME is used by the stdlib
as first priority on *all* platforms, so tests for this behavior are
no longer posix-specific.

closes gh-970
closes gh-747

mattvonrocketstein pushed a commit to mattvonrocketstein/ipython that referenced this issue Nov 3, 2014

Merge pull request #998 from minrk/homedir
defer to stdlib os.path.expanduser('~') for path.get_home_dir()

frozen env still comes first, and My Documents registry query is fallback on Windows, but in all normal cases, expanduser is used, which has $HOME as first priority.

IPython also no longer considers finding no writable home dir as a fatal error.

closes #810
closes #747
closes #970
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment