/usr/bin/flux shouldn't set PYTHONPATH to the system python's site-package

[dongahn]

From a side-installed flux ( /usr/global/tools/flux/toss_3_x86_64_ib/default/bin/flux):

[rztopaz11]:echo $PYTHONPATH

[rztopaz11]:srun --pty -N 1 -n 1 --mpi=none /usr/global/tools/flux/toss_3_x86_64_ib/default/bin/flux start
[rztopaz11]:echo $PYTHONPATH
/collab/usr/global/tools/flux/toss_3_x86_64_ib/flux-0.9.x-20180618/lib64/python2.7/site-packages

The above has a minimal side effect.

From /usr/bin/flux:

[rztopaz11]:echo $PYTHONPATH

[rztopaz11]:srun --pty -N 1 -n 1 --mpi=none flux start
[rztopaz11]:echo $PYTHONPATH
/usr/lib64/python2.7/site-packages

The above PYTHONPATH is for the system python that has a lot of other extra packages that will cause issues with non-system (tce) pythons, which our applications are using. As a result, this user cannot run a large multiphysics code.

From the user who hit this:

You might install the flux module in a share or lib location that contains only the flux module and add this to PYTHONPATH. The issue in general with PYTHONPATH is that it takes precedence over all paths even the python executable defined paths. So it cannot be overridden.

[grondo]

The user's suggestion seems like a good one to me, because I'm not sure how we'd determine reliably when the path we'd add to PYTHONPATH is the "system" path. @SteVwonder or @trws?

[trws]

I may have to think about this a bit. The suggestion is sensible, but means things will only work with PYTHONPATH set.

[grondo]

Would it be possible (and not too much of a kludge) to install Flux's python to the site-package path, but create a link from a flux-specific location (so that PYTHONPATH could be set to flux-specific path, but default system path will work)?

[trws]

We should test it, but I see no reason that shouldn't work. I like that idea much better than moving it out otherwise.

…

On Sat, 09. Feb 17:04, Mark Grondona wrote: Would it be possible (and not too much of a kludge) to install Flux's python to the site-package path, but create a link from a flux-specific location (so that PYTHONPATH could be set to flux-specific path, but default system path will work)? -- You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub: #1999 (comment)

[grondo]

Currently we have the following directories defined in autotools for python modules installation:

AS_VAR_SET(fluxpydir, $pyexecdir/flux)
AC_SUBST(fluxpydir)
AS_VAR_SET(fluxsodir, $pyexecdir/_flux)
AC_SUBST(fluxsodir)
AS_VAR_SET(fluxpymoddir, $pyexecdir/flux/modules)
AC_SUBST(fluxpymoddir)

I think it may be sufficient for this issue if we define one more directory:

AS_VAR_SET(fluxpylinkdir, $fluxlibdir/python)
AC_SUBST(fluxpylinkdir)

Then at install time, create links to fluxpydir and fluxsodir in fluxpylinkdir. Then update INSTALLED_PYTHON_PATH to fluxpylinkdir instead of pyexecdir.

I'll try this out, but someone please speak up if there is a flaw in the plan.