Python compatibility issue #8
Comments
|
I'm not sure I fully understand what you are trying to do. If your only interest is using EUPS, I would suggest installing it from source so that it uses the version of Python shipped with the GridPP VM. Although I don't know for sure, I would say that EUPS does not put strong requirements on Python. That being said, Python v2.6.6 may be a bit too old for EUPS, but it is worth trying. So far I have not had reports on conflicting versions of Python using this distribution of the LSST software stack. Maybe people using it use recent versions of Linux distributions which include a more recent version of Python, such as Python 2.7.5. By the way, the LSST stack is supported on CentOS 7 but is known to work on other distributions such as Ubuntu. A potential solution would be to use a Docker container, but I don't know if the GridPP VM allows you to do that. I'm sorry I'm not able to be more helpful. |
|
I'm trying to leverage EUPS on GridPP. I have a workflow we are trying to get working on GridPP that requires the installation of several pieces of software that aren't standard on GridPP, and EUPS allows us to do that easily. Since your version of EUPS is available from the LSST CVMFS, and this work is under LSST, it made sense to do this. We can simply package up our own EUPS-registered software and ship them off with our workflow to run on GridPP (with access to the LSST CVMFS, since this is work done under LSST VO registered accounts). EUPS works fine with Python 2.6.6, and it also works fine with Python 2.7.8 when done manually, but the issue comes with the way that EUPS is set up on the LSST CVMFS. When we run "setup" on our Python 2.7 package, all of the path variables change to point to the libraries and modules for that Python 2.7 install. However, your install for EUPS doesn't use the version of Python that is in the path (in this case 2.7) but specifically calls the system-wide version of Python regardless. So, in my setup now, I have Python 2.6 looking for modules/libraries/paths created for Python 2.7. This causes major problems. Is there a reason why this is so? I've looked at the versions of EUPS available on github, and they don't seem to do this. It seems to call Python in a general sense. Do you have an old version of EUPS? Is the only option to download, build, install and package the entirety of EUPS as a standalone package and ship that off with our packages? |
|
Thanks for your clarification, I think I can now see what you are trying to accomplish. The LSST software stack available through CernVM FS is installed from source following the instructions here. As part of this installation process, a version of EUPS certified compatible with each version of the LSST software stack is also installed. EUPS requires the path of the Python interpreter to be known at installation time. See here. For your particular case, it would be more convenient if the EUPS Python scripts used the shebang Since the machines where the LSST binary is built run CentOS 7, I use the Python which comes with the system, i.e. The EUPS experts may be able to tell you why the Python path is needed at installation time. See: https://github.com/RobertLuptonTheGood/eups. |
|
Ah I see! I didn't realise it was set at installation. That makes sense
now. It isn't an issue if the version of python that was setup was kept
separate from the system version, but it seems to not be. That might be
something I need to ask a Python expert about. At this stage I'm looking at
a Chicken/Egg situation though if that's not easy to fix. I might have to
use LSST EUPS to setup Python2.7, use that to install EUPS as an EUPS
package that links to the correct Python version (2.7), and bundle it all
off together. This could be fun...
Thanks for your help!
…On 22 March 2017 at 16:15, Fabio Hernandez ***@***.***> wrote:
Thanks for your clarification, I think I can now see what you are trying
to accomplish.
The LSST software stack available through CernVM FS is installed from
source following the instructions here
<https://pipelines.lsst.io/install/newinstall.html>. As part of this
installation process, a version of EUPS certified compatible with each
version of the LSST software stack is also installed.
EUPS requires the path of the Python interpreter to be known at
installation time. See here
<https://github.com/RobertLuptonTheGood/eups/blob/master/configure.ac#L25>.
For your particular case, it would be more convenient if the EUPS Python
scripts used the shebang #!/usr/bin/env python instead of the absolute
path of a Python interpreter. However, I don't know the reason why this is
not done.
Since the machines where the LSST binary is built run CentOS 7, I use the
Python which comes with the system, i.e. /usr/bin/python. This is the
reason /usr/bin/python is the path that you find "hardcoded" in the EUPS
Python scripts.
The EUPS experts may be able to tell you why the Python path is needed at
installation time. See: https://github.com/RobertLuptonTheGood/eups.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
<#8 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AKeL573uFdNPmjaGGiqA7YpDcj0epPYsks5roUksgaJpZM4MlN2c>
.
|
I'm trying to help convert a workflow for use on GridPP, and attempting to use the version of EUPS that comes as part of the LSST CVMFS, but running into issues with creating a new version of Python. The version that is bundled with the GridPP VM (v2.6.6) is too old for use with this workflow, but when I run setup on the new Python install (v2.7.8), everything to do with eups (setup, unsetup, eups list, etc) fails. This seems to be due to the fact that the EUPS version on the LSST CVMFS hardcodes the use of the system version of Python (e.g. /usr/bin/python in eups_setup) instead of the newly-setup version. This causes some issues with compatibility (particularly with future and _weakref), since the system Python is now looking for libraries in places that don't exist, since setup altered the environment variables to suit the new version of Python. Do you know of a way to make sure that the two are kept separate? This isn't technically a CVMFS problem, but I would be surprised if people didn't run into this issue with using it, and I'm hoping someone has an answer.
Cheers
The text was updated successfully, but these errors were encountered: