Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

No module named sqlalchemy.orm #3224

Closed
Jondeen opened this issue Nov 28, 2016 · 5 comments
Closed

No module named sqlalchemy.orm #3224

Jondeen opened this issue Nov 28, 2016 · 5 comments

Comments

@Jondeen
Copy link

Jondeen commented Nov 28, 2016 •
edited
Loading

Hi,

I'm trying to run MACS2 pulled from the Galaxy Main Tool Shed (I've tried both IUC's and pjbriggs' versions) and it works fine, but when the script attempts to update Galaxy metadata after successful run (that is; the proper BAM-files are generated) it fails, complaining about dependencies (which should definitely be available in the .venv, and also globally on the system) as below:

Fatal error: Matched on Error:
Traceback (most recent call last):
  File "/home/galaxy/Galaxy/database/job_working_directory/000/490/set_metadata_FrMKGT.py", line 1, in <module>
    from galaxy_ext.metadata.set_metadata import set_metadata; set_metadata()
  File "/home/galaxy/Galaxy/lib/galaxy_ext/metadata/set_metadata.py", line 23, in <module>
    from sqlalchemy.orm import clear_mappers
ImportError: No module named sqlalchemy.orm

This seems to be similar to #2541 but I couldn't figure out how it was resolved. Any pointers?

UPDATE: In fact sqlalchemy was not installed globally. Installing all required dependencies seem to possibly solve the issue.
@bgruening
Copy link
Member

@Jondeen have you changed your dependency_resolver_conf.xml file?

@nsoranzo
Copy link
Member

And what Galaxy release are you using?

@jmchilton
Copy link
Member

jmchilton commented Dec 21, 2016 •
edited
Loading

I have some logging from Dr. David Morais - this happen on 16.10 and the problem appears to be that sometimes the install of samtools installs setuptools - and this in turn brings in Python.

The following packages will be downloaded:


    package                    |            build
    ---------------------------|-----------------
    libgcc-5.2.0               |                0         1.1 MB
    openssl-1.0.2j             |                0         3.2 MB
    zlib-1.2.8                 |                3         101 KB
    curl-7.49.0                |                1         543 KB
    samtools-1.3.1             |                5         874 KB  bioconda
    setuptools-27.2.0          |           py27_0         521 KB
    ------------------------------------------------------------
                                           Total:         6.2 MB
The following NEW packages will be INSTALLED:
    curl:       7.49.0-1
    libgcc:     5.2.0-0
    openssl:    1.0.2j-0
    pip:        9.0.1-py27_1
    python:     2.7.13-0
    readline:   6.2-2
    samtools:   1.3.1-5       bioconda
    setuptools: 27.2.0-py27_0
    sqlite:     3.13.0-0
    tk:         8.5.18-0
    wheel:      0.29.0-py27_0
    zlib:       1.2.8-3
Fetching packages ...
libgcc-5.2.0-0 100% |###############################| Time: 0:00:00  13.55 MB/s
openssl-1.0.2j 100% |###############################| Time: 0:00:00  20.89 MB/s
zlib-1.2.8-3.t 100% |###############################| Time: 0:00:00   7.11 MB/s
curl-7.49.0-1. 100% |###############################| Time: 0:00:00  10.03 MB/s
samtools-1.3.1 100% |###############################| Time: 0:00:00   3.37 MB/s
setuptools-27. 100% |###############################| Time: 0:00:00  10.30 MB/s
Extracting packages ...
[      COMPLETE      ]|##################################################| 100%
Linking packages ...
[      COMPLETE      ]|##################################################| 100%
#
# To activate this environment, use:
# > source activate __samtools@_uv_
#
# To deactivate this environment, use:
# > source deactivate __samtools@_uv_
#
Cache location: /cvmfs/soft.galaxy/galaxy/tool-dependency/_conda/pkgs
Will remove the following tarballs:
/cvmfs/soft.galaxy/galaxy/tool-dependency/_conda/pkgs
-----------------------------------------------------
enum34-1.1.6-py27_0.tar.bz2                   53 KB
libgcc-5.2.0-0.tar.bz2                       1.1 MB
sqlalchemy-1.1.4-py27_0.tar.bz2              1.4 MB
curl-7.49.0-1.tar.bz2                        543 KB
openssl-1.0.2j-0.tar.bz2                     3.2 MB
ruamel_yaml-0.11.14-py27_0.tar.bz2           352 KB
setuptools-27.2.0-py27_0.tar.bz2             521 KB
samtools-1.3.1-5.tar.bz2                     874 KB
conda-4.2.13-py27_0.tar.bz2                  391 KB
virtualenv-15.0.3-py27_0.tar.bz2             1.8 MB
decorator-4.0.2-py27_0.tar.bz2                11 KB
conda-env-2.6.0-0.tar.bz2                     502 B
sqlparse-0.1.16-py27_0.tar.bz2                51 KB
pip-9.0.1-py27_1.tar.bz2                     1.6 MB
python-2.7.13-0.tar.bz2                     11.5 MB
six-1.9.0-py27_0.tar.bz2                      17 KB
sqlite-3.13.0-0.tar.bz2                      4.0 MB
pbr-1.8.0-py27_0.tar.bz2                      86 KB
zlib-1.2.8-3.tar.bz2                         101 KB
---------------------------------------------------
Total:                                      27.4 MB
Removing enum34-1.1.6-py27_0.tar.bz2
Removing libgcc-5.2.0-0.tar.bz2
Removing sqlalchemy-1.1.4-py27_0.tar.bz2
Removing curl-7.49.0-1.tar.bz2
Removing openssl-1.0.2j-0.tar.bz2
Removing ruamel_yaml-0.11.14-py27_0.tar.bz2
Removing setuptools-27.2.0-py27_0.tar.bz2
Removing samtools-1.3.1-5.tar.bz2
Removing conda-4.2.13-py27_0.tar.bz2
Removing virtualenv-15.0.3-py27_0.tar.bz2
Removing decorator-4.0.2-py27_0.tar.bz2
Removing conda-env-2.6.0-0.tar.bz2
Removing sqlparse-0.1.16-py27_0.tar.bz2
Removing pip-9.0.1-py27_1.tar.bz2

For completeness - a working log such as mine does not include setuptools and therefore doesn't include Python:

Package plan for installation in environment /home/john/workspace/galaxy/database/tool_dependencies/_conda/envs/__samtools@_uv_:

The following packages will be downloaded:

    package                    |            build
    ---------------------------|-----------------
    libgcc-5.2.0               |                0         1.1 MB
    openssl-1.0.2j             |                0         3.2 MB
    zlib-1.2.8                 |                3         101 KB
    curl-7.49.1                |                2         545 KB
    samtools-1.3.1             |                5         874 KB
    ------------------------------------------------------------
                                           Total:         5.7 MB

The following NEW packages will be INSTALLED:

    curl:     7.49.1-2
    libgcc:   5.2.0-0 
    openssl:  1.0.2j-0
    samtools: 1.3.1-5 
    zlib:     1.2.8-3 

Fetching packages ...
libgcc-5.2.0-0 100% |###################################################################################| Time: 0:00:00   3.92 MB/s
openssl-1.0.2j  33% |############################                                                       | Ti127.0.0.1 - - [21/Dec/2016:11:13:53 -0400] "GET /api/histories/6505e875ddb66fd2/contents?v=dev&q=update_time-ge&qv=2016-12-21T16%3A13%3A49.000Z HTTP/1.1" 200 - "http://localhost:8080/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/50.0.2661.102 Safari/537.36"
openssl-1.0.2j 100% |###################################################################################| Time: 0:00:00   4.22 MB/s
zlib-1.2.8-3.t 100% |###################################################################################| Time: 0:00:00 959.62 kB/s
curl-7.49.1-2. 100% |###################################################################################| Time: 0:00:00 989.14 kB/s
samtools-1.3.1 100% |###################################################################################| Time: 0:00:00   1.37 MB/s
Extracting packages ...
127.0.0.1 - - [21/Dec/2016:11:13:57 -0400] "GET /api/histories/6505e875ddb66fd2/contents?v=dev&q=update_time-ge&qv=2016-12-21T16%3A13%3A53.000Z HTTP/1.1" 200 - "http://localhost:8080/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/50.0.2661.102 Safari/537.36"
[      COMPLETE      ]|######################################################################################################| 100%
Linking packages ...
[      COMPLETE      ]|######################################################################################################| 100%
#
# To activate this environment, use:
# $ source activate __samtools@_uv_
#
# To deactivate this environment, use:
# $ source deactivate
#

@mvdbeek
Copy link
Member

mvdbeek commented Dec 21, 2016

Hmmm, this is strange, there's really no dependency on setuptools in the bioconda recipe

Note also that there is conda-4.2.13-py27_0.tar.bz2 391 KB ... maybe conda wants to upgrade itself?!

jmchilton added a commit to jmchilton/galaxy that referenced this issue Dec 21, 2016
@jmchilton
Do not replace Galaxy's Python environment for select tools.
329ca24 
This has never happened in my testing, but there have been multiple reports of Conda crafting environments for the set metadata tool that include Python. When this happens Galaxy's Python environment is lost and the tool cannot function properly.

Fixes galaxyproject#3238.
Fixes galaxyproject#3224.
Fixes https://biostar.usegalaxy.org/p/20865/.
@jmchilton jmchilton mentioned this issue Dec 21, 2016
Merged
jmchilton added a commit to jmchilton/galaxy that referenced this issue Dec 21, 2016
@jmchilton
Do not replace Galaxy's Python environment for select tools.
fe2a1d8 
This has never happened in my testing, but there have been multiple reports of Conda crafting environments for the set metadata tool that include Python. When this happens Galaxy's Python environment is lost and the tool cannot function properly.

Fixes galaxyproject#3238.
Fixes galaxyproject#3224.
Fixes https://biostar.usegalaxy.org/p/20865/.

Rebased to add a comment as suggested by @nsoranzo.
@bgruening
Copy link
Member

David also contacted me via mail and we were able to solve it for him by installing the samtools TS package. This fix will prevent this workaround in the future! Thanks.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@jmchilton @bgruening @Jondeen @nsoranzo @mvdbeek