Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Generate metadata in separate conda environment #2253

Merged
merged 2 commits into from Apr 27, 2016

Conversation

Projects
None yet
3 participants
@mvdbeek
Copy link
Member

commented Apr 27, 2016

Galaxy supports generating metadata in a separate command (enable
beta_tool_command_isolation). This command has its own dependency resolution,
that may interfere with thedependency resolution of the actual tool, if the
same environment is being used. With this PR we create a new environment when
resolving metadata dependencies with conda, that will be active only for
metadata generation.

Should solve issue #2247

Generate metadata in separate conda environment
Galaxy supports generating metadata in a separate command (enable
beta_tool_command_isolation). This command has its own dependency resolution,
that may interfere with thedependency resolution of the actual tool, if the
same environment is being used. With this PR we create a new environment when
resolving metadata dependencies with conda, that will be active only for
metadata generation.
@mvdbeek

This comment has been minimized.

Copy link
Member Author

commented Apr 27, 2016

Fyi, the relevant part of the job_script with this PR would look like this:

mkdir -p working; cd working; /Users/marius/src/galaxy/database/jobs_directory/000/313/tool_script.sh; return_code=$?; cd '/Users/marius/src/galaxy/database/jobs_directory/000/313'; [ "$CONDA_DEFAULT_ENV" = "/Users/marius/src/galaxy/database/jobs_directory/000/313/conda-metadata-env" ] || . /Users/marius/conda/bin/activate '/Users/marius/src/galaxy/database/jobs_directory/000/313/conda-metadata-env' 2>&1 ; python "/Users/marius/src/galaxy/database/jobs_directory/000/313/set_metadata_zYlbEg.py" "/Users/marius/src/galaxy/database/tmp/tmpWpUktm" "/Users/marius/src/galaxy/database/jobs_directory/000/313/working/galaxy.json" "/Users/marius/src/galaxy/database/jobs_directory/000/313/metadata_in_HistoryDatasetAssociation_374_qpxnc0,/Users/marius/src/galaxy/database/jobs_directory/000/313/metadata_kwds_HistoryDatasetAssociation_374_9wJNFk,/Users/marius/src/galaxy/database/jobs_directory/000/313/metadata_out_HistoryDatasetAssociation_374_q43IxKSNIP...; sh -c "exit $return_code"
...

whereas the tool_script.sh looks like this:

#!/bin/bash
[ "$CONDA_DEFAULT_ENV" = "/Users/marius/src/galaxy/database/jobs_directory/000/313/conda-env" ] || . /Users/marius/conda/bin/activate '/Users/marius/src/galaxy/database/jobs_directory/000/313/conda-env' 2>&1 ; [ "$CONDA_DEFAULT_ENV" = "/Users/marius/src/galaxy/database/jobs_directory/000/313/conda-env" ] || . /Users/marius/conda/bin/activate '/Users/marius/src/galaxy/database/jobs_directory/000/313/conda-env' 2>&1 ; [ "$CONDA_DEFAULT_ENV" = "/Users/marius/src/galaxy/database/jobs_directory/000/313/conda-env" ] || . /Users/marius/conda/bin/activate '/Users/marius/src/galaxy/database/jobs_directory/000/313/conda-env' 2>&1 ; [ "$CONDA_DEFAULT_ENV" = "/Users/marius/src/galaxy/database/jobs_directory/000/313/conda-env" ] || . /Users/marius/conda/bin/activate '/Users/marius/src/galaxy/database/jobs_directory/000/313/conda-env' 2>&1 ; ln -f -s "/Users/marius/src/galaxy/database/files/_metadata_files/000/metadata_31.dat" alignment.sorted.bam.bai && ln -f -s "/Users/marius/src/galaxy/database/files/000/dataset_297.dat" alignment.sorted.bam SNIP...

@galaxybot galaxybot added this to the 16.07 milestone Apr 27, 2016

@jmchilton

This comment has been minimized.

Copy link
Member

commented Apr 27, 2016

Beautiful - I think this is a fantastic approach to fixing this problem. Sorry these things we intertwined from the beginning.

@jmchilton jmchilton merged commit a7bcd69 into galaxyproject:dev Apr 27, 2016

4 checks passed

api test Build finished.
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details
framework test Build finished.
Details
toolshed test Build finished.
Details

martenson added a commit that referenced this pull request Jul 26, 2016

Merge pull request #2662 from jmchilton/backport_2253
[16.04] Backport #2253 as requested by Eagle Eyes

@mvdbeek mvdbeek deleted the mvdbeek:conda-metadata-env branch Nov 23, 2016

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.