New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Try next dep. resolver if building env failed #5036

Merged
merged 2 commits into from Jan 25, 2018

Conversation

Projects
None yet
4 participants
@mvdbeek
Member

mvdbeek commented Nov 18, 2017

Otherwise an exception ("Conda dependency seemingly installed but failed to
build job environment." comes to mind) would entirely stop the job submission.
Ping @bgruening.

@mvdbeek

This comment has been minimized.

Member

mvdbeek commented Nov 18, 2017

Two factors are potentially increasing the frequency with which we can see this error: Conda versions newer than 4.3.24 also have problems with --offline, and bcftools (in addition to samtools) being a dependency to "Set external metadata", which requires a mulled environment. The entirely correct thing on the deployer side would be to install these dependencies through the admin panel, but upgrading an instance to find that even simple uploads don't work is not great UX.

kwds['tool_instance'].dependencies = [dep.to_dict() for dep in ordered_dependencies]
except Exception as e:
log.exception(e)
pass

This comment has been minimized.

@bgruening

bgruening Jan 14, 2018

Member

the pass can be removed isn't it?

This comment has been minimized.

@mvdbeek

mvdbeek Jan 25, 2018

Member

right

@mvdbeek mvdbeek added status/review and removed status/WIP labels Jan 25, 2018

mvdbeek added some commits Nov 18, 2017

Try next dep. resolver if building env failed
Otherwise an exception ("Conda dependency seemingly installed but failed to
build job environment." comes to mind) would entirely stop the job submission.
Ping @bgruening.
Avoid duplicate dependency installation
The previous commit changed the resolver logic to try all resolvers, so that
when building an environment fails, the next resolver can be tried. Since
dependency resolution and installation are coupled it would be possible to
install dependencies multiple times for different resolvers (e.g for Conda and
Conda versionless).
To avoid this we stop installation once a set of dependencies has been
resolved.

@galaxybot galaxybot added this to the 18.01 milestone Jan 25, 2018

@jmchilton jmchilton merged commit b24217c into galaxyproject:dev Jan 25, 2018

6 checks passed

api test Build finished. 343 tests run, 4 skipped, 0 failed.
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details
framework test Build finished. 171 tests run, 0 skipped, 0 failed.
Details
integration test Build finished. 79 tests run, 4 skipped, 0 failed.
Details
selenium test Build finished. 118 tests run, 1 skipped, 0 failed.
Details
toolshed test Build finished. 577 tests run, 0 skipped, 0 failed.
Details
@jmchilton

This comment has been minimized.

Member

jmchilton commented Jan 25, 2018

Awesome - thanks @mvdbeek !

dannon added a commit that referenced this pull request Feb 7, 2018

@mvdbeek mvdbeek deleted the mvdbeek:resolver_improvements branch Jun 12, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment