Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

New Dependency resolver for the LMOD environment modules system #4489

Merged
merged 10 commits into from Sep 5, 2017

Conversation

Projects
None yet
4 participants
@arbernard
Copy link
Contributor

commented Aug 24, 2017

Hi everyone,

This Pull Request concern the integration of a new Dependency resolver (with mapping file support) for the LMOD environment modules system.

This new resolver is inspired by the "modules" resolver but takes into account the differences between LMOD and the legacy module system.

I tried to make the commit messages as detailed as possible.

With this PR, 2 new files are included and 2 existing files are modified:

  • (NEW) lib/galaxy/tools/deps/resolvers/lmod.py
  • (NEW) config/lmod_modules_mapping.yml.sample
  • (MOD) config/dependency_resolvers_conf.xml.sample (with explanations on the resolver attributes)
  • (MOD) test/unit/tools/test_tool_deps.py

I've setup the resolver so that it automatically use the lmod_modules_mapping.yml file if it exists in the config folder. Is that the proper way to manage default mapping files in resolvers ?

Since there was no adapted tests for this resolver in test_tool_deps.py script I tried to write some.
However, since I'm defintily not an expert in that matter, feel free to comment them :)

Travis is activated for my fork of the galaxyproject repository and the last build passed without errors (except the .11 build that is in the Allowed Failures section).

arbernard added some commits Aug 23, 2017

LMOD Dependency resolver - Initial commit (lib lmod.py)
Initial commit of a prototype dependency resolver based on the LMOD environment modules system.
Update of the dependency_resolvers_conf.xml.sample file
Addition of the LMOD Dependency resolver to the appropriate sample configuration file (dependency_resolvers_conf.xml.sample). Comment lines have also been added to describe the possible attributes of the resolver as well as their default values.
LMOD Dependency resolver - Default mapping file
If a lmod_modules_mapping.yml file exists in the config folder then it will be taken into consideration automatically to map Galaxy requirements to LMOD modules.
Fix errors reported by the initial Travis tests
Fix incorrect numbers of blank lines.
Fix deprecated syntax.
Syntax fix in the shell_commands method
The built command now reflect more directly the command that is really executed by the the LMOD "module" bash function when a user execute "module load package_name/package_version".
Dependency_resolvers_conf.xml - More explanations
Some more explanations about the versionless mode have been added.
Clearer comment lines in the has_module method
A comment of the has_module method of the AvailModuleChecker class has been modified to be clearer and more explicit.
LMOD Dependency resolver - Unit tests added
The unit/tools/tests_tool_deps.py script has been updated to includ unit tests for the LMOD dependency resolver.
Fix the import order error raised by Travis
The order of the import statements in the test_tool_deps script was incorrect (as stated by travis). That should no longer be the case.
Fix attempt for the modulepath issue in unit tests
On a computer where LMOD is not installed, the MODULEPATH variable is not defined and the modulepath attributes of the LMOD dependency resolver must be used.

@galaxybot galaxybot added the triage label Aug 24, 2017

@galaxybot galaxybot added this to the 17.09 milestone Aug 24, 2017

@jmchilton

This comment has been minimized.

Copy link
Member

commented Aug 28, 2017

@galaxybot test this

@jmchilton

This comment has been minimized.

Copy link
Member

commented Aug 29, 2017

@galaxybot test this

@arbernard

This comment has been minimized.

Copy link
Contributor Author

commented Sep 4, 2017

I'm not familiar with the integration test that is failing.

I checked the log of the failing test but didn't saw anything directly connected to the new resolver.

Is there something I can do on my side to help ?

@mvdbeek

This comment has been minimized.

Copy link
Member

commented Sep 4, 2017

Don't worry about this failing test, I need to refine that test.

@jmchilton

This comment has been minimized.

Copy link
Member

commented Sep 5, 2017

@arbernard this is really awesome - thanks!

@jmchilton jmchilton merged commit 1e3bd2d into galaxyproject:dev Sep 5, 2017

6 checks passed

api test Build finished. 284 tests run, 0 skipped, 0 failed.
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details
framework test Build finished. 161 tests run, 0 skipped, 0 failed.
Details
integration test Build finished. 44 tests run, 0 skipped, 0 failed.
Details
lgtm analysis: JavaScript No alert changes
Details
toolshed test Build finished. 579 tests run, 0 skipped, 0 failed.
Details
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.