Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Hierarchical module naming scheme categorizing modulefiles by 'moduleclass' #1176

Merged
merged 12 commits into from
Feb 16, 2015
106 changes: 106 additions & 0 deletions easybuild/tools/module_naming_scheme/categorized_hmns.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,106 @@
##
# Copyright (c) 2015 Forschungszentrum Juelich GmbH, Germany
#
# All rights reserved.
#
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions are
# met:
#
# * Redistributions of source code must retain the above copyright
# notice, this list of conditions and the following disclaimer.
#
# * Redistributions in binary form must reproduce the above copyright
# notice, this list of conditions and the following disclaimer in the
# documentation and/or other materials provided with the distribution.
#
# * Neither the name of Forschungszentrum Juelich GmbH, nor the names of
# its contributors may be used to endorse or promote products derived
# from this software without specific prior written permission.
#
# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
##
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

as I see it, this is 3-clause BSD, no? http://en.wikipedia.org/wiki/BSD_licenses

It would be nice if it was stated as such somewhere (just me pipedreaming here that all devs would do the same)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@fgeorgatos: Yes, it is 3-clause BSD. I'm so used to it that I thought it was obvious ;-) Not sure where to best add such a statement, though.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

just add an extra comment line, mentioning something like:

# License: 3-clause BSD
# Author: Markus Geimer (JSC)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok, you already have the author bit below, so just an extra comment for the license part is sufficient

"""
Implementation of a hierarchical module naming scheme using module classes.

@author: Markus Geimer (Forschungszentrum Juelich GmbH)
"""

import os

from easybuild.tools.module_naming_scheme.hierarchical_mns import HierarchicalMNS
from easybuild.tools.config import build_option


class CategorizedHMNS(HierarchicalMNS):
"""
Class implementing an extended hierarchical module naming scheme using the
'moduleclass' easyconfig parameter to categorize modulefiles on each level
of the hierarchy.
"""

REQUIRED_KEYS = ['name', 'version', 'versionsuffix', 'toolchain', 'moduleclass']

def det_module_subdir(self, ec):
"""
Determine module subdirectory, relative to the top of the module path.
This determines the separation between module names exposed to users,
and what's part of the $MODULEPATH. This implementation appends the
'moduleclass' easyconfig parameter to the base path of the corresponding
hierarchy level.

Examples:
Core/compiler, Compiler/GCC/4.8.3/mpi, MPI/GCC/4.8.3/OpenMPI/1.6.5/bio
"""
moduleclass = ec['moduleclass']
basedir = super(CategorizedHMNS, self).det_module_subdir(ec)

return os.path.join(basedir, moduleclass)

def det_modpath_extensions(self, ec):
"""
Determine module path extensions, if any. Appends all known (valid)
module classes to the base path of the corresponding hierarchy level.

Examples:
Compiler/GCC/4.8.3/<moduleclasses> (for GCC/4.8.3 module),
MPI/GCC/4.8.3/OpenMPI/1.6.5/<moduleclasses> (for OpenMPI/1.6.5 module)
"""
known_module_classes = build_option('valid_module_classes')
basepaths = super(CategorizedHMNS, self).det_modpath_extensions(ec)

paths = []
for path in basepaths:
for moduleclass in known_module_classes:
paths.extend([os.path.join(path, moduleclass)])

return paths

def det_init_modulepaths(self, ec):
"""
Determine list of initial module paths (i.e., top of the hierarchy).
Appends all known (valid) module classes to the top-level base path.

Examples:
Core/<moduleclasses>
"""
known_module_classes = build_option('valid_module_classes')
basepaths = super(CategorizedHMNS, self).det_init_modulepaths(ec)

paths = []
for path in basepaths:
for moduleclass in known_module_classes:
paths.extend([os.path.join(path, moduleclass)])
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is pretty much copy-paste of det_modpath_extensions, so the body should be fleshed out in another method or function like categorise_paths(paths), which is simply called twice

we already have enough LoC to take care of ;)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

or, alternatively, collapse this into a single line using a Python list comprehension:

paths = [os.path.join(p, mc) for p in basepaths for mc in known_module_classes]

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

and then you don't even need paths or a separate categorize_paths() anymore, just return the list straight away :)

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lovely meeting of Python with the functional languages world ;-)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To be honest, your suggestion looks more like perl than Python... looks awful...

Since I'm in favor of readable code, I opt for the separate method (which in fact makes sense -- and I don't know why I didn't think of it myself).

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is a pretty clean list comprehension, we have many others... ;-)

but sure, go ahead and do it via a separate method then, I don't feel strongly about making it a list comprehension, since I won't have to maintain this bit ;)


return paths