Add module load time env var to user module path #2395

akesandgren · 2018-02-07T12:08:47Z

On systems with multiple architectures sharing a single home directory
for the users there need to be a way to install architecture specific
builds when using the subdir-user-modules extension.

This is a remake of PR #2330 which I messed up by doing an bad pull.

On systems with multiple architectures sharing a single home directory for the users there need to be a way to install architecture specific builds when using the subdir-user-modules extension.

Change from using a specific subdir_arch_env to expanding "{RUNTIME_ENV::SOME_ENV_VAR}" into module runtime equivalences of os.getenv("SOME_ENV_VAR")

…dpath.

…le-path

houndci-bot · 2018-07-03T19:09:10Z

easybuild/tools/module_generator.py

+                        expanded_user_modpath.extend([path])
+			sub_paths = []
+		    expanded_user_modpath.extend(['os.getenv(%s)' % quote_str(matched_re.group(1))])
+                else:


indentation contains mixed spaces and tabs

houndci-bot · 2018-07-03T19:09:10Z

easybuild/tools/module_generator.py

+			path = quote_str(os.path.join(*sub_paths))
+                        expanded_user_modpath.extend([path])
+			sub_paths = []
+		    expanded_user_modpath.extend(['os.getenv(%s)' % quote_str(matched_re.group(1))])


indentation contains tabs
indentation contains mixed spaces and tabs

houndci-bot · 2018-07-03T19:09:10Z

easybuild/tools/module_generator.py

+		    if sub_paths:
+			path = quote_str(os.path.join(*sub_paths))
+                        expanded_user_modpath.extend([path])
+			sub_paths = []


indentation contains tabs
indentation contains mixed spaces and tabs

houndci-bot · 2018-07-03T19:09:10Z

easybuild/tools/module_generator.py

+                if matched_re:
+		    if sub_paths:
+			path = quote_str(os.path.join(*sub_paths))
+                        expanded_user_modpath.extend([path])


indentation contains mixed spaces and tabs

houndci-bot · 2018-07-03T19:09:10Z

easybuild/tools/module_generator.py

+                matched_re = runtime_env_re.match(sub_path)
+                if matched_re:
+		    if sub_paths:
+			path = quote_str(os.path.join(*sub_paths))


indentation contains tabs

houndci-bot · 2018-07-03T19:09:11Z

easybuild/tools/module_generator.py

+            for sub_path in re.split(os.path.sep, user_modpath):
+                matched_re = runtime_env_re.match(sub_path)
+                if matched_re:
+		    if sub_paths:


indentation contains tabs
indentation contains mixed spaces and tabs

houndci-bot · 2018-07-03T19:09:11Z

easybuild/framework/easyblock.py

                self.log.debug("Including user module path extensions returned by naming scheme: %s", user_modpath_exts)
                txt += self.module_generator.use(user_modpath_exts, prefix=self.module_generator.getenv_cmd('HOME'),
-                                                 guarded=True)
+                     guarded=True, user_modpath=user_modpath, mod_path_suffix=mod_path_suffix)


continuation line under-indented for visual indent

houndci-bot · 2018-07-03T19:09:11Z

easybuild/tools/module_generator.py

+                        expanded_user_modpath.extend([path])
+			sub_paths = []
+		    expanded_user_modpath.extend(['os.getenv(%s)' % quote_str(matched_re.group(1))])
+                else:


indentation contains mixed spaces and tabs

houndci-bot · 2018-07-03T19:09:11Z

easybuild/tools/module_generator.py

+			path = quote_str(os.path.join(*sub_paths))
+                        expanded_user_modpath.extend([path])
+			sub_paths = []
+		    expanded_user_modpath.extend(['os.getenv(%s)' % quote_str(matched_re.group(1))])


indentation contains tabs
indentation contains mixed spaces and tabs

houndci-bot · 2018-07-03T19:09:11Z

easybuild/tools/module_generator.py

+		    if sub_paths:
+			path = quote_str(os.path.join(*sub_paths))
+                        expanded_user_modpath.extend([path])
+			sub_paths = []


indentation contains tabs
indentation contains mixed spaces and tabs

houndci-bot · 2018-07-03T19:09:12Z

easybuild/tools/module_generator.py

+                if matched_re:
+		    if sub_paths:
+			path = quote_str(os.path.join(*sub_paths))
+                        expanded_user_modpath.extend([path])


indentation contains mixed spaces and tabs

houndci-bot · 2018-07-03T19:09:12Z

easybuild/tools/module_generator.py

+                matched_re = runtime_env_re.match(sub_path)
+                if matched_re:
+		    if sub_paths:
+			path = quote_str(os.path.join(*sub_paths))


indentation contains tabs

houndci-bot · 2018-07-03T19:09:12Z

easybuild/tools/module_generator.py

+            for sub_path in re.split(os.path.sep, user_modpath):
+                matched_re = runtime_env_re.match(sub_path)
+                if matched_re:
+		    if sub_paths:


indentation contains tabs
indentation contains mixed spaces and tabs

houndci-bot · 2018-07-03T19:09:12Z

easybuild/framework/easyblock.py

                self.log.debug("Including user module path extensions returned by naming scheme: %s", user_modpath_exts)
                txt += self.module_generator.use(user_modpath_exts, prefix=self.module_generator.getenv_cmd('HOME'),
-                                                 guarded=True)
+                     guarded=True, user_modpath=user_modpath, mod_path_suffix=mod_path_suffix)


continuation line under-indented for visual indent

…erator.use

houndci-bot · 2018-07-03T21:01:30Z

test/framework/easyblock.py

                # no per-moduleclass extension for user modules
-                r'\s+prepend_path\("MODULEPATH", pathJoin\(%s, "%s/funky/Compiler/pi/3.14"\)\)' % (home, usermodsdir),
+                r'\s+prepend_path\("MODULEPATH", pathJoin\(%s, "%s", "funky", "Compiler/pi/3.14"\)\)' % (home, usermodsdir),


line too long (124 > 120 characters)

houndci-bot · 2018-07-03T21:01:31Z

test/framework/easyblock.py

                # no per-moduleclass extension for user modules
-                r'\s+prepend_path\("MODULEPATH", pathJoin\(%s, "%s/funky/Compiler/pi/3.14"\)\)' % (home, usermodsdir),
+                r'\s+prepend_path\("MODULEPATH", pathJoin\(%s, "%s", "funky", "Compiler/pi/3.14"\)\)' % (home, usermodsdir),


line too long (124 > 120 characters)

houndci-bot · 2018-07-03T21:04:31Z

test/framework/easyblock.py

                # no per-moduleclass extension for user modules
-                r'\s+prepend_path\("MODULEPATH", pathJoin\(%s, "%s/funky/Compiler/pi/3.14"\)\)' % (home, usermodsdir),
+                r'\s+prepend_path\("MODULEPATH", pathJoin\(%s, "%s", "funky", "Compiler/pi/3.14"\)\)' %
+                    (home, usermodsdir),


continuation line unaligned for hanging indent

houndci-bot · 2018-07-03T21:04:31Z

test/framework/easyblock.py

                # no per-moduleclass extension for user modules
-                r'\s+prepend_path\("MODULEPATH", pathJoin\(%s, "%s/funky/Compiler/pi/3.14"\)\)' % (home, usermodsdir),
+                r'\s+prepend_path\("MODULEPATH", pathJoin\(%s, "%s", "funky", "Compiler/pi/3.14"\)\)' %
+                    (home, usermodsdir),


continuation line unaligned for hanging indent

houndci-bot · 2018-07-03T21:08:52Z

test/framework/easyblock.py

                # no per-moduleclass extension for user modules
-                r'\s+prepend_path\("MODULEPATH", pathJoin\(%s, "%s/funky/Compiler/pi/3.14"\)\)' % (home, usermodsdir),
+                r'\s+prepend_path\("MODULEPATH", pathJoin\(%s, "%s", "funky", "Compiler/pi/3.14"\)\)' %
+			(home, usermodsdir),


indentation contains tabs
indentation contains mixed spaces and tabs
continuation line unaligned for hanging indent

houndci-bot · 2018-07-03T21:08:52Z

test/framework/easyblock.py

                # no per-moduleclass extension for user modules
-                r'\s+prepend_path\("MODULEPATH", pathJoin\(%s, "%s/funky/Compiler/pi/3.14"\)\)' % (home, usermodsdir),
+                r'\s+prepend_path\("MODULEPATH", pathJoin\(%s, "%s", "funky", "Compiler/pi/3.14"\)\)' %
+			(home, usermodsdir),


indentation contains tabs
indentation contains mixed spaces and tabs
continuation line unaligned for hanging indent

…or adds when using subdir-user-modules.

houndci-bot · 2018-07-04T06:29:30Z

easybuild/tools/module_generator.py

        for path in paths:
            quoted_path = quote_str(path)
+            if user_modpath:
+                quoted_path = '[ file join %s %s' ] % (user_modpath, quoted_path)


whitespace before ']'

…ir-user-modules" changes.

houndci-bot · 2018-07-04T07:04:41Z

test/framework/easyblock.py

                # no per-moduleclass extension for user modules
-                r'\s+prepend_path\("MODULEPATH", pathJoin\(%s, "%s/funky/Compiler/pi/3.14"\)\)' % (home, usermodsdir),
+                r'\s+prepend_path\("MODULEPATH", pathJoin\(%s, pathJoin\("%s", "funky", "Compiler/pi/3.14"\)\)\)' %
+                        (home, usermodsdir),


continuation line unaligned for hanging indent

houndci-bot · 2018-07-04T07:04:41Z

test/framework/easyblock.py

@@ -274,18 +274,19 @@ def test_make_module_extend_modpath(self):
            home = r'\$env\(HOME\)'
            regexs.extend([
                # extension for user modules is guarded
-                r'if { \[ file isdirectory \[ file join %s "%s/funky/Compiler/pi/3.14" \] \] } {$' % (home, usermodsdir),
+                r'if { \[ file isdirectory \[ file join %s \[ file join "%s" "funky" "Compiler/pi/3.14" \] \] \] } {$' % (home, usermodsdir),


line too long (141 > 120 characters)

…ning.

houndci-bot · 2018-07-04T07:07:19Z

test/framework/easyblock.py

                # no per-moduleclass extension for user modules
-                r'\s+prepend_path\("MODULEPATH", pathJoin\(%s, "%s/funky/Compiler/pi/3.14"\)\)' % (home, usermodsdir),
+                r'\s+prepend_path\("MODULEPATH", pathJoin\(%s, pathJoin\("%s", "funky", "Compiler/pi/3.14"\)\)\)' %
+                            (home, usermodsdir),


continuation line unaligned for hanging indent

houndci-bot · 2018-07-04T07:07:19Z

test/framework/easyblock.py

                # no per-moduleclass extension for user modules
-                r'^\s+module use \[ file join %s "%s/funky/Compiler/pi/3.14"\ ]$' % (home, usermodsdir),
+                r'^\s+module use \[ file join %s \[ file join "%s" "funky" "Compiler/pi/3.14" \] \]$' % (home, usermodsdir),


line too long (124 > 120 characters)

houndci-bot · 2018-07-04T07:07:19Z

test/framework/easyblock.py

@@ -274,18 +274,20 @@ def test_make_module_extend_modpath(self):
            home = r'\$env\(HOME\)'
            regexs.extend([
                # extension for user modules is guarded
-                r'if { \[ file isdirectory \[ file join %s "%s/funky/Compiler/pi/3.14" \] \] } {$' % (home, usermodsdir),
+                r'if { \[ file isdirectory \[ file join %s \[ file join "%s" "funky" "Compiler/pi/3.14" \] \] \] } {$' %
+                            (home, usermodsdir),


continuation line unaligned for hanging indent

houndci-bot · 2018-07-04T07:07:20Z

test/framework/easyblock.py

                # no per-moduleclass extension for user modules
-                r'\s+prepend_path\("MODULEPATH", pathJoin\(%s, "%s/funky/Compiler/pi/3.14"\)\)' % (home, usermodsdir),
+                r'\s+prepend_path\("MODULEPATH", pathJoin\(%s, pathJoin\("%s", "funky", "Compiler/pi/3.14"\)\)\)' %
+                            (home, usermodsdir),


continuation line unaligned for hanging indent

houndci-bot · 2018-07-04T07:07:20Z

test/framework/easyblock.py

                # no per-moduleclass extension for user modules
-                r'^\s+module use \[ file join %s "%s/funky/Compiler/pi/3.14"\ ]$' % (home, usermodsdir),
+                r'^\s+module use \[ file join %s \[ file join "%s" "funky" "Compiler/pi/3.14" \] \]$' % (home, usermodsdir),


line too long (124 > 120 characters)

houndci-bot · 2018-07-04T07:07:20Z

test/framework/easyblock.py

@@ -274,18 +274,20 @@ def test_make_module_extend_modpath(self):
            home = r'\$env\(HOME\)'
            regexs.extend([
                # extension for user modules is guarded
-                r'if { \[ file isdirectory \[ file join %s "%s/funky/Compiler/pi/3.14" \] \] } {$' % (home, usermodsdir),
+                r'if { \[ file isdirectory \[ file join %s \[ file join "%s" "funky" "Compiler/pi/3.14" \] \] \] } {$' %
+                            (home, usermodsdir),


continuation line unaligned for hanging indent

…enerator.py (& leverage existing getenv_cmd method)

…r-module-path clean up duplicate code

file join/pathjoin, so add a join_str to det_user_modpath.

akesandgren · 2018-07-05T20:20:40Z

As i suspecterd, the Tcl code gets wrong. I'll fix that.

…odpath. Removes the need for an extra argument to det_user_modpath.

…on method

…r-module-path flesh out common part of det_user_modpath into _det_user_modpath_common method

…e $::env(HOME)

…er-module-path fix check for $env(HOME) in test_make_module_extend_modpath, should be $::env(HOME)

akesandgren added 3 commits February 7, 2018 13:00

Add a architecture env variable to the user subdir module path.

2c65cee

On systems with multiple architectures sharing a single home directory for the users there need to be a way to install architecture specific builds when using the subdir-user-modules extension.

Use {RUNTIME_ENV::SOME_ENV_VAR} instead of subdir_arch_env.

0a6e4fe

Change from using a specific subdir_arch_env to expanding "{RUNTIME_ENV::SOME_ENV_VAR}" into module runtime equivalences of os.getenv("SOME_ENV_VAR")

Simplify the code for handling {RUNTIME_ENV::SOME_ENV_VAR} in user_mo…

07e6b3f

…dpath.

akesandgren mentioned this pull request Feb 7, 2018

Add a architecture env variable to the user subdir module path. #2330

Closed

Merge branch 'develop' into add-module-load-time-env-var-to-user-modu…

18b4de3

…le-path

houndci-bot reviewed Jul 3, 2018

View reviewed changes

akesandgren added 3 commits July 3, 2018 21:11

Change tabs to spaces.

7b92aaf

Increase indentation for visual indent as per houndci-bot.

4d2369b

Adjust test for Lua modpath extension to match new code in module_gen…

e95c965

…erator.use

houndci-bot reviewed Jul 3, 2018

View reviewed changes

Adjust line length as per houndci-bot.

fb88ba5

houndci-bot reviewed Jul 3, 2018

View reviewed changes

Increase indentation more since houndci is still complaining.

a17c9e0

houndci-bot reviewed Jul 3, 2018

View reviewed changes

akesandgren added 4 commits July 3, 2018 23:09

Clumsily got tabs into the code, changed to spaces.

04a1620

Forgot the extra pathJoin (in the test regex) that the module generat…

6f6789c

…or adds when using subdir-user-modules.

Missing \ protection for ( causing misbalanced ()

b6e491b

Add Tcl support for {RUNTIME_ENV::SOME_ENV_VAR} in subdir-user-modules.

bece493

houndci-bot reviewed Jul 4, 2018

View reviewed changes

akesandgren added 2 commits July 4, 2018 08:30

Fix "'" position typo.

f38846a

Update test for Tcl style of the "{RUNTIME_ENV::SOME_ENV_VAR} in subd…

9b8fd37

…ir-user-modules" changes.

houndci-bot reviewed Jul 4, 2018

View reviewed changes

Fix long line and increase indentation again since houndci is complai…

3fdb798

…ning.

houndci-bot reviewed Jul 4, 2018

View reviewed changes

One more too long line, and increase indentation alot this time.

4c5579f

easybuilders deleted a comment from boegelbot Jul 5, 2018

boegel added 4 commits July 5, 2018 20:49

minor cleanup in test_make_module_extend_modpath

a7202dc

flesh out duplicate code in implementation of use methods in module_g…

018508b

…enerator.py (& leverage existing getenv_cmd method)

add test for test_det_user_modpath method

98f4498

use $::env rather than to access environment variables in Tcl syntax

73c5180

boegel mentioned this pull request Jul 5, 2018

clean up duplicate code akesandgren/easybuild-framework#8

Merged

akesandgren added 2 commits July 5, 2018 20:55

Merge pull request #8 from boegel/add-module-load-time-env-var-to-use…

71d0100

…r-module-path clean up duplicate code

Tcl and Lua differs in argument separator for their respective

81a59a5

file join/pathjoin, so add a join_str to det_user_modpath.

akesandgren added 3 commits July 6, 2018 07:42

Use self.SYNTAX to determine string to use in join of expanded_user_m…

ece102f

…odpath. Removes the need for an extra argument to det_user_modpath.

Fix incorrect Tcl syntax in expected strings from det_user_modpath.

674b52f

Fix tabs to spaces

1d205c5

easybuilders deleted a comment from houndci-bot Jul 6, 2018

easybuilders deleted a comment from boegelbot Jul 6, 2018

flesh out common part of det_user_modpath into _det_user_modpath_comm…

7c147c5

…on method

boegel mentioned this pull request Jul 6, 2018

flesh out common part of det_user_modpath into _det_user_modpath_common method akesandgren/easybuild-framework#9

Merged

akesandgren and others added 3 commits July 6, 2018 07:37

Merge pull request #9 from boegel/add-module-load-time-env-var-to-use…

3a7c641

…r-module-path flesh out common part of det_user_modpath into _det_user_modpath_common method

fix check for $env(HOME) in test_make_module_extend_modpath, should b…

4dc5f56

…e $::env(HOME)

Merge pull request #10 from boegel/add-module-load-time-env-var-to-us…

af293a8

…er-module-path fix check for $env(HOME) in test_make_module_extend_modpath, should be $::env(HOME)

easybuilders deleted a comment from boegelbot Jul 6, 2018

boegel approved these changes Jul 6, 2018

View reviewed changes

boegel merged commit caf30a4 into easybuilders:develop Jul 6, 2018

akesandgren deleted the add-module-load-time-env-var-to-user-module-path branch July 6, 2018 09:29

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add module load time env var to user module path #2395

Add module load time env var to user module path #2395

akesandgren commented Feb 7, 2018

houndci-bot Jul 3, 2018

houndci-bot Jul 3, 2018

houndci-bot Jul 3, 2018

houndci-bot Jul 3, 2018

houndci-bot Jul 3, 2018

houndci-bot Jul 3, 2018

houndci-bot Jul 3, 2018

houndci-bot Jul 3, 2018

houndci-bot Jul 3, 2018

houndci-bot Jul 3, 2018

houndci-bot Jul 3, 2018

houndci-bot Jul 3, 2018

houndci-bot Jul 3, 2018

houndci-bot Jul 3, 2018

houndci-bot Jul 3, 2018

houndci-bot Jul 3, 2018

houndci-bot Jul 3, 2018

houndci-bot Jul 3, 2018

houndci-bot Jul 3, 2018

houndci-bot Jul 3, 2018

houndci-bot Jul 4, 2018

houndci-bot Jul 4, 2018

houndci-bot Jul 4, 2018

houndci-bot Jul 4, 2018

houndci-bot Jul 4, 2018

houndci-bot Jul 4, 2018

houndci-bot Jul 4, 2018

houndci-bot Jul 4, 2018

houndci-bot Jul 4, 2018

akesandgren commented Jul 5, 2018

Add module load time env var to user module path #2395

Add module load time env var to user module path #2395

Conversation

akesandgren commented Feb 7, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

akesandgren commented Jul 5, 2018