Module hiden with SitePackage.lua is unfindable

[wpoely86]

With the Lmod version in EPEL (Lmod-8.7.53-1.el8.x86_64) we are having issues with modules hidden in the SitePackage.lua. The logic can be found here: https://github.com/vub-hpc/Lmod-config/blob/main/SitePackage.lua#L239-L246

This works well until we updated to 8.7.53. Now the module is not visible at all. Neither ml --show_hidden spider or ml --show_hidden av show it. Module that load it as a dependency fail to load. Removing the lines in our SitePackage.lua fixes the issue (but the module is no longer hidden).

In attachment I've added two files:

• output-broken.txt.gz: This is the output of ml -D --ignore-cache --show_hidden spider JupyterHub.
• output-unhidden.txt.gz. This is the same output but with the line of JupyterHub removed in the SitePackage.lua.

[rtmclay]

I do not have a fix yet but I can reproduce this issue. I'll keep you updated on progress on this issue.

[rtmclay]

I am unclear on what you mean by:

Module that load it as a dependency fail to load.

How are you loading this dependency? Is with the version or not?

[wpoely86]

How are you loading this dependency? Is with the version or not?

As a version locked dependency (it are all EasyBuild generate module files):

depends_on("JupyterHub/4.1.5-GCCcore-12.3.0")

[rtmclay]

I was able to reproduce both issues (ml -A av and the depends_on()). This has been fixed for me on the IS690-hide branch. Please test this branch to see if it works for you.

[wpoely86]

It gives me:

/usr/bin/lua: /usr/share/lmod/lmod/libexec/MRC.lua:535: attempt to index a boolean value (local 'resultT')
stack traceback:
	/usr/share/lmod/lmod/libexec/MRC.lua:535: in upvalue 'l_findHiddenState'
	/usr/share/lmod/lmod/libexec/MRC.lua:716: in function 'MRC.isVisible'
	/usr/share/lmod/lmod/libexec/Spider.lua:635: in local 'l_buildDbT_helper'
	/usr/share/lmod/lmod/libexec/Spider.lua:669: in function 'Spider.buildDbT'
	/usr/share/lmod/lmod/libexec/Cache.lua:631: in function 'Cache.build'
	/usr/share/lmod/lmod/libexec/cmdfuncs.lua:1082: in function 'SpiderCmd'
	/usr/share/lmod/lmod/libexec/lmod:523: in function 'main'
	/usr/share/lmod/lmod/libexec/lmod:594: in main chunk
	[C]: in ?

[rtmclay]

Thanks for running this. Can you give me an example module tree that reproduces this?

[wpoely86]

An empty one works:

$ mkdir /tmp/a
$ export MODULEPATH=/tmp/a
$ ml av
/usr/bin/lua: /usr/share/lmod/lmod/libexec/MRC.lua:535: attempt to index a boolean value (local 'resultT')
stack traceback:
	/usr/share/lmod/lmod/libexec/MRC.lua:535: in upvalue 'l_findHiddenState'
	/usr/share/lmod/lmod/libexec/MRC.lua:716: in function 'MRC.isVisible'
	/usr/share/lmod/lmod/libexec/Spider.lua:635: in local 'l_buildDbT_helper'
	/usr/share/lmod/lmod/libexec/Spider.lua:669: in function 'Spider.buildDbT'
	/usr/share/lmod/lmod/libexec/Cache.lua:631: in function 'Cache.build'
	/usr/share/lmod/lmod/libexec/ModuleA.lua:677: in function 'ModuleA.singleton'
	/usr/share/lmod/lmod/libexec/Hub.lua:1253: in function 'Hub.avail'
	/usr/share/lmod/lmod/libexec/cmdfuncs.lua:145: in function 'Avail'
	/usr/share/lmod/lmod/libexec/lmod:523: in function 'main'
	/usr/share/lmod/lmod/libexec/lmod:594: in main chunk
	[C]: in ?

All of our config stuff can be found in https://github.com/vub-hpc/Lmod-config

[rtmclay]

This is what I get:

% mkdir /tmp/a
% clearMT
% export MODULEPATH=/tmp/a
% ml av       
No module(s) or extension(s) found!
...

I didn't see anything in your Lmod-config variable setting that would make a difference. What happens if you do?

% ml -I av

Also please include ml --config as an attached file.

[wpoely86]

Hmm, using ml --ignore-cache av indeed works. It's something in the cache that breaks it.

[wpoely86]

It's a cache thing. If I add --ignore-cache, the error is gone.

[rtmclay]

Please attach your cache file from your earlier version of Lmod. My attempts to reproduce your issue have not worked. Once I know what is happening, I can create my own cache file to add to the test suite.

[wpoely86]

The issue is a cache build by Lmod 8.7.53 as it's now in EPEL. If I rebuild the cache with the Lmod from the IS690-hide branch, it's fine again and all works as intended. The original issue is gone.

I'm attaching the two cache files:

• this ones gives the error:
  spiderT-broken.lua.gz

• this one does not:
  spiderT-works.lua.gz

[rtmclay]

I would like to figure out why you got the original error. However using your spiderT-broken.lua file as my cache, I am unable to reproduce the error. It is still out there but someone else will have to report it. Thanks for your help.

[wpoely86]

Does it help if I give you a -D -D output of the error? Or a full copy of the module tree?