-
Notifications
You must be signed in to change notification settings - Fork 128
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add sub-command category #600
Conversation
Does/could this support multiple categories, e.g., |
It currently just collects every category it comes across and displays them all. Additionally it is case-sensitive and splits on ",". Therefore "Develop" and "develop" would show as two categories. I assume it should be possible to only display those explicitly requested. Should this be case-insensitive and allow Unfortunately I will be unavailable until 2022-09-20. |
I'm just imagining how a user might use such a sub-command to navigate a particularly dense software tree;
I don't expect you to actually implement this, I'm just imagining how we could usefully leverage this in EasyBuild/EESSI. In our scenarios, there are very large numbers of software packages and only a small subset are actually of real interest to any particular user. EDIT: I always get confused by what the appropriate term is when talking about these things: nomenclature, ontology, taxonomy? After a bit of digging (which is not my first time), I think what I mean here is most accurately represented by a taxonomy |
I currently made the following changes:
For the first part: Would For the second part: Could you give me an example of what you mean with "describing their nomenclature"? Like a brief explanation of what the category contains? Could this be done within a category hook or msgHook? I describe this a bit more at the end.
I added a disclaimer mentioning to use
I imagine this may lead to confusion when there are categories matching that string e.g. Python category and module. Maybe this could be done with an option to But wouldn't that require the exact module name? In that case the user can get the categories with
Maybe a category hook would be helpful? One suggestion:
The table it returns will then be displayed one entry per line. This way you could provide descriptions. If the entries themselves are tables then we could also use ColumnTable.
and it should return the categories and modules that are supposed to be displayed via ColumnTable and Banner. Doesn't have to be any existing category:
If nothing is received then it will do its own matching based on the search strings. |
Yes, I think this is a good approach.
Exactly, a brief explanation of what the category means. I like the idea of being able to use a hook to inject this, that seems more sustainable.
Yes, perfect.
I can see the issues, best to just forget this in the first implementation.
Yes, I think a hook would be really useful, it keeps the main implementation simple but can still allow for additional complexity. You can then use the hook as you described for descriptions, and it could also be able to map subcategories. For example, if
That way you can build up any number of levels. |
With my suggestion for a hook and in case of search strings, the hook could return:
How could a description be added in here? Allow the return of a second optional table? And then add that text below the banner?
|
Sorry, my comment was confusing. I meant one part of the hook that acts on the table of categories Seriously though, you've done more than enough to satisfy me, perfection is the enemy of progress. |
I have added the hook now. Similar to the In the simple case it should return an array:
and in the complex case the structure is:
A mini example for a hook could be:
|
I have started to look at this. It would be very helpful if you could provide example modulefiles and whatever else is needed. If you could use the bugReport script that I could run as an example would greatly speed up this being integrated in the production Lmod. |
The following patch is a minimal example. If you prefer I could add a few more modules so that more rows are visible when using the search for a category. gcc should have a (1) and (2) to show that each modulefile needs the category mentioned in diff --git a/bugReport/bug_report_template.sh b/bugReport/bug_report_template.sh
index f9fd9c6e..6b2ad74e 100755
--- a/bugReport/bug_report_template.sh
+++ b/bugReport/bug_report_template.sh
@@ -8,8 +8,9 @@ export MODULEPATH=$PWD/my_modules/Core
# Put whatever module commands you need to show your issue
# Modify the modules in my_modules/Core if necessary if you are using the software hierarchy
-module load gcc mpich
-module av
+module category
+echo "######## Separation"
+module category dev IO
diff --git a/bugReport/my_modules/Core/Python/3.9.6.lua b/bugReport/my_modules/Core/Python/3.9.6.lua
new file mode 100644
index 00000000..f0e58bc9
--- /dev/null
+++ b/bugReport/my_modules/Core/Python/3.9.6.lua
@@ -0,0 +1 @@
+whatis("Category: Develop, Math")
diff --git a/bugReport/my_modules/Core/gcc/10.0.lua b/bugReport/my_modules/Core/gcc/10.0.lua
index 479ece7f..647d5f1d 100644
--- a/bugReport/my_modules/Core/gcc/10.0.lua
+++ b/bugReport/my_modules/Core/gcc/10.0.lua
@@ -3,3 +3,4 @@ prepend_path("MODULEPATH",pathJoin(MODULEPATH_ROOT,"Compiler/gcc/10"))
+whatis("Category: Develop,IO, App")
diff --git a/bugReport/my_modules/Core/gcc/9.0.lua b/bugReport/my_modules/Core/gcc/9.0.lua
new file mode 100644
index 00000000..e9cb00bc
--- /dev/null
+++ b/bugReport/my_modules/Core/gcc/9.0.lua
@@ -0,0 +1 @@
+whatis("Category: App,IO") |
Thank you @d-kirsten and @ocaisa, for creating this PR and providing feedback. This is will be a very useful feature. I have merged in this PR into a new branch called "600_category". @d-kirsten @ocaisa Please test out this new feature on this branch. Also if @boegel and @wpoely86 get the chance, please test this feature out and provide feedback. Thanks! |
a[#a+1] = [[ | ||
To learn more about a package and how to load it execute: | ||
$ module spider Bar | ||
]] |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
]] | |
where "Bar" is the name of a module. | |
]] |
This is more like how it is done for module keyword
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Maybe package
instead of module
here since that is the word you used previously.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I added the line with the word package as a new commit
@rtmclay I tested this on a flat and hierarchical naming scheme and it worked as expected. I also played around a bit with hooks to affect how the information is displayed and this also seems to work (though I will need to learn some more Lua!). @d-kirsten The one thing that was not clear to me was whether I can use a hook to automatically search subcategories. For example if I know
the table that gets created is actually
Or perhaps that the modules that fall under Bar are also included under Foo (if Bar is not listed on the command line)? I believe the complex table will allow me to do this, I should be able to just add the contents of one entry to the other, but my lua is not good enough to figure out how to do this (and I also wonder if there will be double-counting if a module has both the main and the sub category). |
AFAIK, Lmod has no notion of a sub category. So what do you mean as that something is a sub-category? |
I thought I might be able to define subcategories myself via a hook. As far as Lmod is concerned everything is a category, but when I present things I can organise things according to my scheme. Lmod would see categories Foo and Bar, but in my scheme Bar is a subcategory of Foo. Then I write a hook to get output like
and then if
I actually get
which is (somewhat) similar to the output of |
Ah, I got it to work with
which gives
(as the hook is currently case sensitive). |
Co-authored-by: ocaisa <alan.ocais@cecam.org>
It works for me. @ocaisa does the change for the simple case still allow you to use descriptions the way you wanted to? With the recent change it formats it into columns:
Instead of for key, value in pairs(t["Tools"]) do
a[v][key] = value
end you could also do a.Tools = t.Tools which instead of adding modules it would display the whole Tools category separately. Though your example would require the search term to be an exact category. Maybe something like this (hopefully it works) is what you might want: if (kind == "complex") then
local pargs = masterTbl().pargs
local a = {}
local extra = {
Develop = { "CFD", "IO" },
CFD = { "Material", },
}
for _, arg in ipairs(pargs) do
local term = arg:caseIndependent()
-- Add every matching category
for cat, v in pairs(t) do
if (cat:find(term)) then
a[cat] = v
end
end
-- Add additional categories based on search.
for match, add in pairs(extra) do
if (match:find(term)) then
for _, new in pairs(add) do
a[new] = t[new]
end
end
end
end
return a
end |
@d-kirsten Yes, descriptions was ok. I quickly had a naive implementation:
which looks like
Lmod seems to be pretty smart in how it presents that, with a bit more experience I think I can display it as I like. I tweaked the above hook for the simple case.
which displays ordered subcategories
The hook is pretty verbose, but with a little thought I think you could process a json file with the taxonomy and simplify things. @rtmclay For me, I think everything I would like to see is there and working. |
Note that I have modified "module cat" to use "ColumnTable" instead of a straight list. Please test the latest version of the 600_category branch to see if it works for you. |
@rtmclay Everything seems ok to me, my hooks worked unchanged:
|
Great! I'm merge in this branch onto the main branch and release a new version next week. |
@rtmclay I was just wondering when that release will land? I'd like to hold back a release of https://github.com/EESSI/gentoo-overlay until we can include it |
Soon. I am trying to release this fix and Issue #604 and I have run into problems. I'll have it fixed by Monday 11/7 or sooner. |
This branch has been merged into Lmod 8.7.14. In a week this will become 8.8. Please test Lmod 8.7.14 if you get the chance. |
The commit adds the sub-command
category | cat
which works similar tooverview
but instead lists modules under the categories they are part of and looks at every module reported by Cache.As Robert mentioned on the mailing list I used the Category entry that is being provided by
whatis("Category: Tools, Develop")
.Missing are translations for the help page that should be used in src/lmod.in.lua line 168.
Please let me know if I should make changes to the code.