createmodule.sh / createmodule.py don't work at all.

[hongyi-zhao]

Hi,

The following scripts shipped with the repo:

createmodule.sh / createmodule.py

They were written many years ago, it seems there are some bugs in them.

See the following tests:

modules.git/script$ ./createmodule.py /opt/intel/vtune_profiler/amplxe-vars.sh intel64
WARNING: initscript sent the following to stderr:
/bin/sh: 3: /opt/intel/vtune_profiler/amplxe-vars.sh: Bad substitution

ERROR: Could not parse output line:
Copyright (C) 2009-2019 Intel Corporation. All rights reserved.

[nanobowers]

@hongyi-zhao AFAICT this script works by evaluating your shell script in a new shell and evaluating differences. I found that 'echo' statements (or other print statements) in the shell script can cause the ERROR: Could not parse output line: output. You might wish to comment them out as a workaround.

[hongyi-zhao]

I've solved this issue and do some enhancing/fixes on this script now.

There are some others similar scripts for the same purpose, say, the following ones:

https://github.com/dcblack/env2-code
https://lmod.readthedocs.io/en/latest/260_sh_to_modulefile.html

If I've completed the work on these scripts, I would like to put it on my git repo.
BTW, the the shell version for this type of job is inefficient, all of the others similar script are written by some other script languages than shell/bash, say, tcl/lua/python.

It seems the latter ones are more appropriate for this type of job.

Regards

[xdelaruelle]

Job could be achieved also by shell version (createmodule.sh). Here the script suffers of an issue if there is a shell function defined in the environment. With this issue fixed, it could operate as well as implementation in more elaborated languages.

This put aside, the main issue described here is the inability of both createmodule.sh and createmodule.py to handle script outputting text on stdout. Inhibiting stdout for the evaluation of the script passed as argument will solve the issue.

[hongyi-zhao]

Job could be achieved also by shell version (createmodule.sh). Here the script suffers of an issue if there is a shell function defined in the environment. With this issue fixed, it could operate as well as implementation in more elaborated languages.

This put aside, the main issue described here is the inability of both createmodule.sh and createmodule.py to handle script outputting text on stdout. Inhibiting stdout for the evaluation of the script passed as argument will solve the issue.

Then what's method should be used for finding the diff before and after of sourcing the script?

Regards

[xdelaruelle]

Then what's method should be used for finding the diff before and after of sourcing the script?

Checking env before script source, putting that result in an array, then source script and after that check env again and put that in another array. Diff the two arrays to get the environment diff. This is what createmodule.sh and createmodule.py do.

Regards

[hongyi-zhao]

Yes, this is just the things already done the scripts, but this don't deal with the shell function issue discussed here.

I use the following revised function to filt out the shell functions:

#Will print out array assignment list
printenvarray () {

  env | while read x
  do
    key=${x%%=*}

    # Filter out functions, they cannot be manipulated at present:
    # https://www.networkworld.com/article/2693361/unix-tip-using-bash-s-regular-expressions.html
    if [[ "$key" =~ ^BASH_FUNC_ ]]; then continue; fi
    
    value=`printenv "$key"`
    if [ $? -eq 0 ]
    then
      echo [$key]="'$value'"
    fi
  done
}

Regards

[xdelaruelle]

In 1ef61d6, I have applied the methodology used in createmodule.py to createmodule.sh.

[hongyi-zhao]

This is similar the trick I used, but yours seems more good to do that, but it can be simplified into the following just as I've posted:

   if [[ "$x" =~ '() {' ]]; then
      continue
    fi

Another notes, it seems this don't mergered into the master?

[xdelaruelle]

1ef61d6 is merged into master.

A skip flag is mandatory to exclude the full function definition, not only its header line, as the while loop treats output from env command line by line.

[hongyi-zhao]

See my comments here:
985aa88

[xdelaruelle]

Feel free to send a pull request for your code change proposal. Here are also some generic guidelines:

• an improvement to the createmodule ability should be provided for both createmodule.sh and createmodule.py
• these scripts should not rely on additional external tools, not to introduce additional external dependencies
• users should know an error occurred on the evaluated script to understand something failed somewhere (not to thing everything worked as expected)

[hongyi-zhao]

The touch ups I posted don't meet these requirements.

[hongyi-zhao]

Feel free to send a pull request for your code change proposal. Here are also some generic guidelines:

* an improvement to the `createmodule` ability should be provided for both `createmodule.sh` and `createmodule.py`

Some ideas maybe easily done in python, while it may be difficult or impossible for the same idea implemented in shell.

* these scripts should not rely on additional external tools, not to introduce additional external dependencies

What do you mean by saying ``additional external tools''? The original shell script relies on sort, is this in the range of ``additional external tools'' mentioned by you?

If not, I think that all of the standard *nix tools, say sed/awk, shouldn't be considered as ``additional external tools'' too.

* users should know an error occurred on the evaluated script to understand something failed somewhere (not to thing everything worked as expected)