Some issues about createmodule.sh

[hongyi-zhao]

Hi,

With the latest git version of modules to test the parallel_studio_xe_2020.0.088.
See the following results:

$ ./createmodule.sh /opt/intel/parallel_studio_xe_2020/psxevars.sh intel64 | awk '$2 == "PATH" ' 
append-path	PATH	/opt/intel/parallel_studio_xe_2020.0.088/bin
prepend-path	PATH	/opt/intel/intelpython3/bin:/opt/intel/intelpython3/condabin:/opt/intel/advisor_2020.0.0.604394/bin64:/opt/intel/vtune_profiler_2020.0.0.605129/bin64:/opt/intel/inspector_2020.0.0.603904/bin64:/opt/intel/itac/2020.0.015/intel64/bin:/opt/intel/itac/2020.0.015/intel64/bin:/opt/intel/clck/2019.6/bin/intel64:/opt/intel/compilers_and_libraries_2020.0.166/linux/bin/intel64:/opt/intel/compilers_and_libraries_2020.0.166/linux/bin:/opt/intel/compilers_and_libraries_2020.0.166/linux/mpi/intel64/libfabric/bin:/opt/intel/compilers_and_libraries_2020.0.166/linux/mpi/intel64/bin:/opt/intel/debugger_2020/gdb/intel64/bin

Why the first entry is append-path?

Furthermore, the above results have duplicate in it:

$ ./createmodule.sh /opt/intel/parallel_studio_xe_2020/psxevars.sh intel64 | awk '$2 == "PATH" { print $NF }' | tr ':' '\n' | sort -u | wc
     13      13     619
$ ./createmodule.sh /opt/intel/parallel_studio_xe_2020/psxevars.sh intel64 | awk '$2 == "PATH" { print $NF }' | tr ':' '\n'  | wc
     14      14     658

For more detail:

$ ./createmodule.sh /opt/intel/parallel_studio_xe_2020/psxevars.sh intel64 | awk '$2 == "PATH" { print $NF }' | tr ':' '\n'  
/opt/intel/parallel_studio_xe_2020.0.088/bin
/opt/intel/intelpython3/bin
/opt/intel/intelpython3/condabin
/opt/intel/advisor_2020.0.0.604394/bin64
/opt/intel/vtune_profiler_2020.0.0.605129/bin64
/opt/intel/inspector_2020.0.0.603904/bin64
/opt/intel/itac/2020.0.015/intel64/bin
/opt/intel/itac/2020.0.015/intel64/bin
/opt/intel/clck/2019.6/bin/intel64
/opt/intel/compilers_and_libraries_2020.0.166/linux/bin/intel64
/opt/intel/compilers_and_libraries_2020.0.166/linux/bin
/opt/intel/compilers_and_libraries_2020.0.166/linux/mpi/intel64/libfabric/bin
/opt/intel/compilers_and_libraries_2020.0.166/linux/mpi/intel64/bin
/opt/intel/debugger_2020/gdb/intel64/bin

The following two lines in the above results are duplicate:

/opt/intel/itac/2020.0.015/intel64/bin
/opt/intel/itac/2020.0.015/intel64/bin

Besides, the following var also has duplicate in the results:

$ ./createmodule.sh /opt/intel/parallel_studio_xe_2020/psxevars.sh intel64 | awk '$2 == "LD_LIBRARY_PATH" ' | tr ':' '\n'  | wc
     15      17    1054
$ ./createmodule.sh /opt/intel/parallel_studio_xe_2020/psxevars.sh intel64 | awk '$2 == "LD_LIBRARY_PATH" ' | tr ':' '\n'  | sort -u | wc
     13      15     902

Regards

[xdelaruelle]

Entries are appended or prepended depending on whether the analyzed script adds new path entry to the beginning or the end of the path variable. Duplicates path entries may also be found if analyzed script generates these duplicates path entries.

However as createmodule.py handles the duplication, it may be also expected that createmodule.sh also handles duplicate entries.

[hongyi-zhao]

Xavier Delaruelle <notifications@github.com> 于2020年2月19日周三 上午1:08写道：

Entries are appended or prepended depending on whether the analyzed script adds new path entry to the beginning or the end of the path variable. Duplicates path entries may also be found if analyzed script generates these duplicates path entries. However as createmodule.py handles the duplication, it may be also expected that createmodule.sh also handles duplicate entries.

This is a feature which should be added to the createmodule.sh. But due to it's pure shell based script, doing such thing may be tedious and a headache.

…

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#316>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ACVDTDTUQRW5AL627ZYMHODRDQIYDANCNFSM4KW4HDVQ> .

-- Hongsheng Zhao <hongyi.zhao@gmail.com> Institute of Semiconductors, Chinese Academy of Sciences GnuPG DSA: 0xD108493

[hongyi-zhao]

Xavier Delaruelle <notifications@github.com> 于2020年2月19日周三 上午1:08写道：

Entries are appended or prepended depending on whether the analyzed script adds new path entry to the beginning or the end of the path variable. Duplicates path entries may also be found if analyzed script generates these duplicates path entries. However as createmodule.py handles the duplication, it may be also expected that createmodule.sh also handles duplicate entries.

But I meet the following errors: $ python createmodule.py /opt/intel/parallel_studio_xe_2020/psxevars.sh intel64 WARNING: initscript sent the following to stderr: /bin/sh: 44: /opt/intel/parallel_studio_xe_2020/psxevars.sh: [[: not found /bin/sh: 49: /opt/intel/parallel_studio_xe_2020/psxevars.sh: [[: not found /bin/sh: 54: /opt/intel/parallel_studio_xe_2020/psxevars.sh: [[: not found /bin/sh: 58: /opt/intel/parallel_studio_xe_2020/psxevars.sh: [[: not found /bin/sh: 62: /opt/intel/parallel_studio_xe_2020/psxevars.sh: [[: not found /bin/sh: 66: /opt/intel/parallel_studio_xe_2020/psxevars.sh: [[: not found /bin/sh: 70: /opt/intel/parallel_studio_xe_2020/psxevars.sh: [[: not found #%Module 1.0 set prefix /opt/intel/parallel_studio_xe_2020.0.088/bin append-path PATH $prefix setenv INTEL_LICENSE_FILE /opt/intel/licenses

…

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#316>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ACVDTDTUQRW5AL627ZYMHODRDQIYDANCNFSM4KW4HDVQ> .

-- Hongsheng Zhao <hongyi.zhao@gmail.com> Institute of Semiconductors, Chinese Academy of Sciences GnuPG DSA: 0xD108493

[xdelaruelle]

$ python createmodule.py /opt/intel/parallel_studio_xe_2020/psxevars.sh intel64
WARNING: initscript sent the following to stderr:
/bin/sh: 44: /opt/intel/parallel_studio_xe_2020/psxevars.sh: [[: not found
/bin/sh: 49: /opt/intel/parallel_studio_xe_2020/psxevars.sh: [[: not found
/bin/sh: 54: /opt/intel/parallel_studio_xe_2020/psxevars.sh: [[: not found
/bin/sh: 58: /opt/intel/parallel_studio_xe_2020/psxevars.sh: [[: not found
/bin/sh: 62: /opt/intel/parallel_studio_xe_2020/psxevars.sh: [[: not found
/bin/sh: 66: /opt/intel/parallel_studio_xe_2020/psxevars.sh: [[: not found
/bin/sh: 70: /opt/intel/parallel_studio_xe_2020/psxevars.sh: [[: not found

It seems like your /bin/sh is not a bash shell. psxevars.sh is a bash script.

[hongyi-zhao]

Thanks for pointing this out. On Debian based os, it's as following:

$ realpath /bin/sh
/usr/bin/dash

But I have the following questions on this topic:

In general, I think we should first judged the shell types used by the sourcing script, then use appropriate shell interpreter to sourcing it.

I have checked the createmodule.py, it seems it doesn't do this check.

Regards

[xdelaruelle]

Please point me to an existing Python script that applies what you describe.

Regards,
Xavier

[hongyi-zhao]

Just my ideas/suggestions.