grep commands in pyfragparce.sh and argueparce.sh fail to extract data from input blocks

[lexming]

The scripts pyfragparce.sh and argueparce.sh do not reliably extract the data from within the different blocks (i.e. JOBSUB, ADF, PyFrag) in the input file. The presence in the input file of any text that matches the keywords of those blocks derails the whole process. The issue stems from the grep commands, such as in the file:
https://github.com/sunxb05/PyFrag/blob/5618d5518a8f4a1fc22805feac57b6e6717a08dc/host/standalone/adf_new/pyfragparce.sh#L77-L82

Matching the block keywords without any additional constrain will fail whenever the user uses those strings in their input. For instance, in our system, ADF is loaded with a module called ADF/2019.103. So it is necessary to add the command module load ADF/2019.103 inside the JOBSUB block. Hence, the grep commands will match the ADF in the module name, failing to generate a viable job script.

There are multiple solutions to this problem. We have solved it by replacing those grep commands with sed commands of the form:

sed -n '/^JOBSUB$/,/^JOBSUB END$/{//!p;}' $input > jobsub.txt

• the block keywords are only matched if they are found between the beginning (^) and end of the line ($), so only element in the line
• the code within the blocks is extracted with a single execution instead of 4
• this also avoids needing to define any limit of number of lines

You can find the patch for PyFrag that we developed in https://github.com/lexming/easybuild-easyconfigs/blob/vub-hpc/easybuild/easyconfigs/p/PyFrag/PyFrag-2019_fix_flaky_input_fetch.patch . It might not be complete though.

[SiebeLeDe]

Thanks for letting us know! Added it to the development branch and we will test it further.