Skip to content

LAMMPSBaseParser: Fix parsing for nodes with the script input#60

Merged
JPchico merged 1 commit intodevelopfrom
feature/parser-convert-total-runtime-seconds
Mar 23, 2023
Merged

LAMMPSBaseParser: Fix parsing for nodes with the script input#60
JPchico merged 1 commit intodevelopfrom
feature/parser-convert-total-runtime-seconds

Conversation

@sphuber
Copy link
Member

@sphuber sphuber commented Mar 22, 2023

The BaseLammpsCalculation plugin was recently updated to allow specifying the exact script to run through the script input node. If the default parser is used in this case, which is the LAMMPSBaseParser, a non-zero exit code would always be returned because some of the output files that the parser wouldn't be there. These output files are normally written by the script that the plugin itself would create, but if a complete script is taken from inputs, it cannot be guaranteed that the script will produce these same outputs.

The parser is updated to only return an exit code if an expected output file is not in the retrieved files if the script input was not defined. In addition, the total_wall_time key, if parsed by the log file parser, is parsed and converted to seconds and added as the total_wall_time_seconds key.

@sphuber sphuber requested a review from JPchico March 22, 2023 13:06
@codecov
Copy link

codecov bot commented Mar 22, 2023

Codecov Report

Merging #60 (a8590b5) into develop (bb495bc) will decrease coverage by 0.04%.
The diff coverage is 82.75%.

@@             Coverage Diff             @@
##           develop      #60      +/-   ##
===========================================
- Coverage    90.26%   90.23%   -0.04%     
===========================================
  Files           32       32              
  Lines         2414     2426      +12     
===========================================
+ Hits          2179     2189      +10     
- Misses         235      237       +2     
Flag Coverage Δ
pytests 90.23% <82.75%> (-0.04%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
aiida_lammps/parsers/lammps/lammps_parser.py 81.66% <82.75%> (+0.41%) ⬆️

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

Copy link
Collaborator

@JPchico JPchico left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it looks good, just remove the print statement in the parser when dealing with the global variables.

any idea why the previous tests did not catch this issue?

@JPchico JPchico added the bug label Mar 22, 2023
@sphuber sphuber force-pushed the feature/parser-convert-total-runtime-seconds branch from 17e31ec to c638370 Compare March 22, 2023 15:56
@sphuber
Copy link
Member Author

sphuber commented Mar 22, 2023

I think it looks good, just remove the print statement in the parser when dealing with the global variables.

Done.

any idea why the previous tests did not catch this issue?

Well, as far as I can tell, this parser was not tested whatsoever. I think that this parser was maybe the original one (together with BaseLammpsCalculation) and they were never tested. When you started to refactor things, you added tests, but only for the new plugins that you added (lammps.force, lammps.md, etc.).

We should maybe think how to harmonize all of these, because now there seems to be some duplication.

@sphuber sphuber requested a review from JPchico March 22, 2023 16:06
@JPchico
Copy link
Collaborator

JPchico commented Mar 22, 2023

any idea why the previous tests did not catch this issue?

Well, as far as I can tell, this parser was not tested whatsoever. I think that this parser was maybe the original one (together with BaseLammpsCalculation) and they were never tested. When you started to refactor things, you added tests, but only for the new plugins that you added (lammps.force, lammps.md, etc.).

Well that explains it. It is very nice that you added a test for the parser. I think that the plan was to do that sometime ago, but well clearly I must have forgotten to do it.
I'll look and see if more tests are needed to make sure that no funky business can happen.

We should maybe think how to harmonize all of these, because now there seems to be some duplication.

I agree 100% I think that the idea that @chrisjsewell had was that we should leave the old functions in case somebody needs them. But I think it might actually be best to remove them, or mark them for deprecation and make the test more modular. Since we end up having a lot of duplication and it is easy to miss something.

So I'm all for removing the old parsers and calcjobs and keep the new ones and just run with that.

@sphuber
Copy link
Member Author

sphuber commented Mar 22, 2023

So I'm all for removing the old parsers and calcjobs and keep the new ones and just run with that.

That'd be fine with me as well, as long as I can still port over the functionality that I still added that allow adding a complete script as input and have the wall time seconds parsed. But if you are ok with that as well, then I am fine with opening a PR for that.

@sphuber
Copy link
Member Author

sphuber commented Mar 22, 2023

@JPchico did anything still needed to be done for this PR?

The `BaseLammpsCalculation` plugin was recently updated to allow
specifying the exact script to run through the `script` input node. If
the default parser is used in this case, which is the `LAMMPSBaseParser`,
a non-zero exit code would always be returned because some of the output
files that the parser wouldn't be there. These output files are normally
written by the script that the plugin itself would create, but if a
complete script is taken from inputs, it cannot be guaranteed that the
script will produce these same outputs.

The parser is updated to only return an exit code if an expected output
file is not in the retrieved files if the `script` input was not
defined. In addition, the `total_wall_time` key, if parsed by the log
file parser, is parsed and converted to seconds and added as the
`total_wall_time_seconds` key.
@sphuber sphuber force-pushed the feature/parser-convert-total-runtime-seconds branch from c638370 to a8590b5 Compare March 22, 2023 22:48
Copy link
Collaborator

@JPchico JPchico left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good!

@JPchico JPchico merged commit ed59902 into develop Mar 23, 2023
@sphuber sphuber deleted the feature/parser-convert-total-runtime-seconds branch March 23, 2023 07:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants