Replace format labels with strings #253

sseraj · 2022-12-16T20:31:00Z

Purpose

This PR replaces all format labels with strings. This is necessary to get idempotent formatting with fprettify (that is, the formatting will not change when fprettify is run multiple times in a row on the same code). This is a separate PR from #229 because these changes are manual whereas the fprettify changes will be automated.

If I set the format strings correctly, the output should look the same as before. I checked that this is true for common outputs, like load balancing, cell count, NK switch, adjoint monitor, and negative volumes.

Expected time until merged

One week. This is blocking #229 so it is a bit urgent.

Type of change

Bugfix (non-breaking change which fixes an issue)
New feature (non-breaking change which adds functionality)
Breaking change (non-backwards-compatible fix or feature)
Code style update (formatting, renaming)
Refactoring (no functional changes, no API changes)
Documentation update
Maintenance update
Other (please describe)

Testing

Test coverage for output is non-existent, so it would be great if reviewers could run this code on their regular cases and check that the output looks the same.

Checklist

I have run flake8 and black to make sure the Python code adheres to PEP-8 and is consistently formatted
I have formatted the Fortran code with fprettify or C/C++ code with clang-format as applicable
I have run unit and regression tests which pass locally with my changes
I have added new tests that prove my fix is effective or that my feature works
I have added necessary documentation

codecov · 2022-12-16T20:57:42Z

Codecov Report

Merging #253 (d729491) into main (d5fe462) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##             main     #253   +/-   ##
=======================================
  Coverage   40.77%   40.77%           
=======================================
  Files          13       13           
  Lines        3882     3882           
=======================================
  Hits         1583     1583           
  Misses       2299     2299

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

anilyil

Left a few comments, but I am stopping now to give my big picture feedback: I think we can improve the consistency here a bit. I like the module idea, but I am not sure how necessary it is. Primarily because it looks like we don't need to move all formats to the module, but then again, how do we decide some lives in there compared to the files themselves?

I was going to recommend renaming the format module but I think my current recommendation would be to remove the format module and if a format is used more than one time in a file, define it as a variable. If it is used once, just typing it in the write/print statements is fine.

Let me know what you think. Some of my individual comments are related to this.

Finally, just so that we document this here: Tapenade produced non F2008 standard code when the failure check in BCData was ignored in a certain way. I pushed a fix for this in this PR: fbe2224#diff-9c42070cc0184316dd2c54cc31a4b7fdf4d3b89ee7493c148b844c3b42831d86

In the future, as @sseraj recommended to me in private, we may want to either consider using different compiler flags for the tapenade generated code, or tweak tapenade options to generate only F2008 valid code. I am guessing this issue will be automatically addressed when we update the tapenade version.

src/adjoint/adjointAPI.F90

anilyil · 2022-12-19T01:46:00Z

src/adjoint/adjointUtils.F90


    ! Write the residual norm to stdout every adjMonStep iterations.

    if(mod(n, adjMonStep) ==0 ) then
-       if( myid==0 ) write(*, 10) n, rnorm
+       if( myid==0 ) write(*, monitorFormat) n, 'KSP Residual norm', rnorm


I also realize this is a way to do this formatting. Why didn't you do this everywhere? E.g. in adjoint API, there are formats that repeat. Maybe lets also change those to variables?

anilyil · 2022-12-19T01:46:24Z

src/adjoint/adjointAPI.F90

          else
-             write(*,40) "PETSc solver converged after", adjConvIts, &
-                  "iterations."
+            write(*, "(1X, A, 1X, I5, 1X, A)") "PETSc solver converged after", adjConvIts, "iterations."


It looks like there's some repeated usage in this file. Any particular reason you did not define a variable for these?

I agree, we when we have repeated patterns we should make this a variable, especially for related operations.

src/modules/format.F90

anilyil · 2022-12-19T01:49:05Z

src/modules/format.F90

@@ -0,0 +1,27 @@
+module format
+!
+!       This module contains common format specifiers for printing output.


What is the rule to decide if a format will be defined in here compared to the files they are used? If you have a rule, can you please explain it so people can follow it? If not, we can move the declarations here, but that might be unnecessary effort.

I would say if the same formatting patterns are spanning multiple subroutines within a module then they should be a module variable in the respective module, and if they are spanning multiple modules, it should be moved into this module. The patterns that are currently here should work for a lot of cases.

sseraj

Using a module was originally @eirikurj's idea, so maybe he can comment on my implementation. I put format strings that showed up across different files in the format module, but there ended up being only a few common ones.

I also agree that I was not very consistent in using variables and directly using strings. I would be fine with doing what you suggested with the variables.

src/adjoint/adjointAPI.F90

src/modules/format.F90

eirikurj

I think it looks good overall, but as @anilyil mentioned, there are perhaps places that could be improved.
On the module discussion, I think we should utilize a module for these very common patterns, instead of having the same pattern or similar patters across files. This makes it easier to change and maintain, e.g., by changing print precision etc.
However, I do think that there is a tradeoff, and we should not necessarily force everything to be a subroutine or a module variable, so just defining the pattern for these one-off print/write cases should also be fine.

src/modules/format.F90

eirikurj · 2022-12-20T12:28:04Z

src/modules/format.F90

@@ -0,0 +1,27 @@
+module format
+!
+!       This module contains common format specifiers for printing output.


I would say if the same formatting patterns are spanning multiple subroutines within a module then they should be a module variable in the respective module, and if they are spanning multiple modules, it should be moved into this module. The patterns that are currently here should work for a lot of cases.

eirikurj · 2022-12-20T12:40:51Z

src/adjoint/adjointAPI.F90

          else
-             write(*,40) "PETSc solver converged after", adjConvIts, &
-                  "iterations."
+            write(*, "(1X, A, 1X, I5, 1X, A)") "PETSc solver converged after", adjConvIts, "iterations."


I agree, we when we have repeated patterns we should make this a variable, especially for related operations.

sseraj · 2022-12-20T15:19:34Z

Thanks for the feedback. I will make changes to specify the format patterns more consistently using the following convention:

commonFormats module variable: formats spanning multiple modules
module variable: formats spanning multiple subroutines within a module
subroutine variable: formats that are repeated within a subroutine
string: one-off cases

sseraj · 2022-12-21T17:15:29Z

I think I addressed all the comments. One exception is that I kept instances of "(A)" as strings because this is the most basic format and appears all over the code.

I also checked the outputs again after the changes.

eirikurj

Did another quick pass. Left a few very minor comments.

eirikurj · 2023-01-13T11:34:50Z

src/adjoint/adjointAPI.F90

@@ -1,5 +1,10 @@
 module adjointAPI

+  use constants


Since its imported here at the top, we could remove the same statement from the subroutines.
Also, if its not too difficult, it would be great to use only, although its less important for this module.

src/bcdata/BCData.F90

src/output/tecplotIO.F90

src/partitioning/gridChecking.F90

src/partitioning/loadBalance.F90

src/utils/utils.F90

sseraj · 2023-01-17T16:02:22Z

Thanks for the comments @eirikurj. I will address them by the end of the week.

src/partitioning/loadBalance.F90

src/overset/zipperMesh.F90

eirikurj

Looks good!

sseraj added 30 commits August 25, 2022 15:54

removed one label as example

79c2b87

started format module

cb66011

replaced some more labels in preprocessingAPI

af38cfe

converted remaining labels in preprocessingAPI

f4d8730

added commented out format strings in adtLocalSearch

fd66cf9

removed labels in signals.F90

51a4328

replaced labels in utils.F90

6add56f

removed label in coarseUtils.F90

8ac9536

replaced labels in gridChecking.F90

0b03839

replaced labels in outputMod.F90

489da6d

replaced labels in loadBalance.F90

9fa6ba4

replaced some labels in readCGNSGrid.F90

0dce056

more labels in readCGNSGrid

0974864

line breaks for complexify ceq

d3177f1

print real load balance

6c0ff37

more readCGNSGrid changes

c98f49e

finished readCGNSGrid

7158732

replced labels in partitioning.F90

d55f4ca

replaced labels in BCData.F90

0de81c9

stringOps labels

a9875f8

tecplotIO labels

acc8505

zipperMesh labels

9315809

actuatorRegion labels

7326928

oversetAPI labels

b728b24

initializeFlow labels

a5f7217

variableReading labels

af8d694

writeCGNSSurface labels

706b9d8

unused wallDistance labels

db3da7f

adjointUtils label

4cd5a16

adjointAPI labels

9a246c1

sseraj requested a review from a team as a code owner December 16, 2022 20:31

sseraj requested review from joanibal and anilyil December 16, 2022 20:31

sseraj and others added 2 commits December 16, 2022 17:54

skip bcdata error when using tapenade

1cbd1e6

moved additional part of bc error checking out of tapenade

fbe2224

anilyil reviewed Dec 19, 2022

View reviewed changes

sseraj commented Dec 19, 2022

View reviewed changes

src/adjoint/adjointAPI.F90 Show resolved Hide resolved

src/modules/format.F90 Outdated Show resolved Hide resolved

sseraj requested review from eirikurj and removed request for joanibal December 19, 2022 16:32

eirikurj reviewed Dec 20, 2022

View reviewed changes

sseraj added 3 commits December 21, 2022 10:46

updated documentation in format module

bc4dc86

renamed format module to commonFormats

5bfddbb

improved consistency

10b2a03

sseraj requested review from eirikurj and anilyil December 21, 2022 17:15

eirikurj reviewed Jan 13, 2023

View reviewed changes

sseraj added 4 commits January 18, 2023 13:01

updated constants in adjointAPI

cf4c8ec

more Tapenade checks in BCData

fa5740b

move sci6 to module level in tecplotIO

71b3a14

print instead of write

d729491

eirikurj reviewed Jan 23, 2023

View reviewed changes

src/partitioning/loadBalance.F90 Show resolved Hide resolved

eirikurj reviewed Jan 23, 2023

View reviewed changes

src/overset/zipperMesh.F90 Show resolved Hide resolved

eirikurj approved these changes Jan 25, 2023

View reviewed changes

anilyil approved these changes Feb 3, 2023

View reviewed changes

anilyil merged commit 09d2ffd into main Feb 3, 2023

anilyil deleted the format-string branch February 3, 2023 18:30

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Replace format labels with strings #253

Replace format labels with strings #253

sseraj commented Dec 16, 2022

codecov bot commented Dec 16, 2022 •

edited

Loading

anilyil left a comment

anilyil Dec 19, 2022

anilyil Dec 19, 2022

eirikurj Dec 20, 2022

anilyil Dec 19, 2022

eirikurj Dec 20, 2022

sseraj left a comment

eirikurj left a comment

eirikurj Dec 20, 2022

eirikurj Dec 20, 2022

sseraj commented Dec 20, 2022

sseraj commented Dec 21, 2022

eirikurj left a comment

eirikurj Jan 13, 2023

sseraj Jan 18, 2023

sseraj commented Jan 17, 2023

eirikurj left a comment

Replace format labels with strings #253

Replace format labels with strings #253

Conversation

sseraj commented Dec 16, 2022

Purpose

Expected time until merged

Type of change

Testing

Checklist

codecov bot commented Dec 16, 2022 • edited Loading

Codecov Report

anilyil left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sseraj left a comment

Choose a reason for hiding this comment

eirikurj left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sseraj commented Dec 20, 2022

sseraj commented Dec 21, 2022

eirikurj left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sseraj commented Jan 17, 2023

eirikurj left a comment

Choose a reason for hiding this comment

codecov bot commented Dec 16, 2022 •

edited

Loading