Skip to content

Conversation

@ahijevyc
Copy link
Contributor

@ahijevyc ahijevyc commented Mar 3, 2025

This pull request optimizes the physics tendency code in the fv3_physics_tend module.

  1. Documentation Updates:
  • Updated several figures in the figure directory with blue-white-red colormap.
  • Removed outdated sample data link from fv3_physics.rst.
  • Simplified usage instructions for planview_fv3.py, vert_profile_fv3.py, and cross_section_vert.py.
  1. Code Refactoring:
  • Refactored cross_section_vert.py, planview_fv3.py, and vert_profile_fv3.py to streamline argument parsing and configuration handling.
  • Moved common functions to physics_tend.py for better code reuse.
  1. Configuration Enhancements:
  • New default settings for tendency variables, geographic extent, color maps, time window, valid time, and more.
  • Updated fv3_physics_tend_defaults.yaml for FV3 output /glade/campaign/mmm/parc/ahijevyc/METplotpy/DavidA_fv3_history_etc_files. These data have tendencies that balance the actual change in state variables (i.e. the residual is close to zero).
  • Added a configuration file UFS.yaml to handle FV3 UFS output in May Wong's campaign storage. These data have tendencies that are never reset to zero during the model run nor averaged over the time interval between output. They also have separate DataArrays for each vertical level. For these data, the configuration switch tendencies_were_zeroed_and_averaged_after_every_output must be set to True and the function physics_tend.get_fv3ds is used to stack DataArrays for the same state variable (t, u, v, or q) but different pressure levels in the vertical.
  1. Performance Improvements:
  • Optimized the handling of large datasets by using chunking and lazy loading where appropriate.
  1. TODO:
  • Write test for FV3 UFS output in May Wong's campaign storage.
  • Pass SonarQube Quality Gate check.
  1. Pull Request Testing
    Describe testing already performed for these changes:
  • new Jupyter notebook to plot plan views, vertical cross section, and vertical profiles of tendency using multiple input formats.
  • Plan view, vertical profile, and cross-sections all look the same, aside from the new colormap.
  • Plan view, vertical profile, and cross-section work for /glade/campaign/mmm/parc/mwong/ufs-mrw/2020040812.F024.C768/fv3_history2d.tile3.nc
  • Uploaded new test data from /glade/campaign/mmm/parc/ahijevyc/METplotpy/DavidA_fv3_history_etc_files.

Moved planview, vert_profile, and cross_section_vert out of main().

Moved optional command line args to config file.

try to show requisite yaml mods

space after .rst code-block

add tab to .rst code-block

removed extraneous space

adapted cross_section_vert
with shapefile in config file not argument.
same with xmin and xmax
No more argparse args to functions.

Save image to TMPDIR

override config file with kwargs.
in /glade/campaign/mmm/parc/mwong/ufs-mrw

They are different from previous history file.

vertical pressure levels are split into multiple variables, so they must be combined. Tendencies are accumulated from initialization without resetting to zero along the way. They aren't scaled by how much time has gone by since the most recent reset.
use regex to match FV3-style filename

coordinate reference system option in config file for cross_section_vert
zoom in on vertical_profile x-axis in test case figure
in physics_tend.get_fv3ds. Move to else-block in calling function.
will prepage a single dataset with history and grid spec, given the paths to both. Feed it to planview, vert_profile, and cross_section_vert.

Remove `subtract` option. It is ugly to do it in the functions, and easy to do beforehand.
moved shp inset to left side to avoid covering up power-of-ten in x-axis tick label
rename `fv3` configuration dictionary -> `config`

rename `time_after_time0` -> `time_after_twindow_start`

document get_fv3ds()
        This function handles the fv3 UFS forecasts made by May Wong
        in campaign storage
        It subtracts tendency values across a time window.
        Leaves state variables alone
        Stacks in the vertical
        Now it matches the older fv3 history files METplotpy was
        designed for.

make sure "validtime" and "twindow" are consistent between configuration
dictionary and Dataset .
@ahijevyc
Copy link
Contributor Author

ahijevyc commented Mar 3, 2025

Hi @bikegeek . I refactored the contributed code for fv3 tendency plotting. In addition to moving command-line options to the configuration file, I rewrote a lot of the code. It still works on the older test data and a new input format. I'd like to change the sample data tar file to contain fv3 output for which the physics tendencies properly balance the actual change in temperature, wind, and moisture.
The usual tests started once I submitted this PR, but the SonarQube test is failing, and I don't think it is getting to the tests for fv3 tendency. I'll look through the contributor's guide to look for hints.

@bikegeek
Copy link
Collaborator

bikegeek commented Mar 3, 2025 via email

@ahijevyc
Copy link
Contributor Author

ahijevyc commented Mar 3, 2025

I attached new use case data here based on discussion 954

fv3_physics_tendency.tgz

To be used with configuration file use_case.yaml

@bikegeek bikegeek added this to the METplotpy-3.1.0 milestone Mar 24, 2025
@github-project-automation github-project-automation bot moved this to 🩺 Needs Triage in METplus-6.1 Development Mar 24, 2025
@bikegeek bikegeek moved this from 🩺 Needs Triage to 🏗 In progress in METplus-6.1 Development Mar 24, 2025
@bikegeek bikegeek moved this from 🏗 In progress to 🔎 In review in METplus-6.1 Development Mar 26, 2025
@bikegeek bikegeek marked this pull request as ready for review March 26, 2025 23:15
@bikegeek bikegeek self-requested a review March 26, 2025 23:16
@bikegeek bikegeek merged commit 7a364be into develop Mar 26, 2025
4 of 5 checks passed
@github-project-automation github-project-automation bot moved this from 🔎 In review to 🏁 Done in METplus-6.1 Development Mar 26, 2025
@bikegeek bikegeek deleted the issue449_optimize_physics_tend_config branch April 14, 2025 21:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

No open projects
Status: 🏁 Done

Development

Successfully merging this pull request may close these issues.

Optimize plotting in the Physics Tendency planview, cross-section, and vertical cross-section plots

3 participants