Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

baroclinic initialization #31

Merged
merged 48 commits into from
Nov 24, 2021
Merged

Conversation

rheacangeo
Copy link
Contributor

Purpose

To add the computation of the baroclinic perturbation test case (Jablonowski & Williamson JRMS 2006) to enable running performance scripts without reading serialized data.

This PR:

  • adds validating code (against data generated with the feature/serialize_init branch of fv3gfs-fortran, which will in parallel be PRed into master as data version 7.2.7) that creates the baroclinic initial state in fv3core/fv3core/initialization
  • Two files distinguish the test case -- baroclinic_jablonowski_williamson.py has the base equations found in the paper and DCMIP2016 operating on numpy arrays and baroclinic.py, which calls the functions from the other file, and does the cubed sphere transformation and data shaping sizing to get it in the format we can use in the Dycore.
  • Adds code to test the initial state values when run with baroclinic namelists (not currently blocking PRs, but does or will when it works, get tested with the cache generating cron plan).
  • adds a DycoreState specification using a dataclass and replaces the ArgSpec and get_namespace functionality of fv3core/fv3ocre/stencils/fv_dynamics.py
  • Updated the fv_dynamics test to use the DycoreState
  • updates the dynamics runfile for performance testing to create a DycoreState using the baroclinic initialization and replaced reading the serialized data as inputs.

rheacangeo and others added 30 commits November 16, 2021 00:32
…hrough the grid_data object instead. A future PR may orgnaize these to be computed at a different stage
… launching simulations that are not baroclinic
… on the MetricTerms class. To avoid a circular import with MirrorGrid, changed RIGHT_HAND_GRID to be an argument in the context of mirroring, set to to a constant in out calls of it
fv3core/fv3core/stencils/fv_dynamics.py Show resolved Hide resolved
Comment on lines 297 to 325
lat2, lat3, lat4, lat5 = compute_grid_edge_midpoint_latitude_components(lon, lat)
slice_3d = (slice(0, nx), slice(0, ny), slice(None))
slice_2d = (slice(0, nx), slice(0, ny))
# initialize temperature
t_mean = jablo_init.horizontally_averaged_temperature(eta)
pt[slice_3d] = cell_average_nine_components(
jablo_init.temperature,
[eta, eta_v, t_mean],
lat,
lat_agrid,
lat2,
lat3,
lat4,
lat5,
slice_2d,
)

# initialize surface geopotential
phis[slice_2d] = cell_average_nine_components(
jablo_init.surface_geopotential_perturbation,
[],
lat,
lat_agrid,
lat2,
lat3,
lat4,
lat5,
slice_2d,
)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cell_average_nine_components and compute_grid_edge_midpoint_latitude_components are doing something low level and pretty complex - I can't tell what they are doing. compute_grid_edge_midpoint_latitude_components is also only used for this second function.

Is there any change you can make to have it be more clear what's happening at this function's level, by pushing the low-level logic down one call level? For example, even if it's less efficient, by pushing the call to compute_grid_edge_midpoint_latitude_components down one level (removing many low-level call arguments from this function's level), and by pulling the slicing operation (which is a higher level operation you're already using here in the assignment slice) up to this level (e.g. by passing lat[slice_2d] and lat_agrid[slice_2d])?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sure thing. this duplicates the calculation of these latitude factors, but does make it more understandable. I'll implement the suggestion. Pulling the slicing up is a little tricker, as the resulting shapes of the 9 points being averaged are not by default the same, as they are shifted in different directions. I can do this if you prefer, but want to double check before I iterate on it.

Comment on lines 389 to 390
assert not adjust_dry_mass
assert not hydrostatic
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does it make sense for us to just not implement these as keyword arguments? I don't think we have any plans to implement these, the code itself is easier to understand than the rest of the dycore in terms of if we were to implement a new option, and if we did implement a new option we'd probably do it using new functions rather than if flags?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok sure. The fortran code has a lot of if conditions for different test cases, so it could be potentially confusing to understand and implement the other options. But I am not worried about this as much as I was for the dycore, since this is just an idealized initial state.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm also already not following this p_var subroutine to the letter, since it was recomputing pressure variables in the exact same way they were already computed.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do you mean just for p_var, or for the whole module?

Comment on lines 516 to 517
# TODO: when the dycore state is updated to only include
# quantities and no storages, remove the "_quantity" from phis, u and v
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You could remove this TODO here, when I go to do it this will be using vscode's refactor functionality so it will update everywhere at once (thanks to your use of a dataclass!). As it stands, this comment would probably remain after the refactor unless a reviewer catches it or I've thought long enough about this particular comment location while writing this message.

Suggested change
# TODO: when the dycore state is updated to only include
# quantities and no storages, remove the "_quantity" from phis, u and v

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok thanks

Comment on lines 366 to 377
def init_from_serialized_data(cls, serializer, grid, quantity_factory):
savepoint_in = serializer.get_savepoint("FVDynamics-In")[0]
translate_object = fv3core.testing.TranslateFVDynamics([grid])
input_data = translate_object.collect_input_data(serializer, savepoint_in)
# making just storages for the moment, revisit when making them all
# quantities (maybe use state_from_inputs)
translate_object._base.make_storage_data_input_vars(input_data)
# used for the translate test as inputs, but are generated by the
# MetricsTerms class and are not part of this data class
for delvar in ["ak", "bk", "ptop", "ks"]:
del input_data[delvar]
return cls(**input_data, quantity_factory=quantity_factory)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This method should only be getting used by the tests, and currently fv3core doesn't depend on serialbox other than for testing. Can this get moved to the testing code?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It can. I was thinking it might be useful for performance testing if we end up wanting to test non-baroclinic namelists before we implement reading initial state from disk. But someone wanting to do that can probably figure it out.

@@ -293,6 +296,12 @@ def make_grid_storage(self, pygrid):
if k in self.data:
self.make_composite_var_storage(k, self.data[k], shape)
del self.data[k]
for k in TranslateGrid.ee_vars:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: This was pre-existing, but can you replace k here and below with key? When I read these my mind doesn't stop telling me it's indexing on the vertical direction.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sure thing

@@ -282,15 +284,27 @@ def compute_parallel(self, inputs, communicator):
grid_data.ptop = inputs["ptop"]
grid_data.ks = inputs["ks"]

state = self.state_from_inputs(inputs)
input_storages = self.state_from_inputs(inputs)
# making sure we init DyCOreState with the exact set of variables
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
# making sure we init DyCOreState with the exact set of variables
# making sure we init DyCoreState with the exact set of variables

fv3gfs-util/fv3gfs/util/quantity.py Show resolved Hide resolved
Copy link
Contributor Author

@rheacangeo rheacangeo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the quick turnaround on feedback!

Comment on lines 297 to 325
lat2, lat3, lat4, lat5 = compute_grid_edge_midpoint_latitude_components(lon, lat)
slice_3d = (slice(0, nx), slice(0, ny), slice(None))
slice_2d = (slice(0, nx), slice(0, ny))
# initialize temperature
t_mean = jablo_init.horizontally_averaged_temperature(eta)
pt[slice_3d] = cell_average_nine_components(
jablo_init.temperature,
[eta, eta_v, t_mean],
lat,
lat_agrid,
lat2,
lat3,
lat4,
lat5,
slice_2d,
)

# initialize surface geopotential
phis[slice_2d] = cell_average_nine_components(
jablo_init.surface_geopotential_perturbation,
[],
lat,
lat_agrid,
lat2,
lat3,
lat4,
lat5,
slice_2d,
)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sure thing. this duplicates the calculation of these latitude factors, but does make it more understandable. I'll implement the suggestion. Pulling the slicing up is a little tricker, as the resulting shapes of the 9 points being averaged are not by default the same, as they are shifted in different directions. I can do this if you prefer, but want to double check before I iterate on it.

Comment on lines 389 to 390
assert not adjust_dry_mass
assert not hydrostatic
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok sure. The fortran code has a lot of if conditions for different test cases, so it could be potentially confusing to understand and implement the other options. But I am not worried about this as much as I was for the dycore, since this is just an idealized initial state.

Comment on lines 389 to 390
assert not adjust_dry_mass
assert not hydrostatic
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm also already not following this p_var subroutine to the letter, since it was recomputing pressure variables in the exact same way they were already computed.

Comment on lines 389 to 390
assert not adjust_dry_mass
assert not hydrostatic
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do you mean just for p_var, or for the whole module?

islice, jslice, slice_3d, slice_2d = compute_slices(nx, ny)
# Slices with extra buffer points in the horizontal dimension
# to accomodate averaging over shifted calculations on the grid
isliceb, jsliceb, slice_3db, slice_2db = compute_slices(nx + 1, ny + 1)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sure thing, good point.

Comment on lines 23 to 24
eta_s = 1.0 # surface level
eta_t = 0.2 # tropopause
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sure thing

Comment on lines 15 to 30
# maximum windspeed amplitude - close to windspeed of zonal-mean time-mean
# jet stream in troposphere
u0 = 35.0 # From Table VI of DCMIP2016
# [lon, lat] of zonal wind perturbation centerpoint at 20E, 40N
pcen = [math.pi / 9.0, 2.0 * math.pi / 9.0] # From Table VI of DCMIP2016
u1 = 1.0
pt0 = 0.0
eta_0 = 0.252
eta_s = 1.0 # surface level
eta_t = 0.2 # tropopause
t_0 = 288.0
delta_t = 480000.0
lapse_rate = 0.005 # From Table VI of DCMIP2016
surface_pressure = 1.0e5 # units of (Pa), from Table VI of DCMIP2016
# NOTE RADIUS = 6.3712e6 in FV3 vs Jabowski paper 6.371229e6
R = constants.RADIUS / 10.0 # Perturbation radiusfor test case 13
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks! paid close attention to the paper on this one, which thankfully matches pretty well to what the fortran code does!

Comment on lines 366 to 377
def init_from_serialized_data(cls, serializer, grid, quantity_factory):
savepoint_in = serializer.get_savepoint("FVDynamics-In")[0]
translate_object = fv3core.testing.TranslateFVDynamics([grid])
input_data = translate_object.collect_input_data(serializer, savepoint_in)
# making just storages for the moment, revisit when making them all
# quantities (maybe use state_from_inputs)
translate_object._base.make_storage_data_input_vars(input_data)
# used for the translate test as inputs, but are generated by the
# MetricsTerms class and are not part of this data class
for delvar in ["ak", "bk", "ptop", "ks"]:
del input_data[delvar]
return cls(**input_data, quantity_factory=quantity_factory)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It can. I was thinking it might be useful for performance testing if we end up wanting to test non-baroclinic namelists before we implement reading initial state from disk. But someone wanting to do that can probably figure it out.

@@ -293,6 +296,12 @@ def make_grid_storage(self, pygrid):
if k in self.data:
self.make_composite_var_storage(k, self.data[k], shape)
del self.data[k]
for k in TranslateGrid.ee_vars:
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sure thing

lon,
lat,
lat_agrid,
grid_slice,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The type of this argument is non-intuitive and would be particularly helpful to type hint.

Suggested change
grid_slice,
grid_slice: Tuple[slice, slice],

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you do still want to pull grid_slice up one level, the way you would do it is by passing in lon and lat sliced on their own compute domains - that is, up to nx+1 and ny+1 since they're defined on cell corners. Then below, every access to lat or lon should be clipping off 1 point. pt6 becomes lat[:-1, :-1] while pt9 becomes lat[:-1, 1:].

fv3core/fv3core/initialization/baroclinic.py Outdated Show resolved Hide resolved
rheacangeo and others added 2 commits November 23, 2021 14:42
Co-authored-by: Jeremy McGibbon <jeremym@allenai.org>
Copy link
Collaborator

@mcgibbon mcgibbon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great! Looking forward to making use of this.

@rheacangeo rheacangeo enabled auto-merge (squash) November 23, 2021 23:01
@rheacangeo rheacangeo merged commit 8021a8e into main Nov 24, 2021
@rheacangeo rheacangeo deleted the feature/baroclinic-initialization branch November 24, 2021 00:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants