Read from netcdf #25
Merged
Read from netcdf #25
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Tested against Colleen's test_locs3.m matlab script
instead of filling a yaml file with printed values, matlab can write a netcdf file with output so I updated the matlab-comparison notebook to read this output when doing the comparison.
For columns deeper than cutoff, want t_frac_pelagic = 0, not 1
Formula fixed in 32a71bc is repeated in test_process.py...
combined forcing and biomass into the same netcdf file, so needed to clean up the matlab-comparison notebook. Also did a first pass at low-hanging fruit for performance (adding .data to lots of xarray objects) to get 6x speed-up.
Accidentally appended .data to a non-xarray object but didn't execute that line of code in my driver; pytest caught it.
Closed
Modified matlab code to generate forcing with units "days since 0001-01-01 00:00:00", so xarray converts it to cftime.DatetimeNoLeap object. This required modifying the driver to use these same units for time, which then required updating one of the tests in the driver as well as the time dimension in the idealized forcing testcase.
New test initializes two instances of the idealized test case, one starting on 0001-01-01 and one starting on 0002-01-01, runs both for a single time step, and expects biomass to be identical between the two. Currently this test fails, because the forcing is only defined for year 1 and using xr.DataArray.interp() in year 2 returns a nan which then propogates throughout the biomass array. Proper implementation of an option for cyclic interpolation should allow this test to pass.
Also cleaned up some numpy warnings that were coming about when fish.size_class_bnds_ratio ** (1.0 - (death / somatic_growth_potential)) was causing double-precision overflow (and a similar divide-by-0 error in the matlab-comparison notebook) Lastly, realized that I needed the nc-time-axis package for making plots now that we are using cftime objects
|
As of 05d14d0, the python code offers a mechanism for cyclic forcing (and even tests it as part of the If putting off the improvements to forcing can wait, the last task remaining before opening up this PR will be moving the infrastructure for reading from netCDF out of |
I renamed simulate_testcase() -> config_testcase(), and also copied the function to config_from_netcdf(). Then I removed support for "from_netcdf" as options in config_testcase(), and moved the functions in testcase/from_netcdf.py into driver.py. There's still some cleanup to do in driver.py, though.
Added start_date to docstring for both config_testcase() and config_from_netcdf(); also added tuple to list of valid types for start_date in the simulation constructor.
config_testcase() will determine whether to set cyclic_forcing=True based on which forcing profile is being used; config_from_netcdf() takes an argument (default is False). Also added two new tests that run a single day, and needed to shoehorn in a way to make the forcing actually be cyclic so this commit contains some kludgy code that will be cleaned up in coming commits.
More descriptive name for the class
Note that there is a namespace clash between offline_driver (the class) and offline_driver (the module); the only reason we import the module is to test the _read_settings() routine so I import the module as offline_driver_mod and use that to access _read_settings.
Moved the code to make forcing Dataset cyclic into utils/cyclic_forcing.
|
Thanks @mnlevy1981, this looks great. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Add support for reading forcing / grid information from a netCDF file. This implementation adds
feisty/testcase/from_netcdf.py, though I think @matt-long suggested putting this code in the driver (or maybe incore/?) instead. This is a draft because there are still several updates to make:timeto run from 0 to 364, so if the forcing file begins withtime=1then the interpolation results innans.nanstarting attime=365