Read more daily input #228
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Contributor
N1ckP3rsl3y
commented
Jan 18, 2023
•
N1ckP3rsl3y
commented
- Handles new weather-related input based on SOILWAT2's ability to accept more daily input values.
- Close Daily variables for atmospheric demand #229
added 9 commits
January 10, 2023 15:06
- Added a new constant, "MAX_INPUT_COLUMNS", from SOILWAT2 which holds the maximum possible input columns (14) to be read-in excluding the "DOY" input column
- "p_Year" now takes in new daily cloud cover, wind speed, relative humidity, shortwave radiation, and actual vapor pressure - rSOILWAT2 now transfers listed values above to SOILWAT2 through "allHist" - Updated "cYear" to hold more column head names - Instead of four column, `rSW2_setAllWeather()` now expects 15
- Class "swWeather" handles new flags from SOILWAT2 side: * Monthly value usage * Temperature (components), precipitation, cloud cover, wind speed, relative humidity, specific humidity, dewpoint temperature, actual vapor pressure, and shortwave radiation * Other component flags for wind speed and specific humidity - Added actual vapor pressure and shortwave radiation to "swMonthlyScalingParams" and within "onGet_SW_WTH_setup()" - Translate SOILWAT2 flags to rSOILWAT2 within `onGet_SW_WTH_setup()` - Translate rSOILWAT2 flags to SOILWAT2 within `onSet_SW_WTH_setup()` - Translate scaling factors of actual vapor pressure and shortwave radiation to and from SOILWAT2 and rSOILWAT2 in `onGet_SW_WTH_setup()` and `onSet_SW_WTH_setup()` Note: The program is currently not able to update test input due to the second example generation from `prepare_testInput_objects.R` resulting in a crash due to bad weather input
- `rSW2_setAllWeather()` uses monthly sky input values from the class "swCloud" to get user input monthly values to be interpolated - Mirrored possible calculations from `_read_weather_hist()/readAllWeather()` (SOILWAT2) to add functionality on the R side without other possible difficulties * Clarification: Interpolation and call to `actualVaporPressure1()` which are in `readAllWeather()` on the SOILWAT2 side, have been combined with other calculations within `rSW2_setAllWeather()`
- `onGet_SW_WTH_setup()`, `onSet_SW_WTH_setup()`, and class "swWeather" now hold six new flags which break temp2, hurs2, and windComp into their respective parts - `rSW2_setAllWeather()` translates some flags once they are gathered from the class "swWeather" instead of translating every time the flag is needed - `rSW2_setAllWeather()` tests for a range of input columns from 4 through MAX_INPUT_COLUMNS + 1 (15)
dschlaep
requested changes
Jan 18, 2023
There was a problem hiding this comment.
Nice progress!
I added some comments for specific lines. And here some general comments
-
We will need methods/functions that handle weather data in the previous format, e.g., by upgrading them to the new format on the fly; see
sw_upgrade() -
Some values in the weather data objects are currently not making much sense. For instance, for
Ex1_weather(contains nonsensical values for several of the new variables; and for some that are turned off by default)
> head(Ex1_weather[[1]]@data)
DOY Tmax_C Tmin_C PPT_cm cloudCov windSpeed windSpeed_east windSpeed_north
[1,] 1 -0.52 -15.67 0.22 66.48387 1.3 61 999
[2,] 2 -2.06 -7.86 0.72 66.80645 1.3 61 999
[3,] 3 -4.04 -14.47 0.22 67.12903 1.3 61 999
[4,] 4 -1.26 -9.20 0.55 67.45161 1.3 61 999
[5,] 5 0.65 -10.85 0.02 67.77419 1.3 61 999
[6,] 6 0.35 -18.72 0.00 68.09677 1.3 61 999
rel_H rel_H_max rel_H_min spec_H dewpointTemp_C actVP shortWR
[1,] 0.2024587 0 0 0 0 0 0
[2,] 0.2576992 0 0 0 0 0 0
[3,] 0.1848576 0 0 0 0 0 0
[4,] 0.2524647 0 0 0 0 0 0
[5,] 0.2549731 0 0 0 0 0 0
[6,] 0.1858799 0 0 0 0 0 0
- Create a new function `strip_weather_data_NA_columns()` to eliminate unused columns of the newest version of "swWeatherData" (15 columns) * If an object of this type were to contain only four useful columns of data, nine would be unused and then takes up more space than needed dbW_dataframe_to_weatherData(): - Removed check for input column names to be of length four to account for variability - Function sets column names based on ones sent in and are no longer hardcoded sw_upgrade(): - Now strips unnecessary columns of the input object through the use of `strip_weather_data_NA_columns()` swWeatherData(): - No longer directly copies column names from the created object * Common columns are now detected and set so no error is caused due to unequal amount of columns to be set
- Removed `strip_weather_data_NA_columns()` - Reverted `dbW_dataframe_to_weatherData()` to the previous version to only deal with four columns - Reverted `swWeatherData()` to directly get slots from the local instance of "swWeatherData" - `sw_upgrade()` no longer calls `strip_weather_data_NA_columns()`
- SOILWAT2 contains a struct of the name "allHist" which contains: * max/min/avg temperature and precipitation * cloud cover, wind speed, relative humidity, actual vapor pressure, and shortwave radiation - "swWeatherData" now contains all but average temperature (since it's calculated when data is transferred to SOILWAT2) - "swWeatherData" keeps "DOY"
Codecov Report
@@ Coverage Diff @@
## release/devel_v6.0.0 #228 +/- ##
========================================================
+ Coverage 57.58% 58.49% +0.91%
========================================================
Files 42 42
Lines 7719 8038 +319
========================================================
+ Hits 4445 4702 +257
- Misses 3274 3336 +62
|
including - `solar_radiation()` now handles observed radiation values and missing cloud cover
- use consistent column names - column 7 is for actual vapor pressure and column 8 for shortwave radiation
- class "swWeather" gains `desc_rsds` (description of radiation inputs) and `dailyInputFlags` (logical vector describing which daily weather input variables are active in the associated weather data object) - class "swWeatherData" holds now up to 14 daily weather input variables: "Tmax_C", "Tmin_C", "PPT_cm", "cloudCov_pct", "windSpeed_mPERs", "windSpeed_east_mPERs", "windSpeed_north_mPERs", "rHavg_pct", "rHmax_pct", "rHmin_pct", "specHavg_pct", "Tdewpoint_C", "actVP_kPa", "shortWR" - update C functions that copy values between R and C functionality
- `getWeatherData_folders()` can read new daily weather inputs either via R code (that leaves values and variables as is) or via C code (connecting to SOILWAT2 which processes variables as needed for a simulation run) - new C level `rSW2_readAllWeatherFromDisk()` to use SOIILWAT2 to read (and process) daily weather inputs from disk
- weather data are a list of elements of the class "swWeatherData" -- we cannot use S4 method `sw_upgrade()` --> `upgrade_weatherHistory()` - monthly scaling parameters of class "swMonthlyScalingParams" are contained within "swWeather" -- we cannot use its S4 method `sw_upgrade()` --> `sw_upgrade_MonthlyScalingParams()` - `dbW_check_weatherData()` now checks for valid "swWeatherData" objects carefully and via new argument `check_all` checks carefully for slot "year" in case it is missing
new: - `calc_dailyInputFlags()` estimates which weather variables are active (here, those that are not only missing) - `is_missing_weather()` produces a logical matrix checking for all possible values that could indicate missing weather values - `weatherGenerator_dataColumns()` lists weather variables that the Markov weather generator implements - `weather_dataAggFun()` lists functions that are used to temporally summarize each weather variable (e.g., `mean()` for temperature, `sum()` for precipitation amount) updated: - `dbW_convert_to_GregorianYears()`: argument `name_data` is now by default the result of `weather_dataColumns()` - `dbW_dataframe_to_weatherData()`: argument `weatherDF_dataColumns` is now by default the result of `weather_dataColumns()` (plus "DOY") - `dbW_weatherData_to_monthly()` and `dbW_dataframe_aggregate()`: new argument `funs` with default the result of `weather_dataAggFun()`plus "DOY") - `dbW_weather_to_SOILWATfiles()`: argument `weatherDF_dataColumns` is now by default the result of `weather_dataColumns()` (plus "DOY") - `dbW_weatherData_round()`: argument `weatherDF_dataColumns` is now by default the result of `weather_dataColumns()` - `dbW_check_weatherData()`: now also checks if each "swWeatherData" object is valid and gains argument `check_all` to carefully check for year slot and correct number of days - `dbW_generateWeather()`: now handles correctly new daily weather variables (but only those in `weatherGenerator_dataColumns()` are implemented by the weather generator)
- update scripts that prepare package and test data: they now make sure that weather history objects pass checks
- comparisons of weather data now makes mostly sense for those variables that don't have the potential to be calculated/processed by SOILWAT2
- check that previous versions can be successfully upgraded so that they validity and other checks
- tests for inputs with "standard" variables, and with variable sets from Daymet, gridMET, and MACA
- Vapor pressure and radiation variables from Daymet are now supported inputs
included changes: - Fix `mvnorm()`: avoid NaN as tmin output
- set seed for reproducible output
dschlaep
approved these changes
Jun 3, 2023
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.