Test data sets

[bartnijssen]

We need to develop a test data sets that can be used to ensure that changes to VIC do what they are supposed to do and do not have any unintended side consequences. I will open a series of issues for this, since we'll need different test datasets for different use cases. This issue will serve as the parent issue that can be closed when all the test data sets have been developed. Note that these test cases may need to be updated as we update VIC.

Eventual goal: Something that would allow us to run a series of automated tests after source code changes have been made to VIC (somewhat analogous to travis for compilation).

Note that:

• Simplifying the number of options that are used within VIC will facilitate this testing configuration.
• Small tests that test specific behaviors would be ideal (so they can help in isolating problems).

Test cases:

• no change in output format or calculated values:
  • Comparison with prior VIC output
• change in output format but no change in calculated values:
  • Comparison with prior VIC output
• no change in output format but change in calculated values:
  • Comparison with prior VIC output
  • Comparison with observations

[bartnijssen]

Moved to 5.0 - will need test datasets that can be run with 4.2, but that can be done after the the 4.2 code base is released.

[jhamman]

@bartnijssen and @tbohn,

I've been working on pulling together a bunch of test datasets for this issue. I'm looking for suggestions on datasets for VIC simulations that can be compared to observations. I know there have been projects in the past that have compared VIC output to:

• flux tower observations,
• SNOTEL observations (Chiyu possibly),
• soil temperature observations, and
• streamflow observations.

I'd like to collect established cases of VIC setups and observations to be included in the test datasets. If either of you know where these are, please let me know (personal communication is fine).

[jhamman]

I should also add that we need to think of how we can test against observations for some of the lesser used options in VIC, e.g.:

• Blowing snow,
• Precipitation correction,
• Lakes and wetlands, and
• Carbon cycle.

If we can't test these options, we may want to rethink whether they belong in the released version of the model.

[tbohn]

I have setups for about 60 Ameriflux tower sites. I also have some setups
for some EC towers in Russia, which I used to evaluate the carbon cycle
features.

For lakes and wetlands, this is a harder thing to test because there are so
many parameters - bathymetry is needed. However, I have setups for a) Lake
Chad (comparison with lake level gauges), b) West Siberia (comparison with
satellite surface water observations over large scales), c) multi-year site
observations of water table depth, soil temperature, and some fluxes at a
specific bog in West Siberia.

On Fri, Nov 14, 2014 at 12:16 PM, Joe Hamman notifications@github.com
wrote:

I should also add that we need to think of how we can test against
observations for some of the lesser used options in VIC, e.g.:

• Blowing snow,
• Precipitation correction,
• Lakes and wetlands, and
• Carbon cycle.

If we can't test these options, we may want to rethink whether they belong
in the released version of the model.

—
Reply to this email directly or view it on GitHub
#79 (comment).

[jhamman]

I'm getting close to being done with this work for this issue. How do we feel about putting these datasets in new github repo (https://github.com/UW-Hydro/VIC_test_data)? Total size of the datasets is about 1GB and I'd like to use git-lfs to store the data on github.

I currently have four datasets formatted into netCDF4 ready for the test module:

• Stehekin sample dataset
• Snotel (from Chiyu Lin)
• Ameriflux towers
• Rosemount sample dataset

[tbohn]

Sounds good to me.

On Thu, Jul 23, 2015 at 11:12 AM, Joe Hamman notifications@github.com
wrote:

I'm getting close to being done with this work for this issue. How do we
feel about putting these datasets in new github repo (
https://github.com/UW-Hydro/VIC_test_data)? Total size of the datasets is
about 1GB and I'd like to use git-lfs https://git-lfs.github.com/ to
store the data on github.

I currently have four datasets formatted into netCDF4 ready for the test
module:

• Stehekin sample dataset
• Snotel (from Chiyu Lin)
• Ameriflux towers
• Rosemount sample dataset

—
Reply to this email directly or view it on GitHub
#79 (comment).

[bartnijssen]

Same here - git-lfs may also be a good place for the SUMMA test data. I wasn't aware that they already started this (I know it was in the works)

[jhamman]

partially completed by #387

follow up PRs will add science and release test datasets.

[jhamman]

@dgergel's science testing PR has just been merged (#544). With that done, we just have to put all the test datasets together in one place. I'll work with @dgergel to pull this together. For now, let's just create a directory on our groups FTP. We'll want to put the science and release data in there so we can all start using it.

[jhamman]

This is done. I'll close this for now pointing to this FTP path: ftp://ftp.hydro.washington.edu/pub/gergel/VIC5_test_data/.