-
Notifications
You must be signed in to change notification settings - Fork 17
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Expand & automate testing system #31
Comments
Hi,
One thing that would be useful in (at least some) test cases would be
actual watershed measurement data. It is one thing to try to validate HSP2
against HSPF - but in the longer view HSP2 should be validated against
actual data too. HSPF test 10 didn't come with this. I am not sure if the
colleg test case might have actual data archived somewhere, but I am not
aware of any.
Bob
…On Sun, Apr 19, 2020 at 1:40 PM Anthony Aufdenkampe < ***@***.***> wrote:
As we discussed for our RESPEC-LimnoTech Collaborative Work Plan
<https://docs.google.com/document/d/1jgH_-ly9_VcW_fYRGZOQqv12LY1oALd-tQMLFgOCMYQ/edit#bookmark=id.vhiwqmnyt24j>
during our workshop (March 24-25, 2020), expanding and automating the
testing of HSP2 vs. HSPF is an immediate priority.
Our objective for testing is to ensure that HSP2 provides the same results
as HSPF for:
- all HSP2 releases,
- several relevant operating systems and software environments, and
- a selected group of watershed models that have been calibrated and
examined for real-world water management applications and that represent a
range of watershed properties.
We decided that:
- HSPF “reference” model runs should be added to repo and considered
static/stable
- HSP2 outputs will continually evolve, expanding as new process
modules are implemented
- Comparisons will be point-by-point for major output time series, to
"byte-precision" of about 3 significant figures to allow for rounding
errors
- We can't do traditional unit testing of individual routines
because HSPF doesn't save that data.
RESPEC has two test models to contribute:
- Test10
- Calleg
LimnoTech will add additional models:
- We selected 2 watersheds that we’ve recently modeled in HSPF, and
selected a single sub-watershed (to simplify running)
- Grant River, MI. Relatively simple
- Zumbro River, MN. More complicated. Full water quality suite.
- Hydrological Response Unit (HRU) testing
- 5-10 micro watersheds (1 HRU + a few stream reaches)
Let's use this issue to track progress on all the smaller tasks required
to complete this.
We have already added some reference models and testing code with 49c71f3
<49c71f3>,
60378a7
<60378a7>,
and ***@***.***
<LimnoTech@130bef2>
.
cc: @rheaphy <https://github.com/rheaphy>, @PaulDudaRESPEC
<https://github.com/PaulDudaRESPEC>, @steveskrip
<https://github.com/steveskrip>, @ptomasula <https://github.com/ptomasula>
,
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#31>, or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AFML2ENGI7HBUBDDEXMOBTDRNNHRHANCNFSM4ML5PAFA>
.
|
Bob
I don’t necessarily disagree but HSPF has been around and used to develop applications that are calibrated to data for over 40 years and determined to be a useful tool through that experience. Thus, I somewhat believe constraining the scope to validate HSP2 can reproduce HSPF results has merit.
JASON LOVE, P.E.
Senior Vice President
605.394.6512 office // 605.484.2380 cell
|
Here's a suggestion from @PaulDudaRESPEC for potential tests that we can adapt from the HSPF testing list:
|
For our records... In Bob's May 14 "HSP2 status update" email, he shares the following:
In Bob's May 14 "HSP2 HTML image of calleg test run" email, he wrote:
I added |
I just discovered that Bob emailed @steveskrip the file on May 28 in "Re: calleg HSPF results" thread. I just committed it with LimnoTech@f112f9e. |
So far, the test is implemented for testing conversion codes. I put an instruction as commit in this thread. Yes, it is currently functional in conversion testing. It is intended to be a full suite of testing protocols. |
Merge from respec/HSPsquared-develop Branch on 3/5/2021
PR #61 includes:
|
Although this is a long-term goal, we have sufficiently established a performant and flexible testing system that this issue can be closed with release 0.9.3. We'll open new issues for specific additional enhancements to the testing system. |
As we discussed for our RESPEC-LimnoTech Collaborative Work Plan during our workshop (March 24-25, 2020), expanding and automating the testing of HSP2 vs. HSPF is an immediate priority.
Our objective for testing is to ensure that HSP2 provides the same results as HSPF for:
We decided that:
RESPEC has two test models to contribute:
LimnoTech will add additional models:
Let's use this issue to track progress on all the smaller tasks required to complete this.
We have already added some reference models and testing code with 49c71f3, 60378a7, and LimnoTech@130bef2.
cc: @rheaphy, @PaulDudaRESPEC, @steveskrip, @ptomasula,
The text was updated successfully, but these errors were encountered: