Create standards for test coverage

[jdonges]

Automated tests could make models more reusuable, accessible, interoperable, generally increase software quality. Consider to create standards for test coverage and best practices?

[platipodium]

This is probably more general: do we want to worry about "good software practice" ? There are lots of other consortia talking about this, so maybe we should be more specific on what OMF does beyond this. @cmbarton

[alee]

This was also raised in #102 and IIRC our general consensus was to leave this to organizations like Better Scientific Software

That said if we do start to develop training modules ala #185 we should definitely include some concrete guidance on how to write good tests as it is not always easy to do.

Perhaps a statement in the ideal standards for reusability along the lines of "well-designed automated test suites with good coverage (80%+?) give confidence to potential users of a computational model that care has been taken to ensure it is correct, and that future changes to the model will not break it."

I expect that if we ever do get to a future where scientists contribute to libraries of model components covering our earth & social systems, they would require solid test coverage and integration tests to ensure that things don't break when upstream components change as our knowledge of these systems improves, we encounter the unknown unknowns, or bugs are discovered.

[jdonges]

I agree that a statement on good software standards would be good, also specifically regarding to test coverage etc, but then referring this to other organizations that are taking care of these standards. OMF does not have to create its own good software standards, but should refer to those from trusted organisations.