Testing

[jstayton]

• The preferred testing tool(s) should be stated in the language/framework-specific conventions.
• It should be easy (one command) to run the tests.
• The tests should be run as part of continuous integration through CircleCI.
• Code coverage should be setup (like SimpleCov for Ruby) and reported after the tests are run. Not sure if we should shoot for a specific code coverage %?
• Tests should be written in BDD style syntax.

What else?

[shanebonham]

Not sure if we should shoot for a specific code coverage %?

Seems like we should, otherwise what's the point of the coverage report? Simplecov shows you a red/yellow/green based on how well covered a given file is. I don't know if that's just arbitrary or based on something. I'm not sure why we wouldn't shoot for 100%, but I don't know how realistic that actually is.

Tests should be written in BDD style syntax.

Can you explain what you mean by this?

[jstayton]

Seems like we should, otherwise what's the point of the coverage report? Simplecov shows you a red/yellow/green based on how well covered a given file is. I don't know if that's just arbitrary or based on something. I'm not sure why we wouldn't shoot for 100%, but I don't know how realistic that actually is.

I'd say the point is to give the engineers an idea of how well covered the code is. Even if it's 10%, that's helpful to know.

But I do agree: a big point of that is making sure it increases or stays the same over time. I'm good with saying 100% coverage should be the goal.

Can you explain what you mean by this?

I probably could have worded it better. What I mean is, most tools support a specific test style. Like RSpec is specifically BDD. But some tools support multiple styles, like Buster.JS that ships with xUnit style and BDD style. We should either choose a BDD-specific tool, or one that supports the BDD syntax.

[jstayton]

To expound on the CircleCI convention...

1. Configuration should be done in a circle.yml file, rather than their web interface. At the very least, I think including the file and specifying the target machine language version is good. That way 1) it's clear to engineers that the project is setup on CircleCI, and 2) it prevents any issues when CircleCI updates the default language version.
2. The status badge should be displayed in the README.