Improve Confidence when Deploying

[philschatz]

Since many of us work remote, an async discussion might be easier.

Thoughts on the easiest way to improve confidence when deploying?

Some options (add more!):

• add a coveralls.io webhook
  • easy to drop in
• selenium scripts
  • utility methods would help QA folks
  • would better test how code works between components
  • downside: might require starting up tutor (too slow for travis?) or more mock JSON files (not too bad)
• Selenium + coveralls: see Selenium: add code coverage for selenium tests #924 for how-to

Here is the unit test code coverage as of @a68f082. By changing the reporter in test/config/karma-coverage.config.coffee#L12 (adding coverageReporter: { type: 'lcov', dir: './coverage'} I got the following report: https://openstax.github.io/tutor-js/coverage/

@darrenmason @jivewise @nathanstitt @pandafulmanda ?

[nathanstitt]

I like the approach you've setup in #924 (Selenium + coveralls)

How hard is it to spec network requests with Selenium? Some light googling suggests its possible but I've not personally done that before.

I'm think it would be great if we could test "When this input sequence is conducted, a http PUT matching XXX is issued" Selenium gives us the ability to craft the input sequences, but I'm not sure how to spec the network request.

[gregfitch]

Technically, it doesn't...by design. Hackishly you can form the same requests against the running selenium server.

[philschatz]

Yep, "recording" the requests would be a quick way to create the mock JSON for tests and could be easily updated. Ruby uses vcr/vcr (readme has a list of ports) and I found linkedin/sepia for NodeJS.

For browsers, I created sepia.js a while back to play back cassettes for octokat.js but with some of the recordo code it could record the requests too.

[philschatz]

From the discussion on selecting things via classes, data- attributes, or :nth-of-type(): from an accessibility angle, would it make sense to default to aria- attributes when selecting things in Selenium?

Of course, those attributes are probably not enough to do things but it would be easier than having entirely separate helpers that navigate through the UI.

Adding accessibility is one of the items for this semester so we'll probably start having tests for it too 😄

[philschatz]

I think we've done most of the items in this except for using fixtures files. Those could be added with something like https://github.com/philschatz/sepia.js (mentioned above) /cc @nathanstitt since he mentioned the VCR stuff