Allow pa11y-ci to show failed actions & debug logs

[adams0619]

Expected Behavior

When running pa11y-ci with a sitemap or multiple urls, I would like to be able to determine which actions failed if a test fails or use browser logs from the tests run if the failure is unrelated to any actions. This would provide for a much easier debugging experience when dealing with 5+ urls.

Similar to how pa11y shows logs when enabled inside the config file. A simple flag or option can easily be used to enable logging for tests in pa11y-ci.

Below is an example of logging from pa11y which could be introduced in pa11y-ci:

Welcome to Pa11y

 > Running Pa11y on URL http://localhost:9090/
 > Debug: Launching Headless Chrome
 > Debug: Opening URL in Headless Chrome
 > Debug: Browser Console: You are running Vue in development mode.
Make sure to turn on production mode when deploying for production.
See more tips at https://vuejs.org/guide/deployment.html
 > Debug: Browser Console: TEMP: currentuser not set
 > Running actions
 > Debug: Running action: wait for url to be http://localhost:9090/
 > Debug:   ✔︎ action complete
 > Debug: Running action: wait for element #loader to be hidden

Note: This could be tied to Issue #15 for easier CI support for logging.

Current Behavior

Currently logs are deleted regardless of any options passed in the config for pa11y-ci. The lines below in pa11y-ci outline where the logging option in the config file is deleted inside pa11y-ci.js:

pa11y-ci/lib/pa11y-ci.js

Lines 47 to 51 in 8dd46ed

	// We delete options.log because we don't want it to
	// get passed into Pa11y – we don't want super verbose
	// logs from it
	const log = options.log;
	delete options.log;

When a test does fail because of an action or something else on the page, a generic error is put in place, this makes it very difficult to determine which action failed if there are multiple actions for a particular test or if the issue was unrelated to the actions and had something to do with the test page itself.

Running Pa11y on 1 URLs:
 > http://localhost:9090/ - Failed to run

Errors in http://localhost:9090/:

 • Error: waiting for function failed: timeout 30000ms exceeded

Impact

Lately while reviewing accessibility violationss found by pa11y-ci it has become increasing difficult to debug issues if a particular test has multiple actions or if the error is related to something on the page instead of an action. I've found myself using pa11y on numerous occasions to get better debug output, but this is both tedious & time consuming when testing multiple urls.

[sutt0n]

The option to allow verbose logs, or more debugging in general, would be very beneficial in case there's something wrong on the backend of things.

---

Edit (case scenario): I forked this PR and could not understand why some URLs weren't working in CI versus locally. I pointed my package.json to my fork that did not delete the options.log line and it logged out a CORS problem with the corresponding URLs.

Implementing the option for allowing verbosity while running pa11y-ci would have been beneficial and saved me hours.