Improved reliability and some initial ci configuration (closes: #10, #17, #20, #25, #27, #28, #30)

[kristofferlind]

Overview of changes:

• bail feature
• glob pattern
• fix xvfb race conditions with staggered start
• replace logbased communication with filebased communication as a workaround for intermittent missing test results
• add check to make sure no results are missing
• removed reporting feature of orchestrator, now just pipes output to process
• utilize cypress-multi-reporter so that results orchestration is always on
• added some rudimentary CI

related issues:

• parallel reporting #30 was already solved, now reporter can also be passed in while still summing up results
• Non 0 output for failed tests? (Usage in CI) #28 intermittent missing results should be fixed and fails run on missing results
• Create a test report in html format #25 also already solved, just pass in the wanted reporter and now it should still be summing results and exit correctly
• [Feature request] Support folder structure #20 was already working, but now also has the glob feature mentioned
• empty output and empty parallel-weights.json when tests complete #17 orchestrator reporter was skipped when custom reporter was passed in, now it's always included
• thrown errors on hidden files e.g. .DS_Store #10 glob pattern added, supply better pattern to solve issue

#27 might also be related, the issue I saw with intermittent missing results was that the event fired correctly in the reporter, but was never picked up when reading the logs, switch to filebased communication should hopefully make it a bit more robust

[jtabalujan2]

Hi! I'm trying out this library facing issue #17. Would it be possible to merge this? I'd love to try it out and see if it fixes my issue.

Thanks!

[amcinerneynewf]

Would love to see this merged. issue #28 is a pretty big blocker for use of this package for our CI pipeline

[tnicola]

@kristofferlind Thanks a lot for all the cleanup, feature implementation, CI and bug fixes 👏