Tests in Browser

[raisedadead]

Context:

• We have been dreading this problem for years (see Challenge tests should be done in browser #4201). But it's time we do the tests in browser.

• Stuart (@bouncey) and Oliver (@ojeytonwilliams) have laid the initial foundation by integrating Cypress(see Add cypress.io for client end to end tests #37053). This was not available in our earlier days, and has been quite a popular tool for automation testing.

• Alongside Cypress (which uses puppeteer which in turn enables you to work with headless chrome) we can now do browser tests.

Goal (each requirement below can be a PR of its own)

• Create a test suite that can run against all challenges (only for certifications that we offer).
• Create and add suites for testing user workflows.
  • Sign up / Sign in
  • Email change / update
  • Changes to profile data
  • a11y tests and lighthouse tests?
• Create and add suites for testing navigations and layouts across devices.
• Update documentation about e2e tests
• ... add more.

Nice to have:

• Challenge tests could be run only for changes on PRs and all tests when on master?

[ckujawa]

@raisedadead this sounds interesting and I'd love to help with it...I'm new to the project and still would likely need to pair up so I can learn my way around some of this stuff, but if you want the help just let me know :)

[raisedadead]

Hey Chris! Welcome. Hit us up the in the contributors chat room.

[Twaha-Rahman]

@raisedadead As far as I know, FCC uses Jest for testing. Writing tests for Cypress currently is a little bit annoying as we have write test in Mocha. But there's a proposal on adding Jest support for Cypress. IMHO, if it isn't really important/necessary, can we wait for the proposal to land and then write all the tests in Jest just like in all the other places of FCC? This way, all of the tests written in FCC will be written is one testing language.

(P.S.: The Jest team expressed their interest in the issue to work with Cypress to add support for Jest. See this message)

[ojeytonwilliams]

Based on this comment: cypress-io/cypress#281 (comment) Jest support is not currently being worked on. I think that means that we would be waiting indefinitely before we can use Jest.

[Twaha-Rahman]

@ojeytonwilliams So.... what's the approach then? Should we write tests in Mocha for now and change them in the future? Or stick with Mocha later on even though FCC uses Jest?

[ojeytonwilliams]

I'd say Mocha for now, since I don't see any alternative. If Cypress starts supporting Jest, then we can decide at that point if we want to switch.

[raisedadead]

We would still be using both in some form. Keep in mind that we would have a combination of both because, we would need both integration tests (also called unit tests), and user acceptance tests (also called e2e tests).

At the minute all the challenge tests are unit tests which are run at build time, but we would also want to be able to re-run the same tests in browser using our test runner, as e2e tests to identify broken functionalities like a real user would.

[Twaha-Rahman]

Ok.... I will look into Cypress when I have some time. Thanks for the quick response!

[Twaha-Rahman]

Create a test suite that can run against all challenges (only for certifications that we offer).

@raisedadead Are you referring to the only 5 projects that needs to be submitted to get the certs?

[Sembauke]

Hey,
Going to close this one as outdated.
Will open a new one once new tests can be added.