e2e testing

[thomasplevy]

Description

Adds e2e testing & a utility library to facilitate e2e testing in add-ons

Review the readme in the e2e testing directory.

Reorganizes the existing tests directory, moving php tests into there own directory to keep things organized between the different types of tests.

Types of changes

This does not introduce any code that will affect the end user, this only improves internal dev testing.

Checklist:

• My code has been tested.
• My code passes all existing automated tests.
• My code follows the LifterLMS Coding & Documentation Standards.

[thomasplevy]

7-3 nightly failed because I reran it with SSH to toy with how to add e2e testing to circleci.

[thomasplevy]

@eri-trabiccolo I'd like you to have a look at this mostly to see if the environment setup is clear.

I'm not sure if you've actually installed Docker locally (there's quite a bit of setup steps to get the workflow running) but in the near future we'll be running a lot of e2e tests in add-ons.

Docker isn't required for any of these tests as you could actually point the e2e tests towards any wp environment (even a live site in the cloud) with some environment variables. So as an example tests could be run against a local site "hosted" by vagrant/vbox.

However I'd like to see if you can follow along and get this basic workflow running and let me know if you have any feedback on the process.

Ideally if Docker and Node are both already installed it should just be a few command and minutes (mostly spent waiting for installation and compilation) to get the local environment running.

This isn't high priority!

[eri-trabiccolo]

@thomasplevy
I actually have docker installed and running although I haven't done anything with it...
not sure what I'm missing here, I've gone with:
npm install
npm run env install
npm run env start
npm run test
but I get this:

Test Suites: 4 failed, 1 passed, 5 total
Tests:       1 passed, 1 total
Snapshots:   0 total
Time:        19.421s

where the passed test is: PASS tests/e2e/tests/activate/setup-wizard.test.js (17.122s)
and the failed ones are of this kind:

 FAIL  tests/e2e/tests/activate/config-wp.test.js
  ● Test suite failed to run

    Cannot find module 'llms-e2e-test-utils' from 'config-wp.test.js'

      10 | const { visitAdminPage } = require( '@wordpress/e2e-test-utils' );
      11 | 
    > 12 | const { clickAndWait } = require( 'llms-e2e-test-utils' );
         |                          ^
      13 | 
      14 | 
      15 | describe( 'ConfigureWP', () => {

      at Resolver.resolveModule (../../node_modules/jest-resolve/build/index.js:259:17)
      at Object.<anonymous> (tests/activate/config-wp.test.js:12:26)

hints, resources, love?

[thomasplevy]

@eri-trabiccolo I changed the package from my personal npm to the lifterlms org so it's now @lifterlms/llms-e2e-test-utils. I'll have to update the tests accordingly, sorry.

[eri-trabiccolo]

@thomasplevy
ok, changed tests and packages.json accordingly, no "require" errors anymore <3.
builder.tests.js fails saying:

  ● Builder › should create a new course, title it, publish it, and load the course builder.

    TimeoutError: waiting for selector ".editor-post-publish-panel__header-published" failed: timeout 30000ms exceeded

  ● Builder › should create and save a new section.

    TimeoutError: Navigation timeout of 30000 ms exceeded

but I guess you were more interested on whether I could run e2e tests right!?

[thomasplevy]

yes.... these still need more work as I learn how puppeteer really works. I'm not sure why these are timing out for you though... Enough for now we'll keep digging into it

[eri-trabiccolo]

I'm not sure either, I just lunched the tests again and they passed.

[thomasplevy]

hey look at that programmer magic <3