WIP e2e testing

[s-kennedy]

I finally got around to the end to end testing! 🎉

Dirk, I ended up going with Selenium instead of the Mocha+Puppeteer tests. I think for end to end tests we really need to be able to access the database and Selenium lets us do that easily. I think the mocha/chai test setup is still going to be useful when we get around to unit testing the React components though. I would maybe just get rid of Puppeteer.

I added Selenium to the docker-compose.yml file, but we only want to use that image for testing, and I'm not sure how to do that. What do you think?

P.s. Merry Christmas ☃️

[dirkcuys]

Hey @s-kennedy, happy new year and all that!

Thanks for the update! I think it makes sense to use selenium, it allows you to access the database using the Django ORM like you mention and it keeps the tests more inline with the existing tests.

I'll look at updating the .travis.yml to run selenium for the tests.

[s-kennedy]

Ok great thanks! I tried a couple different ways of updating the travis.yml file but I couldn't get it working, I always ran into this error:

urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='selenium', port=4444): Max retries exceeded with url: /wd/hub/session (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7faabb23f0f0>: Failed to establish a new connection: [Errno -2] Name does not resolve',))

Not sure if that's helpful or not, but I really hope we can get it working because I really liked working with selenium. Also we can add other browsers so we could run our tests on IE and Firefox as well!

[dirkcuys]

That error was cause because we're connecting to a remote Selenium session for the tests. I've update the TravisCI to run a docker container with the selenium server and it's working now.

The only remaining issue is that tests sometimes time out?

[s-kennedy]

Does it time out on the same test every time? Maybe it's missing a wait or something.

I was debugging by removing the --headless flag and using VNC viewer to see what's going on in the browser, maybe try that?

[dirkcuys]

@s-kennedy I've tried a thousand things, but I think I've narrowed down the problem. For some reason it seems like the tests doesn't consistently select a course? I suspect a wait until is needed somewhere?

[s-kennedy]

@dirkcuys ok, maybe it's moving to the next tab too quickly after the "Use this course" button is pressed? I'm not sure why that would matter because clicking the button just adds the course ID to the learning circle object, I don't think it should matter if the UI is updated or not. But you could put in a wait after the button is clicked for the "Remove selection" link to show up to verify the course is actually selected and see if that does the trick.

[s-kennedy]

was something like this already here? the page object is already there but it's not used anywhere else. Anyway this would check that the link to remove the course selection is there, to verify that the course has been selected.

self.wait.until(expected_conditions.presence_of_element_located(LearningCircleCreationPageLocators. REMOVE_COURSE_SELECTION_LINK))

[dirkcuys]

Seems like the problem is with the chrome driver :( SeleniumHQ/selenium#4075

[dirkcuys]

I tried working around this by sending the click via JavaScript:

course_select_button = self.wait.until(expected_conditions.element_to_be_clickable(LearningCircleCreationPageLocators.FIRST_COURSE_BUTTON))
self.driver.execute_script("return arguments[0].click();", course_select_button)

The linked issue says the issue is only with absolute positioned elements, but before I had issues with click on the tabs also.

Because a passing build doesn't mean the issue with inconsistent failures is sorted out, I'm hesitant to merge this PR.