Skip to content
Permalink
Branch: master
Find file Copy path
Find file Copy path
Fetching contributors…
Cannot retrieve contributors at this time
270 lines (176 sloc) 14 KB

Fast and Furious E2E: Using Testcafe for Easy End-to-End Testing

Building automated tests is the "Liver and Onions" of the programming world. Everyone agrees that it's good for you, but no one really wants to get into it. When anyone starts, many questions quickly come to mind: "What testing framework should I use in a chaotic world, and will it still be valid, years from now", "what are the things I should be testing", and "how much time do I really have to set these things up" are three avenues of questioning that quickly become familiar to the developer world.

If you find yourself there, then I have good news for you: There is a testing framework for E2E that has legs, that is easy to set up, and is easy to execute.

And we're going to build some tests. Once we gloss over the types of testing, end where E2E sits, and where Testcafe came from.

Understanding Testing

If you're new to automated testing, you should take a moment to learn the differences between the types of test which you would run. The four distinct categories of tests are "Static Testing', 'Unit Testing', 'Integration Testing', and 'End to End Testing'. The first type, static, deals with errors a programmer may make while typing the code, such as syntax or patterns which linters can observe as bad. Unit Testing is the next step up, where you are looking at a single function in isolation. Integration testing looks at many of these functions when together, and finally, End-to-end (E2E) looks at what results from specific interactions. If you want to learn more about this, then I suggest reading into this article by Kent C Dodds, who brilliantly summarizes this topic.

What is Testcafe

Testcafe Logo

Testcafe is a fully-integrated E2E testing library written for NodeJS (Node). It was borne from the difficulty of building testing E2E platforms which relied on other technologies, so that teams can focus chiefly on the tests, as opposed to the framework which runs the tests. Testcafe comes from Developer Express, a services shop that specializes in dashboards and other user interface components for the Microsoft proprietary ecosystem.

There are many extensions available for this testing suite including for browserstack, and it also offers a UI-based testing tool for automated E2E tests at cost. Because it operates as a standalone E2E entity, it doesn't need to integrate with modern Javascript frameworks to test many things. But for special cases, there is a rich plugin ecosystem that allows users to integrate if this is something that's really needed.

Simply-put, where all the other testing suites require too many steps, Testcafe is easy. Download the library, set up the tests, run. You don't have to assemble the vacuum. You just have to decide what you're going to vacuum, how to turn it on, then point!

lazy-corgi

Building E2E Tests

If you are stuck, this link contains stepped solutions for each of the sections of this tutorial.

In this tutorial, we're going to build fictional E2E tests for Leaflet, and how Testcafe can be used to capture changes to DOM elements, as well as network request. To follow this tutorial, you're going to require the following:

  1. A text editor. My preference is VSCode.
  2. A command line terminal. This tutorial is writen for a Mac or Linux machine.
  3. Git. It comes with Linux or Mac, but Windows users can find it here
  4. NodeJS, at least at version 8.15 (lts/carbon).

This list makes up a common toolsuite for developers. If you're building applications in React, Ember, or Vue, there's a good chance you already have all of these things.

Start the project

To start things, you're going to want to open your preferred terminal, and start a new directory:

  mkdir e2e-sample && cd e2e-sample
  git init
  touch README.md && echo "# End to End Testing with Testcafe" > README.md
  npm init

This will then start up Node's CLI process for building a new application. Run through all of the questions, and once finished, add a .gitignore file to ensure we don't commit the wrong things with the following:

  wget https://raw.githubusercontent.com/github/gitignore/master/Node.gitignore
  mv Node.gitignore .gitignore

This is a good time to commit.

  git add . & git commit -m "🎉 Basic repo" 

Install the Dependencies (branch)

In order to run Testcafe, all you need is something capable of serving HTML from static files, and Testcafe itself. The following are all we need to do our thing!

  npm i -D testcafe http-server

This installs the dependencies as "dev-dependencies". This means that these tools are part of the development environment, but will not be built as part of the production environment. Once you've installed, this is a decent time to commit again.

  git add .
  git commit -m "➕ Testcafe and http-server"

Build the Sandbox (branch)

Because we're testing a Leaflet, a Javascript library, as opposed to a client application, this means that we need to construct a sandbox for the purposes of testing. To start the sandbox, we're going to want to build a new directory, add some HTML and JS that uses the library, then add a NPM command which serves the sandbox:

  mkdir sandbox && touch ./sandbox/index.html ./sandbox/index.js

And from inside your text editor, add the HTML and JS that makes up the sandbox, which we will gloss over as this is a testing tutorial. You can find it in the following gist: Link

Finally, modify the package.json file to run the sandbox with http-server. Add the following line into the "scripts" object:

"sandbox": "http-server ./sandbox -p 3001"

With this line in your package.json file, you should be able to run npm run sandbox in the terminal window. sandbox

If the server starts, you should be able to go to http://localhost:3001. There, you should see something like this:

sandbox

This is a sandbox that tests the leaflet library. It simulates a pan to a random coordinate, a random zoom, changing the layer, and adding a marker to the centre of the map at the last pan.

Now is a good time to commit.

  git add . && git commit -m "✨  Functional sandbox in leaflet"

Create Your First Test (branch)

At this point, we have a testing toolchain, and we have a sandbox environemnt to run those tests. The next step here is to build the tests! The first test ensures that we have, in fact, a working E2E environemnt. We're going to add the file, we're going to add a script, and then we're going to write the test!

  mkdir tests && touch ./tests/1-map-render.js

Now, we're going to modify our lone "test" script in our package.json file to run all files in the test directory:

  "test": "testcafe chrome ./tests/** --app \"http-server ./sandbox -p 3001 -s\"",

This gets us the test command. In lieu of chrome, Testcafe can also test against other browsers. Browsers such as "chrome:headless", "firefox", "firefox:headless", "safari", "ie", "edge" will all work so long as the browser is installed on your computer. Finally, we're going to write the tests. In the new JS file, we first want to declare the fixture, or the thing we wish to test against. Add the following to your test file:

  fixture `Leaflet Map - Testing Render`
    .page `http://localhost:3001`;

This indicates the name of the fixture, and where the mapping library can load it from. With this, we can add the test:

  test('The testing environment works', async t => {});

This is a boilerplate test. Using this, we can run the command npm run test in the terminal. If successful, we should see the following:

test-1

This proves that we have a functional E2E environemnt, and we can go forth and build real tests! At this point, it is a good time to commit:

  git add . && git commit -m "✅  My first test"

Now we can add more tests!

Test DOM Elements (branch)

Now that we have a testing environment, we can start to test all the things! Here, we're going to test whether a marker appears when we press the marker appearance button.

The first step is to have Testcafe press a button. In the sandbox, I've given all the buttons on the right hand side unique ID's for this very purpose! First, to try yourself, we're going to need to import the Selector object from Testcafe. Write the following at the top of the JS file:

  import { Selector } from 'testcafe';

Now add the following code to the bottom:

const addMarker = Selector('#addMarker');

test('The marker is visible', async t => {
  await t
  .click(addMarker)
  .wait(500)
})

Now try running npm run test. If successful, you should see the following for half a second:

test-sandbox-leaflet

Notice that you can see a mouse cursor clicking on the DOM element #addMarker. We now have made an action, but we need to go further and ensure that the marker populates, and that it comes from the correct source. To verify whether it populates, we can write the following:

test('The marker is visible', async t => {
  await t
  .click(addMarker)
  .expect(Selector('img.leaflet-marker.icon'))
  .ok('The marker must exist')

})

This is our first expectation. We're expecting the selector to exist, and not return null. If it isn't ok, then we will indicate to the tester that this fails. Now we want to add another test that determines whether the marker populates from the correct source:

test('The marker image pulls from the correct source', async t => {
  await t
    .click(addMarker)
    .expect(Selector('.leaflet-marker-icon')
    .withAttribute('src', 'https://cdnjs.cloudflare.com/ajax/libs/leaflet/1.0.0/images/marker-icon.png'))
    .ok('The marker must come from the proper source')
})

Run this test. Once you have 3/3 working tests, this would be an excellent time to commit before we move to the final section: Network Requests.

  git add .
  git commit -m "✅ Test marker properties"

Testing Network Requests (branch)

Because this introduction also seeks to test the Leaflet JS library, it would be remiss if we didn't test for outgoing network requests on certain events. We want to know if leaflet loads the desired tiles on load, and we want to know whether it loads a different set of tiles when we load a second layer. Let's start by creating a new test file, and adding some code:

  touch ./tests/2-tile-loading.js

For this part, we want to import the request logger object from Testcafe, with a fixture and a selector. Start your file with the following:

import { RequestLogger, Selector } from 'testcafe';


fixture`Test network requests for leaflet`
  .page`http://localhost:3001`;
  
  const addCycleLayer = Selector('#addCycleLayer');

We have the start, and we now have a fixture. Now we're going to build two request logger objects:

const osmLogger = RequestLogger(/\.tile\.openstreetmap\.org/, {
  logRequestHeaders: true,
});


const cycleMapLogger = RequestLogger(/tile\.opencyclemap\.org/, {
  logRequestHeaders: true,
});

These are objects which capture outgoing requests, and use regular expressions (regex) to compare strings to patterns. Finally, we use the first logger to build a test:

test.requestHooks(osmLogger)('Test outgoing osm requests on map load...', async t => {
  await t
    .wait(200)
    .expect(osmLogger.count(() => true))
    .gt(5, 'Must request more than 5 tiles from OSM')
})

Because we're monitoring network requests as opposed to DOM elements, the syntax changes. We're using the requestHooks higher-order function, as opposed to the vanilla test function for the testing, and we're injecting a logger which monitors network requests that follow a specific pattern. In this test, we want to ensure that at least 5 tiles are being pulled from the correct serve.

Let's now test what happens when we load tiles from a different source. Add the following to the end of the file:

test.requestHooks(cycleMapLogger)('Test outgoing opencycle requests on layer load', async t => {
  await t
  .click(addCycleLayer)
  .wait(200)
  .expect(cycleMapLogger.count(() => true))
  .gt(5, 'Must request more than 5 tiles from OpenCycleMap')
})

With this, if you're able to get 5/5 tests working, then you've succeeded in building an End-to-End testing environment for a client library using a custom sandbox. This is one of the tools which we use to ensure that we're tracking your web map use, and that we don't degrade existing features accidently when we improve and iterate. Congratulations!

happycorgi

You can’t perform that action at this time.