Skip to content

Testing

Dave Lockhart edited this page Feb 19, 2019 · 18 revisions

This article is all about testing strategies and setting up automated tests for your web components.

test tubes

Topics:

Linting

Linting typically involves doing an analysis of your code that looks for syntax errors, missing files, non-adherence to our code format rules and other non-functional code health checks.

lint brush

For web components, there are two types of linting we care about:

polymer lint

Polymer CLI's polymer lint command checks for various lint errors that are specific to Polymer elements.

It's important to tell the linter what type of element you have (Polymer 1.x/2.x hybrid, Polymer 2 or Polymer 3). The best way to do this is in the polymer.json file:

{
  "lint": {
    "rules": ["polymer-3"]
  }
}

Possible values are: polymer-3, polymer-2, polymer-2-hybrid

By default, polymer lint will lint all the code in the project, however it's been known to travel deep into node_modules and error out on dependencies that you don't care about. If this happens, you can tell it explicitly which file(s) to lint:

polymer lint -i file1.js file2.js file3.js

Put this command into an NPM script in your package.json:

{
  "scripts": {
    "lint:wc": "polymer lint -i file1.js file2.js file3.js"
  }
}

More: Polymer Project: Polymer CLI Commands -> Lint

eslint

ESLint is a popular JavaScript linting library with a great plugin ecosystem. We bring in the eslint-plugin-html plugin to extend its JavaScript linting to the HTML files which contain our web components.

eslint logo

D2L also has our own configurations defined for different types of projects at: eslint-config-brightspace.

To use eslint in your web component, include the following NPM devDependencies:

npm install --save-dev eslint
npm install --save-dev eslint-plugin-html
npm install --save-dev eslint-config-brightspace

Then create an .eslintrc.json file in the root of your project which extends our Polymer configuration:

{
  "extends": "brightspace/polymer-3-config"
}

Inside your test directory, use the WCT configuration to make it aware of the Mocha globals:

{
  "extends": "brightspace/wct-polymer-3-config"
}

Now run the eslint command, passing it a list of files to lint. Put this into an NPM script in your package.json:

{
  "scripts": {
    "lint:html": "eslint *.js demo/*.html test/*.html"
  }
}

Testing with polymer test

The Polymer CLI comes with a test command that runs something called Web Component Tester. It leverages some other libraries which are likely familiar to you -- Mocha, Chai and Sinon -- which allow you to write tests for your web component.

To learn about how to organize and write your tests, check out Polymer Project: Test your elements.

If you used one of the polymer init generators to bootstrap your component, you should already have most of the pieces in place to use polymer test.

Configuration

The wct.conf.json file in the root of your project contains the different plugins and browser configurations that get run as part of polymer test. Typically you'll want a local plugin which runs only headless Chrome and a sauce plugin which defines the different browsers you'd like to test in Sauce Labs. More on Sauce Labs down below.

If one wasn't generated for you, here's a sample file to get you started.

The polymer test command

Once you have your tests defined and a wct.conf.json configuration, run the polymer test command to execute the tests.

If you don't want to run all the plugins, use the --skip-plugin flag.

# run all plugins
polymer test

# run only local plugin
polymer test --skip-plugin sauce

# run only sauce plugin
polymer test --skip-plugin local

It's probably easiest to throw each of these commands into a script in your package.json for easy reference:

{
  "scripts": {
    "test:polymer:local": "polymer test --skip-plugin sauce",
    "test:polymer:sauce": "polymer test --skip-plugin local"
  }
}

For Windows users, if running polymer test is slow, it may help to define these two environment variables, where LAUNCHPAD_CHROME obviously points to the Chrome binary on your machine:

LAUNCHPAD_BROWSERS=chrome
LAUNCHPAD_CHROME='C:\Program Files (x86)\Google\Chrome\Application\chrome.exe'

Running tests locally

The command line is great for automation, but sometimes you want to see how your tests are behaving in an actual browser. To load them up, run:

polymer serve

The serve command will start a local web server that you can visit in your browser. That's the root of your project, so append test/ to it to visit the index.html file inside your test directory.

Shady and Shadow DOM

The concept of shadow DOM and its magic of scoped DOM subtrees is integral to the web component story. It gives us scoped styles, encapsulation and isolation from the other elements on a page. Oh, and <slot>s 🎉!

shadow dom

Dive deep into shadow DOM awesomeness: Google Web Fundamentals: Shadow DOM v1

Shady DOM (and shady CSS) on the other hand, is a term used for the polyfills that attempt to "fake out" shadow DOM in browsers who don't yet natively support it. If you're curious, read more about how they work.

The important takeaway is: when building and testing your web components, you need to make sure you test them in both a shadow and shady DOM environment.

Browser Support

First, be aware of which browsers support shadow DOM v1 natively: CanIUse.com: Shadow DOM v1.

These are the only browsers with which you can actually test that your component works with shadow DOM -- all the others will be all shady DOM, all the time.

Polymer Version

The version of Polymer also matters. If your project is using Polymer 1, shadow DOM is off by default. To enable it, you need to either set the ?dom=shadow query-string parameter or enable it in code:

window.Polymer = {
  dom: 'shadow'
};

Polymer 2 and later has shadow DOM enabled by default. To forcibly disable it, set the ?wc-shadydom=true query-string parameter or disable it in code:

window.ShadyDOM = {
  force: true
};

Note: never actually disable shadow DOM support globally in your project, only use this for testing shady DOM. Your web components need to work properly in both modes.

Testing Continuously with Travis CI

We use Travis CI as our continuous integration system for web component repos.

travis ci logo

Enabling Travis CI

Once you create your repo, to enable Travis CI:

  1. Log into Travis CI using your GitHub credentials
  2. Go to your profile
  3. Press the "Sync Account" button to refresh the list of repos
  4. Click on your repo's organization (either BrightspaceUI or BrightspaceHypermediaComponents)
  5. Find your repo in the list on the right
  6. Press the toggle to "on"

You can now click on the name of the repo to see its build dashboard, which for a new repo should be empty.

More: Travis CI Documentation: Getting Started

.travis.yml

In the root of your project, create a .travis.yml configuration file with node_js as the language.

Under the 'script' block, run any linting and test tasks that you have.

A typical .travis.yml file looks like this:

language: node_js
node_js: node
script:
- npm run lint
- npm run myothertests

That's it! Whenever you push to a branch, create a pull request or merge to master, Travis CI will automatically do a build. It will also integrate into your GitHub pull request workflows, allowing you to ensure that your tests pass before merging.

Cross-browser testing with Sauce Labs

Sauce Labs is a cloud platform for running your tests on different operating systems, browsers and devices. It allows us to have more confidence that our web components work in the different browsers we support.

sauce labs logo

Username and API Key

Because Sauce Labs isn't free, to use Sauce Labs' APIs you'll need an account username and a secret API key for that user.

If you don't know which username or API key to use, reach out on the #sauce-labs Slack channel and someone can help you.

IMPORTANT: shh, the API key is a secret! Never check it into source control as plain text, even in private repos. Always encrypt it. Instructions on how to do that below.

Configuring polymer test for Sauce Labs

Just as with running tests locally, running tests in Sauce Labs leverages the polymer test command. Make sure you're already familiar with it.

Add a sauce plugin in wct.conf.json next to your local one that has an entry for each browser we support. Only worry about the latest release of evergreen browsers, and to save Sauce Labs minutes you only need to run each browser on a single operating system. Here's an example.

The sauce plugin can also be run from your local developer machine, but you'll need to set the SAUCE_USERNAME and SAUCE_ACCESS_KEY environment variables to correspond with the username and API key.

Running Sauce tests from Travis CI

To run your tests in Sauce Labs from a Travis CI build, you need several things in the .travis.yml file:

  1. Include the "chrome" stable addon to allow for Chrome headless testing
  2. Execute the NPM script that you set up to call polymer test --skip-plugin local (see example below)
  3. The SAUCE_USERNAME needs to be defined as an environment variable
  4. The SAUCE_ACCESS_KEY needs to be defined as an encrypted environment variable

To encrypt the API key, Travis CI has a Ruby Gem, which you install once on your machine:

gem install travis

Then inside your project, use the travis encrypt command to encrypt the API key and automatically add it to the .travis.yml file:

travis encrypt SAUCE_ACCESS_KEY="secretvalue" --add

More: Travis CI Documentation: Encryption keys

A note about forks: Because forks don't have access to encrypted environment variables, Sauce Labs tests cannot be run from them. This used to be possible via JWT addon, but it's since been deprecated.

When you're finished, the .travis.yml file should look something like this:

language: node_js
node_js: node
addons:
  chrome: stable
script:
- npm run lint
- |
  if [ $TRAVIS_PULL_REQUEST != false ] && [ $TRAVIS_SECURE_ENV_VARS == true ]; then
    echo "Pull request with secure environment variables, running Sauce tests...";
    npm run test:polymer:sauce || travis_terminate 1;
  else
    echo "Not a pull request and/or no secure environment variables, running headless tests...";
    npm run test:polymer:local || travis_terminate 1;
  fi
env:
  global:
  - SAUCE_USERNAME: Desire2Learn
  - secure: <encrypted-access-key>

This runs the full set of browser tests in Sauce Labs for pull requests when secure environment variables are available, and falls back to running the tests in Chrome headless (no Sauce Labs) otherwise.

Debugging Sauce Results

When your tests fail in Sauce Labs, it's sometimes difficult to determine from the log what happened.

The best way to debug the failure is to log into the Sauce Labs UI, find the job that failed and review the detailed logs and video recording of the test run. If you don't have the credentials to log in, reach out on the #sauce-labs Slack channel.

Another approach is to simply run polymer serve and visit the test page locally in the browser(s) that failed.

Supported Browsers

Ensure your component works correctly in all supported browsers. The list of supported browsers can be found at https://documentation.brightspace.com/EN/brightspace/requirements/all/browser_support.htm

You can’t perform that action at this time.