Permalink
Switch branches/tags
merge_pr_14113 merge_pr_14108 merge_pr_14106 merge_pr_14105 merge_pr_14102 merge_pr_14098 merge_pr_14097 merge_pr_14096 merge_pr_14092 merge_pr_14090 merge_pr_14086 merge_pr_14085 merge_pr_14084 merge_pr_14080 merge_pr_14079 merge_pr_14078 merge_pr_14077 merge_pr_14076 merge_pr_14075 merge_pr_14071 merge_pr_14068 merge_pr_14067 merge_pr_14065 merge_pr_14063 merge_pr_14062 merge_pr_14061 merge_pr_14058 merge_pr_14057 merge_pr_14056 merge_pr_14055 merge_pr_14053 merge_pr_14052 merge_pr_14051 merge_pr_14050 merge_pr_14049 merge_pr_14048 merge_pr_14047 merge_pr_14046 merge_pr_14045 merge_pr_14042 merge_pr_14040 merge_pr_14039 merge_pr_14038 merge_pr_14036 merge_pr_14035 merge_pr_14034 merge_pr_14032 merge_pr_14031 merge_pr_14030 merge_pr_14028 merge_pr_14027 merge_pr_14026 merge_pr_14024 merge_pr_14023 merge_pr_14022 merge_pr_14021 merge_pr_14020 merge_pr_14019 merge_pr_14018 merge_pr_14017 merge_pr_14016 merge_pr_14015 merge_pr_14011 merge_pr_14010 merge_pr_14009 merge_pr_14008 merge_pr_14007 merge_pr_14006 merge_pr_14005 merge_pr_14004 merge_pr_14002 merge_pr_14001 merge_pr_14000 merge_pr_13997 merge_pr_13996 merge_pr_13995 merge_pr_13994 merge_pr_13993 merge_pr_13992 merge_pr_13991 merge_pr_13988 merge_pr_13987 merge_pr_13986 merge_pr_13985 merge_pr_13984 merge_pr_13983 merge_pr_13981 merge_pr_13980 merge_pr_13979 merge_pr_13978 merge_pr_13977 merge_pr_13974 merge_pr_13972 merge_pr_13969 merge_pr_13968 merge_pr_13967 merge_pr_13962 merge_pr_13961 merge_pr_13960 merge_pr_13959
Nothing to show
Find file Copy path
Fetching contributors…
Cannot retrieve contributors at this time
81 lines (58 sloc) 2.89 KB
layout title
page
Running Tests

In simple cases individual tests can be run by simply loading the page in a browser window. For running larger groups of tests, or running tests frequently, this is not a practical approach and several better options exist.

From the Command Line

The simplest way to run tests is to use the wpt run command from the root of the repository. This will automatically load the tests in the chosen browser, and extract the test results. For example to run the dom/historical.html tests in a local copy of Chrome:

./wpt run chrome dom/historical.html

Or to run in a specified copy of Firefox:

./wpt run --binary ~/local/firefox/firefox firefox dom/historical.html

For details on the supported products and a large number of other options for customising the test run:

./wpt run --help

Additional browser-specific documentation:

  • [Chrome][chrome]

  • [Chrome for Android][chrome android]

  • [Safari][safari]

From Inside a Browser

Tests that have been merged on GitHub are mirrored at [http://w3c-test.org/][w3c-test].

For running multiple tests inside a browser, there is a test runner located at /tools/runner/index.html.

This allows all the tests, or those matching a specific prefix (e.g. all tests under /dom/) to be run. For testharness.js tests, the results will be automatically collected, while the runner provides a simple UI for manually comparing reftest rendering and running manual tests.

Note, however, it does not currently handle more complex reftests with more than one reference involved.

Because it runs entirely in-browser, this runner cannot deal with edge-cases like tests that cause the browser to crash or hang.

Writing Your Own Runner

Most test runners have two stages: finding all tests, followed by executing them (or a subset thereof).

To find all tests in the repository, it is strongly recommended to use the included wpt manifest tool: the required behaviors are more complex than what are documented (especially when it comes to precedence of the various possibilities and some undocumented legacy ways to define test types), and hence its behavior should be considered the canonical definition of how to enumerate tests and find their type in the repository.

For test execution, please read the documentation for the various test types very carefully and then check your understanding on the [mailing list][public-test-infra] or [IRC][] ([webclient][web irc], join channel #testing). It's possible edge-case behavior isn't properly documented!

[chrome]: {{ site.baseurl }}{% link _running-tests/chrome.md %} [chrome android]: {{ site.baseurl }}{% link _running-tests/chrome_android.md %} [safari]: {{ site.baseurl }}{% link _running-tests/safari.md %} [public-test-infra]: https://lists.w3.org/Archives/Public/public-test-infra/ [IRC]: irc://irc.w3.org:6667/testing [web irc]: http://irc.w3.org [w3c-test]: http://w3c-test.org