Skip to content
Permalink

Comparing changes

Choose two branches to see what’s changed or to start a new pull request. If you need to, you can also or learn more about diff comparisons.

Open a pull request

Create a new pull request by comparing changes across two branches. If you need to, you can also . Learn more about diff comparisons here.
base repository: browserstack/browserstack-runner
Failed to load repositories. Confirm that selected base ref is valid, then try again.
Loading
base: master
Choose a base ref
...
head repository: qunitjs/browserstack-runner
Failed to load repositories. Confirm that selected head ref is valid, then try again.
Loading
compare: main
Choose a head ref
Able to merge. These branches can be automatically merged.
  • 5 commits
  • 6 files changed
  • 2 contributors

Commits on Jun 6, 2021

  1. Update js-reporters to 2.1.0 

    Important changes:

* Mocha 8 and later are now supported. Previously, it's pending tests
  were incorrectly reported as undefined or failing.

* `QUnit.todo()` is now supported. Previously, these intentional
  failures were reported by browserstack-runner as failing tests.

  With js-reported 2.1 the runEnd report counts them as `todo` instead
  of as `failed`.

The API that browserstack-runner depends on has not changed since 1.1.0.
<https://github.com/js-reporters/js-reporters/blob/v2.1.0/CHANGELOG.md>.

Specifically, browserstack-runner captures the following data in
lib/_patch/reporter.js, and proccesses it in lib/server.js under
the "_progress" and "_report" handlers:

* The `testEnd` event, reading:
  - `name` string,
  - `suiteName` string,
  - `status` string,
  - `errors` array.

* The `runEnd` event, reading:
  - `status` string,
  - `testCounts` object with ints `total`, `passed`, `failed`, `skipped`,
  - `runtime` int.

These still exist the same way in the latest release:
<https://github.com/js-reporters/js-reporters/blob/v2.1.0/spec/cri-draft.adoc#runend>

Fixes #248.
Ref #247.
    @Krinkle
    Krinkle committed Jun 6, 2021
    Copy the full SHA
    8d262e3 View commit details

  2. Fix reporting of QUnit todo tests 

    The update to js-reporters 2.1.0 in the previous commit has fixed
the most important problem, which is the build status reported
by the `runEnd` event, handled via "_report" submission.

However, the todo tests were still printed as errors in the output
which is confusing.

Before:

```
[Windows 8, Chrome 91.0] Error: "test.todo.each [0]" failed
  Expected: true
  Actual: false
[Windows 8, Chrome 91.0] Passed: 332 tests, 321 passed, 0 failed, 7 skipped; ran for 1255ms
```

After:

```
[Windows 10, Firefox 88.0] Passed: 332 tests, 321 passed, 0 failed, 7 skipped; ran for 1910ms
```

Fixes #247.
    @Krinkle
    Krinkle committed Jun 6, 2021
    Copy the full SHA
    dab609e View commit details

  3. Tag 0.9.5-qunitjs.1

    @Krinkle
    Krinkle committed Jun 6, 2021
    Copy the full SHA
    019946e View commit details

Commits on May 29, 2024

  1. Use the bs-local.com hostname on OS X workers 

    Ref #263.
    @Krinkle
    Krinkle committed May 29, 2024
    Copy the full SHA
    529d9db View commit details

  2. Tag 0.9.5-qunitjs.2

    @Krinkle
    Krinkle committed May 29, 2024
    Copy the full SHA
    5ee0c5a View commit details
Showing with 37 additions and 26 deletions.
  1. +0 −3 .travis.yml
  2. +5 −2 README.md
  3. +2 −1 bin/cli.js
  4. +18 −8 lib/server.js
  5. +5 −5 package-lock.json
  6. +7 −7 package.json
3 changes: 0 additions & 3 deletions .travis.yml
View file
Original file line number Diff line number Diff line change
@@ -3,9 +3,6 @@ language: node_js
node_js:
- 'stable'

before_install:
- npm install -g grunt-cli jshint gulp

script:
- npm run-script test-ci

7 changes: 5 additions & 2 deletions README.md
View file
Original file line number Diff line number Diff line change
@@ -1,9 +1,12 @@
# BrowserStack Runner

[![Build Status](https://travis-ci.org/browserstack/browserstack-runner.svg?branch=master)](https://travis-ci.org/browserstack/browserstack-runner)

A command line interface to run browser tests over BrowserStack.

## Hotfixes from [@qunitjs/browserstack-runner](https://github.com/qunitjs/browserstack-runner):

1. Fix Mocha 8+ compat. [#248](https://github.com/browserstack/browserstack-runner/issues/248)
2. Fix QUnit.todo breakage. [#247](https://github.com/browserstack/browserstack-runner/issues/247)

## Usage

Install globally:
3 changes: 2 additions & 1 deletion bin/cli.js
View file
Original file line number Diff line number Diff line change
@@ -106,7 +106,8 @@ function getTestBrowserInfo(browserString, path) {

function buildTestUrl(test_path, worker_key, browser) {
var host;
if (browser.os.toLowerCase() === 'ios' ){
// https://github.com/browserstack/browserstack-runner/issues/263
if (browser.os.toLowerCase() === 'ios' || browser.os === 'OS X') {
host = 'bs-local.com';
} else {
host = 'localhost';
26 changes: 18 additions & 8 deletions lib/server.js
View file
Original file line number Diff line number Diff line change
@@ -387,17 +387,27 @@ exports.Server = function Server(bsClient, workers, config, callback) {

logger.trace('[%s] _progress', worker.id, CircularJSON.stringify(query));

if (query && query.test && query.test.errors) {
if (query && query.test) {
// Include all tests in the report
var browserReport = getBrowserReport(browserInfo);
browserReport.tests.push(query.test || {});

query.test.errors.forEach(function(error) {
logger.info('[%s] ' + chalk.red('Error:'), browserInfo, formatTraceback({
error: error,
testName: query.test.name,
suiteName: query.test.suiteName
}));
});
// Only print errors of failed tests
// (errors from todo tests are expected and will not be reported as
// "failed" to the "_report" handler either, so printing them here
// would create a confusing output with error printed but a success
// status at the end)
// https://github.com/browserstack/browserstack-runner/issues/247
if (query.test.errors && query.test.status !== 'todo') {
query.test.errors.forEach(function(error) {
logger.info('[%s] ' + chalk.red('Error:'), browserInfo, formatTraceback({
testName: query.test.name,
suiteName: query.test.suiteName,
status: query.test.status,
error: error
}));
});
}
}
response.end();
},
10 changes: 5 additions & 5 deletions package-lock.json
View file

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

14 changes: 7 additions & 7 deletions package.json
View file
Original file line number Diff line number Diff line change
@@ -1,17 +1,17 @@
{
"name": "browserstack-runner",
"description": "A command line interface to run browser tests over BrowserStack",
"version": "0.9.5",
"homepage": "https://github.com/browserstack/browserstack-runner",
"name": "@qunitjs/browserstack-runner",
"description": "[Hotfix] run browser tests over BrowserStack",
"version": "0.9.5-qunitjs.2",
"homepage": "https://github.com/qunitjs/browserstack-runner",
"repository": {
"type": "git",
"url": "https://github.com/browserstack/browserstack-runner.git"
"url": "https://github.com/qunitjs/browserstack-runner.git"
},
"dependencies": {
"browserstack": "1.3.0",
"chalk": "0.4.0",
"circular-json": "0.3.1",
"js-reporters": "1.1.0",
"js-reporters": "2.1.0",
"mime": "1.6.0",
"resolve": "1.1.7",
"send": "0.16.2",
@@ -26,7 +26,7 @@
"licenses": [
{
"type": "MIT",
"url": "https://github.com/browserstack/browserstack-runner/blob/master/MIT-LICENSE.txt"
"url": "https://github.com/qunitjs/browserstack-runner/blob/master/MIT-LICENSE.txt"
}
],
"bin": {