[test-runner] Sauce Labs launcher #472

justinfagnani · 2020-08-30T01:46:36Z

It would be great to have a Sauce Labs launcher similar to the Browserstack launcher.

daKmoR · 2020-08-30T06:34:30Z

We are still waiting for our sauce labs open source account to get activated 🙈

it should be very similar to the browserstack one so there is already a possible implementation but we could not verify it yet 🙈

#445

marvinhagemeister · 2020-08-30T12:48:35Z

We're very interested in that too over at the Preact team. It's the last remaining point for us before we make the switch.

cc @saucelabs @christian-bromann

christian-bromann · 2020-08-31T07:21:39Z

Thanks for looping me in @marvinhagemeister

@LarsDenBakker I upgraded the account. Please let me know if you need more concurrency for the project.

I am happy to assist in this effort! For updating jobs, add metadata information and interact with our API you should use our NPM package. Let me know if there are any questions ☺️

LarsDenBakker · 2020-08-31T16:38:49Z

@christian-bromann Thanks a lot! I got something up and running now.

One question about the url to point selenium to. I'm in the EU datacenter so I picked https://ondemand.eu-central-1.saucelabs.com/wd/hub, but how can I automate this for users of the browser launcher? I tried using the region option from the sauce labs options object but https://ondemand.us.saucelabs.com/wd/hub gave me an error.

Just using https://ondemand.saucelabs.com/wd/hub also worked to run tests, but the test results didn't show up in the dashboard. So I'm not sure what the best path is here.

We're really grateful for having a free open source account, but since you're offerring.. having more concurrency would be great as two concurrent sessions would start to queue up pretty quickly as the project grows. 👍

christian-bromann · 2020-08-31T16:47:16Z

@LarsDenBakker Sauce Labs has currently 3 WebDriver endpoints:

https://ondemand.eu-central-1.saucelabs.com/wd/hub
https://ondemand.us-west-1.saucelabs.com/wd/hub
https://ondemand.us-east-1.saucelabs.com/wd/hub (only FF and Chrome headless)

Just using https://ondemand.saucelabs.com/wd/hub also worked to run tests, but the test results didn't show up in the dashboard. So I'm not sure what the best path is here.

This is our legacy endpoint and we are in the process of deprecating it. It also points to our us-west-1 DC.

I tried using the region option from the sauce labs options object but https://ondemand.us.saucelabs.com/wd/hub gave me an error.

I could expose the ondemand url so that you wouldn't need to build it by yourself in the saucelabs package. What do you think?

having more concurrency would be great as two concurrent sessions would start to queue up pretty quickly as the project grows. 👍

Just bumped it to 10 concurrent sessions.

LarsDenBakker · 2020-08-31T16:51:11Z

@christian-bromann Thanks! 🙏 Exposing the URL would be great, also helps with migration between endpoints.

LarsDenBakker · 2020-08-31T19:59:42Z

I released a 0.0.1 of @web/test-runner-saucelabs, switching between eu-central-1 and us-west-1 for now.

@marvinhagemeister @justinfagnani it would be great if you could give it a try and see if it works for you. Docs are here: https://modern-web.dev/docs/test-runner/browsers/saucelabs/

LarsDenBakker · 2020-09-03T16:11:55Z

@christian-bromann just checking, it still shows "Team Concurrency" as 2 under My Account. Am I looking at the right place?

marvinhagemeister · 2020-09-15T16:41:18Z

So I've finally had some time to play around with the plugin. It's a fantastic start and I was able to connect to saucelabs and run some tests 🎉

Problems encountered:

1) Unable to start firefox on Windows 10.

Not sure what's happening here, but I haven't been able to instantiate a firefox browser at all. It works fine with our karma setup, so I'm assuming there is some difference somewhere. The error I'm seeing is:

(node:4017) UnhandledPromiseRejectionWarning: InvalidArgumentError: Expected "handle" to be a string, got [object Undefined] undefined
    at Object.throwDecodedError (/home/runner/work/***/***/node_modules/selenium-webdriver/lib/error.js:550:15)
    at parseHttpResponse (/home/runner/work/***/***/node_modules/selenium-webdriver/lib/http.js:565:13)
    at Executor.execute (/home/runner/work/***/***/node_modules/selenium-webdriver/lib/http.js:491:26)
    at processTicksAndRejections (internal/process/task_queues.js:93:5)
    at async mixin.execute (/home/runner/work/***/***/node_modules/selenium-webdriver/lib/webdriver.js:700:17)
    at async WindowManager.switchToAvailableWindow (/home/runner/work/***/***/node_modules/@web/test-runner-selenium/dist/WindowManager.js:92:9)
    at async WindowManager.startSession (/home/runner/work/***/***/node_modules/@web/test-runner-selenium/dist/WindowManager.js:97:30)
(Use `node --trace-warnings ...` to show where the warning was created)
(node:4017) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). To terminate the node process on unhandled promise rejection, use the CLI flag `--unhandled-rejections=strict` (see https://nodejs.org/api/cli.html#cli_unhandled_rejections_mode). (rejection id: 1)
(node:4017) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.

test/browser/cloneElement.test.js:

 ❌ The browser was unable to open the test page after 300000ms.  (failed on Windows 10 firefox latest)

2) Increased tests times

Spent a while debugging this and checking what karma is doing and it seems like karma loads all test files at once in the same browser instance, whereas web-test-runner opens a browser for each test file. The timing data I have here is heavily skewed by 1), so I'm not sure how valid this is. Probably best to wait until 1) is resolved. Our karma setup roughly takes around 2-3s whereas current runs with web-test-runner take a few minutes.

web-padawan · 2020-09-15T17:24:53Z

I can confirm the problem with Firefox on Windows.

Also, I was only able to get Safari and iOS Simulator builds passing after I set concurrency to 1. See example.
Note, platform and version worked for me, but not platformName and browserVersion.

LarsDenBakker · 2020-09-15T17:33:25Z

Thanks for trying it out. The firefox issue looks like a problem opening a browser tab, will see if we can do something about that.

For the speed the roundtrips between saucelabs and your local machine are a big bottleneck. There are two things to consider:

Your current karma config uses webpack to bundle before serving to saucelabs, this reduces the total roundtrips significantly. We could use rollup to quickly bundle the test files and see if that improves the speed.

Because we run tests in separate pages there are more requests from the browser. When watch is not turned on, we do agressive caching so once its served once it should be served from the browser cache. I need to double check whether this works correctly with selenium.

We can add a flag to run all tests in a single page, like karma. This would break encapsulation between tests and efficient refresh in watch mode - but that would be fine for tests in the CI.

marvinhagemeister · 2020-09-15T17:39:18Z

Yeah, personally I'm not sure if running all tests in a single page is the way to go. I'd leave that of the table for now.

aomarks · 2020-09-16T19:11:12Z

Hello! I'm trying out @web/test-runner-saucelabs for the lit-html repo, thanks for putting it together @LarsDenBakker! I'm getting various errors on all the browsers I've tried so far.

Here's our wtr config:

import {createSauceLabsLauncher} from '@web/test-runner-saucelabs';

const sauceLabsLauncher = createSauceLabsLauncher({
  user: process.env.SAUCE_USERNAME,
  key: process.env.SAUCE_ACCESS_KEY,
});

const sharedCapabilities = {
  'sauce:options': {
    name: 'my test name',
    build: `my project ${process.env.GITHUB_REF ?? 'local'} build ${
      process.env.GITHUB_RUN_NUMBER ?? ''
    }`,
  },
};

export default {
  rootDir: '../../',
  nodeResolve: true,
  browsers: [
    sauceLabsLauncher({
      ...sharedCapabilities,
      browserName: 'chrome',
      browserVersion: 'latest',
      platformName: 'linux',
    }),
  ],
  testFramework: {
    config: {
      ui: 'tdd',
      timeout: '2000',
    },
  }
};

With chrome/latest/linux we get:

WebDriverError: unknown error: cannot find dict 'desiredCapabilities'
  (Driver info: chromedriver=2.21.371459 (36d3d07f660ff2bc1bf28a75d1cdabed0983e7c4),platform=Linux 3.13.0-83-generic x86)
    at Object.checkLegacyResponse (/Users/aomarks/code/web/node_modules/selenium-webdriver/lib/error.js:585:15)
    at parseHttpResponse (/Users/aomarks/code/web/node_modules/selenium-webdriver/lib/http.js:556:13)
    at Executor.execute (/Users/aomarks/code/web/node_modules/selenium-webdriver/lib/http.js:491:26)
    at processTicksAndRejections (internal/process/task_queues.js:93:5) {
  remoteStacktrace: ''
}

With safari/11/macos 10.13 we get:

Error while running tests:
WebDriverError: Infrastructure Error -- The Sauce VMs failed to start the browser or device.
For more info, please check https://wiki.saucelabs.com/display/DOCS/Common+Error+Messages
    at Object.checkLegacyResponse (/Users/aomarks/code/web/node_modules/selenium-webdriver/lib/error.js:585:15)
    at parseHttpResponse (/Users/aomarks/code/web/node_modules/selenium-webdriver/lib/http.js:556:13)
    at Executor.execute (/Users/aomarks/code/web/node_modules/selenium-webdriver/lib/http.js:491:26)
    at processTicksAndRejections (internal/process/task_queues.js:93:5) {
  remoteStacktrace: ''
}

With microsoftedge/17/windows 10 we get:

(node:83578) UnhandledPromiseRejectionWarning: Error: Something went wrong opening a new browser window. New window never appeared
    at Timeout._onTimeout (/Users/aomarks/code/web/packages/test-runner-selenium/dist/utils.js:10:20)
    at listOnTimeout (internal/timers.js:551:17)
    at processTimers (internal/timers.js:494:7)
(Use `node --trace-warnings ...` to show where the warning was created)
(node:83578) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). To terminate the node process on unhandled promise rejection, use the CLI flag `--unhandled-rejections=strict` (see https://nodejs.org/api/cli.html#cli_unhandled_rejections_mode). (rejection id: 1)
(node:83578) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.

I've also noticed that the Saucelabs tunnels stay open after this crash (maybe after a success too?), and it creates a new tunnel every time, so after a few attempts I have to go to the Saucelabs website to close the tunnels or else I'll get a connection error due to all our tunnels being used up.

I'm investigating, but let me know if any of this looks familiar!

LarsDenBakker · 2020-09-16T20:43:48Z

@aomarks

These are working for us in the CI right now:

     sauceLabsLauncher({
       ...sharedCapabilities,
       browserName: 'chrome',
       browserVersion: 'latest',
       platformName: 'Windows 10',
    }),  

    sauceLabsLauncher({
      ...sharedCapabilities,
      browserName: 'internet explorer',
      browserVersion: '11.0',
      platformName: 'Windows 7',
    }),

IE11 requires a legacy plugin.

I added an example project here: https://github.com/modernweb-dev/example-projects/tree/master/saucelabs

I'm getting similar errors like you with the other configurations, we need to investigate that. The error on edge: Error: Something went wrong opening a new browser window. New window never appeared can be due to the configuration not allowing selenium to open new tabs/pages. For IE11 we're adding some extra selenium capabilities to allow that.

web-padawan · 2020-09-18T12:59:16Z

As mentioned above, I got the tests passing on MacOS and iOS Simulator with the following config:

  const sharedCapabilities = {
    'sauce:options': {
      name: 'vaadin-date-picker unit tests',
      build: `build ${process.env.TRAVIS_JOB_NUMBER || ''}`
    }
  };

  config.concurrency = 1;
  config.browsers = [
    sauceLabsLauncher({
      ...sharedCapabilities,
      browserName: 'safari',
      platform: 'macOS 10.14',
      version: '13'
    }),
    sauceLabsLauncher({
      ...sharedCapabilities,
      browserName: 'safari',
      platform: 'iOS Simulator',
      version: '13.1'
    })
  ];

There is a logging output problem when using same browserName, here is what happens:

safari: |██████████████████████████████| 24/12 test files | 608 passed, 0 failed, 2 skipped
safari: |██████████████████████████████| 24/12 test files | 608 passed, 0 failed, 2 skipped

Finished running tests in 241.8s, all tests passed! 🎉

Build example: https://github.com/vaadin/vaadin-date-picker/pull/733/checks?check_run_id=1133888849

LarsDenBakker · 2020-09-19T11:26:32Z

I released @web/test-runner-saucelabs@0.0.4 which improves the generated browser name in the reporter and fixes the double test files reported. I also improved the logic which closes saucelabs proxy, though I still see in some cases the proxy remains open even though we closed it on our side.

Browsers

I dug into some different browsers configurations, this is what I've found so far.

Firefox: saucelabs uses a version of geckodriver which is incompatible with Firefox 80. Firefox 79 seems to work fine. See mozilla/geckodriver#1771 and https://firefox-source-docs.mozilla.org/testing/geckodriver/Support.html.

Edge: on edge saucelabs blocks popups. it works with concurrency set to 1 since we don't open extra pages in that case. For browserstack we set some special capabilities: https://github.com/modernweb-dev/web/blob/master/packages/test-runner-browserstack/src/browserstackLauncher.ts#L82 I'm trying to find a similar settings at saucelabs.

chrome/latest/linux and safari/11/macos 10.13: I haven't been able to find out if we're doing something wrong here. We just pass the capabilities as is, so they might be errors on saucelabs side. Linux isn't in their platform configurator: https://wiki.saucelabs.com/display/DOCS/Platform+Configurator

@christian-bromann anything you can help out with here?

Speed

I added a plugin for bundling on the fly with rollup: https://modern-web.dev/docs/dev-server/plugins/rollup/#bundling I'm going to test it out on some projects to see if that makes testing on a remote service faster.

A general improvement I also want to make is to open browsers sequentially, instead of all in parallel. This should improve speed when running tests in a lot of remote browsers. Tracking this in #602

web-padawan · 2020-09-19T11:39:11Z

Confirmed that the browser names issue is now fixed and the test report is looking better, thanks!

The only thing is that I just noticed it celebrates the build success twice 🙂

iOS Simulator safari: |██████████████████████████████| 12/12 test files | 304 passed, 0 failed, 1 skipped
macOS 10.14 safari:   |██████████████████████████████| 12/12 test files | 304 passed, 0 failed, 1 skipped

Finished running tests in 243.7s, all tests passed! 🎉

iOS Simulator safari: |██████████████████████████████| 12/12 test files | 304 passed, 0 failed, 1 skipped
macOS 10.14 safari:   |██████████████████████████████| 12/12 test files | 304 passed, 0 failed, 1 skipped

Finished running tests in 243.7s, all tests passed! 🎉

Build: https://github.com/vaadin/vaadin-date-picker/pull/733/checks?check_run_id=1137688725

aomarks · 2020-09-22T04:01:30Z

Hmm, I can't get the safari configs @web-padawan had luck with to work for us. If I try macOS 10.14/safari/13, I get:

WebDriverError: Misconfigured -- Unsupported OS/browser/version/device combo: OS: 'Mac 10.14', Browser: 'safari', Version: '13.', Device: 'unspecified'

Note that the version number has a redundant trailing ., I wonder if that's a clue? I also tried 13.1 and 13.1.2, all of which return similar errors with the trailing ..

aomarks · 2020-09-22T04:19:16Z

Also FYI, I just noticed an uncaught promise error from WebDriver in a test run that reported success:

Windows 10 firefox 68: |██████████████████████████████| 25/25 test files | 333 passed, 0 failed, 11 skipped
Chromium:              |██████████████████████████████| 25/25 test files | 334 passed, 0 failed, 10 skipped
Firefox:               |██████████████████████████████| 25/25 test files | 333 passed, 0 failed, 11 skipped
Webkit:                |██████████████████████████████| 25/25 test files | 333 passed, 0 failed, 11 skipped

Finished running tests in 141.5s, all tests passed! 🎉


(node:8607) UnhandledPromiseRejectionWarning: WebDriverError: Invalid message: The test with session id 4ce27dd892474192bcfc9585389c2ad7 has already finished, and can't receive further commands.
For help, please check https://wiki.saucelabs.com/display/DOCS/Common+Error+Messages
    at Object.checkLegacyResponse (/home/runner/work/lit-html/lit-html/packages/tests/node_modules/selenium-webdriver/lib/error.js:585:15)
    at parseHttpResponse (/home/runner/work/lit-html/lit-html/packages/tests/node_modules/selenium-webdriver/lib/http.js:556:13)
    at Executor.execute (/home/runner/work/lit-html/lit-html/packages/tests/node_modules/selenium-webdriver/lib/http.js:491:26)
    at processTicksAndRejections (internal/process/task_queues.js:93:5)
    at async mixin.execute (/home/runner/work/lit-html/lit-html/packages/tests/node_modules/selenium-webdriver/lib/webdriver.js:700:17)
    at async WindowManager.stopSession (/home/runner/work/lit-html/lit-html/packages/tests/node_modules/@web/test-runner-selenium/dist/WindowManager.js:136:9)

https://github.com/Polymer/lit-html/pull/1305/checks?check_run_id=1147546061

LarsDenBakker · 2020-09-22T06:34:03Z

Thanks for reporting, will look into those.

I also looked into improving the speed. I did some tests and while the speed varies between runs, bundling didn't seem to make things faster for both the Lion and Preact projects. Lion has a lot of tests, so if anything I expected to see an improvement there.

Running all tests from a single file, like karma did, instead of running each test in a separate file is faster on saucelabs. This in part due to a limitation in selenium where it can only send commands to the current focused window. Locally using puppeteer or playwright it's faster.

I'm going to investigate if there are improvements we can make there. Otherwise we may need to offer the option to run tests from a single file. We might be able to use iframes, though each iframe will still request the files again.

web-padawan · 2020-09-22T06:36:27Z

@aomarks please note that I'm getting WebDriverError: Misconfigured with "platformName" set, but not "platform".

These are the configs that worked:

browserName: 'safari',
platform: 'macOS 10.14',
browserVersion: '13'

browserName: 'safari',
platform: 'iOS Simulator',
browserVersion: '13.1'

aomarks · 2020-09-22T18:49:33Z

Ah, thank you @web-padawan! I blanked that part of your earlier comment. It does seem like using platform instead of platformName makes some configurations work -- weird!

Another experience to share, on chrome@latest-3/Windows 10 (and probably other configurations, this is just the one I happened to see the effect on), I got a failure with no explanation. This has happened a few times:

Windows 10 chrome latest-3: |██████████████████████████████| 25/25 test files | 311 passed, 
0 failed, 7 skipped

Error while running tests.

Maybe there are some extra errors/logs being caught somewhere that aren't getting displayed? It looks like everything passed, yet it still reports an error.

- Set up the rigging for Sauce with `@web/test-runner`. - Adds Firefox 68 (ESR) on Windows to our CI. - For now, running the sauce browsers in a separate job, because I think it's likely we'll see some failures, and it will be less disruptive if we can quickly see whether its local vs sauce that's failing. - The other browsers still don't work, since the plugin is very new. Firefox 68 is the only one I've gotten to work from the preliminary set for lit-next. modernweb-dev/web#472 is tracking the problems, so we'll just keep an eye on it and uncomment browsers as they start working. - Adds a `BROWSER` environment variable, read by our wtr config. Makes it easy to pick the browsers from the commandline. You can specify Sauce ones like `SAUCE_USERNAME=polymer-devs SAUCE_ACCESS_KEY=<redacted> BROWSERS="sauce:Windows 10/firefox@68" npm test`, Playwright ones like `BROWSERS=chromium,firefox npm test`, or the same set we run in CI with `BROWSERS=preset:sauce npm test`.

LarsDenBakker · 2020-09-26T11:18:22Z

I'm experimenting with running tests in a single page using iframes (web component tester did this too). This keeps the encapsulation intact and avoids the overhead I'm seeing with selenium jumping between tabs. I'm not sure if browsers execute iframes on a separate thread, but that would retain some of the concurrency too.

It also avoids some rendering issues I was seeing where tests that rely on layout/rendering were timing out because unfocused tabs are deprioritized. This isn't an issue with puppeteer or playwright though.

It seems to speed up the tests on preact and lit-html repositories, going to do some more testing. Hope to wrap this up soon!

marvinhagemeister · 2020-09-26T11:34:13Z

@LarsDenBakker this is fantastic news! Thank you so much for working on this 🙌

LarsDenBakker · 2020-09-30T20:36:26Z

I've enabled the new iframe mode by default. This makes the tests much more stable and faster, as mentioned above.

As a side effect, because we're no longer switching tabs, we don't suffer from the Edge and Firefox issues in saucelabs mentioned above. There are all the browser configurations that I've tested to be working (and now run in a CI for us): https://github.com/modernweb-dev/example-projects/blob/master/saucelabs/web-test-runner.config.mjs#L33 Other browsers errors look like errors on saucelabs side, but let me know if you think otherwise.

I also reworked the concurrency mode of the test runner itself. Before we were booting up all browsers at once, which is a problem when testing in remote browsers. Now there is a new concurrentBrowsers option which controls how many browsers we test at the same time. The concurrency option controls how many test files are tested in that browser at a time.

Locally you probably don't want to set concurrentBrowsers any higher than the default of 2, unless you're working on an an extremely high-end machine. But with remote testing, you can set this as high as your service plan allows. I've found that a concurrency of 6 works well on saucelabs. See updated docs at https://modern-web.dev/docs/test-runner/browser-launchers/saucelabs/#usage

For preact testing on the latest chrome, firefox and edge it takes 100sec. For lit-html testing on firefox 68, chrome latest-3 and safari takes 43sec (2x for dev and prod tests). This is with concurrentBrowsers set to 2. I think there are still some things to improve here, I'm looking into those.

For IE11 and other browsers that don't use modules there are test failures on both projects. This might be because of conflicting polyfills, we will make this configurable (#659). Let me know if there are other issues on these browsers.

web-padawan · 2020-10-01T08:34:06Z

Did anyone manage to get iPhone Simulator working in Sauce Labs? I just realised my above example might be wrong.

This is what I previously got working with Karma:

  'sl-ios-12': {
    base: 'SauceLabs',
    browserName: 'iphone',
    platform: 'OS X 10.13',
    version: '12.2'
  },
  'sl-ios-13': {
    base: 'SauceLabs',
    browserName: 'iphone',
    platform: 'iPhone X Simulator',
    version: '13.0'
  }

But when I try the above combinations with WTR, there is an error:

WebDriverError: Infrastructure Error -- The Sauce VMs failed to start the browser or device.

UPD: the above config is valid for karma-sauce-launcher 2.x which was using older saucelabs version (1.5.0).
After I upgraded karma-sauce-launcher to the latest version which uses saucelabs 4.3.0 it also failed 😕

Enables the new iframe mode (see modernweb-dev/web#472 (comment)). This drops the sauce test time from ~7m30s to ~3m40s! Also adds npm run nuke command to bump all dependencies in all packages.

LarsDenBakker · 2020-10-07T16:04:22Z

I will close this issue since we have a basic functioning version now. If anyone finds any bugs or has new feature requests - please open new issues :)

@web-padawan if you find anything actionable from your investigations into the iPhone simulator, let me know

aomarks · 2020-10-07T16:06:16Z

Thanks for the great work here!

LarsDenBakker added enhancement New feature or request priority: high labels Aug 30, 2020

LarsDenBakker added this to the test runner v1 milestone Aug 30, 2020

christian-bromann mentioned this issue Aug 31, 2020

Expose ondemand url from client object saucelabs/node-saucelabs#94

Merged

aomarks mentioned this issue Sep 22, 2020

[lit-next] Set up Sauce with Firefox ESR lit/lit#1305

Merged

LarsDenBakker mentioned this issue Sep 24, 2020

[test-runner] rework concurrency mechanism #622

Closed

LarsDenBakker mentioned this issue Sep 28, 2020

feat(test-runner): rework concurrent scheduling logic #649

Merged

aomarks mentioned this issue Sep 30, 2020

[lit-next] Bump to latest wtr and increase browser concurrency lit/lit#1338

Merged

LarsDenBakker closed this as completed Oct 7, 2020

web-padawan mentioned this issue Oct 13, 2020

Breaking change for iPhone Simulators caused by node-saucelabs version bump karma-runner/karma-sauce-launcher#220

Closed

abdonrd mentioned this issue Nov 2, 2020

[lit-html] Review browser test matrix lit/lit#1404

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[test-runner] Sauce Labs launcher #472

[test-runner] Sauce Labs launcher #472

justinfagnani commented Aug 30, 2020

daKmoR commented Aug 30, 2020 •

edited

Loading

marvinhagemeister commented Aug 30, 2020 •

edited

Loading

christian-bromann commented Aug 31, 2020

LarsDenBakker commented Aug 31, 2020

christian-bromann commented Aug 31, 2020

LarsDenBakker commented Aug 31, 2020

LarsDenBakker commented Aug 31, 2020 •

edited

Loading

LarsDenBakker commented Sep 3, 2020

marvinhagemeister commented Sep 15, 2020

web-padawan commented Sep 15, 2020 •

edited

Loading

LarsDenBakker commented Sep 15, 2020

marvinhagemeister commented Sep 15, 2020

aomarks commented Sep 16, 2020

LarsDenBakker commented Sep 16, 2020 •

edited

Loading

web-padawan commented Sep 18, 2020

LarsDenBakker commented Sep 19, 2020 •

edited

Loading

web-padawan commented Sep 19, 2020 •

edited

Loading

aomarks commented Sep 22, 2020

aomarks commented Sep 22, 2020

LarsDenBakker commented Sep 22, 2020

web-padawan commented Sep 22, 2020

aomarks commented Sep 22, 2020

LarsDenBakker commented Sep 26, 2020 •

edited

Loading

marvinhagemeister commented Sep 26, 2020

LarsDenBakker commented Sep 30, 2020 •

edited

Loading

web-padawan commented Oct 1, 2020 •

edited

Loading

LarsDenBakker commented Oct 7, 2020

aomarks commented Oct 7, 2020

[test-runner] Sauce Labs launcher #472

[test-runner] Sauce Labs launcher #472

Comments

justinfagnani commented Aug 30, 2020

daKmoR commented Aug 30, 2020 • edited Loading

marvinhagemeister commented Aug 30, 2020 • edited Loading

christian-bromann commented Aug 31, 2020

LarsDenBakker commented Aug 31, 2020

christian-bromann commented Aug 31, 2020

LarsDenBakker commented Aug 31, 2020

LarsDenBakker commented Aug 31, 2020 • edited Loading

LarsDenBakker commented Sep 3, 2020

marvinhagemeister commented Sep 15, 2020

web-padawan commented Sep 15, 2020 • edited Loading

LarsDenBakker commented Sep 15, 2020

marvinhagemeister commented Sep 15, 2020

aomarks commented Sep 16, 2020

LarsDenBakker commented Sep 16, 2020 • edited Loading

web-padawan commented Sep 18, 2020

LarsDenBakker commented Sep 19, 2020 • edited Loading

Browsers

Speed

web-padawan commented Sep 19, 2020 • edited Loading

aomarks commented Sep 22, 2020

aomarks commented Sep 22, 2020

LarsDenBakker commented Sep 22, 2020

web-padawan commented Sep 22, 2020

aomarks commented Sep 22, 2020

LarsDenBakker commented Sep 26, 2020 • edited Loading

marvinhagemeister commented Sep 26, 2020

LarsDenBakker commented Sep 30, 2020 • edited Loading

web-padawan commented Oct 1, 2020 • edited Loading

LarsDenBakker commented Oct 7, 2020

aomarks commented Oct 7, 2020

daKmoR commented Aug 30, 2020 •

edited

Loading

marvinhagemeister commented Aug 30, 2020 •

edited

Loading

LarsDenBakker commented Aug 31, 2020 •

edited

Loading

web-padawan commented Sep 15, 2020 •

edited

Loading

LarsDenBakker commented Sep 16, 2020 •

edited

Loading

LarsDenBakker commented Sep 19, 2020 •

edited

Loading

web-padawan commented Sep 19, 2020 •

edited

Loading

LarsDenBakker commented Sep 26, 2020 •

edited

Loading

LarsDenBakker commented Sep 30, 2020 •

edited

Loading

web-padawan commented Oct 1, 2020 •

edited

Loading