Skip to content

Commit

Permalink
build: polish up bazel karma saucelabs info tools/saucelabs/README.md (
Browse files Browse the repository at this point in the history
…#35667)

PR Close #35667
  • Loading branch information
gregmagolan authored and mhevery committed Feb 26, 2020
1 parent f25d00a commit d58b5ce
Showing 1 changed file with 22 additions and 7 deletions.
29 changes: 22 additions & 7 deletions tools/saucelabs/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,17 +6,28 @@ Setup your `SAUCE_USERNAME`, `SAUCE_ACCESS_KEY` & `SAUCE_TUNNEL_IDENTIFIER` envi

To run tests use:

```
``` bash
yarn bazel run //tools/saucelabs:sauce_service_setup
yarn bazel test //path/to:saucelabs_test_target_1 --config=saucelabs [--config=ivy]
yarn bazel test //path/to:saucelabs_test_target_2 --config=saucelabs [--config=ivy]
```

or if the tests are combined into a test suite:
`bazel query` is required gather up all karma saucelabs test labels so they can be run in one command as they are tagged `manual`.

Running all ViewEngine karma tests in Saucelabs:

``` bash
yarn bazel run //tools/saucelabs:sauce_service_setup
TESTS=$(./node_modules/.bin/bazel query --output label '(kind(karma_web_test, ...) intersect attr("tags", "saucelabs", ...)) except attr("tags", "ivy-only", ...) except attr("tags", "fixme-saucelabs-ve", ...)')
yarn bazel test --config=saucelabs ${TESTS}
```

Running all Ivy karma tests in Saucelabs:

``` bash
yarn bazel run //tools/saucelabs:sauce_service_setup
yarn bazel test //path/to:saucelabs_test_suite --config=saucelabs [--config=ivy]
TESTS=$(./node_modules/.bin/bazel query --output label '(kind(karma_web_test, ...) intersect attr("tags", "saucelabs", ...)) except attr("tags", "no-ivy-aot", ...) except attr("tags", "fixme-saucelabs-ivy", ...)')
yarn bazel test --config=saucelabs --config=ivy ${TESTS}
```

To see the test output while the tests are running as these are long tests, add the `--test_output=streamed` option. Note, this option will also prevent bazel from using the test cache and will force the test to run.
Expand All @@ -31,21 +42,25 @@ script which configures the saucelabs environment and starts Sauce Connect befor

For example,

```
``` python
karma_web_test(
name = "saucelabs_core_acceptance_tests",
timeout = "long",
karma = "//tools/saucelabs:karma-saucelabs",
tags = [
"exclusive",
"manual",
"no-remote-exec",
"saucelabs",
],
deps = [
"//packages/core/test/acceptance:acceptance_lib",
],
)
```

These saucelabs targets must be tagged `no-remote-exec` as they cannot be executed remotely since
they require a local Sauce Connect process. They should also be tagged `manual` so they are not
automatically tested with `//...`.
These saucelabs targets must have a few important tags:
* `no-remote-exec` as they cannot be executed remotely since they require a local Sauce Connect process
* `manual` so they are not automatically tested with `//...`
* `exclusive` as they must be run serially in order to not over-provision Saucelabs browsers
* `saucelabs` so that they can be easily gathered up for testing in a `bazel query`

0 comments on commit d58b5ce

Please sign in to comment.