Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add a "mach test-wpt --chaos" mode for reproducing intermittent failures using rr #9607

Merged
merged 3 commits into from Feb 11, 2016

Conversation

@mbrubeck
Copy link
Contributor

mbrubeck commented Feb 11, 2016

This adds a new --chaos flag to the test-wpt and test-css commands. This will run a test repeatedly until it fails, recording it with the rr debugger in chaos mode. Requires a recent master build of rr.

./mach test-wpt --chaos test.html

is shorthand for:

./mach test-wpt --repeat-until-unexpected \
                --debugger rr \
                --debugger-args "record --chaos" \
                --include test.html

This PR currently includes changes to wptrunner which were also submitted upstream as w3c/wptrunner#170. We should not merge this until that upstream PR is merged. CC @jgraham

Review on Reviewable

@jdm
Copy link
Member

jdm commented Feb 11, 2016

FYI: the wptrunner changes should be upstreamed, too.

@jgraham
Copy link
Contributor

jgraham commented Feb 11, 2016

Reviewed 2 of 2 files at r1, 1 of 1 files at r3.
Review status: all files reviewed at latest revision, 1 unresolved discussion.


python/servo/testing_commands.py, line 349 [r3] (raw file):
Maybe pop this argument since it doesn't exist further down the chain.

I guess it's OK for these to overwrite any user-supplied options?


Comments from the review on Reviewable.io

@mbrubeck mbrubeck force-pushed the mbrubeck:chaos branch from 1aadb67 to 7c740d3 Feb 11, 2016
@mbrubeck
Copy link
Contributor Author

mbrubeck commented Feb 11, 2016

@bors-servo r=jgraham

-S-awaiting-review

FYI: the wptrunner changes should be upstreamed, too.

Done.


Review status: 2 of 3 files reviewed at latest revision, 1 unresolved discussion.


python/servo/testing_commands.py, line 349 [r3] (raw file):
Done.


Comments from the review on Reviewable.io

@bors-servo
Copy link
Contributor

bors-servo commented Feb 11, 2016

📌 Commit 7c740d3 has been approved by jgraham

@bors-servo
Copy link
Contributor

bors-servo commented Feb 11, 2016

Testing commit 7c740d3 with merge 151842e...

bors-servo added a commit that referenced this pull request Feb 11, 2016
Add a "mach test-wpt --chaos" mode for reproducing intermittent failures using rr

This adds a new `--chaos` flag to the `test-wpt` and `test-css` commands.  This will run a test repeatedly until it fails, recording it with the rr debugger in chaos mode. Requires a recent master build of rr.

```
./mach test-wpt --chaos test.html
```
is shorthand for:
```
./mach test-wpt --repeat-until-unexpected \
                --debugger rr \
                --debugger-args "record --chaos" \
                --include test.html
```
This PR currently includes changes to wptrunner which were also submitted upstream as w3c/wptrunner#170.  We should not merge this until that upstream PR is merged. CC @jgraham

<!-- Reviewable:start -->
[<img src="https://reviewable.io/review_button.svg" height="40" alt="Review on Reviewable"/>](https://reviewable.io/reviews/servo/servo/9607)
<!-- Reviewable:end -->
@bors-servo
Copy link
Contributor

bors-servo commented Feb 11, 2016

💔 Test failed - linux-rel

@jdm
Copy link
Member

jdm commented Feb 11, 2016

@bors-servo
Copy link
Contributor

bors-servo commented Feb 11, 2016

Previous build results for android, gonk, linux-dev, mac-dev-unit, mac-rel-css, mac-rel-wpt are reusable. Rebuilding only linux-rel...

@bors-servo
Copy link
Contributor

bors-servo commented Feb 11, 2016

@bors-servo
Copy link
Contributor

bors-servo commented Feb 11, 2016

👀 Test was successful, but fast-forwarding failed: 422 Update is not a fast forward

@jdm
Copy link
Member

jdm commented Feb 11, 2016

@bors-servo: retry

  • git problem?
@bors-servo
Copy link
Contributor

bors-servo commented Feb 11, 2016

Testing commit 7c740d3 with merge faa349f...

bors-servo added a commit that referenced this pull request Feb 11, 2016
Add a "mach test-wpt --chaos" mode for reproducing intermittent failures using rr

This adds a new `--chaos` flag to the `test-wpt` and `test-css` commands.  This will run a test repeatedly until it fails, recording it with the rr debugger in chaos mode. Requires a recent master build of rr.

```
./mach test-wpt --chaos test.html
```
is shorthand for:
```
./mach test-wpt --repeat-until-unexpected \
                --debugger rr \
                --debugger-args "record --chaos" \
                --include test.html
```
This PR currently includes changes to wptrunner which were also submitted upstream as w3c/wptrunner#170.  We should not merge this until that upstream PR is merged. CC @jgraham

<!-- Reviewable:start -->
[<img src="https://reviewable.io/review_button.svg" height="40" alt="Review on Reviewable"/>](https://reviewable.io/reviews/servo/servo/9607)
<!-- Reviewable:end -->
@bors-servo
Copy link
Contributor

bors-servo commented Feb 11, 2016

@bors-servo bors-servo merged commit 7c740d3 into servo:master Feb 11, 2016
2 checks passed
2 checks passed
continuous-integration/travis-ci/pr The Travis CI build passed
Details
homu Test successful
Details
@mbrubeck mbrubeck deleted the mbrubeck:chaos branch May 17, 2016
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

None yet

5 participants
You can’t perform that action at this time.