Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WPT tests now show debug ouput for passing tests as well #15124

Merged
merged 1 commit into from Jan 24, 2017

Conversation

@cynicaldevil
Copy link
Contributor

cynicaldevil commented Jan 20, 2017


  • ./mach build -d does not report any errors
  • ./mach test-tidy does not report any errors
  • These changes fix (partially) #12808

@jdm I thought it would be better to make a PR for my local changes anyway, since you wanted it :)


This change is Reviewable

@highfive
Copy link

highfive commented Jan 20, 2017

Heads up! This PR modifies the following files:

  • @jgraham: tests/wpt/grouping_formatter.py
@jgraham
Copy link
Contributor

jgraham commented Jan 20, 2017

I guess I don't exactly know what you want, but this seems like a pretty surprising change; the point of the grouping formatter was to have as little output as possible. Maybe a simpler alternative would be just to switch to using --log-mach by default locally and use this one only in CI?

@jdm
Copy link
Member

jdm commented Jan 20, 2017

@jgraham The default formatter is in an interesting position. On one hand, minimizing the output for passing tests is often valuable. On the other hand, we regularly have contributors adding println! and debug! to code that executes during a passing test and becoming confused when no output appears. --log-mach is completely undiscoverable, so I lean towards addressing that problem. Maybe we could change the default based on whether a singles test is being run versus multiple ones.

@jgraham
Copy link
Contributor

jgraham commented Jan 20, 2017

That seems very possible. Just change the default in tests/wpt/run.py according to len(kwargs.get("tests", [])) == 1 or something.

@jdm
Copy link
Member

jdm commented Jan 20, 2017

@cynicaldevil Want to revert the changes in this PR and make the one suggested in the previous comment instead?

@cynicaldevil
Copy link
Contributor Author

cynicaldevil commented Jan 20, 2017

Sure, I'll do it tomorrow.

@cynicaldevil cynicaldevil force-pushed the cynicaldevil:pass-output branch from a162fc8 to 3fd5122 Jan 21, 2017
@cynicaldevil
Copy link
Contributor Author

cynicaldevil commented Jan 21, 2017

Updated the PR.

@cynicaldevil
Copy link
Contributor Author

cynicaldevil commented Jan 22, 2017

Using print messages I found that wptrunner.setup_logging() does create two logger objects, but only the second one is used. If I commented out https://github.com/cynicaldevil/servo/blob/3fd5122930e2d009769ac455e0d5ce4768b5a0af/tests/wpt/run.py#L34 and ran a single test, I found that aside from some stderr messages and test metadata, everything was being printed.

@jdm
Copy link
Member

jdm commented Jan 22, 2017

What if we make the code clearer by moving the other logger into an else branch?

@cynicaldevil cynicaldevil force-pushed the cynicaldevil:pass-output branch from 3fd5122 to fa3639d Jan 22, 2017
@cynicaldevil
Copy link
Contributor Author

cynicaldevil commented Jan 22, 2017

@jdm done.

@jdm
Copy link
Member

jdm commented Jan 22, 2017

@bors-servo
Copy link
Contributor

bors-servo commented Jan 22, 2017

📌 Commit fa3639d has been approved by jdm

@bors-servo
Copy link
Contributor

bors-servo commented Jan 22, 2017

Testing commit fa3639d with merge 478b4f2...

bors-servo added a commit that referenced this pull request Jan 22, 2017
WPT tests now show debug ouput for passing tests as well

<!-- Please describe your changes on the following line: -->

---
<!-- Thank you for contributing to Servo! Please replace each `[ ]` by `[X]` when the step is complete, and replace `__` with appropriate data: -->
- [X] `./mach build -d` does not report any errors
- [X] `./mach test-tidy` does not report any errors
- [X] These changes fix (partially) #12808
<!-- Either: -->

@jdm I thought it would be better to make a PR for my local changes anyway, since you wanted it :)
<!-- Pull requests that do not address these steps are welcome, but they will require additional verification as part of the review process. -->

<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/servo/servo/15124)
<!-- Reviewable:end -->
@jdm jdm closed this Jan 23, 2017
@jdm jdm reopened this Jan 23, 2017
@jdm
Copy link
Member

jdm commented Jan 23, 2017

@bors-servo
Copy link
Contributor

bors-servo commented Jan 23, 2017

💡 This pull request was already approved, no need to approve it again.

  • There's another pull request that is currently being tested, blocking this pull request: #15142
@bors-servo
Copy link
Contributor

bors-servo commented Jan 23, 2017

📌 Commit fa3639d has been approved by jdm

@mbrubeck mbrubeck reopened this Jan 24, 2017
@emilio emilio closed this Jan 24, 2017
@emilio emilio reopened this Jan 24, 2017
@emilio
Copy link
Member

emilio commented Jan 24, 2017

@bors-servo r- try- clean

@emilio
Copy link
Member

emilio commented Jan 24, 2017

@bors-servo r=jdm

@bors-servo
Copy link
Contributor

bors-servo commented Jan 24, 2017

📌 Commit fa3639d has been approved by jdm

@emilio
Copy link
Member

emilio commented Jan 24, 2017

Let's try syncing homu

@emilio emilio closed this Jan 24, 2017
@emilio emilio reopened this Jan 24, 2017
@Ms2ger
Copy link
Contributor

Ms2ger commented Jan 24, 2017

@cynicaldevil please find me on IRC when you can? Thanks

@Ms2ger Ms2ger closed this Jan 24, 2017
@Ms2ger Ms2ger reopened this Jan 24, 2017
@cynicaldevil cynicaldevil force-pushed the cynicaldevil:pass-output branch from fa3639d to 1551dba Jan 24, 2017
@Ms2ger
Copy link
Contributor

Ms2ger commented Jan 24, 2017

@bors-servo r=jdm

@bors-servo
Copy link
Contributor

bors-servo commented Jan 24, 2017

📌 Commit 1551dba has been approved by jdm

@bors-servo
Copy link
Contributor

bors-servo commented Jan 24, 2017

Testing commit 1551dba with merge 8511258...

bors-servo added a commit that referenced this pull request Jan 24, 2017
WPT tests now show debug ouput for passing tests as well

<!-- Please describe your changes on the following line: -->

---
<!-- Thank you for contributing to Servo! Please replace each `[ ]` by `[X]` when the step is complete, and replace `__` with appropriate data: -->
- [X] `./mach build -d` does not report any errors
- [X] `./mach test-tidy` does not report any errors
- [X] These changes fix (partially) #12808
<!-- Either: -->

@jdm I thought it would be better to make a PR for my local changes anyway, since you wanted it :)
<!-- Pull requests that do not address these steps are welcome, but they will require additional verification as part of the review process. -->

<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/servo/servo/15124)
<!-- Reviewable:end -->
@bors-servo
Copy link
Contributor

bors-servo commented Jan 24, 2017

💔 Test failed - windows-msvc-dev

@cynicaldevil
Copy link
Contributor Author

cynicaldevil commented Jan 24, 2017

@bors-servo retry

@bors-servo
Copy link
Contributor

bors-servo commented Jan 24, 2017

Previous build results for arm32, arm64, linux-dev, linux-rel-css, linux-rel-wpt, mac-dev-unit, mac-rel-css, mac-rel-wpt1, mac-rel-wpt2, windows-gnu-dev are reusable. Rebuilding only android, windows-msvc-dev...

@bors-servo
Copy link
Contributor

bors-servo commented Jan 24, 2017

@bors-servo bors-servo merged commit 1551dba into servo:master Jan 24, 2017
3 checks passed
3 checks passed
continuous-integration/appveyor/pr AppVeyor build succeeded
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details
homu Test successful
Details
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

None yet

9 participants
You can’t perform that action at this time.