Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use bail out instead of throw #27

Merged
merged 1 commit into from
Nov 21, 2019

Conversation

jbhoosreddy
Copy link
Collaborator

@jbhoosreddy jbhoosreddy commented Nov 21, 2019

Summary of changes

  1. Currently, there are 2 reason the comparison will bail out. a) incorrect args and b) different length of tests.
  2. When this happens, the circle CI output shows no test failure which may be fine since no test262 tests were run. But even looking at diff.tap only shows TAP version 13. The only way to see what happened is to sift through the logs.

With this change:
diff.tap should look like:

TAP version 13
Bail out! master branch and PR artifacts are out of sync. Possible reasons include intermittent-tests.txt or test262 sha was updated. Consider re-running the master branch test262 job to update the master branch artifact first!

0..0

This will still report it as an error in CircleCI job.

ref: https://testanything.org/tap-version-13-specification.html

node lib/compare-results master.tap test262.tap | $(npm bin)/tap-merge | $(npm bin)/tap-mocha-reporter xunit
<testsuite name="TAP Tests" tests="0" failures="0" errors="0" skipped="0" timestamp="Thu, 21 Nov 2019 13:50:26 GMT" time="9.568">
</testsuite>

babel-test262-runner on  master [$!?] is 📦 v1.0.0 via ⬢ v10.0.0 took 11s
➜ echo $?
1

@nicolo-ribaudo nicolo-ribaudo merged commit 87976f5 into babel:master Nov 21, 2019
@jbhoosreddy jbhoosreddy deleted the bail-out-errors branch November 21, 2019 17:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants