Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Merged by Bors] - Correctly run async tests #2683

Closed
wants to merge 5 commits into from
Closed

Conversation

jedel1043
Copy link
Member

So, while working on #2658 I apparently broke 1280 tests... except I didn't! Turns out we were considering async tests that didn't call the print function as passed tests, but the async section of https://github.com/tc39/test262/blob/main/INTERPRETING.md#flags says:

The test must not be considered complete until the implementation-defined print function has been invoked or some length of time has passed without any such invocation.

@jedel1043 jedel1043 added bug Something isn't working test Issues and PRs related to the tests. labels Mar 16, 2023
@jedel1043 jedel1043 added this to the v0.17.0 milestone Mar 16, 2023
@codecov
Copy link

codecov bot commented Mar 16, 2023

Codecov Report

Merging #2683 (99800de) into main (c9ae4dc) will increase coverage by 0.31%.
The diff coverage is 0.00%.

@@            Coverage Diff             @@
##             main    #2683      +/-   ##
==========================================
+ Coverage   49.17%   49.49%   +0.31%     
==========================================
  Files         397      393       -4     
  Lines       39748    39473     -275     
==========================================
- Hits        19545    19536       -9     
+ Misses      20203    19937     -266     
Impacted Files Coverage Δ
boa_tester/src/exec/mod.rs 0.00% <0.00%> (ø)

... and 12 files with indirect coverage changes

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

@jedel1043
Copy link
Member Author

Test262 conformance changes

Test result main count PR count difference
Total 94,277 94,277 0
Passed 71,994 70,714 -1,280
Ignored 17,324 17,324 0
Failed 4,959 6,239 +1,280
Panics 14 14 0
Conformance 76.36% 75.01% -1.36%
Broken tests (1280):
(1280 tests)

boa_tester/src/exec/mod.rs Outdated Show resolved Hide resolved
Copy link
Member

@Razican Razican left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch! I think we can merge it with a bit of documentation in the new result structure.

boa_tester/src/exec/mod.rs Show resolved Hide resolved
Copy link
Member

@HalidOdat HalidOdat left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me! :)

@Razican
Copy link
Member

Razican commented Mar 17, 2023

bors r+

bors bot pushed a commit that referenced this pull request Mar 17, 2023
So, while working on #2658 I apparently broke 1280 tests... except I didn't! Turns out we were considering async tests that didn't call the `print` function as passed tests, but the `async` section of https://github.com/tc39/test262/blob/main/INTERPRETING.md#flags says:
> The test must not be considered complete until the implementation-defined print function has been invoked or some length of time has passed without any such invocation.
@bors
Copy link

bors bot commented Mar 17, 2023

Pull request successfully merged into main.

Build succeeded:

@bors bors bot changed the title Correctly run async tests [Merged by Bors] - Correctly run async tests Mar 17, 2023
@bors bors bot closed this Mar 17, 2023
@bors bors bot deleted the good-async-tests branch March 17, 2023 16:44
bors bot pushed a commit that referenced this pull request Mar 21, 2023
~Depends on #2683.~ Merged.

This Pull Request fixes #2658.

It changes the following:

- Makes `for .. of` loop execution more spec compliant.
- Rewrites iterator related opcodes to be able to use it on all for .. of/in loops.
- Adds some utility op codes.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working test Issues and PRs related to the tests.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants