-
Notifications
You must be signed in to change notification settings - Fork 453
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
add parser tests #559
add parser tests #559
Conversation
I'm reluctant to accept this. On the one hand, I love tests, and more tests First, there's the issue of validation. As the official test suite for the More fundamentally, though, I'm concerned about test coverage. Quite a lot of effort goes in to organizing tests. This is crucial for the Because the tests in this patch eschew the organization we are working to Accepting this patch work would reflect an approach to project maintenance that That hypothetical may seem abstract, so I'll point out that it applies even
I think we have a responsibility to optimize the resources (in both space and I have a few ideas for alternatives that leverage the work here:
To be clear: I am not a member of TC-39. I'd really like to hear @goyakin and |
@jugglinmike, thanks for the feedback. I'm certainly interested in the conversation about the usefulness of these tests, how they should best be organized, and if they belong in test262 at all; that's part of the point of this PR. I'm not sure how to organize these tests according to the existing schema, particularly the failing tests. For example, it's not obvious to me what section of the spec Also, duplication is less of an issue than you might expect: that's the purpose of normalization. Only passing (and early error) tests are normalized, hence the duplication you see above, but then, it's not obvious to me what it means for two distinct texts to be duplicate failing tests. |
I believe resolution from the committee meeting was that these should go in a new repository under tc39, which would still be under the purview of the test262 project but not necessarily maintained in the same way. I'm leaving this PR open pending advancement on that front. |
@littledan Can you help create this repository and grant the appropriate parties write permission? |
Ping @littledan @bterlson. This is blocked on getting a new repo under the tc39 organisation. |
I guess I should read the notes, but in the meantime I can certainly create the repo if someone can tell me what to call it? |
I'd say |
Note that |
https://github.com/tc39/test262-parser-tests is there with the test262 group + @michaelficarra and @bakkot as admins. 💯 |
Parser tests are now published at https://github.com/tc39/test262-parser-tests. This can be closed. |
Closing as completed elsewhere. Sorry for the delay in getting this finalized. |
This PR adds several thousand tests designed to stress parsers lacking a corresponding evaluation engine, drawn from the test suites of the following open source projects:
See parser-tests/README.md for the full description. Briefly, there are source texts which should pass, fail, and produce early errors, and passing source texts have an alternative more legible version.
Tests have been rendered in a relatively uniform way, where possible, using a public node.js package. Because these tests are of a different kind than most of test262, their format is correspondingly different: files are not given descriptive names, and contain no header. Copyright information is provided in bulk in
licenses.md
.