Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement full .errors verification in tests #2838

Closed
RossPatterson opened this issue Sep 22, 2022 · 3 comments · Fixed by #2855
Closed

Implement full .errors verification in tests #2838

RossPatterson opened this issue Sep 22, 2022 · 3 comments · Fixed by #2855

Comments

@RossPatterson
Copy link
Contributor

In grammars-v4 issue 2821, I wrote:

I'm experimenting now with making trgen verify ... the .errors reference files, as antlr4test-maven-plugin does in AssertErrorsErrorListener ... I'm interested in the .errors capability because it seems like it should be fully supported.

As I said, I've got an experiment in progress to do this.

@teverett
Copy link
Member

@RossPatterson this is great news. Thank-you!

@RossPatterson
Copy link
Contributor Author

RossPatterson commented Sep 29, 2022

I encountered the grammars-v4/csharp example issue-2612 error described in antlr4 issue #3700 during work on this. I expect I'll have to add it to grammars-v4/_scripts/skip_csharp.txt.

@RossPatterson
Copy link
Contributor Author

At this point, I believe this work is complete, and PR #2855 is ready to merge.

  • The build-csharp CI job now skips csharp, due to antlr4 issue #3700.
  • The build-php CI job now skips logo/logo, due to issue [logo/logo] Unexpected error from example logo_feature_butfail_2.txt #2859.
  • The build-php CI job and all JavaScript build-pwsh CI jobs now skip python/python3alt, as it has Java code in the @members sections. All other non-Java builds already skipped it.
  • All test drivers (e.g., _scripts/*/Program.*) now dump out the detected errors to stdout in the expected format, and only dump the parse tree if there are no errors.
  • All test scripts (i.e., _scripts/*/test.sh, _scripts/*/tester.psm1) now check the detected errors against the reference files, and only check the parse tree if there are no errors.
  • The Java test driver now dumps out the errors in UTF-8. All the other test drivers already did so.
  • All the non-skipped grammars that have .errors files in their examples directory generate all and only those errors.
  • All non-skipped tests are succeeding.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants