Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix and enable the PatternsDataSet_GenerateInputsWithNonBacktracking_MatchWithAllEngines test #79373

Open
stephentoub opened this issue Dec 8, 2022 · 1 comment
Labels
area-System.Text.RegularExpressions disabled-test The test is disabled in source code against the issue
Milestone

Comments

@stephentoub
Copy link
Member

#79364 added a new test that iterates through all of the patterns we have in our patterns database and for each:

  • uses the NonBacktracking engine's SampleMatches function to generate inputs that match the pattern
  • calls IsMatch with each of those inputs on a Regex created for that pattern/option with each of our engines

The test is still disabled, however, as there's a variety of work required to get it "production ready":

  • SampleMatches ends up asserting on some patterns and hanging on others. This is likely one or more bugs in the SampleMatches implementation.
  • The test takes a really long time to execute.
  • It's only currently able to validate inputs expected to match. We also want to validate inputs expected to not match.
  • It's possible for some patterns it could generate inputs that take too long to process. We'll want to explore using timeouts.

cc: @olsaarik, @veanes

@stephentoub stephentoub added area-System.Text.RegularExpressions disabled-test The test is disabled in source code against the issue labels Dec 8, 2022
@stephentoub stephentoub added this to the Future milestone Dec 8, 2022
@ghost
Copy link

ghost commented Dec 8, 2022

Tagging subscribers to this area: @dotnet/area-system-text-regularexpressions
See info in area-owners.md if you want to be subscribed.

Issue Details

#79364 added a new test that iterates through all of the patterns we have in our patterns database and for each:

  • uses the NonBacktracking engine's SampleMatches function to generate inputs that match the pattern
  • calls IsMatch with each of those inputs on a Regex created for that pattern/option with each of our engines

The test is still disabled, however, as there's a variety of work required to get it "production ready":

  • SampleMatches ends up asserting on some patterns and hanging on others. This is likely one or more bugs in the SampleMatches implementation.
  • The test takes a really long time to execute.
  • It's only currently able to validate inputs expected to match. We also want to validate inputs expected to not match.
  • It's possible for some patterns it could generate inputs that take too long to process. We'll want to explore using timeouts.

cc: @olsaarik, @veanes

Author: stephentoub
Assignees: -
Labels:

area-System.Text.RegularExpressions, disabled-test

Milestone: Future

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area-System.Text.RegularExpressions disabled-test The test is disabled in source code against the issue
Projects
None yet
Development

No branches or pull requests

1 participant