Add tests for the RegExp Modifiers proposal

[rbuckton]

This adds tests for the Stage 3 RegExp Modifiers proposal.

[rbuckton]

Is there any action I need to take to help move this along? I'd like to unblock V8 if possible.

[ptomato]

Thanks for asking! What I think will need to happen here, that you could help with, is:

• Compare this PR with proposal-regexp-modifiers: Tests for syntax errors #3807 (which contains syntax tests for this feature) to see if there are any duplicated parts, and if so, which version to use
• Check if this PR covers the remainder of the cases from the testing plan in Tests for regular expression modifiers #3756

Also if we could get a look from a proposal champion or someone else familiar with the proposal, that would help. In this case it probably doesn't make sense for you to review your own PR, and it looks like you're the only champion, but maybe one of the proposal's stage 3 reviewers could lend a hand?

[rbuckton]

• Compare this PR with proposal-regexp-modifiers: Tests for syntax errors #3807 (which contains syntax tests for this feature) to see if there are any duplicated parts, and if so, which version to use

It looks like the following files in this PR are already covered by #3807:

• test/built-ins/RegExp/regexp-modifiers/syntax/invalid/add-modifiers-duplicate-add-modifiers.js
• test/built-ins/RegExp/regexp-modifiers/syntax/invalid/add-modifiers-invalid-add-modifier.js
• test/built-ins/RegExp/regexp-modifiers/syntax/invalid/add-modifiers-invalid-flag-in-add-modifier.js
• test/built-ins/RegExp/regexp-modifiers/syntax/invalid/add-remove-modifiers-both-empty.js
• test/built-ins/RegExp/regexp-modifiers/syntax/invalid/add-remove-modifiers-duplicate-add-modifiers.js
• test/built-ins/RegExp/regexp-modifiers/syntax/invalid/add-remove-modifiers-duplicate-remove-modifiers.js
• test/built-ins/RegExp/regexp-modifiers/syntax/invalid/add-remove-modifiers-invalid-add-modifier.js
• test/built-ins/RegExp/regexp-modifiers/syntax/invalid/add-remove-modifiers-invalid-flag-in-add-modifier.js
• test/built-ins/RegExp/regexp-modifiers/syntax/invalid/add-remove-modifiers-invalid-flag-in-remove-modifier.js
• test/built-ins/RegExp/regexp-modifiers/syntax/invalid/add-remove-modifiers-invalid-remove-modifier.js
• test/built-ins/RegExp/regexp-modifiers/syntax/invalid/add-remove-modifiers-same-modifier-in-both.js
• test/built-ins/RegExp/regexp-modifiers/syntax/invalid/regexp-constructor-add-modifiers-duplicate-add-modifiers.js
• test/built-ins/RegExp/regexp-modifiers/syntax/invalid/regexp-constructor-add-modifiers-invalid-add-modifier.js
• test/built-ins/RegExp/regexp-modifiers/syntax/invalid/regexp-constructor-add-modifiers-invalid-flag-in-add-modifier.js
• test/built-ins/RegExp/regexp-modifiers/syntax/invalid/regexp-constructor-add-remove-modifiers-both-empty.js
• test/built-ins/RegExp/regexp-modifiers/syntax/invalid/regexp-constructor-add-remove-modifiers-duplicate-add-modifiers.js
• test/built-ins/RegExp/regexp-modifiers/syntax/invalid/regexp-constructor-add-remove-modifiers-duplicate-remove-modifiers.js
• test/built-ins/RegExp/regexp-modifiers/syntax/invalid/regexp-constructor-add-remove-modifiers-invalid-add-modifier.js
• test/built-ins/RegExp/regexp-modifiers/syntax/invalid/regexp-constructor-add-remove-modifiers-invalid-flag-in-add-modifier.js
• test/built-ins/RegExp/regexp-modifiers/syntax/invalid/regexp-constructor-add-remove-modifiers-invalid-flag-in-remove-modifier.js
• test/built-ins/RegExp/regexp-modifiers/syntax/invalid/regexp-constructor-add-remove-modifiers-invalid-remove-modifier.js
• test/built-ins/RegExp/regexp-modifiers/syntax/invalid/regexp-constructor-add-remove-modifiers-same-modifier-in-both.js

Since #3807 is more comprehensive for those cases, I would defer to that PR and will remove the duplicates from this PR.

• Check if this PR covers the remainder of the cases from the testing plan in Tests for regular expression modifiers #3756

This covers some, but not all, of the remaining test cases:

• Valid syntax
  • Basic regular expression flags parse correctly
    • Source text uses unicode escape sequences to express code points i, m, s
  • Source text with any valid combination of flags or arithmetic flags - reasonable to enumerate
• Behavior
  • Disabling flag in subexpression behaves correctly when corresponding top-level flag is and isn't already set
  • Enabling flag in subexpression behaves correctly when corresponding top-level flag is and isn't already set
  • Constructing a RegExp from a literal but changing flags by an argument to the RegExp constructor does (or does not) correctly change behavior of a subexpression that enables or removes flags.
  • i
    • Ignore case applies appropriately inside subexpression, but not outside; when turned on, off, and when nested inside a subexpression that has previously modified behavior
    • Behavior as normal when other flags modified but i flag not modified
    • Callers of Canonicalize:
      • Backreferences ignore case in captures
      • Individual characters ignore case
      • Character sets ignore case
      • Character escapes ignore case
      • Character class escapes ignore case
      • \w class, \b, \B all ignore case
  • m
    • ^ and $ apply appropriately inside subexpression, but not outside; when turned on, off, and when nested inside a subexpression that has previously modified behavior
  • s
    • . applies appropriately inside subexpression, but not outside; when turned on, off, and when nested inside a subexpression that has previously modified behavior
  • Subexpressions with flags set do not cause RegExp()....flags or /.../.flags to have the flags set, e.g. (new RegExp("(?i:a)")).flags does not include i.
  • ^ for RegExp.prototype.dotAll, .multiline, ignoreCase

I'll look into adding the missing test cases over the next few days.

Also if we could get a look from a proposal champion or someone else familiar with the proposal, that would help. In this case it probably doesn't make sense for you to review your own PR, and it looks like you're the only champion, but maybe one of the proposal's stage 3 reviewers could lend a hand?

I reviewed #3807, but it looks like it's blocked by a test generation issue.

[rbuckton]

The last commit should cover the majority of the remaining portions of the test plan, and removes tests which are redundant with #3807. The only thing not yet covered is:

Source text uses unicode escape sequences to express code points i, m, s

This is listed as a test for valid syntax, but as far as I am aware /(?\u0069:)/ should be a syntax error and should be added to #3807.

[ioannad]

@rbuckton about the overlapping tests between this and #3807 , I think we should prioritise landing the tests in this PR first. So I will spend the next few days taking a look at this PR and try to make sure we can land it asap.

If there are parts of the test plan in #3756 that are not covered here, then we can think of a follow up PR. I could even offer to create such a follow up, unless you prefer to do it.

[rbuckton]

@ioannad I think the only thing missing between this and #3807 is this:

Source text uses unicode escape sequences to express code points i, m, s

Though I would argue the test plan should indicate that this is not supported, i.e.

Source text cannot use unicode escape sequences to express code points i, m, s

Since #3807 seems mostly to cover syntax errors, it seems that would be a better place for this test.

[rbuckton]

@gibson042: As a reviewer for the proposal, I'd appreciate a review of the tests if you have the time.

[ioannad]

Apart from the suggested changes, looks good to me. I am not sure whether the frontmatter should be more customised to each test, but it's probably not that important, and I think it's good to be merged as is, modulo suggestions.

[ptomato]

My review is more superficial but Ioanna's detailed review makes me confident we can merge this. Here's the maintainer stamp. @gibson042 if you still plan to do a review, give a shout by tomorrow, otherwise let's plan to merge this.

[ptomato]

Wow, this was a spicy one. Good that you included it.