Implement RegExp `v` flag with set notation + properties of strings #10984

msaboff · 2023-03-03T06:12:51Z

`270c824`

Implement RegExp `v` flag with set notation + properties of strings
https://bugs.webkit.org/show_bug.cgi?id=241593
rdar://100337109

Reviewed by Yusuke Suzuki.

This change implements the TC39 stage 3 proposal RegExp v flag with set notation + properties of strings,
https://github.com/tc39/proposal-regexp-v-flag.  It adds a new "unicodeSets" compile mode for the Yarr engine.
Like the prior Unicode Yarr features, this change is driven by Unicode Database Files (UCD).
This change includes two such new files, JavaScriptCore/ucd/{emoji-sequences.txt & emoji-zwj-sequences.txt}.

The newly added properties include lists of strings.  These strings are processed via the character class syntax
through.  When it comes to matching however, there is some desuguraing that turns such a property of strings into
a list of alternations.  For example, say a property has strings str1...strN plus a traditional character class,
single-character-class, we create the pattern equivalent of:
     (?:str1|str2|...|strN|[single-character-class])
Per the spec, longer strings appear earlier in the alternation, and before the traditional character class.
This allows for searching for longer properties in a property list where substrings of other strings are included
in that list.

There are new set of combining operators allowed in the class set character classes.  Two character class elements
that appear adjacent to each other implicitly have the Union combining operations.  There is also an Intersection
operation with the && operator and a Subtraction operation with the || operator.

There is new ClassSet parsing that follows new "cleaner" rules that traditional character classes.
The prior ccharacter class constructor and delegates are mostly unchanged, except for the compile mode
now being switched on an enum instead of a bool.

Added check that both 'u' and 'v' flags don't appear in the same RegExp.

Added unicodeSets getter watchpoint to the m_regExpPrimordialPropertiesWatchpointSet.

* JSTests/es6/Proxy_internal_get_calls_RegExp.prototype.flags.js:
* JSTests/stress/regexp-vflag-property-of-strings.js: Added.
(arrayToString):
(objectToString):
(dumpValue):
(compareArray):
(compareGroups):
(testRegExp):
(testRegExpSyntaxError):
* JSTests/stress/static-getter-in-names.js:
* JSTests/test262/config.yaml:
* LayoutTests/js/Object-getOwnPropertyNames-expected.txt:
* LayoutTests/js/script-tests/Object-getOwnPropertyNames.js:
* Source/JavaScriptCore/JavaScriptCore.xcodeproj/project.pbxproj:
* Source/JavaScriptCore/builtins/BuiltinNames.h:
* Source/JavaScriptCore/builtins/RegExpPrototype.js:
(linkTimeConstant.hasObservableSideEffectsForRegExpMatch):
(linkTimeConstant.hasObservableSideEffectsForRegExpSplit):
(overriddenName.string_appeared_here.split):
* Source/JavaScriptCore/builtins/StringPrototype.js:
(linkTimeConstant.hasObservableSideEffectsForStringReplace):
* Source/JavaScriptCore/bytecode/LinkTimeConstant.h:
* Source/JavaScriptCore/dfg/DFGAbstractInterpreterInlines.h:
(JSC::DFG::AbstractInterpreter<AbstractStateType>::executeEffects):
* Source/JavaScriptCore/dfg/DFGFixupPhase.cpp:
(JSC::DFG::FixupPhase::addStringReplacePrimordialChecks):
* Source/JavaScriptCore/dfg/DFGOperations.cpp:
(JSC::DFG::JSC_DEFINE_JIT_OPERATION):
* Source/JavaScriptCore/dfg/DFGStrengthReductionPhase.cpp:
(JSC::DFG::StrengthReductionPhase::handleNode):
* Source/JavaScriptCore/runtime/CachedTypes.cpp:
* Source/JavaScriptCore/runtime/CommonIdentifiers.h:
* Source/JavaScriptCore/runtime/JSGlobalObject.cpp:
(JSC::JSGlobalObject::init):
* Source/JavaScriptCore/runtime/JSGlobalObject.h:
* Source/JavaScriptCore/runtime/JSGlobalObjectInlines.h:
(JSC::JSGlobalObject::regExpProtoUnicodeSetsGetter const):
* Source/JavaScriptCore/runtime/RegExp.h:
* Source/JavaScriptCore/runtime/RegExpCache.h:
* Source/JavaScriptCore/runtime/RegExpObject.cpp:
(JSC::RegExpObject::matchGlobal):
* Source/JavaScriptCore/runtime/RegExpPrototype.cpp:
(JSC::RegExpPrototype::finishCreation):
(JSC::JSC_DEFINE_HOST_FUNCTION):
* Source/JavaScriptCore/ucd/emoji-sequences.txt: Added.
* Source/JavaScriptCore/ucd/emoji-zwj-sequences.txt: Added.
* Source/JavaScriptCore/yarr/Yarr.h:
* Source/JavaScriptCore/yarr/YarrErrorCode.cpp:
(JSC::Yarr::errorMessage):
(JSC::Yarr::errorToThrow):
* Source/JavaScriptCore/yarr/YarrErrorCode.h:
* Source/JavaScriptCore/yarr/YarrFlags.h:
* Source/JavaScriptCore/yarr/YarrInterpreter.cpp:
(JSC::Yarr::ByteTermDumper::ByteTermDumper):
(JSC::Yarr::ByteTermDumper::unicode const):
(JSC::Yarr::ByteTermDumper::unicodeSets const):
(JSC::Yarr::ByteTermDumper::eitherUnicode const):
(JSC::Yarr::Interpreter::tryConsumeBackReference):
(JSC::Yarr::Interpreter::matchCharacterClass):
(JSC::Yarr::Interpreter::backtrackCharacterClass):
(JSC::Yarr::Interpreter::matchDisjunction):
(JSC::Yarr::Interpreter::Interpreter):
(JSC::Yarr::Interpreter::isLegacyCompilation const):
(JSC::Yarr::Interpreter::isUnicodeCompilation const):
(JSC::Yarr::Interpreter::isUnicodeSetsCompilation const):
(JSC::Yarr::Interpreter::isEitherUnicodeCompilation const):
(JSC::Yarr::ByteTermDumper::dumpTerm):
(JSC::Yarr::ByteTermDumper::unicode): Deleted.
* Source/JavaScriptCore/yarr/YarrInterpreter.h:
(JSC::Yarr::BytecodePattern::BytecodePattern):
(JSC::Yarr::BytecodePattern::compileMode const):
(JSC::Yarr::BytecodePattern::unicodeSets const):
(JSC::Yarr::BytecodePattern::eitherUnicode const):
* Source/JavaScriptCore/yarr/YarrJIT.cpp:
* Source/JavaScriptCore/yarr/YarrParser.h:
(JSC::Yarr::Parser::CharacterClassParserDelegate::CharacterClassParserDelegate):
(JSC::Yarr::Parser::ClassSetParserDelegate::ClassSetParserDelegate):
(JSC::Yarr::Parser::ClassSetParserDelegate::begin):
(JSC::Yarr::Parser::ClassSetParserDelegate::nestedClassBegin):
(JSC::Yarr::Parser::ClassSetParserDelegate::doneAfterCharacterClassEnd):
(JSC::Yarr::Parser::ClassSetParserDelegate::setUnionOp):
(JSC::Yarr::Parser::ClassSetParserDelegate::setSubtractOp):
(JSC::Yarr::Parser::ClassSetParserDelegate::setIntersectionOp):
(JSC::Yarr::Parser::ClassSetParserDelegate::flushCachedCharacterIfNeeded):
(JSC::Yarr::Parser::ClassSetParserDelegate::atomPatternCharacter):
(JSC::Yarr::Parser::ClassSetParserDelegate::atomBuiltInCharacterClass):
(JSC::Yarr::Parser::ClassSetParserDelegate::end):
(JSC::Yarr::Parser::ClassSetParserDelegate::error):
(JSC::Yarr::Parser::ClassSetParserDelegate::assertionWordBoundary):
(JSC::Yarr::Parser::ClassSetParserDelegate::atomBackReference):
(JSC::Yarr::Parser::ClassSetParserDelegate::atomNamedBackReference):
(JSC::Yarr::Parser::ClassSetParserDelegate::atomNamedForwardReference):
(JSC::Yarr::Parser::ClassStringDisjunctionParserDelegate::ClassStringDisjunctionParserDelegate):
(JSC::Yarr::Parser::ClassStringDisjunctionParserDelegate::atomPatternCharacter):
(JSC::Yarr::Parser::ClassStringDisjunctionParserDelegate::newAlternative):
(JSC::Yarr::Parser::ClassStringDisjunctionParserDelegate::end):
(JSC::Yarr::Parser::ClassStringDisjunctionParserDelegate::assertionWordBoundary):
(JSC::Yarr::Parser::ClassStringDisjunctionParserDelegate::atomBackReference):
(JSC::Yarr::Parser::ClassStringDisjunctionParserDelegate::atomNamedBackReference):
(JSC::Yarr::Parser::ClassStringDisjunctionParserDelegate::atomNamedForwardReference):
(JSC::Yarr::Parser::ClassStringDisjunctionParserDelegate::atomBuiltInCharacterClass):
(JSC::Yarr::Parser::Parser):
(JSC::Yarr::Parser::isIdentityEscapeAnError):
(JSC::Yarr::Parser::parseEscape):
(JSC::Yarr::Parser::consumePossibleSurrogatePair):
(JSC::Yarr::Parser::parseAtomEscape):
(JSC::Yarr::Parser::parseCharacterClassEscape):
(JSC::Yarr::Parser::parseClassSetEscape):
(JSC::Yarr::Parser::parseClassStringDisjunctionEscape):
(JSC::Yarr::Parser::parseCharacterClass):
(JSC::Yarr::Parser::parseClassSet):
(JSC::Yarr::Parser::parseClassStringDisjunction):
(JSC::Yarr::Parser::parseParenthesesEnd):
(JSC::Yarr::Parser::parseTokens):
(JSC::Yarr::Parser::handleIllegalReferences):
(JSC::Yarr::Parser::tryConsumeUnicodeEscape):
(JSC::Yarr::Parser::tryConsumeUnicodePropertyExpression):
(JSC::Yarr::Parser::isLegacyCompilation const):
(JSC::Yarr::Parser::isUnicodeCompilation const):
(JSC::Yarr::Parser::isUnicodeSetsCompilation const):
(JSC::Yarr::Parser::isEitherUnicodeCompilation const):
(JSC::Yarr::compileMode):
(JSC::Yarr::parse):
* Source/JavaScriptCore/yarr/YarrPattern.cpp:
(JSC::Yarr::CharacterClassConstructor::CharacterClassConstructor):
(JSC::Yarr::CharacterClassConstructor::reset):
(JSC::Yarr::CharacterClassConstructor::combiningSetOp):
(JSC::Yarr::CharacterClassConstructor::append):
(JSC::Yarr::CharacterClassConstructor::appendInverted):
(JSC::Yarr::CharacterClassConstructor::putRange):
(JSC::Yarr::CharacterClassConstructor::atomClassStringDisjunction):
(JSC::Yarr::CharacterClassConstructor::performSetOpWith):
(JSC::Yarr::CharacterClassConstructor::performSetOpWithStrings):
(JSC::Yarr::CharacterClassConstructor::performSetOpWithMatches):
(JSC::Yarr::CharacterClassConstructor::hasInverteStrings):
(JSC::Yarr::CharacterClassConstructor::compareUTF32Strings):
(JSC::Yarr::CharacterClassConstructor::sort):
(JSC::Yarr::CharacterClassConstructor::charClass):
(JSC::Yarr::CharacterClassConstructor::mergeRangesFrom):
(JSC::Yarr::CharacterClassConstructor::unionStrings):
(JSC::Yarr::CharacterClassConstructor::intersectionStrings):
(JSC::Yarr::CharacterClassConstructor::subtractionStrings):
(JSC::Yarr::CharacterClassConstructor::asciiOpSorted):
(JSC::Yarr::CharacterClassConstructor::unicodeOpSorted):
(JSC::Yarr::YarrPatternConstructor::YarrPatternConstructor):
(JSC::Yarr::YarrPatternConstructor::resetForReparsing):
(JSC::Yarr::YarrPatternConstructor::atomPatternCharacter):
(JSC::Yarr::YarrPatternConstructor::atomBuiltInCharacterClass):
(JSC::Yarr::YarrPatternConstructor::atomCharacterClassAtom):
(JSC::Yarr::YarrPatternConstructor::atomCharacterClassRange):
(JSC::Yarr::YarrPatternConstructor::atomCharacterClassBuiltIn):
(JSC::Yarr::YarrPatternConstructor::atomClassStringDisjunction):
(JSC::Yarr::YarrPatternConstructor::atomCharacterClassSetOp):
(JSC::Yarr::YarrPatternConstructor::atomCharacterClassPushNested):
(JSC::Yarr::YarrPatternConstructor::atomCharacterClassPopNested):
(JSC::Yarr::YarrPatternConstructor::atomCharacterClassEnd):
(JSC::Yarr::YarrPattern::compile):
(JSC::Yarr::PatternTerm::dump):
(JSC::Yarr::YarrPattern::dumpPatternString):
(JSC::Yarr::YarrPattern::dumpPattern):
* Source/JavaScriptCore/yarr/YarrPattern.h:
(JSC::Yarr::CharacterClass::CharacterClass):
(JSC::Yarr::CharacterClass::hasNonBMPCharacters const):
(JSC::Yarr::CharacterClass::hasOneCharacterSize const):
(JSC::Yarr::CharacterClass::hasOnlyNonBMPCharacters const):
(JSC::Yarr::CharacterClass::hasStrings const):
(JSC::Yarr::CharacterClass::hasSingleCharacters const):
(JSC::Yarr::ClassSet::ClassSet):
(JSC::Yarr::YarrPattern::unicodeSets const):
(JSC::Yarr::YarrPattern::eitherUnicode const):
(JSC::Yarr::YarrPattern::compileMode const):
(JSC::Yarr::CharacterClass::hasNonBMPCharacters): Deleted.
(JSC::Yarr::CharacterClass::hasOneCharacterSize): Deleted.
(JSC::Yarr::CharacterClass::hasOnlyNonBMPCharacters): Deleted.
* Source/JavaScriptCore/yarr/YarrSyntaxChecker.cpp:
(JSC::Yarr::SyntaxChecker::atomClassStringDisjunction):
(JSC::Yarr::SyntaxChecker::atomCharacterClassSetOp):
(JSC::Yarr::SyntaxChecker::atomCharacterClassPushNested):
(JSC::Yarr::SyntaxChecker::atomCharacterClassPopNested):
(JSC::Yarr::checkSyntax):
* Source/JavaScriptCore/yarr/YarrUnicodeProperties.cpp:
(JSC::Yarr::unicodeMatchProperty):
(JSC::Yarr::createUnicodeCharacterClassFor):
(JSC::Yarr::characterClassMayContainStrings):
* Source/JavaScriptCore/yarr/YarrUnicodeProperties.h:
* Source/JavaScriptCore/yarr/generateYarrUnicodePropertyTables.py:
(PropertyData.__init__):
(PropertyData.addMatchString):
(PropertyData.stringsCompare):
(PropertyData):
(PropertyData.sortStrings):
(PropertyData.dumpMatchData):
(PropertyData.convertStringToCppFormat):
(PropertyData.dump):
(PropertyData.dumpAll):
(PropertyData.dumpMayContainStringFunc):
(BinaryProperty.dump):
(SequenceProperty):
(SequenceProperty.__init__):
(SequenceProperty.parsePropertyFile):
(SequenceProperty.dump):
* Source/WebCore/contentextensions/URLFilterParser.cpp:
(WebCore::ContentExtensions::PatternParser::atomClassStringDisjunction):
(WebCore::ContentExtensions::PatternParser::atomCharacterClassSetOp):
(WebCore::ContentExtensions::PatternParser::atomCharacterClassPushNested):
(WebCore::ContentExtensions::PatternParser::atomCharacterClassPopNested):
(WebCore::ContentExtensions::URLFilterParser::addPattern):

Canonical link: https://commits.webkit.org/261188@main

033aaa0

Misc	iOS, tvOS & watchOS	macOS	Linux	Windows
~~🧪 style~~	~~🛠 ios~~	~~🛠 mac~~	~~🛠 wpe~~	~~🛠 wincairo~~
~~🧪 bindings~~	~~🛠 ios-sim~~	~~🛠 mac-AS-debug~~	~~🧪 wpe-wk2~~
✅ 🧪 webkitperl	~~🧪 ios-wk2~~	~~🧪 api-mac~~	~~🛠 gtk~~
	~~🧪 api-ios~~	~~🧪 mac-wk1~~	~~🧪 gtk-wk2~~
~~🛠 🧪 jsc~~	~~🛠 tv~~	~~🧪 mac-wk2~~	~~🧪 api-gtk~~
✅ 🛠 🧪 jsc-arm64	~~🛠 tv-sim~~	~~🧪 mac-AS-debug-wk2~~	✅ 🛠 jsc-armv7
	~~🛠 watch~~	~~🧪 mac-wk2-stress~~	~~🧪 jsc-armv7-tests~~
	~~🛠 watch-sim~~		✅ 🛠 jsc-mips
❌ 🛠 🧪 unsafe-merge			~~🧪 jsc-mips-tests~~

webkit-early-warning-system · 2023-03-03T06:14:23Z

EWS run on previous version of this PR (hash 64ed139)

Misc	iOS, tvOS & watchOS	macOS	Linux	Windows
❌ 🧪 style	❌ 🛠 ios	❌ 🛠 mac	✅ 🛠 wpe	✅ 🛠 wincairo
✅ 🧪 bindings	❌ 🛠 ios-sim	❌ 🛠 mac-AS-debug	❌ 🧪 wpe-wk2
✅ 🧪 webkitperl	❌ 🧪 ios-wk2	❌ 🧪 api-mac	❌ 🛠 gtk
	❌ 🧪 api-ios	❌ 🧪 mac-wk1	❌ 🧪 gtk-wk2
❌ 🛠 🧪 jsc	⏳ 🛠 tv	❌ 🧪 mac-wk2	❌ 🧪 api-gtk
❌ 🛠 🧪 jsc-arm64	❌ 🛠 tv-sim	❌ 🧪 mac-AS-debug-wk2	✅ 🛠 jsc-armv7
	❌ 🛠 watch	❌ 🧪 mac-wk2-stress	❌ 🧪 jsc-armv7-tests
	❌ 🛠 watch-sim		✅ 🛠 jsc-mips
			❌ 🧪 jsc-mips-tests

Constellation · 2023-03-03T06:18:12Z

Source/JavaScriptCore/builtins/StringPrototype.js

+    var regexpUnicode = @tryGetById(regexp, "unicodeSets");
+    if (regexpUnicode !== @regExpProtoUnicodeGetter)
+        return true;
+


You need to add optimization for unicodeSets. See JSGlobalObject.cpp etc. This is comparing with @regExpProtoUnicodeGetter, which is unicode getter.

Also need to add optimization done for unicode getter etc. Like, DFGAbstractInterpreterInlines.h etc. for example.

See DFGFixupPhase.cpp and DFGAbstractInterpreterInlines.h too. Let's grep with propertyNames->global for example.

I fixed these.

Constellation · 2023-03-03T06:19:42Z

Source/JavaScriptCore/builtins/RegExpPrototype.js

+    var regexpUnicode = @tryGetById(regexp, "unicodeSets");
+    if (regexpUnicode !== @regExpProtoUnicodeGetter)
+        return true;


Constellation · 2023-03-03T06:20:37Z

Source/JavaScriptCore/builtins/RegExpPrototype.js

+    var regexpUnicode = @tryGetById(regexp, "unicodeSets");
+    if (regexpUnicode !== @regExpProtoUnicodeGetter)
+        return true;


Ditto. This is unicode getter.

rkirsling

Please be sure to update the test262 results in expectations.yaml; in addition to the 7(x2) outstanding failures introduced by #10355 (displayed here), this patch appears to introduce an additional 2(x2) failures:

test/built-ins/RegExp/prototype/unicodeSets/uv-flags-constructor.js:
  default: 'Test262Error: Expected a SyntaxError to be thrown but no exception was thrown at all'
  strict mode: 'Test262Error: Expected a SyntaxError to be thrown but no exception was thrown at all'
test/built-ins/RegExp/prototype/unicodeSets/uv-flags.js:
  default: 'Test262: This statement should not be evaluated.'
  strict mode: 'Test262: This statement should not be evaluated.'

webkit-early-warning-system · 2023-03-03T19:33:27Z

EWS run on previous version of this PR (hash b632354)

Misc	iOS, tvOS & watchOS	macOS	Linux	Windows
❌ 🧪 style	✅ 🛠 ios	✅ 🛠 mac	✅ 🛠 wpe	✅ 🛠 wincairo
✅ 🧪 bindings	✅ 🛠 ios-sim	✅ 🛠 mac-AS-debug	~~🧪 wpe-wk2~~
✅ 🧪 webkitperl	~~🧪 ios-wk2~~	~~🧪 api-mac~~	✅ 🛠 gtk
	~~🧪 api-ios~~	✅ 🧪 mac-wk1	~~🧪 gtk-wk2~~
✅ 🛠 🧪 jsc	⏳ 🛠 tv	~~🧪 mac-wk2~~	~~🧪 api-gtk~~
✅ 🛠 🧪 jsc-arm64	✅ 🛠 tv-sim	~~🧪 mac-AS-debug-wk2~~	✅ 🛠 jsc-armv7
	✅ 🛠 watch	~~🧪 mac-wk2-stress~~	✅ 🧪 jsc-armv7-tests
	⏳ 🛠 watch-sim		✅ 🛠 jsc-mips
			✅ 🧪 jsc-mips-tests

msaboff · 2023-03-03T19:36:44Z

Please be sure to update the test262 results in expectations.yaml; in addition to the 7(x2) outstanding failures introduced by #10355 (displayed here), this patch appears to introduce an additional 2(x2) failures:

test/built-ins/RegExp/prototype/unicodeSets/uv-flags-constructor.js:
  default: 'Test262Error: Expected a SyntaxError to be thrown but no exception was thrown at all'
  strict mode: 'Test262Error: Expected a SyntaxError to be thrown but no exception was thrown at all'
test/built-ins/RegExp/prototype/unicodeSets/uv-flags.js:
  default: 'Test262: This statement should not be evaluated.'
  strict mode: 'Test262: This statement should not be evaluated.'

@rkirsling I'll update test262 results in a follow on PR.

Constellation · 2023-03-03T19:48:41Z

Source/JavaScriptCore/dfg/DFGAbstractInterpreterInlines.h

@@ -3555,6 +3555,9 @@ bool AbstractInterpreter<AbstractStateType>::executeEffects(unsigned clobberLimi
                    if (attemptToFold(m_vm.propertyNames->unicode.impl(), globalObject->regExpProtoUnicodeGetter()))
                        break;

+                    if (attemptToFold(m_vm.propertyNames->unicode.impl(), globalObject->regExpProtoUnicodeSetsGetter()))


Not unicode.

Copy paste error - Fixed.

webkit-early-warning-system · 2023-03-03T22:28:00Z

EWS run on previous version of this PR (hash 0045ca6)

Misc	iOS, tvOS & watchOS	macOS	Linux	Windows
❌ 🧪 style	✅ 🛠 ios	⏳ 🛠 mac	⏳ 🛠 wpe	⏳ 🛠 wincairo
✅ 🧪 bindings	✅ 🛠 ios-sim	~~🛠 mac-AS-debug~~	⏳ 🧪 wpe-wk2
✅ 🧪 webkitperl	~~🧪 ios-wk2~~	⏳ 🧪 api-mac	✅ 🛠 gtk
	~~🧪 api-ios~~	⏳ 🧪 mac-wk1	~~🧪 gtk-wk2~~
⏳ 🛠 🧪 jsc	⏳ 🛠 tv	⏳ 🧪 mac-wk2	~~🧪 api-gtk~~
✅ 🛠 🧪 jsc-arm64	⏳ 🛠 tv-sim	~~🧪 mac-AS-debug-wk2~~	✅ 🛠 jsc-armv7
	✅ 🛠 watch	⏳ 🧪 mac-wk2-stress	✅ 🧪 jsc-armv7-tests
	⏳ 🛠 watch-sim		✅ 🛠 jsc-mips
			✅ 🧪 jsc-mips-tests

msaboff · 2023-03-03T22:36:18Z

Please be sure to update the test262 results in expectations.yaml; in addition to the 7(x2) outstanding failures introduced by #10355 (displayed here), this patch appears to introduce an additional 2(x2) failures:

test/built-ins/RegExp/prototype/unicodeSets/uv-flags-constructor.js:
  default: 'Test262Error: Expected a SyntaxError to be thrown but no exception was thrown at all'
  strict mode: 'Test262Error: Expected a SyntaxError to be thrown but no exception was thrown at all'
test/built-ins/RegExp/prototype/unicodeSets/uv-flags.js:
  default: 'Test262: This statement should not be evaluated.'
  strict mode: 'Test262: This statement should not be evaluated.'

Fixed these failures in the latest version.

Constellation · 2023-03-03T23:31:51Z

Source/JavaScriptCore/runtime/JSGlobalObject.cpp

@@ -1416,6 +1416,9 @@ capitalName ## Constructor* lowerName ## Constructor = featureFlag ? capitalName
    GetterSetter* regExpProtoUnicodeGetter = getGetterById(this, m_regExpPrototype.get(), vm.propertyNames->unicode);
    catchScope.assertNoException();
    m_linkTimeConstants[static_cast<unsigned>(LinkTimeConstant::regExpProtoUnicodeGetter)].set(vm, this, regExpProtoUnicodeGetter);
+    GetterSetter* regExpProtoUnicodeSetsGetter = getGetterById(this, m_regExpPrototype.get(), vm.propertyNames->unicodeSets);
+    catchScope.assertNoException();
+    m_linkTimeConstants[static_cast<unsigned>(LinkTimeConstant::regExpProtoUnicodeSetsGetter)].set(vm, this, regExpProtoUnicodeSetsGetter);


I think RegExpPrototype watchpoint optimization is probably missing. Please grep all use of existing getters carefully and align the new one with this.

I grep'ed the other uses of the RegExp getters and only find uses in DFGAbstractInterpreterInlines.h and DFGFixupPhase. The unicodeSets getter is already added to those sections.

Constellation

rs=me with watchpoint fix.

webkit-early-warning-system · 2023-03-04T00:56:49Z

EWS run on current version of this PR (hash 033aaa0)

Misc	iOS, tvOS & watchOS	macOS	Linux	Windows
~~🧪 style~~	~~🛠 ios~~	~~🛠 mac~~	~~🛠 wpe~~	~~🛠 wincairo~~
~~🧪 bindings~~	~~🛠 ios-sim~~	~~🛠 mac-AS-debug~~	~~🧪 wpe-wk2~~
✅ 🧪 webkitperl	~~🧪 ios-wk2~~	~~🧪 api-mac~~	~~🛠 gtk~~
	~~🧪 api-ios~~	~~🧪 mac-wk1~~	~~🧪 gtk-wk2~~
~~🛠 🧪 jsc~~	~~🛠 tv~~	~~🧪 mac-wk2~~	~~🧪 api-gtk~~
✅ 🛠 🧪 jsc-arm64	~~🛠 tv-sim~~	~~🧪 mac-AS-debug-wk2~~	✅ 🛠 jsc-armv7
	~~🛠 watch~~	~~🧪 mac-wk2-stress~~	~~🧪 jsc-armv7-tests~~
	~~🛠 watch-sim~~		✅ 🛠 jsc-mips
❌ 🛠 🧪 unsafe-merge			~~🧪 jsc-mips-tests~~

https://bugs.webkit.org/show_bug.cgi?id=241593 rdar://100337109 Reviewed by Yusuke Suzuki. This change implements the TC39 stage 3 proposal RegExp v flag with set notation + properties of strings, https://github.com/tc39/proposal-regexp-v-flag. It adds a new "unicodeSets" compile mode for the Yarr engine. Like the prior Unicode Yarr features, this change is driven by Unicode Database Files (UCD). This change includes two such new files, JavaScriptCore/ucd/{emoji-sequences.txt & emoji-zwj-sequences.txt}. The newly added properties include lists of strings. These strings are processed via the character class syntax through. When it comes to matching however, there is some desuguraing that turns such a property of strings into a list of alternations. For example, say a property has strings str1...strN plus a traditional character class, single-character-class, we create the pattern equivalent of: (?:str1|str2|...|strN|[single-character-class]) Per the spec, longer strings appear earlier in the alternation, and before the traditional character class. This allows for searching for longer properties in a property list where substrings of other strings are included in that list. There are new set of combining operators allowed in the class set character classes. Two character class elements that appear adjacent to each other implicitly have the Union combining operations. There is also an Intersection operation with the && operator and a Subtraction operation with the || operator. There is new ClassSet parsing that follows new "cleaner" rules that traditional character classes. The prior ccharacter class constructor and delegates are mostly unchanged, except for the compile mode now being switched on an enum instead of a bool. Added check that both 'u' and 'v' flags don't appear in the same RegExp. Added unicodeSets getter watchpoint to the m_regExpPrimordialPropertiesWatchpointSet. * JSTests/es6/Proxy_internal_get_calls_RegExp.prototype.flags.js: * JSTests/stress/regexp-vflag-property-of-strings.js: Added. (arrayToString): (objectToString): (dumpValue): (compareArray): (compareGroups): (testRegExp): (testRegExpSyntaxError): * JSTests/stress/static-getter-in-names.js: * JSTests/test262/config.yaml: * LayoutTests/js/Object-getOwnPropertyNames-expected.txt: * LayoutTests/js/script-tests/Object-getOwnPropertyNames.js: * Source/JavaScriptCore/JavaScriptCore.xcodeproj/project.pbxproj: * Source/JavaScriptCore/builtins/BuiltinNames.h: * Source/JavaScriptCore/builtins/RegExpPrototype.js: (linkTimeConstant.hasObservableSideEffectsForRegExpMatch): (linkTimeConstant.hasObservableSideEffectsForRegExpSplit): (overriddenName.string_appeared_here.split): * Source/JavaScriptCore/builtins/StringPrototype.js: (linkTimeConstant.hasObservableSideEffectsForStringReplace): * Source/JavaScriptCore/bytecode/LinkTimeConstant.h: * Source/JavaScriptCore/dfg/DFGAbstractInterpreterInlines.h: (JSC::DFG::AbstractInterpreter<AbstractStateType>::executeEffects): * Source/JavaScriptCore/dfg/DFGFixupPhase.cpp: (JSC::DFG::FixupPhase::addStringReplacePrimordialChecks): * Source/JavaScriptCore/dfg/DFGOperations.cpp: (JSC::DFG::JSC_DEFINE_JIT_OPERATION): * Source/JavaScriptCore/dfg/DFGStrengthReductionPhase.cpp: (JSC::DFG::StrengthReductionPhase::handleNode): * Source/JavaScriptCore/runtime/CachedTypes.cpp: * Source/JavaScriptCore/runtime/CommonIdentifiers.h: * Source/JavaScriptCore/runtime/JSGlobalObject.cpp: (JSC::JSGlobalObject::init): * Source/JavaScriptCore/runtime/JSGlobalObject.h: * Source/JavaScriptCore/runtime/JSGlobalObjectInlines.h: (JSC::JSGlobalObject::regExpProtoUnicodeSetsGetter const): * Source/JavaScriptCore/runtime/RegExp.h: * Source/JavaScriptCore/runtime/RegExpCache.h: * Source/JavaScriptCore/runtime/RegExpObject.cpp: (JSC::RegExpObject::matchGlobal): * Source/JavaScriptCore/runtime/RegExpPrototype.cpp: (JSC::RegExpPrototype::finishCreation): (JSC::JSC_DEFINE_HOST_FUNCTION): * Source/JavaScriptCore/ucd/emoji-sequences.txt: Added. * Source/JavaScriptCore/ucd/emoji-zwj-sequences.txt: Added. * Source/JavaScriptCore/yarr/Yarr.h: * Source/JavaScriptCore/yarr/YarrErrorCode.cpp: (JSC::Yarr::errorMessage): (JSC::Yarr::errorToThrow): * Source/JavaScriptCore/yarr/YarrErrorCode.h: * Source/JavaScriptCore/yarr/YarrFlags.h: * Source/JavaScriptCore/yarr/YarrInterpreter.cpp: (JSC::Yarr::ByteTermDumper::ByteTermDumper): (JSC::Yarr::ByteTermDumper::unicode const): (JSC::Yarr::ByteTermDumper::unicodeSets const): (JSC::Yarr::ByteTermDumper::eitherUnicode const): (JSC::Yarr::Interpreter::tryConsumeBackReference): (JSC::Yarr::Interpreter::matchCharacterClass): (JSC::Yarr::Interpreter::backtrackCharacterClass): (JSC::Yarr::Interpreter::matchDisjunction): (JSC::Yarr::Interpreter::Interpreter): (JSC::Yarr::Interpreter::isLegacyCompilation const): (JSC::Yarr::Interpreter::isUnicodeCompilation const): (JSC::Yarr::Interpreter::isUnicodeSetsCompilation const): (JSC::Yarr::Interpreter::isEitherUnicodeCompilation const): (JSC::Yarr::ByteTermDumper::dumpTerm): (JSC::Yarr::ByteTermDumper::unicode): Deleted. * Source/JavaScriptCore/yarr/YarrInterpreter.h: (JSC::Yarr::BytecodePattern::BytecodePattern): (JSC::Yarr::BytecodePattern::compileMode const): (JSC::Yarr::BytecodePattern::unicodeSets const): (JSC::Yarr::BytecodePattern::eitherUnicode const): * Source/JavaScriptCore/yarr/YarrJIT.cpp: * Source/JavaScriptCore/yarr/YarrParser.h: (JSC::Yarr::Parser::CharacterClassParserDelegate::CharacterClassParserDelegate): (JSC::Yarr::Parser::ClassSetParserDelegate::ClassSetParserDelegate): (JSC::Yarr::Parser::ClassSetParserDelegate::begin): (JSC::Yarr::Parser::ClassSetParserDelegate::nestedClassBegin): (JSC::Yarr::Parser::ClassSetParserDelegate::doneAfterCharacterClassEnd): (JSC::Yarr::Parser::ClassSetParserDelegate::setUnionOp): (JSC::Yarr::Parser::ClassSetParserDelegate::setSubtractOp): (JSC::Yarr::Parser::ClassSetParserDelegate::setIntersectionOp): (JSC::Yarr::Parser::ClassSetParserDelegate::flushCachedCharacterIfNeeded): (JSC::Yarr::Parser::ClassSetParserDelegate::atomPatternCharacter): (JSC::Yarr::Parser::ClassSetParserDelegate::atomBuiltInCharacterClass): (JSC::Yarr::Parser::ClassSetParserDelegate::end): (JSC::Yarr::Parser::ClassSetParserDelegate::error): (JSC::Yarr::Parser::ClassSetParserDelegate::assertionWordBoundary): (JSC::Yarr::Parser::ClassSetParserDelegate::atomBackReference): (JSC::Yarr::Parser::ClassSetParserDelegate::atomNamedBackReference): (JSC::Yarr::Parser::ClassSetParserDelegate::atomNamedForwardReference): (JSC::Yarr::Parser::ClassStringDisjunctionParserDelegate::ClassStringDisjunctionParserDelegate): (JSC::Yarr::Parser::ClassStringDisjunctionParserDelegate::atomPatternCharacter): (JSC::Yarr::Parser::ClassStringDisjunctionParserDelegate::newAlternative): (JSC::Yarr::Parser::ClassStringDisjunctionParserDelegate::end): (JSC::Yarr::Parser::ClassStringDisjunctionParserDelegate::assertionWordBoundary): (JSC::Yarr::Parser::ClassStringDisjunctionParserDelegate::atomBackReference): (JSC::Yarr::Parser::ClassStringDisjunctionParserDelegate::atomNamedBackReference): (JSC::Yarr::Parser::ClassStringDisjunctionParserDelegate::atomNamedForwardReference): (JSC::Yarr::Parser::ClassStringDisjunctionParserDelegate::atomBuiltInCharacterClass): (JSC::Yarr::Parser::Parser): (JSC::Yarr::Parser::isIdentityEscapeAnError): (JSC::Yarr::Parser::parseEscape): (JSC::Yarr::Parser::consumePossibleSurrogatePair): (JSC::Yarr::Parser::parseAtomEscape): (JSC::Yarr::Parser::parseCharacterClassEscape): (JSC::Yarr::Parser::parseClassSetEscape): (JSC::Yarr::Parser::parseClassStringDisjunctionEscape): (JSC::Yarr::Parser::parseCharacterClass): (JSC::Yarr::Parser::parseClassSet): (JSC::Yarr::Parser::parseClassStringDisjunction): (JSC::Yarr::Parser::parseParenthesesEnd): (JSC::Yarr::Parser::parseTokens): (JSC::Yarr::Parser::handleIllegalReferences): (JSC::Yarr::Parser::tryConsumeUnicodeEscape): (JSC::Yarr::Parser::tryConsumeUnicodePropertyExpression): (JSC::Yarr::Parser::isLegacyCompilation const): (JSC::Yarr::Parser::isUnicodeCompilation const): (JSC::Yarr::Parser::isUnicodeSetsCompilation const): (JSC::Yarr::Parser::isEitherUnicodeCompilation const): (JSC::Yarr::compileMode): (JSC::Yarr::parse): * Source/JavaScriptCore/yarr/YarrPattern.cpp: (JSC::Yarr::CharacterClassConstructor::CharacterClassConstructor): (JSC::Yarr::CharacterClassConstructor::reset): (JSC::Yarr::CharacterClassConstructor::combiningSetOp): (JSC::Yarr::CharacterClassConstructor::append): (JSC::Yarr::CharacterClassConstructor::appendInverted): (JSC::Yarr::CharacterClassConstructor::putRange): (JSC::Yarr::CharacterClassConstructor::atomClassStringDisjunction): (JSC::Yarr::CharacterClassConstructor::performSetOpWith): (JSC::Yarr::CharacterClassConstructor::performSetOpWithStrings): (JSC::Yarr::CharacterClassConstructor::performSetOpWithMatches): (JSC::Yarr::CharacterClassConstructor::hasInverteStrings): (JSC::Yarr::CharacterClassConstructor::compareUTF32Strings): (JSC::Yarr::CharacterClassConstructor::sort): (JSC::Yarr::CharacterClassConstructor::charClass): (JSC::Yarr::CharacterClassConstructor::mergeRangesFrom): (JSC::Yarr::CharacterClassConstructor::unionStrings): (JSC::Yarr::CharacterClassConstructor::intersectionStrings): (JSC::Yarr::CharacterClassConstructor::subtractionStrings): (JSC::Yarr::CharacterClassConstructor::asciiOpSorted): (JSC::Yarr::CharacterClassConstructor::unicodeOpSorted): (JSC::Yarr::YarrPatternConstructor::YarrPatternConstructor): (JSC::Yarr::YarrPatternConstructor::resetForReparsing): (JSC::Yarr::YarrPatternConstructor::atomPatternCharacter): (JSC::Yarr::YarrPatternConstructor::atomBuiltInCharacterClass): (JSC::Yarr::YarrPatternConstructor::atomCharacterClassAtom): (JSC::Yarr::YarrPatternConstructor::atomCharacterClassRange): (JSC::Yarr::YarrPatternConstructor::atomCharacterClassBuiltIn): (JSC::Yarr::YarrPatternConstructor::atomClassStringDisjunction): (JSC::Yarr::YarrPatternConstructor::atomCharacterClassSetOp): (JSC::Yarr::YarrPatternConstructor::atomCharacterClassPushNested): (JSC::Yarr::YarrPatternConstructor::atomCharacterClassPopNested): (JSC::Yarr::YarrPatternConstructor::atomCharacterClassEnd): (JSC::Yarr::YarrPattern::compile): (JSC::Yarr::PatternTerm::dump): (JSC::Yarr::YarrPattern::dumpPatternString): (JSC::Yarr::YarrPattern::dumpPattern): * Source/JavaScriptCore/yarr/YarrPattern.h: (JSC::Yarr::CharacterClass::CharacterClass): (JSC::Yarr::CharacterClass::hasNonBMPCharacters const): (JSC::Yarr::CharacterClass::hasOneCharacterSize const): (JSC::Yarr::CharacterClass::hasOnlyNonBMPCharacters const): (JSC::Yarr::CharacterClass::hasStrings const): (JSC::Yarr::CharacterClass::hasSingleCharacters const): (JSC::Yarr::ClassSet::ClassSet): (JSC::Yarr::YarrPattern::unicodeSets const): (JSC::Yarr::YarrPattern::eitherUnicode const): (JSC::Yarr::YarrPattern::compileMode const): (JSC::Yarr::CharacterClass::hasNonBMPCharacters): Deleted. (JSC::Yarr::CharacterClass::hasOneCharacterSize): Deleted. (JSC::Yarr::CharacterClass::hasOnlyNonBMPCharacters): Deleted. * Source/JavaScriptCore/yarr/YarrSyntaxChecker.cpp: (JSC::Yarr::SyntaxChecker::atomClassStringDisjunction): (JSC::Yarr::SyntaxChecker::atomCharacterClassSetOp): (JSC::Yarr::SyntaxChecker::atomCharacterClassPushNested): (JSC::Yarr::SyntaxChecker::atomCharacterClassPopNested): (JSC::Yarr::checkSyntax): * Source/JavaScriptCore/yarr/YarrUnicodeProperties.cpp: (JSC::Yarr::unicodeMatchProperty): (JSC::Yarr::createUnicodeCharacterClassFor): (JSC::Yarr::characterClassMayContainStrings): * Source/JavaScriptCore/yarr/YarrUnicodeProperties.h: * Source/JavaScriptCore/yarr/generateYarrUnicodePropertyTables.py: (PropertyData.__init__): (PropertyData.addMatchString): (PropertyData.stringsCompare): (PropertyData): (PropertyData.sortStrings): (PropertyData.dumpMatchData): (PropertyData.convertStringToCppFormat): (PropertyData.dump): (PropertyData.dumpAll): (PropertyData.dumpMayContainStringFunc): (BinaryProperty.dump): (SequenceProperty): (SequenceProperty.__init__): (SequenceProperty.parsePropertyFile): (SequenceProperty.dump): * Source/WebCore/contentextensions/URLFilterParser.cpp: (WebCore::ContentExtensions::PatternParser::atomClassStringDisjunction): (WebCore::ContentExtensions::PatternParser::atomCharacterClassSetOp): (WebCore::ContentExtensions::PatternParser::atomCharacterClassPushNested): (WebCore::ContentExtensions::PatternParser::atomCharacterClassPopNested): (WebCore::ContentExtensions::URLFilterParser::addPattern): Canonical link: https://commits.webkit.org/261188@main

webkit-commit-queue · 2023-03-04T01:04:59Z

Committed 261188@main (270c824): https://commits.webkit.org/261188@main

Reviewed commits have been landed. Closing PR #10984 and removing active labels.

msaboff requested a review from a team as a code owner March 3, 2023 06:12

msaboff self-assigned this Mar 3, 2023

msaboff added the JavaScriptCore For bugs in JavaScriptCore, the JS engine used by WebKit, other than kxmlcore issues. label Mar 3, 2023

Constellation reviewed Mar 3, 2023

View reviewed changes

rkirsling requested changes Mar 3, 2023

View reviewed changes

msaboff force-pushed the eng/Implement-RegExp-v-flag-with-set-notation--properties-of-strings branch from 64ed139 to b632354 Compare March 3, 2023 19:33

Constellation reviewed Mar 3, 2023

View reviewed changes

msaboff force-pushed the eng/Implement-RegExp-v-flag-with-set-notation--properties-of-strings branch from b632354 to 0045ca6 Compare March 3, 2023 22:24

msaboff requested a review from rkirsling March 3, 2023 22:27

Constellation reviewed Mar 3, 2023

View reviewed changes

Constellation approved these changes Mar 4, 2023

View reviewed changes

msaboff force-pushed the eng/Implement-RegExp-v-flag-with-set-notation--properties-of-strings branch from 0045ca6 to 033aaa0 Compare March 4, 2023 00:35

msaboff added the unsafe-merge-queue Applied to send a pull request to merge-queue, but skip building and testing label Mar 4, 2023

webkit-early-warning-system force-pushed the eng/Implement-RegExp-v-flag-with-set-notation--properties-of-strings branch from 033aaa0 to b309ad8 Compare March 4, 2023 01:01

webkit-early-warning-system force-pushed the eng/Implement-RegExp-v-flag-with-set-notation--properties-of-strings branch from b309ad8 to 270c824 Compare March 4, 2023 01:04

webkit-early-warning-system merged commit 270c824 into WebKit:main Mar 4, 2023

webkit-commit-queue removed the unsafe-merge-queue Applied to send a pull request to merge-queue, but skip building and testing label Mar 4, 2023

en0ent1ty mentioned this pull request Jun 5, 2023

Support new ECMAScript RegExp v flag oven-sh/bun#3212

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Implement RegExp `v` flag with set notation + properties of strings #10984

Implement RegExp `v` flag with set notation + properties of strings #10984

msaboff commented Mar 3, 2023 •

edited by webkit-early-warning-system

webkit-early-warning-system commented Mar 3, 2023 •

edited

Constellation Mar 3, 2023 •

edited

Constellation Mar 3, 2023

Constellation Mar 3, 2023

msaboff Mar 3, 2023

Constellation Mar 3, 2023

msaboff Mar 3, 2023

Constellation Mar 3, 2023

msaboff Mar 3, 2023

rkirsling left a comment

webkit-early-warning-system commented Mar 3, 2023 •

edited

msaboff commented Mar 3, 2023

Constellation Mar 3, 2023

msaboff Mar 3, 2023

webkit-early-warning-system commented Mar 3, 2023 •

edited

msaboff commented Mar 3, 2023

Constellation Mar 3, 2023

msaboff Mar 4, 2023

msaboff Mar 4, 2023

Constellation left a comment

webkit-early-warning-system commented Mar 4, 2023 •

edited

webkit-commit-queue commented Mar 4, 2023

Implement RegExp v flag with set notation + properties of strings #10984

Implement RegExp v flag with set notation + properties of strings #10984

Conversation

msaboff commented Mar 3, 2023 • edited by webkit-early-warning-system

webkit-early-warning-system commented Mar 3, 2023 • edited

Constellation Mar 3, 2023 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

rkirsling left a comment

Choose a reason for hiding this comment

webkit-early-warning-system commented Mar 3, 2023 • edited

msaboff commented Mar 3, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

webkit-early-warning-system commented Mar 3, 2023 • edited

msaboff commented Mar 3, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Constellation left a comment

Choose a reason for hiding this comment

webkit-early-warning-system commented Mar 4, 2023 • edited

webkit-commit-queue commented Mar 4, 2023

Implement RegExp `v` flag with set notation + properties of strings #10984

Implement RegExp `v` flag with set notation + properties of strings #10984

msaboff commented Mar 3, 2023 •

edited by webkit-early-warning-system

webkit-early-warning-system commented Mar 3, 2023 •

edited

Constellation Mar 3, 2023 •

edited

webkit-early-warning-system commented Mar 3, 2023 •

edited

webkit-early-warning-system commented Mar 3, 2023 •

edited

webkit-early-warning-system commented Mar 4, 2023 •

edited