Full ECMA-262 regex literal grammar sample

[STRd6]

Ports the ECMA-262 regex grammar from #11 onto current main, fixed up so it compiles and parses accurately.

What changed from the #11 version

• Fixed PatternCharacter not excluding \ — escapes never parsed via AtomEscape; a trailing lone backslash was silently accepted
• Added the missing DecimalDigits rule (was an undefined reference)
• Removed the unfinished proposal sections — the set-notation rules contained pasted spec prose ([lookahead ∉ ClassReservedDouble] SourceCharacter but not ...) referencing undefined rules, and the trailing modifiers Atom rule duplicated (and would clobber) the real Atom
• Fixed QuantifierPrefix to reject {1,2,} (was ( "," DecimalDigits )? ","?)
• Strict-mode alignment: IdentityEscape restricted to SyntaxCharacter / /, \0 requires no following digit, Annex B forms (a{, lone ], \q, legacy octal) rejected

Modernized

• \u{...} code point escapes, \p{...}/\P{...} property escapes (hera compiles terminals with the u flag, so these work natively)
• ES2025 pattern modifiers (?ims-ims:...) — now standard, integrated into Atom (subsumes (?:)
• Named groups, \k<name>, lookbehind kept from starting on optimizer #11

The start rule is now RegExpLiteral (/pattern/flags per the lexical grammar — unescaped / only inside classes, no line terminators, // is not a regex), which keeps the perf/compare.civet input /foo[abc](?:bar)\d+/g parsing.

Set-notation (v-flag ClassSetExpression) syntax is left as a TODO; the flag itself is accepted.

Testing

test/regex.civet: 25 valid literals — every accepted literal is cross-checked against the JS engine via new Function — and 22 rejections (each a SyntaxError in Unicode mode). Recursive rules carry ::any annotations so the generated parser passes the strict tsc -p tsconfig.parsers.json check.

Note: pnpm test:typed-parser-samples currently fails on main locally on inference.fixture.hera ($C/$S overload arity); unrelated to this change.

🤖 Generated with Claude Code

[greptile-apps]

Greptile Summary

This PR replaces the 45-line placeholder regex grammar with a 189-line full ECMA-262 Pattern grammar (strict Unicode mode), adding a new test file that cross-checks every accepted literal against the real JS engine and confirms all 22 rejected literals throw parse errors.

• Grammar (samples/regex.hera): Correctly adds PatternCharacter exclusions, fixed QuantifierPrefix ({1,2,} rejected), strict IdentityEscape, \\u{...}/\\p{...} escapes, named groups, lookbehind, and ES2025 pattern modifiers; removes unfinished Annex-B/set-notation sections from starting on optimizer #11.
• Tests (test/regex.civet): 25 valid literals (each JS-engine verified) + 22 invalid literals covering the perf benchmark input, named captures, backreferences, property escapes, quantifiers, and assertions.
• Known gap: The CodePoint rule (/[0-9a-fA-F]+/) has no 0x10FFFF upper-bound, so \\u{110000} and larger values parse successfully even though they are SyntaxError in JS Unicode mode; no corresponding invalid test case exists to guard this.

Confidence Score: 3/5

The grammar is a well-thought-out rewrite that substantially improves on the original, but the CodePoint rule silently accepts code points above 0x10FFFF that JS engines reject in Unicode mode, and the test suite has no invalid cases exercising that path.

The CodePoint rule accepts hex strings of arbitrary value, meaning /\u{110000}/u and friends parse without error even though they are SyntaxError in every Unicode-aware JS engine. The grammar header documents other known omissions but does not mention this one, and the invalid test array has no entry that would expose the gap. The rest of the grammar change looks correct and is well-covered by tests.

The CodePoint rule in samples/regex.hera (lines 124-125) and the invalid array in test/regex.civet (around line 43) both need attention to close the out-of-range code-point gap.

Important Files Changed

Filename	Overview
samples/regex.hera	Full ECMA-262 Pattern grammar rewrite (45 to 189 lines): well-structured with correct QuantifierPrefix, PatternCharacter exclusions, strict IdentityEscape, named groups, lookbehind, Unicode escapes, and pattern modifiers; however the CodePoint rule has no 0x10FFFF upper-bound and UnicodePropertyName omits digits from its character set.
test/regex.civet	New test file with 25 valid (JS-engine cross-checked) and 22 invalid regex literals; good coverage overall but the invalid set has no out-of-range CodePoint cases to guard the CodePoint upper-bound gap.

_{Reviews (1): Last reviewed commit: "Replace WIP regex sample with full ECMA-..." | Re-trigger Greptile}

[greptile-apps]

CodePoint has no upper-bound — accepts \u{110000} and beyond

The rule /[0-9a-fA-F]+/ matches any-length hex string, so \u{110000} (0x110000 > 0x10FFFF) and \u{1FFFFF} parse successfully even though both are SyntaxError in JS Unicode mode. This is not one of the context-dependent errors the header disclaims (those are: nonexistent back-references, duplicate names/flags, [z-a] ranges, and {3,1} bounds). Code-point range is a purely local static check on a single token value.

The parallel in this repo's own url.hera is instructive — DecimalOctet uses an explicit regex to stay within 0-255 rather than just /[0-9]+/. A constrained pattern like /[0-9a-fA-F]{1,5}|10[0-9a-fA-F]{4}/ would enforce the ceiling. Alternatively, at minimum, add /\\u{110000}/u to the invalid array in test/regex.civet to document and guard the gap.

[greptile-apps]

UnicodePropertyName excludes digits, deviating from ECMA-262

ECMA-262 defines UnicodePropertyNameCharacter as AsciiLetter | $ | _ | DecimalDigit, so digits are technically legal in property names (though no currently-assigned Unicode property name happens to use one). The UnicodePropertyValue rule already includes digits (/[A-Za-z0-9_]+/), so the asymmetry is easy to miss. Aligning both rules removes the spec deviation at no practical cost.

Suggested change

	UnicodePropertyName
	/[A-Za-z_]+/
	UnicodePropertyName
	/[A-Za-z0-9_]+/

[greptile-apps]

Missing invalid test cases for out-of-range CodePoint

The 22 rejection cases cover Annex-B constructs, malformed escapes, and structural errors, but none exercise a \u{...} escape whose numeric value exceeds 0x10FFFF. Because the cross-check via new Function only runs for items in the valid array, the CodePoint upper-bound gap goes undetected. Adding entries like '/\\u{110000}/u' and '/\\u{1FFFFF}/u' would either catch the grammar gap or document it explicitly once the rule is tightened.

[STRd6]

This could come in handy when / if we want to do fancier things with regexes in Hera

[edemaine]

Looka like a nice realistic example! And potentially useful, as you say.

[edemaine]

GPT says: Maybe mention pattern modifier early errors here too. The grammar accepts forms like (?ii:x), (?i-i:x), and (?-:x), while ECMA-262 rejects them via modifier early-error rules. That seems consistent with the “purely syntactic” scope, but it’s not quite covered by “duplicate flags” as written.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 100.00%. Comparing base (1f6a93d) to head (9ee97eb).
⚠️ Report is 5 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff            @@
##              main       #99   +/-   ##
=========================================
  Coverage   100.00%   100.00%           
=========================================
  Files            9         9           
  Lines         1783      1783           
  Branches       300       300           
=========================================
  Hits          1783      1783           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.