Release 3.8 · ocaml-community/sedlex

CHANGES:

New features

Named capture groups: (r as x) binds x in the action to a
Sedlexing.submatch, extracted with Sedlexing.Utf8.of_submatch,
Latin1.of_submatch, Utf16.of_submatch or lexeme_of_submatch. Both
branches of an or-pattern must bind the same names, and as is not allowed
under Star, Plus, Opt, Rep, Compl, Sub or Intersect. Submatches
are chosen leftmost-greedy within the longest match, using a tagged DFA; a
boundary at a fixed offset from the start or end of the lexeme costs nothing
at runtime. A submatch is only valid inside its action (#177, #178, #204,
#208, #209, #212)
Nested let .. in definitions of [%sedlex.regexp?] (#176)
Unicode 18.0.0 (#205)
Compl, Sub and Intersect accept Rep (c, 1 .. 1) (#201)
Clearer errors for a bare Star, Plus, Opt, Utf8, Latin1 or Ascii
and for unknown operators (#196)

Rule priority with eof: a rule matching s, eof no longer beats an
earlier rule matching s (#211)
End of input is read once: Plus eof and Star ('a' | eof) no longer loop
forever at end of input, and eof, eof no longer matches (#211)
Empty character sets (Chars "", Compl any, Sub or Intersect of
coinciding classes) are a compile error instead of crashing the ppx or
producing a rule that never matches (#201)
Rule sets where every rule is nullable, such as a lone "", no longer
generate invalid code (#201)
UTF-8 decoding: the surrogate range rejected by Utf8.Helper.check_three
ended at U+DF00 instead of U+DFFF (#184)
Sedlexing.sub_lexeme and its Latin1, Utf8 and Utf16 versions raise
Invalid_argument when the range is not within the lexeme, instead of
returning code points of the rest of the buffer or, for a negative length,
an empty string (#219)

Faster lexeme, sub_lexeme and of_submatch in Sedlexing.Latin1,
Sedlexing.Utf8 and Sedlexing.Utf16 (#219)
The memory cells of as bindings are no longer cleared at the start of
every token (#220)