Repository navigation
CHANGES:
- Breaking: errors are structured values.
parseandparse_sourcereturnError ewhereeis aParseff.error:{ kind; range; location; contexts; notes; excerpt }, wherelocationis the line and column in bytes.locationisNonewhen the line is not known (streaming, older than the buffered input).kind : error_kindisExpected { items; found; more },Failure,Custom,Depth_limit_exceededorBacktrack_window_exceeded, so a match on it is exhaustive.foundisText,End,ShortorValue. The result type is('a, error) Stdlib.result, andParseff.OkandParseff.Errorare nowStdlib.OkandStdlib.Error. The typeerrorand its parts areprivate: you can read and match them, not build them.noteandrangestay buildable, forfail ~notes. See "Migrating from 0.4" below. - Breaking:
errortakes an extensible variant.errorwas'e -> 'aand the result carried a free'e, so a parser could raise anintand the caller could read astring: a segfault.custom_erroris now an extensible variant,error : ?from:int -> ?notes:note list -> custom_error -> 'a, anderror_kindhasCustom of custom_error.render_error ?customanderror_message ?customtake a printercustom_error -> string. Declare your cases withtype Parseff.custom_error += Too_large of intand matchCustom (Too_large n). ACustommatch needs a final_case. - Breaking: diagnostics are typed by a collector.
warn : 'd -> unithad the same hole.Diagnostics.create ()makes a'd Diagnostics.t;warn d xandwarn_at d ~pos xrecord into it; pass it to the runner with~diagnostics:dand readDiagnostics.to_list dafter the parse, onOkand onError. The runner empties the collector first.warninto a collector the runner did not get does nothing. Diagnostics are still rolled back with a branch that backtracks. A collector in use by one parse raisesInvalid_argumentif another parse gets it, or ifDiagnostics.clearis called on it. - Breaking: two runners.
parseandparse_sourcetake?consume:[ `All | `Partial ](default`All) and?diagnostics. By default the parse fails withExpected End_of_inputwhen input remains.parse_until_end s pis nowparse s p, andparse_source_until_endis nowparse_source. To parse a leading part and ignore the rest, which was whatparse s pdid in 0.4, pass~consume:`Partial. Removed:parse_until_end,parse_source_until_end, theresultalias (it shadowedStdlib.resultunderopen Parseff),diagnostic,error_with_diagnosticsandresult_with_diagnostics. - Add
render_erroranderror_message.render_error ?name ?source ?custom eprints aname:line:col: [error] messageheader, the source line with carets under the failure, a╭─╯and╰──^bar that joins the opening token to the caret for the construct that started on an earlier line,* while parsinglines for the opencontexts, and notes. Control bytes, invalid UTF-8 and C1 controls are escaped in the excerpt as in the message, tabs keep the carets aligned, long lines are cut to 240 bytes around the failure, and the columns of the header and of the* while parsinglines count Unicode characters, whilelocation.colcounts bytes. When streaming has dropped the start of the line and~sourceis not given, the header isname: byte N: [error] ...and the context line saysfrom byte N. Source.of_function,of_chunksandof_seqstart with a smaller read buffer and grow it. The firstreadcall gets alenof 2040 instead of 4096; the buffer grows to 4096 after a read that fills it.- Failures at the end of the input keep what was expected.
[1, 2reportsexpected ',' or ']', found end of inputinstead of the old end-of-input error without items, including throughexpectandone_of_labeled. - A successful
look_aheadforgets the failures of its probe.look_ahead (many (char 'a')); char 'b'onaacreportsexpected 'b'at offset 0 instead of the probe'sexpected 'a'at offset 2. A failinglook_aheadis a failure like any other and merges with the earlier ones. commitforgets earlier failures only if it sealed a frame, andexpectkeeps relabeling correctly when acommitran inside it.- Expected items past 64 are counted. An error that expected more than 64 alternatives at one position keeps 64 distinct ones and counts the rest in
Expected.more; duplicates are removed first, so they never hide an alternative.expectandone_of_labeledstay correct when the set overflows. - A truncated read says how much is left.
take 5onabcreportsexpected 5 bytes, found only 3 (end of input), andfound end of inputwhen nothing is left.foundisShort 3orEndfor it. - Fix the error report of a streaming window overflow and of
~fromor notes that point before the buffer. The first raisedInvalid_argument "String.sub"when the window was exceeded on a long line.~fromolder than the buffered input keeps its position, andrender_errorprintsbyte Ninstead of a line it cannot know. - Fix the excerpt: a line cut at 240 bytes always ends with
..., a lone carriage return at the end of the input is not shown, and C1 control characters are escaped in messages. - Breaking: removed API. Each removed item has a replacement:
- The type
backtrack_window_erroris removed.Backtrack_window_exceededis an inline record{ limit; oldest }, so patternsBacktrack_window_exceeded { limit; _ }replaceBacktrack_window_exceeded info. catchis removed. It caught onlyfail, neverExpected, and dropped~fromand~notes. Check the condition before callingfail, or useoptionalandor_on parsers whose failure is anExpected.Utf8.digitis removed. Usedigit, which parses the same ASCII digits.BE.any_uint8,BE.any_int8,LE.any_uint8andLE.any_int8becomeany_uint8andany_int8at the top level.
- The type
Backtrack_window_exceededreports the parser position. Its error position, line and excerpt are those of the parser when the window was exceeded, not the end of the read that did not fit, and the open* while parsinglines follow the usual rule.- Polish the error output.
- A streaming error whose line is not known, such as a
~fromolder than the buffered input, printsinput: byte N: [error] ...with no excerpt instead of a line and column computed from the partial buffer.render_erroralso indents the*rows by two spaces then. - A mismatch of
BE.int16,BE.int32,BE.int64and theLEversions reportsexpected int16be 0x0001, found 0x0002: the endianness is in the label andfoundis the integer read in the same width.foundisValuefor it. 'in a quoted character is escaped as'\'', a non-UTF-8 byte fromchar '\xE9'prints as'\xE9', and a truncatedfoundis'abcdef'....- Non-breaking spaces, format characters, line and paragraph separators, and a combining mark at the start of the quoted text are escaped as
\u{...}infoundand in expected items. - A
Customerror without a~customprinter sayscustom error (no ~custom printer given). - A message lists at most 8 expected items and then
, or N more alternatives(or 1 more alternativefor one).Expected.moreholds the count of alternatives past the 64 kept. - The excerpt shows control bytes, invalid UTF-8 and C1 controls as the escapes of the message, and the carets and the corner bar count the escape.
- The rows the renderer adds never end with a space, including empty source rows and an empty message. Source text is printed byte for byte, so a source line with trailing spaces keeps them.
Depth_limit_exceededandBacktrack_window_exceededkeep the* while parsinglines that were open and end with* help: raise ~max_depthor* help: call commit or cut earlier, or raise ~backtrack_window.* ... and 1 moreprints the frame instead, and the...row for hidden lines is aligned with the gutter.- The label of a regex without
~labelistext matching the pattern, andone_of []expects nothing: alone it fails withFailure "no alternative matched", and next to other failures at the same position it adds nothing to them.
- A streaming error whose line is not known, such as a
- Add
?labeltomatch_regex. A failed match reportsexpected <label>and merges with the other alternatives expected at the same position. Without~labela failed match expectstext matching the pattern. - Add
?fromand?notestofailanderror.~frommakes the error cover the bytes fromfromto the current position, and~notesattaches annotations: a note with a range is underlined in the error excerpt, a note without one is printed as a* note:line. Plain calls behave as before. The error of afailalso lists thecontexts it unwound through. - Add
context.context ?delimited name parserrunsparserinside a named construct. A failure inside it remembers the open constructs, outermost first, with their start offsets, for the error report. A parser that never callscontextpays nothing; a call costs about 100 instructions. - Add
cut.cut ()declares the running construct decided. It seals everyor_,one_of,many,sep_by,optionalandfold_*frame opened since the innermost enclosingcontext, and stops atlook_ahead. If thatcontextthen fails, the frames outside it are sealed too, somany (fun () -> context "statement" (fun () -> consume "let"; cut (); ...))reports the error of a broken statement instead of stopping the loop and ignoring the rest of the input.commit ()still seals the nearest frame only. - Report the furthest failure. A failed parse now returns the error at the highest position where any primitive failed, also inside branches that
or_,many,sep_by,optionalorfold_leftabandoned.[1, 2, 3,]reports the missing value at the], andsep_by (consume "ab") (char ',')followed byend_of_inputonab,ab,axreports thexat offset 7 instead of the trailing input at offset 5. Code that comparesposor the message of an error sees different values. - Merge, sort and deduplicate what was expected. Failures at the same position give one message:
expected "else", "if", or "while"instead ofexpected "if" or expected "else" or expected "while", andor_ digit digitsaysexpected digitonce. The order of the branches does not change the message. Characters print as'x', literals as"abc". consumefails at the first byte that differs.consume "ab"onaxfails at offset 1 withexpected "b", andconsume "hello"onhelfails at offset 3 withexpected "lo", found end of input. Both used to report offset 0. When fewer bytes remain than the literal needs but the available bytes already differ,consumefails at the first differing byte, soor_composes the messages of its branches. Only input that ends inside a prefix of the literal hasfoundset toEnd. A mismatch inside a multi-byte character is reported at the start of the character:consume "é"onèfails at the character, expecting"é".expectandone_of_labeledrelabel only failures at their start. A parser that got past its first byte keeps its own, more precise message.expect "key" (fun () -> consume "id:"; digit ())onid:xreportsexpected digitat offset 3.expectandtake_while ~labelmessages now start withexpected, andone_of_labeledreportsexpected number or stringinstead ofexpected one of: number, string.- Report a custom error from an abandoned branch when it is the furthest failure.
errorstill letsor_,manyandoptionalbacktrack. At the same position a custom error wins overExpected, and the first of two custom errors wins. It used to be the second. commitforgets the failures at or beyond its position. A committed branch reports only what it expected itself, not the failure of the previous branch.take_while ~at_leastwithout~labelsaysexpected at least n charactersinstead oftake_while.Utf8.charno longer formats a label on every call. Its failure reports the expected character, formatted only when an error is built.- Reduce the cost of failures. Primitives record the furthest failure in the parse state instead of allocating an exception that carries a message,
or_reuses its rewind frame for the right branch, and a finished parse allocates one closure less. - Improve runtime performance. Combinators read the parse state without a
Domain.DLSlookup, backtracking frames no longer allocate, internal failures skip backtrace recording, and streaming compaction no longer rebuilds line starts. - Fix
BE.int16andLE.int16for values above0x7FFF. They compared the signed result ofany_int16, soBE.int16 0xFFFEnever matched"\xFF\xFE". They now compare the 16 bits, accept-32768to65535and raiseInvalid_argumentoutside that range.int16,int32andint64mismatches are now reported at the start of the field instead of after it. - Fix concurrent parses on one domain. Two
parse_sourcecalls whosereadsuspends, for example in two Eio fibers, corrupted each other's state and raisedInvalid_argument "Parseff.State.pop_rewind_frame". Parseff now installs the parse state again after everyread, and a finished parse no longer leaves its state behind. Onlyreadmay suspend; a suspension inside a parser callback is still not supported. - Fix streaming memory growing with the input. Compaction kept one line index entry per newline for the whole stream. The index now keeps only the retained buffer and a count of dropped lines, so
locationstays correct and memory is bounded by the window. - Fix
match_regexhiding a full backtrack window. When the window was full and the source had not ended, the regex ran on the short buffer and failed, soor_took another branch. It now returns a match that ends before the buffered end, and otherwise fails with`Backtrack_window_exceeded, astakedoes, because more input could change the result. - Check streaming arguments.
parse_sourceraisesInvalid_argumentfor~backtrack_windowbelow2,Source.of_channelfor~buf_sizebelow1, and a parse for areadresult outside0..len. A window of0or1used to report a false end of input. - Fix streaming documentation.
spanis an owned copy in streaming,match_regexreads ahead half a window instead of retrying at chunk boundaries, and a source is for one parse. - Fix
expectandone_of_labeledrelabeling custom errors. A custom error raised through anor_orone_ofwith several branches was replaced by theexpectorone_of_labeledmessage. It now passes through unchanged, as documented. - Fix
many,sep_by,end_byandfold_leftlooping forever on parsers that consume no input. The loop now stops after an iteration that consumes nothing and drops that iteration's result and diagnostics.~at_leaststill applies to the kept results. - Fix
take nraisingInvalid_argumentfor a very largen. The length check overflowed.takenow fails with anExpectederror whosefoundisShortorEnd. A negative count also fails withExpected,expected non-negative length, found -1, as in 0.4, soor_recovers from it. - Document the cost of
match_regexfailures.Researches forward when the regex does not match at the current position, so a failed match scans the rest of the input and a loop of failing matches takes quadratic time.
Migrating runners, error and warn
| Before | After |
|---|---|
parse_until_end s p |
parse s p |
parse_source_until_end src p |
parse_source src p |
parse s p |
parse ~consume:`Partial s p |
parse_source src p |
parse_source ~consume:`Partial src p |
Ok (v, ds) and Error { error; diagnostics } |
Ok v and Error e; read the diagnostics from the collector |
warn x |
let d = Diagnostics.create (), parse ~diagnostics:d, warn d x |
warn_at ~pos x |
warn_at d ~pos x |
'd Parseff.diagnostic = { pos; diagnostic } |
'd Parseff.Diagnostics.entry = { pos; value } |
error (`Too_large n) |
type Parseff.custom_error += Too_large of int, then error (Too_large n) |
Error { pos; error = `Too_large n } |
Error { kind = Custom (Too_large n); _ }, with a final _ case |
('a, 'e) Parseff.result |
('a, Parseff.error) result |
Migrating from 0.4
The common case is one pattern. Replace the match on the built-in variants by a match on the error value:
(* 0.4 *)
match Parseff.parse input parser with
| Ok v -> v
| Error { pos; error = `Expected msg } -> Printf.eprintf "%d: %s\n" pos msg; exit 1
| Error { pos; error = `Unexpected_end_of_input } -> Printf.eprintf "%d: end of input\n" pos; exit 1
| Error _ -> exit 1
(* 0.5 *)
match Parseff.parse input parser with
| Ok v -> v
| Error e -> prerr_endline (Parseff.render_error ~name:"input" e); exit 1Error { pos; error }becomesError e. The position ise.range.start, the message isParseff.error_message e.- Custom errors move into
kind:Error { error = `Too_large n; _ }becomesError { kind = Custom (Too_large n); _ }, withtype Parseff.custom_error += Too_large of int. `Failure msgbecomesFailure msg. The 0.4 messages of primitives that were not expectations (invalid UTF-8, a failedmatch_regexwithout a label, a mismatchedBE.int16) areExpecteditems now (valid UTF-8,text matching the pattern,int16be 0xFFFE), so they merge with the alternatives expected at the same position.`Expected msgbecomesExpected { items; found; more }.Parseff.error_message egivesexpected A, B, or C, found Xinstead ofexpected A or expected B, sorted and without duplicates.`Unexpected_end_of_inputbecomesExpected { found = End; _ }and now hasitems.`Depth_limit_exceeded msgbecomesDepth_limit_exceeded { limit }, and`Backtrack_window_exceeded infobecomesBacktrack_window_exceeded { limit; oldest }.Parseff.OkandParseff.ErrorareStdlib.OkandStdlib.Error. Code that writesOk vandError ekeeps compiling.parse_until_endandparse_source_until_endbecomeparseandparse_source.parseandparse_sourcenow require all input to be consumed unless you pass~consume:`Partial. Diagnostics are read from aDiagnostics.tcollector. See the table above.- Tests that compared error strings change: messages now name what was expected at the furthest failure, and
consumefails at the first differing byte.