Repository navigation
OpenDocKit 0.3.4
A second review of the whole source. The sample outputs in examples/ are unchanged.
Security
- Reading files:
convert_file, the CLI and the LangChain and LlamaIndex adapters read a pipe or a device (/dev/zero) only up tomax_file_bytes; before, the size check came after reading it to the end.IsolatedConverterrefuses an oversized or empty input before it is sent to a worker. - XLS: merge ranges written back to front no longer cancel out the merged-cell limit (a 461 KB file took 16 seconds).
- PDF: a small crafted file no longer holds one page for tens of seconds through thousands of FreeText annotations, one long letter-spaced line, thousands of check-box shapes, faint watermark characters, shading beside a ruled table, a long zebra table, many highlighted lines, a borderless "table" of a thousand columns, a long paragraph or stacked text drawn many times. The running character limit also counts sideways text and form values, so such a document stops at the limit.
- PDF layout analysis: pdfminer's grouping of lines into text boxes and of text boxes into reading order is bounded per page (the page itself and every form drawn on it); past the bound the page is read in pdfminer's plain order with the existing "read without text-box reading-order analysis" warning. A 20 KB page took 62 seconds, a 68 KB page two minutes. A single huge character, or two characters far apart, no longer makes pdfminer fill millions of index cells.
- XLSX: scanning a sheet for formula results, the shared strings and the sheet parser no longer keep every row in memory (a 600,000-row sheet took 948 MB more), and a sheet without
<dimension>is no longer parsed twice to measure it. - Mail: mail forwarded inside forwarded mails is limited to 100 levels, counted along the structure the mail parser follows (part headers quoted in a body or in a text attachment are text).
- Release: the PyPI publish action is pinned to a commit.
Fixed
- PDF: a row of a borderless table that pdfminer grouped differently came out in two tables, splitting the table in two, and the header of the lower of two stacked borderless tables also ended the table above as a row of its own; columns to the right of a nested table were shifted; FreeText labels were lost on pages whose MediaBox does not start at 0,0 or that are rotated, and image clips were ignored when the MediaBox starts right of 0; text at a non-finite position rejected the whole document or leaked off-page copies into the body; a ruled table continued on the next page without its header lost the text of a merged cell left open at the page end.
- DOCX/PPTX: one damaged picture member rejected the whole document; an empty equation wrote
$$, which swallowed the following text, and$ % # { } \typed in an equation were read as LaTeX; a footnote reference with no note left a dangling[^1]; a struck superscript ending in a space gave~~^2 ~~; a comment quoted moved-away, deleted or hidden text. - Spreadsheets: an integer too large for a float rejected the whole XLSX; an unclosed quote in a small CSV swallowed the rest of the file without a warning; a large hyperlink range forced streaming and lost merges;
1E+400in Excel 2003 XML becameinf; a literal between decimal places (0.0\E+0) was lost. - HTML:
visibility:visiblecontent inside avisibility:hiddenelement, and pages hidden with<body style="visibility:hidden">until a script runs, came out empty; a page title written over several lines broke the heading; a# commentin a code block kept the title out;~~,^and_marks were put inside code; an empty list item holding a nested list lost its indentation;rowspan="0"spanned one row; alt text could close[Image: …]and form a link; struck elements next to each other (<del>a</del><del>b</del>) gave~~a~~~~b~~, which is not strikethrough for a renderer. - Mail: one HTML-only mail left a whole
.txtmailbox undecoded; an HTML attachment was read instead of the body of an MHTML-shaped mail; a forwarded mail that is itself a forward lost its body; meeting invitations took the reminder's title and attendees, and a recurrence rule ending in;stayed raw; short broken UTF-8 could be read as UTF-16; a multipart mail with an empty boundary (boundary="") was rejected, as was a mail whose parts were counted deeper than they nest. - ODF: hidden text (hidden spans, hidden text fields and paragraphs, hidden conditional sections) came out as body text; a nested list with its own style never restarted its numbering; a heading's number was glued to it (
# 1.Intro), and a heading in a numbered list was not counted;text:continue-listnow continues the list it names. - HWP/HWPX: a broken DocInfo stream, or an HWPX member with an unknown compression method or the encryption flag, rejected the whole document instead of only that part.
- RTF: a code page that does not read ASCII as ASCII (EBCDIC) turned the text into noise; a
\ufallback that a writer counted in characters left half a double-byte character. detect_formatcalled on its own raisesCorruptedDocumentErrorinstead of library exceptions.describe_imagesno longer re-renders the document every few pictures when many are judged decorative nearmax_chars.
Changed
- HWP documents protected with DRM or a public certificate raise
EncryptedDocumentError. - The MCP server keeps its last four results (by file content), so
document_outlinefollowed byconvert_documentby sections converts the file once. detect_formatraisesTypeErrorfor content that is not bytes, bytearray or memoryview, and for a file name that is not astr, asconvertdoes.
Full list: CHANGELOG.