Skip to content

v0.3.14

Latest

Choose a tag to compare

@github-actions github-actions released this 19 Nov 07:45
· 2 commits to main since this release
7fb338a

Fixes from five audit rounds, most of them small files that cost a lot of time or memory, and content that was lost. One sample output changed: score-table.hwpx no longer writes the guide text of an unfilled click-here field (날짜).

Added

  • CLI: opendockit --version, as the other two commands have, for bug reports.

Changed

  • CLI: output to a closed pipe or a closed standard output, --help and --version included, ends quietly with exit code 1; a short output to a closed pipe ended with Exception ignored … BrokenPipeError and exit code 120. With standard error closed, messages are no longer written into the output.

Fixed

  • PDF shaded tables: in a striped table, the cells beside a cell merged over two rows with its text centred on them no longer run into one cell (South Lee).
  • PDF forms: a field is named by its own /TU or /T. The tooltip of a field grouping others named every field under it, so their values could not be told apart.
  • HTML: an element with the role main or article is the main content, as <main> is: the footnotes, topics and sidebars of Sphinx and Read the Docs pages (asides since docutils 0.18) are no longer removed as side menus. A plain <button> question whose folded answer is kept is kept too, instead of leaving the answer without it.
  • Mail: a sent mail with multipart/related at the top (Outlook and Apple Mail, with pictures in the body) is read as mail, so its meeting invitation, forwarded mails and the plain body of a picture-only HTML are no longer lost.
  • DOCX / PPTX: a cell marked as continuing a merge that has no cell to continue (a w:vMerge in the first row, a PPTX vMerge/hMerge cell no cell spans) keeps its text.
  • XLSX: a cell without an r address that holds a malformed value, and a comment with an empty ref, no longer reject the whole workbook. A value under a malformed format index is checked like any other (1E400 was inf). Weekday-only formats (aaa) show the day, as dddd and XLS do, not the date and time.
  • PDF text: a word in fake bold on part of a line (drawn twice 0.3pt apart, common in PDFs exported from Hangul) is read once; it came out again as a paragraph of its own, or split its line in two. Lifting a line pushed down in a single column no longer compares every pair of lines on the page: a 480-line page took 7 seconds, outside pdf_time_budget (an 11 KB file took 22 seconds); it takes 0.2.
  • PDF tables: a borderless table that pdfminer merged differently from row to row is one table again, not three, with every cell in its column (one part had a column too few and its values stood one column to the left). The next page of such a table gets the header of its top part, not a data row of its lower part, which came out twice.
  • Character limit: the quote marks of a quoted line holding only a picture that is erased (too small, a repeated logo) no longer count, so a document within max_chars is no longer reported exceeded and its pictures are decoded.
  • Pictures: alt text Office or Hangul wrote itself (그림 3) is recognised in decomposed (NFD) Hangul too. A picture inside an HTML <pre> no longer adds a blank line to the code block.
  • Markdown: the data: URI pictures inside a code block are left alone in every code block cleaning sees — a fence indented in a list item, or closed by a longer fence — and inline code over two lines of a CRLF file is code.
  • RTF: a file behind a UTF-8 byte order mark (PowerShell 5, .NET) is read as RTF, not as its source text. A Shift_JIS, GBK or Big5 character whose second byte is written as \\, \{ or \} is one character, not �\.
  • DOCX / PPTX: a styles or document-properties relationship pointing back at the body, and a slide layout, master or notes relationship pointing at another kind of part, no longer reject the document or drop the slide. A merge continuation under the second column of a wider cell repeats its text instead of coming out empty.
  • XLSX / XLS / Excel 2003 XML: a workbook with very many small merges (490,000: 20 seconds, 558 MB) is streamed. A What-If data table read as formulas is =TABLE(…), not an object address that changed on every run. Zero padding is kept with a thousands separator (00,000 → 01,234). [hhh]:mm no longer loses a day for serial 60. Serial 0 under a date or weekday format is 0 everywhere, and Excel 2003 XML reads weekday-only formats as dates.
  • HTML: every formula of a saved Wikipedia article is kept, once, without its {\displaystyle …} wrapper. A $ or brace in MathML (also in ODF) is escaped. Selects left open no longer take quadratic time (8,000: 20 seconds).
  • Mail: a forwarded mail of type message/global is read. A base64 body cut off at one letter past a whole group of four is decoded up to the cut, not shown as base64. A cid: address in upper case or percent-encoded finds its picture.
  • EPUB: only one branch of an <epub:switch> is read, not the formula and its text fallback.
  • HTML: what follows a table that sits inside a table left open (a missing </table>) stays after it, as in a browser; the headings and paragraphs after the second table came out ahead of both. The words of a MathML <mtext> are \text{…} with their spaces (also in ODF): 1 if x>0 read 1ifx>0.
  • Mail: a body picture sent as application/octet-stream with a Content-ID is extracted; it was neither extracted nor reported.
  • PDF tables: on a two-column page, a column of a borderless table lying in the gap between the page columns is written once; it was read into a table on each side.
  • PDF text: a paragraph drawn through a Form XObject beside a paragraph on the page goes after that paragraph, as when both are drawn on the page; it was wedged in after its first line.
  • PPTX: a numbered paragraph whose start number differs from the run's starts over at it (three items, then two set to start at 7, read 4. 5.).
  • XLS: a text box with an emoji ends where its text ends; its formatting was read as text after it (on time(@).
  • XLSX / XLS / Excel 2003 XML: a stored -0 shows as 0.00 or $0.00 under its format, not -0.00. A text cell shows the literals of its format's text section (@" 님" → 홍길동 님).
  • Limits: max_uncompressed_bytes of sys.maxsize or more no longer rejects every compressed HWP and ODT as corrupted, nor drops the compressed pictures of an XLS.
  • Open WebUI loader: an app from create_app mounted in another Starlette or FastAPI app converts in worker processes too; Starlette runs no lifespan of a mounted app, and every file was converted in the server process without the 2-minute and 2 GB limits.
  • PDF text: small text with tight character spacing or condensed width keeps both letters of ll and its spaces; the second letter was taken for a fake-bold copy (Al staffwil folow). A line rebuilt around a word in fake bold keeps the spaces the PDF draws (Theplanisimportant). A drop cap drawn after its lines (as LibreOffice writes it) joins its word (O / nce upon a time), and one word in synthetic italic no longer keeps a letter-spaced line spaced out.
  • PDF tables: a borderless cell joined from the pieces of one line (cells centred over a wrapped cell, as Hangul exports them) is no longer repeated as a paragraph under the table. The header is carried to the next page when that page's first row has an empty cell. An open-sided table whose last row merges across every column rule keeps that row, and a table shaded with one rectangle for the header row and one for the first column is no longer read as two cells.
  • Tables (HWP, HWPX and every format): a few cells far apart (one spanning 10,000 rows, another 1,000 columns) no longer build a dense grid of empty cells: a 1.9 KB HWPX took 6.5 seconds per table. A table is drawn from the rows and columns that hold text, and the empty cells of one conversion share one budget.
  • CSV: a run of empty lines is skipped in one step and the lines are no longer all held in memory: a header and then 50 MB of line breaks took 32 seconds and 4 GB.
  • XLSX / XLS / Excel 2003 XML: #.## shows 0.5 as .5 and #,###.00 as .50, as Excel does. A bare % stays where the format puts it (0.0 % → 12.5 %, %0 → %50), and each one multiplies by 100 (0%% → 1250%%). A huge negative under h:mm, and a serial past 9999-12-31 under a date format with a quoted [h], no longer reject the workbook.
  • DOCX: a footnote with a custom mark (Insert Footnote > Custom mark) no longer writes the mark twice (Text[^1]*). First-page and even-page headers that Word does not show (the option unticked, the part kept) are left out. Lower-level numbers in full-width digits (1.) get their bullet; they ran into the item above.
  • PPTX: unwrapping mc:AlternateContent takes linear time (40,000 wrapped shapes: 19.6 seconds), and a wrapped text box on a notes page is part of the notes.
  • RTF / ODT: an empty numbered paragraph takes its number, as in Word, LibreOffice and DOCX (1. then 3. after an empty item). An RTF list override starts at the number it sets (\listoverridestartat).
  • HTML: a link or emphasis around the nested list of a list item is no longer written across the list. Every caption of a table is kept. Many <xmp> or <textarea> elements no longer take quadratic time (400,000: 10.8 seconds).
  • Mail: the HTML of a mail is searched for visible text only when a plain-text branch waits on it; a 37 MB HTML mail was parsed whole, 9.9 seconds and 1 GB.
  • EPUB: many <epub:switch> elements no longer take quadratic time.
  • Pictures: a 1-bit or palette picture (a fax TIFF, a bilevel scan, a GIF) is blended when it is shrunk; Pillow shrank it by nearest neighbour, and 2 px rules and thin strokes vanished. Scanned pages that are copies of one picture are each counted in the warning (100 such pages were '10 page(s)').
  • Markdown files: a data: URI picture whose alt text holds brackets (![Figure [1]](…)) or a line break, or an <img> whose quoted alt holds >, is extracted; its base64 stayed in the text.
  • MCP server: a long document without pages, slides or sheets is no longer sent to document_outline and sections, which gave the same first characters again; the note says to raise max_chars, and the outline no longer says it is returned whole.
  • LlamaIndex reader: warnings and exceeded stay out of the text LlamaIndex embeds and sends to the LLM with every node, and source out of the embedded text; a file with many warnings was refused by SentenceSplitter.
  • DOCX: an emoji or symbol Word 2016 writes as w16se:symEx is read; it vanished. A start override of one list (Set numbering value) no longer becomes the start of that level in every list sharing its definition. Two paragraphs joined with Track Changes on are one paragraph, not a sentence cut in two. A zero-padded custom number format (001, 002, 003, ...) keeps its width. A paragraph of invisible characters is empty, in PPTX too.
  • DOCX / PPTX packages: a part named in another case than the relationship that points at it is read, as Office does; a DOCX lost it to the repair and a PPTX lost a picture or a slide. One relationship without its Id, Type or Target no longer rejects the whole document, and one without a Type to a styles, numbering or other known part takes that part's type.
  • HWP / HWPX: the one-character controls of a paragraph are decoded with the text around them, without the control characters they held; the guide text of an unfilled click-here field is left out; an empty numbered paragraph takes its number; notes count toward max_chars while the body is read, and uncompressed streams toward max_uncompressed_bytes. Shape positions past the end of the text are dropped before the shapes are ordered, an atop inside an atop keeps its braces, and HWPX members are looked up in constant time.
  • RTF: the older \pn numbering starts at its \pnstart; a list in a header or footer no longer takes the body's first number; the number of a numbered heading is counted; the ANSI font of the macOS text system reads as CP1252; the Unicode line separator is a line break.
  • ODP: the paragraphs of a text box are numbered as Impress draws them: lists that follow each other count on (1. a) b) 2. 3., not 1. a) b) 1. 2.), also across a plain paragraph, and a list of another number format or start restarts.
  • XLSX / XML Spreadsheet: a negative section that differs from the positive one only by a space keeps its minus; an escaped quote before a quoted literal is a literal in date checks; the cell comments of an Excel 2003 XML workbook come after the sheet.
  • HTML: closed dialogs and popovers, the permalink marks of documentation headings and the fallback text of <video> / <audio> are left out; white-space: pre-wrap keeps its line breaks; a line-numbered code block is the code alone; the TeX of a MathJax script is the formula; <q> gets its quote marks; a table drawn inside a list item keeps its lines; the buttons of a heading are unwrapped in linear time, and a list item with very many code blocks is split in linear time (40,000 <pre> blocks: 21 seconds).
  • MathML: a script base that is itself a script or a group is braced ({x^2}^3), also in ODF, and the pairs of an <mmultiscripts> are written without double scripts (R_{i}^{j}{}_{k}, {}_{6}^{14}C).
  • Mail: the parts of a forwarded mail are read like those of the outer mail; delivery and read reports, and the returned headers of a bounce, are read; named inline text files (a patch, a CSV) are listed in the attachment warning; a named meeting invitation beside the text is kept, and written once when it is sent in two parts; a header with unclosed encoded words is decoded in linear time, and a picture shared by several references is decoded once.
  • Outlook .msg: rejected with a message that says how to save the mail instead of as an unknown format.
  • EPUB: a member whose name is in another Unicode normalization than its reference is found.
  • PDF: a footer of a label and Page 3 of 4 side by side, and roman page numbers (i, ii, iii) that count with the pages, are removed from every page. Prose in three or more columns that pdfminer reads column by column is prose, not tables, and a data row with a wrapped cell is no longer joined into the header.
  • Detection: a DOCTYPE with an internal subset is one DOCTYPE. A note that lists a whole PDF, named as text, is read as that text instead of the PDF it lists.
  • Markdown: the Unicode line and paragraph separators are line breaks outside code blocks (kept in code and data files such as JSON); title rows of a long table are lifted in linear time.
  • Pictures: a Lab picture is prepared as opaque, and the split pages of a TIFF count toward max_uncompressed_bytes.
  • MCP server: an answer lists at most 20 warnings, and the outline lists pages, slides or sheets as far as the server's --max-chars and counts the rest; with --root, a folder that may be passed through but not listed no longer refuses the files under it.
  • LangChain / LlamaIndex / Open WebUI: a file of pictures that were not described gives no Document of [Image] placeholders, as a scanned PDF gives none of page markers; nor does a sheet of pictures with their bare cells, or a bare [Chart], [Video] or [OLE object].
  • PDF limits: a Form XObject that draws itself more than once no longer makes pdfium expand it (an 800-byte file grew past 10 GB); the page is left out with the drawing-elements warning. A content stream listed many times in /Contents is charged to max_uncompressed_bytes every time, a page whose content would take minutes to interpret (a form of a million q Q drawn 16 times: 88 seconds) and a page of very many characters (800,000: 25 seconds and 2 GB) are read with pdfium's plain text extraction, with the same warning. A page is opened in pdfium at the number pdfium gives it, so an odd page tree (a missing or non-page entry in /Kids) cannot hand it a page that was refused. The data of an inline image is read in linear time (5 MB: 8.5 seconds). A page that adds more than 150,000 pieces to paths (a detailed map) is read with pdfium too: pdfminer kept about 1.4 KB a segment (600,000: 942 MB).
  • PDF tables: a table shaded with grey set as DeviceGray (0.85 g) is read as shaded, as one in RGB is; each row came out as a table of its own.
  • Mail: a Content-Type with many parameters no longer takes quadratic time (an 8 KB mail: 20 seconds), nor does a header of many encoded words on one line (1 MB: 13 seconds). An attachment name written in raw 8-bit bytes or in an RFC 2231 Korean charset (ks_c_5601-1987''…) is read in the warning, not as ���, and such a name on a part without a usable boundary or on a report part no longer rejects the whole mail; nor, on Python 3.12 and 3.13, does a name the standard library cannot read (filename*= beside filename*1=).
  • Character limit (spreadsheets and CSV): a sheet is drawn only up to max_chars, so a merged cell repeated down 20,000 rows (108 KB: 31 seconds and 3.5 GB), a CSV of short rows under a wide header (1 MB: 19 seconds and 1.4 GB) and sparse rows no longer build the whole table first; a cell holding only > counts.
  • XLSX / XLS / Excel 2003 XML: a number format of unclosed brackets no longer costs milliseconds per cell (10,000 cells: 93 seconds); one long shared formula is moved to a cell only when its row is read (8,000 cells: up to 74 seconds); the column-limit warning is looked up once per sheet.
  • HWP / XLS / DOC: an OLE stream is read no further than the file, so a chain that loops with a huge declared size no longer builds gigabytes (a 26 KB HWP: 5.4 GB); a compressed HWP stream counts its stored bytes too, a picture stream named by many items is read once, and a file of very many streams opens in linear time (16,000 sections: 27 seconds).
  • HWP / HWPX: control characters are kept out of an equation script (\x05a+b\x06 read as struck text), and long atop chains with limits or over no longer take quadratic time (20 equations: 9 seconds).
  • RTF: shapes nested hundreds deep inside a table cell no longer redraw the cell at every level (35 seconds for 946 KB).
  • HTML: sub- and superscripts, formulas and form values inside a ruby annotation are read (水(H_2O)). Rows that open inside an element left in an open row (a <form> between rows) are read; they vanished with no warning. A <q> inside a superscript, subscript or ruby gets its quote marks too.

Full list: CHANGELOG.