Skip to content

Releases

v0.8.7 — canonical Kruti Dev 010 legacy Devanagari recovery

Choose a tag to compare

@Detective-XH Detective-XH released this 30 Jun 16:30
v0.8.7
fbb9b61

Added

  • GoPDF now recovers real, searchable Unicode Devanagari from a canonical-coded simple Kruti Dev 010 font — a legacy non-Unicode font common in South Asian (Hindi) statistical PDFs whose bytes otherwise decode to Latin gibberish (for example fodk instead of विका, or ,u-;w-,y-,e- instead of एन.यू.एल.एम.). Where the previous release only flagged such garble, Page.Words(), Page.Lines(), Page.Blocks(), Page.Tables(), and Reader.GetPlainText() now emit the intended script for these fonts. The remap uses a built-in byte→Unicode transducer ported from the SIL wsresources mapping data (MIT — see NOTICE); no font files or external dependencies are bundled, and zero-width joiners are omitted so the recovered text stays searchable. The legacy_font_text warning no longer fires on text that is cleanly recovered.
  • The remap is deliberately conservative: it fires only for a strict-legacy-named, simple, canonical-coded font whose current decode is non-Indic. Composite (Type0) Kruti, subsetted Walkman-Chanakya, and DevLys fonts are not remapped — they still decode to gibberish and still warn. Extraction output for every non-matched font is byte-identical to before. One edge case: on a page that overlaps a text column with a table in the same vertical band, a recovered table label's leading glyphs can bucket into an adjacent row; the numeric data and surrounding prose are unaffected.

Verify this release

This tag is signed with an ED25519-SK FIDO2 hardware key:

256 SHA256:duCP4h22hb2oNAZMaFhUlpq0j8+qBbZuaXnS99yUhkY (ED25519-SK)

All of the maintainer's trusted keys are published on the GitHub account, so the recipe below works for any release regardless of which key signed it:

curl -s https://api.github.com/users/Detective-XH/ssh_signing_keys \
  | python3 -c "import json,sys; [print('*', k['key']) for k in json.load(sys.stdin)]" > allowed_signers
git -c gpg.ssh.allowedSignersFile=./allowed_signers tag -v v0.8.7
# expect: Good "git" signature for * with ED25519-SK key SHA256:duCP4h22hb2oNAZMaFhUlpq0j8+qBbZuaXnS99yUhkY

Module integrity is independently guaranteed by the Go checksum database:

go mod download github.com/Detective-XH/gopdf@v0.8.7 && go mod verify

v0.8.6 — honest text/table quality

Choose a tag to compare

@Detective-XH Detective-XH released this 29 Jun 22:01
v0.8.6
0ed76ba

v0.8.6 — 2026-06-29

Added

  • Page.Tables() and Reader.Warnings() now flag text drawn through a legacy non-Unicode font — a
    pattern common in South Asian statistical PDFs where script labels (such as Devanagari/Hindi) are
    rendered with fonts like Kruti Dev, DevLys, or Walkman-Chanakya, whose glyphs map to Latin
    characters rather than the intended script. Page.Tables() emits a per-table
    TableWarningLegacyFont (legacy_font_text) code on Table.Warnings when a table's alphabetic
    cell text is predominantly garbled in this way; Reader.Warnings() emits the same
    legacy_font_text code at the document level so consumers of Words(), Lines(), and
    GetPlainText() also receive the signal. The warning is strictly additive — Tables().Cells is
    byte-identical — and tells a caller that the garbled text cannot be recovered on a pure-text path.
  • Page.Words(), Page.Lines(), and Page.Blocks() now omit glyphs from a recurring diagonal
    watermark printed across most pages of a document — for example an agency stamp repeated on every
    page. GoPDF detects a recurring watermark by sampling up to 16 evenly-spaced pages; when the same
    angled-text signature recurs on at least half of them, its glyphs are excluded from word assembly,
    so a value like 11144 no longer arrives fused with watermark characters. Documents without a
    recurring page-spanning watermark are byte-unchanged; Page.Content(), Page.Texts(), and
    DebugJSON always return every glyph unfiltered. Page.Tables() is unaffected and byte-identical
    — it already unconditionally excludes diagonal text from cell content.
  • Page.Tables() now reports per-table quality: each Table carries a Confidence
    (TableConfidence — "high" or "low") and a Warnings slice ([]TableWarning, each with a
    TableWarningCode), so an LLM/RAG consumer can tell a trustworthy grid from a defective one
    without re-reading the page. Confidence is detection-relative: "high" means no quality
    detector flagged the table — not that it is verified correct — so a table may move from "high"
    to "low" in a future release as more detectors are added. The first detector flags a
    phantom_table: a grid whose columns are mostly empty (≥ 60% entirely blank), the signature of a
    bar chart or infographic misread as a ruled table (a genuinely very sparse table can also trip it,
    and the warning text says so). The flag never alters the extracted cells.
  • Page.TableRegions() returns the page-space bounding box of each detected table, 1:1 with
    Page.Tables() by index, so a consumer can locate a low-confidence table on the page (for example
    to crop and re-read it). Boxes are in the same Y-up display space as Page.Words() and
    Page.Content(). These additions do not change the reconstructed grids — Page.Tables().Cells is
    unchanged.

Fixed

  • Page.Tables() now recovers the data rows of "comb-body" tables — a layout, common in statistical
    yearbooks, where the header is ruled into columns but the data body is ruled only with vertical lines
    extending downward (no horizontal row rules and no closing bottom rule). Previously every data value
    fell outside the closed cells and was dropped from the grid, even though Page.Lines() held the data
    and the page rendered correctly. Page.Tables() now synthesizes the data cells at the header's column
    positions across the vertically-ruled data rows, recovering the full grid. False-positive guards ensure
    that non-table layouts — tables of contents, formula and prose blocks, bar charts, and infographic maps
    — are left untouched; Page.Words(), Page.Lines(), and Page.Blocks() are unaffected.
  • Page.Tables() now recovers the data rows of "headers-only" ruled tables — a layout, common in
    statistical yearbooks (e.g. quarterly tables), where the header is ruled into columns but the data body
    is ruled only down the leftmost row-label column. Previously every data value fell outside the closed
    cells and was dropped, so the grid came out with populated headers and entirely empty data columns even
    though Page.Lines() held the data and the page renders correctly. Page.Tables() now synthesizes the
    missing data cells at the header's column positions across the ruled data rows, but only for a column
    whose values recur across two or more rows — so genuinely empty cells stay empty and a lone stray word
    inside the table rectangle (e.g. a footnote) is not promoted to data. Recovery is confined to this
    structural pattern; normally- and partially-ruled tables are left untouched, and Page.Words(),
    Page.Lines(), and Page.Blocks() are unaffected. On tables whose columns are themselves only
    whitespace-separated, the recovered values may land in a single fused column — the data is no longer
    lost, but column splitting there remains a separate limitation.
  • Page.Tables() now reconstructs landscape tables that are embedded on a portrait page via a 90°
    counter-clockwise text matrix — a layout common in statistical yearbooks. Previously the rotation made
    each value extract character-reversed (Birganj→jnagriB, 1130.1→1.0311) and the whole grid came
    out transposed, so the table was unusable even though the page renders correctly. Page.Tables() now
    detects a predominantly-90°-rotated page and remaps it to reading orientation before reconstruction, so
    cells come out de-reversed and in the correct row/column order. The remap is internal to Page.Tables();
    Page.Words(), Page.Lines(), and Page.Blocks() continue to report such text in its raw rotated page
    geometry. On very dense or sparsely-ruled rotated tables the data is recovered in the correct cells but
    the grid may remain imperfect (a long number can be split, or a row flattened), so treat such a grid as
    advisory.
  • Page.Tables() no longer pulls diagonal watermark or rotated chart-label text into table cells. When a
    document carries a watermark printed diagonally across the page, or a chart with slanted axis/arc labels,
    the individual letters are scattered over the page at an angle; where one fell within a data row it was
    absorbed into that table cell — turning a value like 8,9 into G8,98,9, merging adjacent rows, or
    adding a spurious empty column. Page.Tables() now excludes text whose baseline is more than ~10° off an
    axis (horizontal or vertical) from cell content, so diagonal decoration can no longer corrupt the grid.
    Axis-aligned text is always kept, including 90°/270° landscape and vertical tables and headers.
    Page.Words(), Page.Lines(), and Page.Blocks() are unaffected — they still return every glyph,
    including diagonal text — so this narrows only the structured table view. Tables on heavily watermarked
    statistical pages, previously garbled, now extract cleanly.

Verify this release

This tag is signed with the maintainer's ED25519-SK FIDO2 hardware key:

256 SHA256:duCP4h22hb2oNAZMaFhUlpq0j8+qBbZuaXnS99yUhkY (ED25519-SK)

All of the maintainer's trusted keys are published on the GitHub account, so the recipe below verifies any release regardless of which key signed it:

curl -s https://api.github.com/users/Detective-XH/ssh_signing_keys \
  | python3 -c "import json,sys; [print('*', k['key']) for k in json.load(sys.stdin)]" > allowed_signers
git -c gpg.ssh.allowedSignersFile=./allowed_signers tag -v v0.8.6
# expect: Good "git" signature for * with ED25519-SK key SHA256:duCP4h22hb2oNAZMaFhUlpq0j8+qBbZuaXnS99yUhkY...

Module integrity is independently guaranteed by the Go checksum database:

go mod download github.com/Detective-XH/gopdf@v0.8.6 && go mod verify

v0.8.5 — table extraction column fixes

Choose a tag to compare

@Detective-XH Detective-XH released this 27 Jun 19:43
v0.8.5
2f8935a

Fixed

  • Page.Tables() now recovers the trailing groups of space-separated thousands numbers that overflow a
    column's rule. In some statistical tables the figures are typeset on a slightly wider spacing than the
    ruled columns, so the last group of a number — the 315 in 66 315 — crossed the column's right rule
    and was dropped, truncating the rightmost numeric column on every row. Such a trailing digit group is
    now re-attached to the number it belongs to, recovering the full value. The recovery is deliberately
    narrow: only an all-digit group that straddles a numeric cell's right rule within the normal
    intra-number spacing is pulled back, so labels, units, footnote markers, and adjacent columns are never
    drawn across a boundary. The change is additive — Page.Words() and Page.Lines() are unaffected, and
    every regression-locked corpus fixture is byte-for-byte unchanged.

  • Page.Tables() no longer injects a phantom column into a banded table whose header background is
    drawn as two side-by-side filled rectangles. The seam between those two header rectangles was being
    mistaken for a column rule; where it fell inside a data column it split the value there — for example
    a space-separated thousands figure 5 417 724 was broken into 5 417 and 724, and the table
    reported one column too many. The header band is now reconciled so the spurious column is dropped and
    the value stays whole, with all header and data text preserved. This corrects single-row seam headers;
    multi-row seam headers (two stacked header bands) are handled by a later fix below, while a genuine
    grouped header carrying real per-column sub-labels is always left intact. Ruled tables and every
    regression-locked corpus fixture are byte-for-byte unchanged.

  • Page.Tables() no longer emits phantom empty columns where a drawn vertical rule cuts across a
    wide spanning cell. When a real rule splits a cell that actually encloses several columns, the
    lattice produced an all-empty "column" that the gutter filter could not drop — the spanning cell
    made it look too wide to be a gutter. Such a column is now recognised structurally (an empty
    column whose drawn span contains another column's position is a mis-split artifact) and removed.
    Only all-empty columns are ever dropped, so no extracted text is affected. For example a Serbian
    statistical-yearbook crop-production table reported 8 columns where the page has 6. Tables whose
    empty columns are genuine, and every regression-locked corpus fixture, are unchanged.

  • Page.Tables() now also removes phantom seam columns from banded tables whose header is drawn as
    TWO stacked bands (e.g. a title row over a year-label row), extending the single-row seam fix above.
    Each header row is checked independently and only a row carrying a decorative fill-rectangle seam
    (a boundary with no corresponding data column) is collapsed; a genuine grouped/spanning header whose
    sub-column edges align with real data columns is left intact, so no header or data text is lost. For
    example an exchange-rate table and several monthly-digest tables that reported one or two extra empty
    columns now match the printed column count. Regression-locked corpus fixtures are byte-for-byte
    unchanged.

Verify this release

Signed with an ED25519-SK FIDO2 hardware key:

256 SHA256:duCP4h22hb2oNAZMaFhUlpq0j8+qBbZuaXnS99yUhkY (ED25519-SK)

All of the maintainer's trusted keys are published on the GitHub account, so this recipe works regardless of which key signed the tag — nothing needs copying from these notes:

curl -s https://api.github.com/users/Detective-XH/ssh_signing_keys \
  | python3 -c "import json,sys; [print('*', k['key']) for k in json.load(sys.stdin)]" > allowed_signers
git -c gpg.ssh.allowedSignersFile=./allowed_signers tag -v v0.8.5
# expect: Good "git" signature for * with ED25519-SK key SHA256:duCP4h2...

Module integrity is independently guaranteed by the Go checksum database:

go mod download github.com/Detective-XH/gopdf@v0.8.5 && go mod verify

v0.8.4

Choose a tag to compare

@Detective-XH Detective-XH released this 21 Jun 10:37
v0.8.4
a34e74e

Fixed

  • Page.Words() and Page.Tables() now read proportional glyph widths for composite (Type0) fonts that declare a uniform default width but also carry a per-character width array — common for Latin/Cyrillic (and other alphabetic) text wrapped in a Type0/Identity-H subset font, especially in non-US statistical PDFs. Previously every glyph was given the uniform default advance, so a long row label drifted into the adjacent right-aligned number and the two were welded character-by-character (for example a label instituce and a value 483 extracted as institu4c8e3). Across a fresh multilingual corpus this character-interleaving dropped sharply and net extracted numeric content rose; ruled tables, CJK tables, and the character set of Page.Words() are byte-for-byte unchanged. Composite fonts with a non-Identity encoding, and the degenerate zero-default-width case, are unaffected.
    • Known limitation: because this adjusts glyph positions on every page, a minority of documents may extract worse — specifically a banded table whose rows can be re-bucketed by the more-accurate positions; the font widths are correct, the sensitivity is in banded-table reconstruction, which remains best-effort.

Docs: the Page.Tables() scope in README/EXAMPLES is brought current (fully-ruled + rect-bordered locked classes; stroke-free per-cell-grid reconstruction + proportional Type0/Identity-H text as best-effort/improving; banded/borderless out of documented scope).

v0.8.3 — per-cell-grid table reconstruction & composite-font glyph placement

Choose a tag to compare

@Detective-XH Detective-XH released this 21 Jun 01:21
v0.8.3
4e40fe9

v0.8.3 — 2026-06-21

Added

  • Page.Tables() now merges the interleaved blank columns that appear when a stroke-free
    per-cell-grid table's rules sub-divide each logical column into a wide (header-bearing) cell nested
    with a narrow (data) cell. Such a table reconstructs with roughly twice its real column count — the
    header label landing in one grid column and its data in the adjacent one (a header read with no data,
    and data read with no header). GoPDF now merges each such nested pair back into a single column,
    realigning the header with its data. The merge is conservative and loss-free: it fires only when the
    two columns share a right wall (the nested sub-cell signature, not an ordinary column boundary), never
    hold content in the same row (so no cell is ever overwritten), and one column is a sparse header while
    the other is a dense data column (so two genuine data columns are never merged, and a small table is
    not collapsed). It was validated against the nested-sub-cell signature by unit tests; every ruled,
    rect-bordered, and previously-handled table is byte-for-byte unchanged, and reconstruction remains
    best-effort per the Page.Tables() documentation.

  • Page.Tables() now also recovers the rows of dense per-cell-grid statistical tables in which many
    stacked records had collapsed into a single tall cell (every value for a column landing in one row).
    Common in stroke-free government statistics — for example Japanese MHLW wage tables and US Census trade
    releases — GoPDF now splits that cell into its separate records. The split is conservative: it fires only
    where the would-be rows carry numeric values across two or more columns (so a wrapped multi-line label, a
    column header, or a footnote line is left intact), and only for a genuinely stacked block of several rows.
    It was validated against a blind-authored held-out wage-table fixture, where the recovered rows match the
    printed table. Known limitations: a multi-row block of numeric column headers, or a non-tabular text block
    (a cover page, a table of contents, or running prose) that reaches this reconstruction path, can be
    over-split. Every ruled, rect-bordered, and previously-handled table is byte-for-byte unchanged, and
    reconstruction remains best-effort per the Page.Tables() documentation.

  • Page.Tables() now recovers the columns of dense per-cell-grid statistical tables whose numeric
    columns previously collapsed into a single cell per row. In many stroke-free statistical tables — common
    in non-US government statistics, for example German Destatis and Taiwan DGBAS releases — the data band
    closed as one wide cell, so every figure in a row landed in the same column; GoPDF now splits that cell
    at the table's own vertical column rules, restoring the separate columns (measured against a blind-authored
    German insolvency regression fixture, the great majority of data cells now land in the right column). The split is conservative:
    it fires only at column boundaries the table itself establishes elsewhere (so a rule from a different
    table on the page cannot mis-split a cell) and only when the cell's own content actually spans those
    columns (so a spanning label or merged-header title is left intact). Every ruled, rect-bordered, and
    previously-handled table is byte-for-byte unchanged, and reconstruction remains best-effort per the
    Page.Tables() documentation.

  • Page.Tables() now reconstructs fill-banded "staircase" tables — statistical tables whose rows
    are separated by alternating background shading (filled rectangles) rather than ruled lines, drawn as
    a staircase of nested fills that share a common bottom edge (for example U.S. EIA Annual Energy Review
    energy-price tables). Such a table previously collapsed to roughly two-thirds of its rows — the lattice
    closed one cell per shading step and fused the rows that share a step — and GoPDF now splits those bands
    back into individual rows and columns, recovering the full grid. Detection is deliberately conservative:
    it requires the nested-fill staircase signature, so bar/column charts, shaded callout boxes, and shaded
    prose are not promoted to tables, and every other table (ruled, rect-bordered, or borderless) is
    byte-for-byte unchanged. Banded reconstruction remains best-effort (per the Page.Tables()
    documentation) — this covers the common-bottom staircase style; other banded layouts may still yield an
    incomplete grid.

  • Page.Tables() now also reconstructs the opposite banded style — dense per-cell-grid tables drawn
    with one filled rectangle per cell and no ruled lines (for example U.S. BEA Survey of Current
    Business GDP tables). Such a table's grid already closes on its own, but the page's title and footnote
    banners were swept in as extra phantom rows; GoPDF now restricts the result to the table's multi-column
    data body, dropping those banner rows and recovering the intended grid. Trimming is conservative: a row
    is removed only when it is a single full-width banner lying outside the data body — a real multi-column
    row is never dropped, even when blank — and a genuine table with no phantom banners is left byte-for-byte
    unchanged. As above, banded reconstruction is best-effort; cell values in these tables can still be
    limited by a separate word-spacing issue when adjacent numeric columns are laid out with no real gap
    between them.

Fixed

  • Page.Words(), Page.Tables(), and Page.Content() now place glyphs correctly for composite
    (Type0) CIDFonts whose default glyph width (/DW) is zero — a quirk of some
    Microsoft-Office-exported PDFs (for example a bold Cambria heading font). Such a font advances
    every glyph by zero, so a whole label stacked at one position and the geometry-sorted extractors
    read it scrambled — Personal consumption expenditures came out as
    esxpendituronsumption eersonal cP. GoPDF now reads the font's per-CID width array (/W) for these
    zero-/DW fonts, so the glyphs are spaced and the label reads in order. Fonts with a normal
    (non-zero) /DW are unchanged, and GetPlainText — which reads in stream order and never used
    glyph widths — was already correct and is byte-for-byte unaffected.

Verify this release

This tag is signed with an ED25519-SK FIDO2 hardware key:
256 SHA256:duCP4h22hb2oNAZMaFhUlpq0j8+qBbZuaXnS99yUhkY (ED25519-SK)

curl -s https://api.github.com/users/Detective-XH/ssh_signing_keys \
  | python3 -c "import json,sys; [print('*', k['key']) for k in json.load(sys.stdin)]" > allowed_signers
git -c gpg.ssh.allowedSignersFile=./allowed_signers tag -v v0.8.3
# expect: Good "git" signature for * with ED25519-SK key SHA256:duCP4h22hb2oNAZMaFhUlpq0j8+qBbZuaXnS99yUhkY

Module integrity:

go mod download github.com/Detective-XH/gopdf@v0.8.3 && go mod verify

v0.8.2 — table extraction quality + robustness

Choose a tag to compare

@Detective-XH Detective-XH released this 19 Jun 18:35
v0.8.2
dcd3118

v0.8.2 — 2026-06-19

Fixed

  • Page.Tables() now recovers row labels whose dot leader is rendered with its glyphs interleaved
    into the label text. Some publishers (for example USDA statistical tables) lay out a row label and
    its trailing dot leader as overlapping glyphs, so the label arrived fused with filler dots —
    Alabama extracting as Alaba...m...a..... GoPDF now detects this fused-leader signature (a run
    of three or more dots flanked by a letter on both sides) and strips the filler dots, recovering
    the label. Legitimate data text is left untouched: decimal points, abbreviations such as U.S., a
    trailing ellipsis (continued...), and a dot-separated range (1...3) are all preserved. The
    change affects only Page.Tables() cell text; plain text extraction (Words/Lines) is
    unaffected.

  • Page.Tables() no longer splits a cell value across adjacent columns when the PDF renders the
    token with an embedded zero-advance space — for example a comma-grouped number laid out as
    2,1+20 (2,120), or a character-spaced word laid out as addr+ess (address). Such tokens
    previously fragmented at the space, with the tail bleeding into the neighbouring cell; GoPDF now
    re-joins same-row pieces that abut with no real gap, so the value lands whole in one cell. Two
    genuinely distinct values that meet at a ruled cell border are kept apart — the re-join never
    crosses a vertical rule — and tables with normal column spacing are byte-for-byte unchanged.

  • A font whose /ToUnicode CMap is malformed — for example, a stray PostScript def keyword
    inside a dictionary literal — no longer empties the entire document. Previously the malformed
    CMap raised a parse error that propagated up through font loading and left every page's text
    empty; GoPDF now recovers, discards only the broken CMap, and falls back to the same handling as
    a font with no /ToUnicode (so Adobe CID and Identity fonts still decode correctly). A new
    malformed_tounicode warning is raised — alongside the existing missing_tounicode — so the
    degraded font stays visible rather than failing silently. A genuine internal fault is still
    re-raised rather than being masked as malformed input, and valid PDFs are unaffected.

  • Type0 CJK fonts that use /Encoding /Identity-H or /Identity-V with an Adobe-Japan1 CID
    ordering (/Registry (Adobe) /Ordering (Japan1)) and no /ToUnicode are no longer garbled.
    Their 2-byte show-string codes are Adobe-Japan1 CIDs, not Unicode; GoPDF now decodes them through
    Adobe's published CID→Unicode table instead of mangling each CID into two replacement characters.
    On real Japanese government/academic PDFs this took the U+FFFD replacement-character rate from
    ~30–46% to 0, recovering coherent Japanese that matches an independent extractor character for
    character. Font selection stays byte-identical for every other font: the path requires Type0 +
    Identity-H/V + the genuine Adobe registry and ordering + no usable /ToUnicode, so private CID
    collections, Identity-ordering fonts, and non-CJK fonts are unaffected. Vertical-writing fonts
    still raise the vertical-writing-mode warning (reading order for vertical text is unchanged). The
    Adobe-Japan1 mapping data is derived from Adobe's cid2code.txt (BSD-3-Clause; see NOTICE).

Changed

  • DocumentSummary's DecodeRatios.FallbackRatio now also counts glyphs decoded through the Adobe
    CID→Unicode table described above (it already counted predefined-CMap charset approximations).
    Both decode paths share the fallback_encoding warning, so the ratio and the warning stay
    consistent. Documents without such fonts are unaffected.

Verify this release

Signed with an ED25519-SK FIDO2 hardware key:

256 SHA256:duCP4h22hb2oNAZMaFhUlpq0j8+qBbZuaXnS99yUhkY (ED25519-SK)

All of the maintainer's trusted keys are published on the GitHub account, so this recipe verifies the tag regardless of which key signed it — nothing needs to be copied from these notes:

curl -s https://api.github.com/users/Detective-XH/ssh_signing_keys \
  | python3 -c "import json,sys; [print('*', k['key']) for k in json.load(sys.stdin)]" > allowed_signers
git -c gpg.ssh.allowedSignersFile=./allowed_signers tag -v v0.8.2
# expect: Good "git" signature for * with ED25519-SK key SHA256:duCP4h22hb2oNAZMaFhUlpq0j8+qBbZuaXnS99yUhkY

Module integrity is independently guaranteed by the Go checksum database:

go mod download github.com/Detective-XH/gopdf@v0.8.2 && go mod verify

v0.8.1 — text-extraction quality fixes

Choose a tag to compare

@Detective-XH Detective-XH released this 18 Jun 01:47
v0.8.1
3c38d09

v0.8.1 — 2026-06-18

Added

  • Page.Tables() now recovers rect-bordered tables — those with full column rules and an
    outer frame but no horizontal rules between data rows, which previously collapsed every column
    into a single cell and dropped the open row-label column. The detector infers row boundaries
    from the data text and recovers the open label column, so a frame-ruled, row-unruled grid
    extracts with its rows and label column intact. False positives on framed non-table regions
    (callout boxes, two-column prose, a ruled table's wrapped last row) are guarded against, and
    existing ruled-table output is unchanged. This remains Experimental — it is validated on
    table geometry, not yet a quality contract for rect-bordered tables generally: a region whose
    geometry is a single tall multi-line cell (one wrapped row rather than many rows) is
    indistinguishable from a data grid and will be split into rows.

Changed

  • Page.Tables() gains blocking accuracy regression gates on the public surface and a
    precise, honest documented scope; it remains Experimental. Reconstruction accuracy is
    now locked by corpus regression tests so it cannot silently drift: EPA eGRID 476/476 cells,
    IRS SOI 160/160 (including the recovered half-open right column, 32/32), and a 17×2 NIST
    HB44 grid at 33/34 content. The documented scope is made precise: ruled (lattice)
    tables
    — interior cells closed by visible rules between rows and columns, plus
    structurally-recovered half-open edge columns. Borderless and partially-ruled or banded
    tables (ruled only at group boundaries, rows separated by shading) are out of scope —
    best-effort, not a contract: they may return no table or an incomplete grid. A superscript
    renders at a distinct vertical position and font size, so it extracts as a spaced token
    (cm² → cm 2) — a Y-offset effect specific to superscripts, not a general spacing
    artifact, and it never changes which cell a value lands in. Graduation to a Stable API is
    gated on a forthcoming extraction-quality program. See API-STABILITY.md and EXAMPLES.md.

Fixed

  • GetPlainText, Page.Words(), and Page.Tables() no longer garble a composite (Type0) font
    that declares /Encoding /Identity-H and /ToUnicode as the name /Identity-H (rather than
    a CMap stream) — a pattern produced by some tools (e.g. certain LibreOffice/FreeType output) whose
    two-byte codes are in fact UCS-2 big-endian Unicode. Such a font previously decoded each two-byte
    code as two single bytes, emitting a U+FFFD before every character (roughly half the output), so a
    whole document came out unreadable; it now decodes directly to text. The recovery is narrowly
    gated — it applies only when the font is Type0, both /Encoding and /ToUnicode name
    Identity-H/Identity-V, and the descendant CIDSystemInfo ordering is explicitly Identity
    — so a genuine CJK CIDFont (Adobe-Japan1/GB1/CNS1/Korea1), whose Identity-H codes are glyph
    identifiers rather than Unicode, is unaffected and continues to report a missing-ToUnicode warning.
    There is no API change and every existing corpus golden is unchanged.

  • Page.Tables() no longer emits phantom empty columns from decorative double-wall borders.
    A frame drawn with two close-set parallel rules — common on report cover and navigation pages —
    leaves a thin gap that the lattice detector turned into an entirely empty column in the grid (a
    cover frame could reconstruct with eight such phantom columns). A thin, entirely empty gutter
    column is now dropped. The drop is width-gated: a column is removed only when it is both entirely
    empty AND narrow — below both a fraction of the table's median column width and an absolute
    ceiling — so a legitimately empty, normal-width data column is always preserved. There is no API
    change, and every ruled data table is unchanged.

  • Page.Tables() no longer emits tabular dot-leader filler — a run of four or more leader
    dots (.) that visually connects a row label to its value — into cell text. Such a run is page
    typography, not data: a Year label that previously extracted as 1973 ...................... now
    reads 1973, and a spurious extra column built from an all-leader overflow run is no longer
    fabricated. The trim is conservative — it fires only on a token of four or more consecutive .
    and nothing else, so decimals (4.0), abbreviations (U.S.A.), and three-dot ellipses are
    untouched, and a cell whose entire content is a dot run is preserved rather than emptied. The
    contamination was observed across multiple publishers' statistical and regulatory tables. There
    is no API change and every corpus golden is unchanged.

  • Page.Lines() and Page.Blocks() now keep columns in the correct reading order on dense
    multi-column pages. Previously a column whose text began slightly left of the detected
    inter-column gutter was mis-assigned to the previous column, so Lines() merged two columns into
    one line and Blocks() (which reads each column top-to-bottom) interleaved them — scrambling the
    reading order on layouts such as multi-column government, regulatory, and journal pages. A line or
    word is now assigned to the column a gutter opens only when the majority of it lies to the right of
    the gutter, so a genuine column start is placed correctly while a short or indented line in the
    left column stays put. There is no API change; single-column and CJK pages are unaffected. Reading
    order around a full-width line that spans the gutters (a masthead or heading) remains best-effort.

  • GetPlainText, Page.Words(), and Page.Tables() no longer emit U+FFFD replacement
    characters for a simple (Type1/TrueType) font whose embedded ToUnicode map marks a glyph as
    U+FFFD — a quirk of some PDF producers (e.g. a decimal point or other punctuation the subsetter
    could not map back to Unicode). Such a code now falls back to the font's own /Encoding,
    recovering the real character, so values that previously read as 4<U+FFFD>0 now extract as
    4.0. A code genuinely absent from the ToUnicode still reads as U+FFFD, and composite (Type0)
    fonts are unaffected; the fallback is used only where the font's encoding gives a confident
    mapping (a known base encoding or explicit /Differences), never to guess an unknown one.

  • Page.Words(), Page.Lines(), and Page.Tables() now correctly segment text from PDFs
    using composite (Type0 / CIDFont) fonts. Previously the glyph advance was always 0 for
    these fonts — GoPDF read only the simple-font /Widths array, which does not exist in
    CIDFont dicts — causing the word segmenter to split every character into its own word and
    break table cell matching. Glyph advance is now read from the descendant CIDFont's /DW
    entry (falling back to 1000 per PDF spec §9.7.4.3 when absent). Per-CID /W lookup is
    deferred; the uniform /DW approximation is sufficient for segmentation and is locked by a
    corpus regression gate.

  • GetPlainText no longer emits U+FFFD replacement characters for text drawn with composite
    (Type0 / Identity-H, two-byte CMap) fonts in two cases:

    • Text shown after a q/Q graphics-state save/restore — the active text encoder is now
      saved on q and restored on Q, so text that relies on the outer font restored by Q
      (with no fresh Tf) decodes through the correct encoder instead of the inner font's.
    • A T* line-move under such a font — the line break is now emitted as a newline instead of
      being decoded through the two-byte CMap and turning into a replacement character.

Verify this release

Signed with an ED25519-SK FIDO2 hardware key:

256 SHA256:duCP4h22hb2oNAZMaFhUlpq0j8+qBbZuaXnS99yUhkY (ED25519-SK)

The trusted signing keys live on the GitHub account, so verification needs nothing copied from here:

curl -s https://api.github.com/users/Detective-XH/ssh_signing_keys \
  | python3 -c "import json,sys; [print('*', k['key']) for k in json.load(sys.stdin)]" > allowed_signers
git -c gpg.ssh.allowedSignersFile=./allowed_signers tag -v v0.8.1
# expect: Good "git" signature for * with ED25519-SK key SHA256:duCP4h22hb2oNAZMaFhUlpq0j8+qBbZuaXnS99yUhkY

Module integrity is independently guaranteed by the Go checksum database:

go mod download github.com/Detective-XH/gopdf@v0.8.1 && go mod verify

v0.8.0 — Page.Tables() ruled-table extraction

Choose a tag to compare

@Detective-XH Detective-XH released this 14 Jun 17:07
v0.8.0
1d38caa

Added

  • Page.Content() now reports stroked ruling lines — table cell borders, underlines,
    and other vector rules drawn with the content stream's stroke operators — as
    Content.Stroke, a slice of Stroke{From, To Point} segments, alongside the existing
    text and rectangles. Endpoints are in the page's upright display space (honoring page
    /Rotate and cm transforms), the same coordinate space as Content.Rect. Bézier
    curves break the run rather than forming a chord; fill-only and clip paths are excluded;
    rectangles drawn with the re operator stay in Content.Rect (even when stroked), so a
    table-grid consumer should read both Stroke and Rect — some documents rule tables
    with thin filled rectangles rather than stroked lines. This is the raw vector signal for
    detecting table structure without rendering. Marked experimental: the type and field
    are additive-stable, but which segments are emitted may be refined as table support
    matures.

  • Page.Tables() extracts ruled tables from a page, returning each as a Table whose
    Cells[row][col] holds the text of that cell. It reconstructs the grid from the page's ruling
    lines — both stroked lines and thin filled rectangles — and assigns each word to its cell by
    position, so dense numeric values that fragment under text-only heuristics land whole in the
    right cell. Half-open edge columns common in statistical tables — the row-label column and the
    last data column, which have no outer vertical rule — are recovered when the table's own
    horizontal row rules extend into them (and only then, so margin text such as page numbers or a
    sidebar label is never mistaken for a column). Only ruled tables are detected: borderless and
    partially-ruled tables, and open columns whose rules stop at the inner border, return nothing
    rather than guessing. Marked experimental — the Table type and the detection geometry may
    gain fields or be refined in a future minor release. See EXAMPLES.md for usage.

Performance

  • Text extraction allocates less and runs faster on large pages. The content-stream
    interpreter previously allocated a fresh operand slice for every operator; it now
    reuses a per-pass scratch buffer, cutting garbage on the structured-extraction paths
    (Page.Content, Reader.DebugJSON, ExtractionSummary, and the CJK Words/Texts
    paths). Output is byte-identical. Measured same-machine A/B (Apple M4 Pro):
    DebugJSON on a 22-page CJK PDF uses −13.9 % memory with −8.0 % allocations
    and runs −5.1 % faster; CJK word extraction uses −10.4 % memory.
    GetPlainText is unaffected.

Fixed

  • Text extraction now recovers simple fonts (Type1/TrueType/Type3/MMType1) whose embedded
    ToUnicode CMap declares a 2-byte codespacerange over single-byte character codes
    — a
    common pattern in Adobe-generated PDFs (notably subset TrueType fonts named CIDFont+F*).
    Previously the decoder chunked the content bytes by the ToUnicode codespace width, so it read
    the 1-byte codes two at a time, matched none of the 1-byte bfchar entries, and emitted
    U+FFFD (the replacement character) for the entire run; an affected page extracted as almost
    pure replacement characters even though its text layer was fully valid (external tools such as
    pdftotext/pdfminer recovered the text). A simple font's character codes are single-byte by
    definition (PDF 32000-1:2008 §9.6), so its ToUnicode is now decoded one byte per code,
    independent of the declared codespace width. Composite (Type0) fonts are unchanged — their CMap
    codespace width is authoritative. The change is byte-identical for simple fonts whose ToUnicode
    already declares a 1-byte codespace, and for every font without a ToUnicode stream.

  • Page.Content() and Page.Words() no longer emit a U+FFFD replacement character for the
    word/line separators a TJ array synthesizes
    . Showing text with a TJ array appends a
    newline after the array (and a space across a wide kerning gap) as a layout marker; the
    interpreter routed that interpreter-chosen ASCII separator through the content font's encoder,
    and a simple font whose ToUnicode CMap has no entry for the 0x0A/0x20 byte decoded it to
    U+FFFD — so a table laid out with one TJ array per cell leaked one replacement character per
    array into the structured-extraction stream (a real IRS tax-table page carried 190 of them in
    Content().Text and Words()). The separator is already its own Unicode, so an unmappable
    decode now falls back to the literal separator, matching the byte-identical passthrough every
    other encoder already gave it. GetPlainText was never affected (it synthesizes no such
    separator); cell-grid values were unaffected (the zero-advance markers fall at cell edges).

Verify this release

This tag is signed with a hardware-backed ED25519-SK key:

256 SHA256:duCP4h22hb2oNAZMaFhUlpq0j8+qBbZuaXnS99yUhkY (ED25519-SK)

All of the maintainer's trusted signing keys are published on the GitHub account, so the recipe below verifies the tag regardless of which key signed it — nothing needs to be copied from these notes:

curl -s https://api.github.com/users/Detective-XH/ssh_signing_keys \
  | python3 -c "import json,sys; [print('*', k['key']) for k in json.load(sys.stdin)]" > allowed_signers
git -c gpg.ssh.allowedSignersFile=./allowed_signers tag -v v0.8.0
# expect: Good "git" signature for * with ED25519-SK key SHA256:duCP4h22hb2oNAZMaFhUlpq0j8+qBbZuaXnS99yUhkY...

Module integrity is independently guaranteed by the Go checksum database:

go mod download github.com/Detective-XH/gopdf@v0.8.0 && go mod verify

v0.7.9 — rotated and vertical text geometry

Choose a tag to compare

@Detective-XH Detective-XH released this 13 Jun 14:39
v0.7.9
057036a

Added

  • Text.H and Text.Rotation complete the per-glyph geometry on the Text struct
    (returned by Page.Texts(), Page.Content(), and Reader.GetStyledTexts).
    Text.H is the nominal font-box height in points — the magnitude of the text
    rendering matrix's up-vector, so it equals the font size for ordinary horizontal
    text and stays positive and rotation-invariant for rotated runs, where
    FontSize (taken from the matrix x-scale) collapses toward zero: on a 90°-rotated
    run whose FontSize reads 0, H still reports the true 12 pt. Text.Rotation is
    the baseline's angle in degrees, counter-clockwise-positive from horizontal
    (0 for upright text) — the rendering matrix's baseline angle, distinct from and
    opposite-signed to the page /Rotate attribute (clockwise), which is not applied
    here. Both fields are additive (Text stays Stable-tier); no existing field value,
    golden, or DebugJSON output changes. Word.H/Line.H are unchanged (still the
    first-glyph font size); unifying them with the rotation-aware Text.H is left to a
    later vertical-layout change. Carrying the two new float64 fields widens Text
    from 64 to 80 bytes, so the structured-extraction paths
    (Texts/GetStyledTexts/Content/DebugJSON) allocate +8–13 % more bytes and
    run +2–3.5 % slower, with the allocation count unchanged; GetPlainText is a
    separate path and is unaffected (measured same-machine A/B, Apple M4 Pro). A
    fast-path that skips the per-glyph trigonometry on horizontal text was measured and
    gave no benefit, confirming the cost is the wider struct rather than the
    computation — so the simpler unconditional form ships. See EXAMPLES.md.

  • Vertical writing-mode text (a -V predefined CMap) now advances down the page
    instead of horizontally. Previously such text decoded correctly but every glyph
    advanced along the x-axis (or, for composite fonts, did not advance at all), so a
    multi-glyph vertical run overprinted at a single point. The per-glyph advance and the
    TJ kerning adjustment now translate along the vertical axis using the PDF default
    one-em displacement, and text following a TJ array in the same text block is no
    longer pushed an extra line down. The vertical_writing_mode warning message now
    states that the advance is applied with a default-metrics approximation rather than
    ignored. Per-glyph vertical metrics (/W2), the glyph position vector, and vertical
    word/line grouping are not yet modelled; horizontal text, GetPlainText, and all
    existing field values and goldens are unchanged.

  • Page /Rotate (the clockwise display-rotation attribute) is now honored, so
    extracted text and image coordinates land in the page's upright display space instead
    of the raw content space that previously ignored the rotation. A page authored sideways
    with /Rotate set to display upright now extracts with a horizontal baseline — the font
    size comes back, the rotated_text warning stops, and reading-order grouping works. A
    new Page.Rotate() int accessor returns the applied clockwise rotation
    (0/90/180/270); /Rotate is inheritable and normalized (a non-multiple of 90,
    an absent, or a non-integer value reads as 0). Text.Rotation now reflects the combined
    text-matrix and page rotation in display space. The change is additive and
    golden-neutral: an unrotated page (/Rotate 0) is bit-for-bit unchanged,
    GetPlainText is unaffected, and every corpus, DebugJSON, and signal golden is
    unchanged; the per-page cost is within measurement noise. See EXAMPLES.md.

Changed

  • Content().Rect (the re path-construction rectangles from Page.Content()) now
    honors the CTM: each rectangle's four corners are mapped through the current
    transformation matrix and returned as their axis-aligned display-space bounding box, so
    it agrees with the /Rotate-honored text and image geometry on a rotated or
    cm-transformed page — the last mixed-frame surface after the page-/Rotate work. An
    unrotated page with no cm transform is bit-for-bit unchanged; the returned bbox is now
    always normalized (Min <= Max), which additionally corrects un-normalized output for a
    negative-dimension re. DebugJSON does not emit content rectangles, so its goldens are
    unaffected.

  • Word.H and Line.H (and the Block and DebugJSON boxes derived from them) are
    now the rotation-aware nominal font height — the same basis as Text.H — finishing
    the unification noted when Text.H shipped. Previously they came from the text
    matrix's x-scale (FontSize), which collapses toward 0 on a 90°-rotated run, doubles
    under horizontal text scaling, and goes negative under a horizontal flip; they now
    report the text up-vector magnitude, which is rotation-invariant and always
    non-negative
    , so a word, line, or block reports the same nominal height as the glyphs
    in it. For ordinary horizontal text this value equals the font size exactly, so all
    existing extraction output — every corpus golden, DebugJSON, and Page.Words()/
    Page.Lines() result — is byte-for-byte unchanged; only rotated, flipped, or scaled
    runs (none of which were previously locked by a test) take the new value. One
    consequence on the Experimental Page.Blocks() API: because the corrected line height
    also feeds block grouping, the way non-horizontal lines group into blocks can
    change — in particular a 90°-rotated column, which used to collapse into a single block
    because its old height read 0, can now split into separate blocks.

Performance

  • The text-extraction paths that group glyphs into lines — Page.Words,
    Page.Lines, Page.Blocks, Reader.DebugJSON, and Page.ExtractionSummary —
    now allocate substantially less memory. The internal y-band grouping step used to
    copy the page's glyph slice twice per page (a defensive copy plus per-band
    value copies); it now sorts a lightweight index permutation and references the
    original glyphs in place, leaving the page's content untouched. Output is
    byte-for-byte identical. On a 22-page Traditional-Chinese document,
    Reader.DebugJSON drops −21.6 % bytes allocated (and ~1 % fewer allocations),
    and the word/line path drops −29–31 % bytes. Time per operation is flat to
    slightly lower — the workload is dominated by garbage-collection pressure, so
    fewer bytes chiefly relieve GC. This more than offsets the +8–13 % bytes the new
    Text.H/Text.Rotation fields added (above).

Verify this release

This is a signed git tag (ED25519-SK / FIDO2 hardware key). This release was
signed with:

256 SHA256:BlsTid0viIuF2db9zc/gY1JerD7mK47KH19brGGFKWM (ED25519-SK)

The maintainer's trusted signing keys are published on the GitHub account, so
nothing needs to be copied from these notes:

curl -s https://api.github.com/users/Detective-XH/ssh_signing_keys \
  | python3 -c "import json,sys; [print('*', k['key']) for k in json.load(sys.stdin)]" > allowed_signers
git -c gpg.ssh.allowedSignersFile=./allowed_signers tag -v v0.7.9
# expect: Good "git" signature for * with ED25519-SK key SHA256:BlsTid0...

Module integrity is independently guaranteed by the Go checksum database:

go mod download github.com/Detective-XH/gopdf@v0.7.9 && go mod verify

v0.7.8 — document-level ToUnicode CMap cache

Choose a tag to compare

@Detective-XH Detective-XH released this 12 Jun 01:51
v0.7.8
8d4a978

Performance

  • CJK and other ToUnicode-based text extraction is substantially faster and
    lighter: a font's ToUnicode character map is now decoded once per document
    instead of re-parsed on every page of every extraction pass. Repeated
    extraction calls and multi-page documents reuse the decoded map, which is
    bounded in memory and safe for concurrent use. Output is byte-identical — the
    cache only changes when a map is parsed, never what it decodes to. On a
    22-page Traditional-Chinese document, Reader.GetPlainText drops −28 % time,
    −35 % bytes, −18 % allocations
    ; cold open-and-extract −26 % / −31 % /
    −17 %
    ; and Reader.DebugJSON a further −12 % / −10 % / −11 %.

  • Reader.DebugJSON / Page.DebugJSON got faster and lighter again: each page's
    words are no longer assembled twice. Building on the v0.7.7 single-interpret change,
    the structured-export path now reuses the words it already grouped into lines to
    drive its routing summary, instead of re-grouping the page from scratch. On a
    22-page CJK document Reader.DebugJSON drops from 67.1 to 59.7 ms/op (−11 %),
    74.8 to 59.3 MB/op (−21 %), and allocations from 921k to 863k per call
    (−6 %)
    . The output is byte-identical.

Verify this release

This is a signed git tag (ED25519-SK / FIDO2 hardware key). The maintainer's
trusted signing keys are published on the GitHub account, so nothing needs to be
copied from these notes:

curl -s https://api.github.com/users/Detective-XH/ssh_signing_keys \
  | python3 -c "import json,sys; [print('*', k['key']) for k in json.load(sys.stdin)]" > allowed_signers
git -c gpg.ssh.allowedSignersFile=./allowed_signers tag -v v0.7.8
# expect: Good "git" signature for * with an ED25519-SK key

Module integrity is independently guaranteed by the Go checksum database:

go mod download github.com/Detective-XH/gopdf@v0.7.8 && go mod verify