Repository navigation
v0.8.0 — Page.Tables() ruled-table extraction
Added
-
Page.Content()now reports stroked ruling lines — table cell borders, underlines,
and other vector rules drawn with the content stream's stroke operators — as
Content.Stroke, a slice ofStroke{From, To Point}segments, alongside the existing
text and rectangles. Endpoints are in the page's upright display space (honoring page
/Rotateandcmtransforms), the same coordinate space asContent.Rect. Bézier
curves break the run rather than forming a chord; fill-only and clip paths are excluded;
rectangles drawn with thereoperator stay inContent.Rect(even when stroked), so a
table-grid consumer should read bothStrokeandRect— some documents rule tables
with thin filled rectangles rather than stroked lines. This is the raw vector signal for
detecting table structure without rendering. Marked experimental: the type and field
are additive-stable, but which segments are emitted may be refined as table support
matures. -
Page.Tables()extracts ruled tables from a page, returning each as aTablewhose
Cells[row][col]holds the text of that cell. It reconstructs the grid from the page's ruling
lines — both stroked lines and thin filled rectangles — and assigns each word to its cell by
position, so dense numeric values that fragment under text-only heuristics land whole in the
right cell. Half-open edge columns common in statistical tables — the row-label column and the
last data column, which have no outer vertical rule — are recovered when the table's own
horizontal row rules extend into them (and only then, so margin text such as page numbers or a
sidebar label is never mistaken for a column). Only ruled tables are detected: borderless and
partially-ruled tables, and open columns whose rules stop at the inner border, return nothing
rather than guessing. Marked experimental — theTabletype and the detection geometry may
gain fields or be refined in a future minor release. SeeEXAMPLES.mdfor usage.
Performance
- Text extraction allocates less and runs faster on large pages. The content-stream
interpreter previously allocated a fresh operand slice for every operator; it now
reuses a per-pass scratch buffer, cutting garbage on the structured-extraction paths
(Page.Content,Reader.DebugJSON,ExtractionSummary, and the CJKWords/Texts
paths). Output is byte-identical. Measured same-machine A/B (Apple M4 Pro):
DebugJSONon a 22-page CJK PDF uses −13.9 % memory with −8.0 % allocations
and runs −5.1 % faster; CJK word extraction uses −10.4 % memory.
GetPlainTextis unaffected.
Fixed
-
Text extraction now recovers simple fonts (Type1/TrueType/Type3/MMType1) whose embedded
ToUnicodeCMap declares a 2-bytecodespacerangeover single-byte character codes — a
common pattern in Adobe-generated PDFs (notably subset TrueType fonts namedCIDFont+F*).
Previously the decoder chunked the content bytes by the ToUnicode codespace width, so it read
the 1-byte codes two at a time, matched none of the 1-bytebfcharentries, and emitted
U+FFFD (the replacement character) for the entire run; an affected page extracted as almost
pure replacement characters even though its text layer was fully valid (external tools such as
pdftotext/pdfminerrecovered the text). A simple font's character codes are single-byte by
definition (PDF 32000-1:2008 §9.6), so its ToUnicode is now decoded one byte per code,
independent of the declared codespace width. Composite (Type0) fonts are unchanged — their CMap
codespace width is authoritative. The change is byte-identical for simple fonts whose ToUnicode
already declares a 1-byte codespace, and for every font without a ToUnicode stream. -
Page.Content()andPage.Words()no longer emit a U+FFFD replacement character for the
word/line separators aTJarray synthesizes. Showing text with aTJarray appends a
newline after the array (and a space across a wide kerning gap) as a layout marker; the
interpreter routed that interpreter-chosen ASCII separator through the content font's encoder,
and a simple font whoseToUnicodeCMap has no entry for the0x0A/0x20byte decoded it to
U+FFFD — so a table laid out with oneTJarray per cell leaked one replacement character per
array into the structured-extraction stream (a real IRS tax-table page carried 190 of them in
Content().TextandWords()). The separator is already its own Unicode, so an unmappable
decode now falls back to the literal separator, matching the byte-identical passthrough every
other encoder already gave it.GetPlainTextwas never affected (it synthesizes no such
separator); cell-grid values were unaffected (the zero-advance markers fall at cell edges).
Verify this release
This tag is signed with a hardware-backed ED25519-SK key:
256 SHA256:duCP4h22hb2oNAZMaFhUlpq0j8+qBbZuaXnS99yUhkY (ED25519-SK)
All of the maintainer's trusted signing keys are published on the GitHub account, so the recipe below verifies the tag regardless of which key signed it — nothing needs to be copied from these notes:
curl -s https://api.github.com/users/Detective-XH/ssh_signing_keys \
| python3 -c "import json,sys; [print('*', k['key']) for k in json.load(sys.stdin)]" > allowed_signers
git -c gpg.ssh.allowedSignersFile=./allowed_signers tag -v v0.8.0
# expect: Good "git" signature for * with ED25519-SK key SHA256:duCP4h22hb2oNAZMaFhUlpq0j8+qBbZuaXnS99yUhkY...Module integrity is independently guaranteed by the Go checksum database:
go mod download github.com/Detective-XH/gopdf@v0.8.0 && go mod verify