Repository navigation
中文
本次为纯修复,无新增功能。
- PDF → Markdown 乱码与假表格:矢量图里的文字常来自 Visio/Word 子集字体(ArialMT、TimesNewRomanPSMT 等),这些字体没有 Unicode 映射,任何提取器都只能得到
!"#$%&'这类乱码。现在按「字体族 + 字母数字占比」识别图内文字,改为输出一条> [图内文字无法可靠提取,已省略(图形区域文字缺少 Unicode 映射)]占位,不再输出乱码;被图形框线误判成表格的图区也只留占位,不再生成假表格。 - 数学公式符号还原:LaTeX 的 txfonts(txex / txsyc 等)同样缺少 Unicode 映射,此前
∑会变成X、≠会变成,、矩阵括号变成私有区字符或(cid:N)。现在按 PDF/Differences的字形名还原:∑、≠、⎡⎢⎣⎤⎥⎦、[]/()均可正确输出;同时保留 pdfminer 的空格标注,避免整行单词粘连(AdversarialFiltering...)。 - 版面清理:自动剔除页脚与页码;双栏页面按「左栏读到底再读右栏」重排(不再左右交错);合并行尾连字符(
Evasion at-\ntacks→Evasion attacks)。 - Markdown 安全:表格单元格转义
$_*`[]~&<>#等字符,并在表格块之间留空行,避免转换出的$...$被前端 texmath 当作公式、渲染出 KaTeX ParseError。 - 升级方式不变(按 README「版本迁移」覆盖文件);本次不涉及配置或数据库迁移。
English
This is a fix-only release with no new features.
- PDF → Markdown garbage and fake tables: text inside vector figures often comes from Visio/Word subset fonts (ArialMT, TimesNewRomanPSMT, …) with no Unicode mapping, so every extractor can only produce garbage such as
!"#$%&'. Figure text is now detected by "font family + alphanumeric ratio" and replaced with a single> [figure text could not be extracted reliably, omitted]placeholder instead of garbage; figure regions whose borders were mistaken for tables also become a placeholder rather than a fake table. - Math glyph recovery: LaTeX txfonts (txex / txsyc, …) also lack Unicode mappings, so
∑used to come out asX,≠as,, and matrix brackets as private-use characters or(cid:N). They are now restored from the PDF/Differencesglyph names:∑,≠,⎡⎢⎣⎤⎥⎦,[]/()all render correctly; pdfminer's space annotations are preserved so words no longer fuse intoAdversarialFiltering.... - Layout cleanup: page footers and page numbers are dropped; two-column pages are reordered as "read the left column to the bottom, then the right column" (no more interleaving); end-of-line hyphenation is joined (
Evasion at-\ntacks→Evasion attacks). - Markdown safety: table cells escape
$_*`[]~&<>#and table blocks are separated by blank lines, so a converted$...$is no longer treated as a formula and rendered as a KaTeX ParseError. - Upgrading is unchanged (see README "Version migration"); this release has no config or database migration.