Skip to content

PDF Font Types and Encodings

Jump to bottom
Marc Jones edited this page Dec 22, 2025 · 1 revision

PDF Font Types and Encodings

This page documents how fonts work in PDF and how character codes map to Unicode text.

References

The Font Problem for Redaction

Why fonts matter for redaction:

  1. Character Codes ≠ Unicode: The byte 0x41 might mean 'A', 'α', or a ligature depending on the font
  2. PdfPig handles decoding: It reads font encoding and produces correct Unicode values
  3. Our challenge: Map PdfPig's decoded Unicode back to content stream bytes for modification
Content Stream → [Font Encoding] → Unicode
    (Hello)    →    [Decode]    → "Hello"
      ↓                            ↓
   Bytes to           ← Match ←   PdfPig
   modify                         Letters

Font Categories

Simple Fonts (Single-byte)

Each character code is exactly one byte (0-255).

Type Description Outline Format
Type 1 Adobe's original format PostScript
TrueType Apple/Microsoft format TrueType
Type 3 Glyphs as content streams PDF operators
MMType1 Multiple Master Type 1 PostScript

Encoding: 256-entry table mapping byte → glyph name or Unicode.

Composite Fonts (Multi-byte)

Character codes can be 1-4 bytes, enabling CJK and Unicode.

Type Description Glyph Format
Type 0 Composite font (references CIDFont) CMap → CIDFont
CIDFontType0 CID font with Type 1 outlines Type 1/CFF
CIDFontType2 CID font with TrueType outlines TrueType

Encoding: CMap that maps variable-length codes → CID values.

Simple Font Encoding

Standard Encodings

PDF defines four standard encodings:

Encoding Description Usage
StandardEncoding Adobe Latin subset Legacy Type 1 fonts
MacRomanEncoding Mac OS Roman Mac-created PDFs
WinAnsiEncoding Windows CP1252 Most common in Windows PDFs
MacExpertEncoding Expert glyphs Small caps, fractions

WinAnsiEncoding is most common for Western documents.

Encoding in Font Dictionary

/Font <<
  /Type /Font
  /Subtype /TrueType
  /BaseFont /Arial
  /Encoding /WinAnsiEncoding   % Named encoding
  /FirstChar 32
  /LastChar 255
  /Widths [...]
>>

Custom Encoding with Differences

Override specific character codes:

/Encoding <<
  /Type /Encoding
  /BaseEncoding /WinAnsiEncoding
  /Differences [
    128 /Euro          % Code 128 → Euro sign
    144 /bullet        % Code 144 → bullet
    160 /space         % Code 160 → space (NBSP)
  ]
>>

The Differences Array:

  • Integer: Starting code point
  • Names: Glyph names for consecutive codes

Example: [65 /A /B /C] means:

  • 65 → 'A'
  • 66 → 'B'
  • 67 → 'C'

Symbolic Fonts

Fonts like Wingdings or Symbol don't use standard encodings:

/Font <<
  /Type /Font
  /Subtype /TrueType
  /BaseFont /Wingdings
  /Encoding /Identity-H        % Or no encoding
  /Flags 4                     % Symbolic flag
>>

ToUnicode CMap

The most reliable way to extract Unicode text from a PDF.

Purpose

Maps character codes to Unicode values. This is what PdfPig primarily uses for text extraction.

Structure

/ToUnicode stream:

/CIDInit /ProcSet findresource begin
12 dict begin
begincmap
/CMapName /Adobe-Identity-UCS def
/CMapType 2 def
1 begincodespacerange
<00> <FF>
endcodespacerange

% Single character mappings
3 beginbfchar
<01> <0041>            % Code 01 → U+0041 (A)
<02> <0042>            % Code 02 → U+0042 (B)
<03> <0043>            % Code 03 → U+0043 (C)
endbfchar

% Range mappings
1 beginbfrange
<04> <08> <0044>       % Codes 04-08 → U+0044-U+0048 (D-H)
endbfrange

endcmap

Key Operators

Operator Purpose Example
beginbfchar / endbfchar Single mappings <01> <0041>
beginbfrange / endbfrange Range mappings <04> <08> <0044>
begincodespacerange Valid code ranges <00> <FF>

Ligatures and Multi-Character Mappings

A single glyph can map to multiple Unicode characters:

<FB00> <0066 0066>     % ff ligature → "ff"
<FB01> <0066 0069>     % fi ligature → "fi"
<FB02> <0066 006C>     % fl ligature → "fl"

Redaction implication: One glyph = multiple Unicode characters.

CID Fonts (CJK Support)

Structure

/Type0 font
    ↓
   CMap (encoding)
    ↓
CIDFont (glyph descriptions)
    ↓
   Glyphs

Predefined CMaps

CMap Languages Description
Identity-H Universal Horizontal, CID = code
Identity-V Universal Vertical, CID = code
UniJIS-UTF16-H Japanese UTF-16 to CID
UniGB-UTF16-H Chinese (Simplified) UTF-16 to CID
UniKS-UTF16-H Korean UTF-16 to CID
UniCNS-UTF16-H Chinese (Traditional) UTF-16 to CID

Identity-H/V

The most common CMap. Character code directly equals CID:

% Code 0x0041 → CID 65 → Glyph for 'A'

With a ToUnicode map, this becomes:

% Code 0x0001 → Unicode U+0041 (via ToUnicode)

CIDToGIDMap

For TrueType-based CID fonts, maps CID to Glyph Index:

/CIDToGIDMap /Identity      % CID equals Glyph Index
% or
/CIDToGIDMap stream         % Custom mapping table

How PdfPig Decodes Text

PdfPig's decoding priority:

  1. ToUnicode CMap (if present) - Most reliable
  2. Encoding + Differences (for simple fonts)
  3. Standard encoding (fallback)
  4. Glyph names to Unicode (via Adobe Glyph List)

This is why PdfPig's text extraction is reliable - it uses multiple fallback strategies.

Font Subsetting

PDF generators often embed only used glyphs:

/BaseFont /BCDFEE+Arial-Bold     % Subset prefix: BCDFEE+

The 6-letter prefix indicates a subset. The font contains only needed glyphs.

Implication: Glyph indices may differ from full font.

Embedded vs. Referenced Fonts

Fully Embedded

Font data included in PDF:

/FontDescriptor <<
  /FontFile2 stream           % TrueType font data
  % or
  /FontFile stream            % Type 1 font data
  % or
  /FontFile3 stream           % CFF or OpenType
>>

Referenced (System Fonts)

Font must exist on system:

/Font <<
  /BaseFont /Helvetica        % Standard 14 fonts always available
>>

Standard 14 Fonts (always available):

  • Times-Roman, Times-Bold, Times-Italic, Times-BoldItalic
  • Helvetica, Helvetica-Bold, Helvetica-Oblique, Helvetica-BoldOblique
  • Courier, Courier-Bold, Courier-Oblique, Courier-BoldOblique
  • Symbol, ZapfDingbats

Type 3 Fonts

Glyphs defined as PDF content streams:

/Font <<
  /Type /Font
  /Subtype /Type3
  /FontBBox [0 0 1000 1000]
  /FontMatrix [0.001 0 0 0.001 0 0]
  /CharProcs <<
    /A stream                  % Content stream for 'A'
    /B stream                  % Content stream for 'B'
  >>
  /Encoding <<
    /Differences [65 /A /B /C]
  >>
>>

Redaction challenge: The glyph itself is a mini-PDF. To redact, we'd need to:

  1. Identify Type 3 font usage
  2. Modify the glyph stream (or replace with empty glyph)
  3. Or remove the text operator entirely

Practical Examples

Simple ASCII Text (Most Common)

/F1 12 Tf
(Hello World) Tj
  • Font F1 uses WinAnsiEncoding
  • Each byte maps directly: 0x48 → 'H', 0x65 → 'e', etc.
  • PdfPig returns "Hello World"

Kerned Professional Text

/F1 12 Tf
[(W) -80 (A) -40 (TER)] TJ
  • Same encoding as above
  • Numbers are kerning adjustments
  • PdfPig returns "WATER" with correct spacing

CJK Text

/F2 12 Tf
<4E2D6587> Tj
  • Font F2 is Type 0 with Identity-H CMap
  • Two 2-byte codes: 0x4E2D, 0x6587
  • ToUnicode maps to U+4E2D U+6587
  • PdfPig returns "中文"

Symbolic Font

/Wingdings 12 Tf
<6C> Tj
  • Code 0x6C in Wingdings = envelope symbol
  • ToUnicode would map to appropriate Unicode
  • PdfPig returns the Unicode character

Our Codebase Approach

Since PdfPig handles all decoding, our approach is:

  1. Trust PdfPig: Use its Letter objects for Unicode values and positions
  2. Match by content: Find content stream operators by matching decoded text sequences
  3. Modify at byte level: When removing text, we modify the content stream bytes

We do NOT need to:

  • Parse font dictionaries ourselves
  • Decode character codes ourselves
  • Handle encoding variations ourselves

PdfPig does all of this for us.

Codebase Locations

File Purpose
CharacterMatcher.cs Matches PdfPig letters to operators
ContentStreamParser.cs Parses text operators (not font decoding)
PdfPig (external) Handles all font/encoding decoding

Last updated: 2024-12-22

Clone this wiki locally