Redaction Library Implementation

Redaction Library Implementation

This page tracks the implementation status of the PdfEditor.Redaction library - a modular TRUE glyph-level redaction engine.

Overview

The Redaction Library (PdfEditor.Redaction) is a standalone, reusable component for TRUE content-level PDF redaction. Unlike visual overlay redaction (drawing black boxes), this library removes text from the PDF content stream.

Key Design Principles:

• Glyph-Level Removal: Individual characters are removed from content streams
• PdfPig for Positioning: Use PdfPig as source of truth for letter positions
• PDFsharp for Modification: Use PDFsharp 6.x for PDF structure modification
• Modular Operator Handlers: Each PDF operator has its own handler class

References

• PDF 32000-1:2008 - PDF Reference (Section 9 covers text)
• PDF Operators Cheat Sheet - PDF Association quick reference
• PDF Text Operators - Our detailed operator documentation

Current Status: v1.4.0

All core operators implemented. Glyph-level redaction working.

Implemented Operators

Operator	Description	Status	Notes
BT/ET	Begin/end text object	✅ Done
Tf	Set font and size	✅ Done
Tm	Set text matrix	✅ Done
Td	Move text position	✅ Done
TD	Move + set leading	✅ Done
TL	Set text leading	✅ Done
T*	Move to next line	✅ Done
Tj	Show text string	✅ Done
TJ	Show array w/kerning	✅ Done
'	Quote (newline + show)	✅ Done	Issue #81
"	Double quote (spacing + show)	✅ Done	Issue #82
Tc	Set character spacing	✅ Done
Tw	Set word spacing	✅ Done
Tz	Set horizontal scaling	✅ Done
Ts	Set text rise	✅ Done
Tr	Set text rendering mode	✅ Done

Test Coverage

Project	Tests	Status
PdfEditor.Redaction.Tests	300+	✅ Passing
PdfEditor.Redaction.Cli.Tests	74	✅ Passing
Birth Certificate Tests	8	✅ Passing

Known Limitations

Feature	Status	Issue
Rotated pages	⚠️ Limited	#151
Inline images	❌ Not supported	#152
Clipping paths	❌ Not tracked	#154
Shading patterns	❌ Not supported	#163

Architecture

Project Structure

PdfEditor.Redaction/
├── ITextRedactor.cs              # Main public interface
├── TextRedactor.cs               # Implementation
├── IContentStreamParser.cs       # Parser interface
├── ContentStream/
│   ├── ContentStreamTokenizer.cs # Low-level parsing
│   ├── ContentStreamRedactor.cs  # Core redaction engine
│   └── Building/
│       └── ContentStreamBuilder.cs # Rebuilds content streams
├── GlyphLevel/
│   ├── GlyphRemover.cs           # Orchestrates glyph removal
│   ├── LetterFinder.cs           # Matches PdfPig letters to operations
│   ├── TextSegmenter.cs          # Splits text into keep/remove
│   └── OperationReconstructor.cs # Rebuilds operations
├── Operators/
│   ├── IOperatorHandler.cs       # Handler interface
│   ├── PdfParserState.cs         # Parser state
│   ├── OperatorRegistry.cs       # Handler registration
│   ├── TextObject/               # BT, ET
│   ├── TextState/                # Tf, TL, Tc, Tw, Tz, Ts, Tr
│   ├── TextPositioning/          # Tm, Td, TD, T*
│   └── TextShowing/              # Tj, TJ, ', "
├── PdfPig/
│   └── PdfPigHelper.cs           # Letter extraction
├── PdfADetector.cs               # PDF/A detection
└── PdfAMetadataPreserver.cs      # PDF/A metadata preservation

Modular Operator Pattern

Each operator has its own handler implementing IOperatorHandler:

public interface IOperatorHandler
{
    string OperatorName { get; }
    PdfOperation? Handle(IReadOnlyList<object> operands, PdfParserState state);
}

Handlers are registered in OperatorRegistry:

var registry = OperatorRegistry.CreateDefault();
// Registers all 16+ operators automatically

Glyph-Level Redaction Flow

1. Extract letters from PDF (via PdfPig)
2. Find letters that match redaction area
3. Map letters back to content stream operations
4. Split operations into keep/remove segments
5. Reconstruct operations without removed glyphs
6. Build new content stream
7. Replace page content
8. Draw visual marker (black rectangle)

TRUE Redaction Strategy

The Problem with Black Box Redaction

Original PDF:    "SSN: 123-45-6789"
Black box:       "SSN: ████████████"  ← Text still in PDF!
Text extraction: "SSN: 123-45-6789"  ← Security failure!

Our Solution: Glyph Removal

1. Parse content stream → Identify text operations
2. Extract letters → Get exact positions from PdfPig
3. Match letters to redaction area
4. Remove matching glyphs from operations
5. Rebuild content stream without removed glyphs

After TRUE redaction:

Text extraction: "SSN: "  ← Sensitive data GONE

Verification

Every redaction test MUST verify text is actually removed:

var textAfter = PdfTestHelpers.ExtractAllText(redactedPdf);
textAfter.Should().NotContain("123-45-6789",
    "Text must be REMOVED from PDF structure, not just hidden");

CLI Tool: pdfer

The pdfer command-line tool provides scripted access to redaction:

# Redact text from PDF
pdfer redact input.pdf output.pdf --text "CONFIDENTIAL"

# Search for text
pdfer search input.pdf --text "CONFIDENTIAL"

# Verify redaction
pdfer verify output.pdf --text "CONFIDENTIAL"

# Get PDF info
pdfer info input.pdf

See CLI-Tool for detailed documentation.

Adding a New Operator

1. Create handler class in appropriate subdirectory:

   public class NewOperatorHandler : IOperatorHandler
   {
       public string OperatorName => "Op";
       public PdfOperation? Handle(IReadOnlyList<object> operands, PdfParserState state)
       {
           // Implementation
       }
   }

2. Register in OperatorRegistry.CreateDefault():

   registry.Register(new NewOperatorHandler());

3. Add unit tests

4. Update this wiki page

---

Last updated: 2024-12-29