Skip to content

v0.1.0

Choose a tag to compare

View all tags
@github-actions github-actions released this 29 Jan 21:57
· 27 commits to main since this release
v0.1.0
ab860f3

docprims v0.1.0

Release Date: 2026-01-28
Status: Initial Release

Summary

GPL-free document text extraction primitives for Rust with Go bindings. Extract text from DOCX, XLSX, PPTX, Markdown, HTML, and XML with provenance tracking and schema-validated output.

Highlights

  • Schema-Validated Output: All extractors emit DocprimsExtract JSON conforming to versioned schemas
  • Provenance Tracking: Every text block includes locators back to source (paragraph index, cell ref, etc.)
  • Defensive Parsing: Built for untrusted input with configurable resource limits
  • GPL-Free: All dependencies permissively licensed (MIT/Apache-2.0)

Architecture

┌─────────────────────────────────────────────────────────────┐
│                        Consumers                            │
├─────────────┬─────────────┬─────────────┬──────────────────┤
│  Rust API   │  Go Binding │  CLI User   │  Future: TS/Py   │
└──────┬──────┴──────┬──────┴──────┬──────┴──────────────────┘
       │             │             │
       ▼             ▼             ▼
┌─────────────────────────────────────────────────────────────┐
│                      docprims crates                        │
├─────────────┬─────────────┬─────────────┬──────────────────┤
│ docprims-   │ docprims-   │ docprims-   │ docprims-cli     │
│ core        │ text        │ ooxml       │                  │
└─────────────┴─────────────┴─────────────┴──────────────────┘
       │
       ▼
┌─────────────────────────────────────────────────────────────┐
│                    DocprimsExtract                          │
│  { schema_version, source, document: { text, blocks } }     │
└─────────────────────────────────────────────────────────────┘

Supported Formats

Format Crate Block Kinds Locator Type
Markdown docprims-text markdown:heading, markdown:paragraph, markdown:code, markdown:list_item markdown:locator
HTML docprims-text html:block html:locator
XML docprims-text xml:text xml:locator
DOCX docprims-ooxml docx:paragraph, docx:table_cell docx:locator
XLSX docprims-ooxml xlsx:cell xlsx:locator
PPTX docprims-ooxml pptx:shape_text pptx:locator

Installation

Rust

[dependencies]
docprims-core = "0.1.0"
docprims-text = "0.1.0"
docprims-ooxml = "0.1.0"

CLI

cargo install --path crates/docprims-cli

Go

go get github.com/3leaps/docprims/bindings/go/docprims@v0.1.0

Usage

CLI

# Plain text extraction
docprims extract document.docx

# JSON with structured blocks
docprims extract document.md --format json --include-blocks

# Multi-file NDJSON
docprims extract *.docx --format json

# With logging
docprims --log-level debug extract document.xlsx

Rust API

use docprims_text::extract_markdown_v0;
use docprims_core::ExtractLimits;

let limits = ExtractLimits::default();
let extract = extract_markdown_v0("README.md", limits)?;

println!("Text: {}", extract.document.text);
for block in &extract.document.blocks {
    println!("Block {}: {}", block.id, block.text);
}

Go API

package main

import (
    "fmt"
    "log"

    "github.com/3leaps/docprims/bindings/go/docprims"
)

func main() {
    limits := docprims.ExtractLimits{
        MaxInputBytes: 100 * 1024 * 1024, // 100MB
    }

    result, err := docprims.ExtractMarkdownV0("README.md", limits)
    if err != nil {
        log.Fatal(err)
    }

    fmt.Println("Text:", result.Document.Text)
    for _, block := range result.Document.Blocks {
        fmt.Printf("Block %s: %s\n", block.ID, block.Text)
    }
}

Output Schema

DocprimsExtract

{
  "schema_id": "https://schemas.3leaps.dev/docprims/extract/v0/docprims-extract.schema.json",
  "schema_version": "1.0.0",
  "generator": {
    "name": "docprims",
    "version": "0.1.0"
  },
  "source": {
    "uri": "./document.md",
    "format": {
      "family": "text",
      "kind": "markdown"
    },
    "sha256": "abc123..."
  },
  "document": {
    "quality": {
      "status": "complete"
    },
    "text": "Full extracted text joined from blocks...",
    "blocks": [
      {
        "id": "markdown:heading:0",
        "kind": "markdown:heading",
        "text": "Document Title",
        "doc_text_range": {
          "start_byte": 0,
          "end_byte": 14
        },
        "attrs": {
          "level": 1
        },
        "loc": {
          "kind": "markdown:locator",
          "container": {
            "kind": "file",
            "path": "./document.md"
          },
          "hints": {
            "block_index": 0
          }
        }
      }
    ]
  }
}

Security

docprims is designed for untrusted input:

Threat Mitigation
Zip bombs Decompression ratio limits
XML bombs (billion laughs) Entity expansion disabled
Path traversal Archive path validation
Memory exhaustion Configurable max_input_bytes, max_output_bytes
Excessive blocks Configurable max_blocks

Resource Limits

pub struct ExtractLimits {
    pub max_input_bytes: usize,  // Default: 100MB
    pub max_output_bytes: usize, // Default: 50MB
    pub max_blocks: usize,       // Default: 100000
}

Platform Support

Go Bindings

Platform Architecture Library Build
macOS arm64 Prebuilt Vendored
Linux amd64 Prebuilt glibc 2.17+
Linux arm64 Prebuilt glibc 2.17+
Linux amd64-musl Prebuilt Alpine
Linux arm64-musl Prebuilt Alpine
Windows amd64 Prebuilt MinGW/GNU

Rust

  • MSRV: Rust 1.85.0
  • Tested on: Linux, macOS, Windows

Dependencies

All dependencies are permissively licensed (MIT or Apache-2.0):

Crate Version License Purpose
quick-xml 0.39 MIT XML parsing
zip 2.x MIT ZIP archive handling
pulldown-cmark 0.13 MIT Markdown parsing
scraper 0.25 MIT HTML parsing
encoding_rs 0.8 MIT/Apache-2.0 Character encoding

Known Limitations

  • PDF extraction not supported (planned for v0.2)
  • OOXML table structure (row/column indices) not yet in blocks
  • Span annotations (bold/italic/links) planned for future release
  • RTF format not supported

Related Documentation

Changelog

See CHANGELOG.md for detailed changes.

Assets 20
Loading