Skip to content

v2.2.0 — OpenDocument, YouTube proxies and typed extraction errors

Latest

Choose a tag to compare

@lfnovo lfnovo released this 03 Oct 19:55
c4725c0

Content Core 2.2.0 adds OpenDocument support, YouTube cookies and proxies, and configurable audio segments. Extraction failures now raise typed errors across the library, CLI and MCP instead of returning empty content.

New capabilities

  • Extract OpenDocument files: .odt, .ods and .odp (#43, #96). Text documents become Markdown with paragraphs, headings, lists and tables. Spreadsheets become one table per sheet (# Sheet: <name>), and presentations become one block per slide (# Slide N), the same structure XLSX and PPTX already use. Detection reads the archive's mimetype, so a wrong or missing extension still works. odfpy is a new core dependency.
  • YouTube cookies and proxies for transcripts (#46, #95). youtube_cookies_file (CCORE_YOUTUBE_COOKIES_FILE) loads a Netscape cookies.txt, which unblocks networks YouTube has flagged. youtube_proxy (CCORE_YOUTUBE_PROXY) routes the transcript paths and the title lookup through a proxy; use a residential proxy, because datacenter proxies are blocked. With neither set, behavior is unchanged. Cookie values are never logged.
  • audio_segment_minutes (CCORE_AUDIO_SEGMENT_MINUTES) sets the length of the segments long audio is split into before transcription. The default stays 10 minutes. 0 sends the file whole, for self-hosted STT endpoints without an upload limit (#94).

⚠️ Behavior changes

Hiding an error is treated as a bug, so these changes ship in 2.x (#60, #97). If your code checked for empty content to detect a failure, catch ContentCoreError (or one of its subclasses) instead. content == "" now only means the source was empty.

  1. URL extraction raises. A named url_engine that fails raises one of these:

    • NetworkError for connection, timeout or DNS failures
    • NotFoundError for HTTP 404/410 with the simple engine
    • ExternalServiceError for engine API and HTTP errors
    • ConfigurationError when crawl4ai is selected but not installed

    auto still falls back through firecrawl → jina → crawl4ai → simple and raises only when every engine fails. A malformed URL raises InvalidInputError. The simple engine no longer returns error text as page content.

  2. File and media extraction raise. Corrupted PDF and EPUB files, and text files that can't be read, raise FileOperationError. ffmpeg/ffprobe failures raise FileOperationError; video extraction no longer returns empty content. STT provider failures raise ExternalServiceError after retries.

  3. YouTube raises. When no transcript can be fetched, YouTube extraction raises NoTranscriptFound. When YouTube blocks or fails the request on both paths, it raises ExternalServiceError, and a URL without a video ID raises InvalidInputError.

  4. An explicit STT model is honored or raises. If audio_provider/audio_model are set and the model can't be created, Content Core raises ConfigurationError. It no longer silently transcribes with the default model (#99).

  5. Retries are classified by error type. Configuration, input and file errors are permanent, and network errors are transient. Blocked YouTube requests are retried, so a rotating proxy can get a new IP. Other transcript errors that name a state of the video (unavailable, disabled, age-restricted) are not retried (#99).

Fixes

  • youtube_proxy now also covers the video title lookup (#99).
  • requests is declared as a direct dependency instead of arriving transitively (#99).
  • A failed Crawl4AI crawl (navigation timeout, blocked page) raises ExternalServiceError instead of crashing with a pydantic ValidationError, and auto falls through to the next engine (#101).

Security

  • Minimum versions raised to releases without known vulnerabilities: aiohttp>=3.14.3, pillow>=12.3.0, click>=8.3.3, pydantic-settings>=2.14.2; in the extras, docling>=2.94.0, crawl4ai>=0.9.0 and langchain-core>=1.3.3. Upgrading content-core upgrades these in an existing environment (#102).
  • Two advisories remain in the locked set: nltk (no fixed release yet) and transformers, which the docling extra keeps below 5 until docling compatibility is validated (#103).

Validation

  • Tests: 526 unit and integration tests passed on the release commit. CI passed on Python 3.10, 3.11 and 3.12.
  • Built package: the built wheel passed isolated library and CLI smokes (TXT and PDF). Over MCP stdio, a real extract_content tool call returned the fixture text, with fastmcp 4 and mcp 2. Versions and runtime assets were checked. The docling, crawl4ai and langchain extras installed; the LangChain extract_content_tool ran with langchain-core 1.6.6.
  • Live checks, all passed:
    • YouTube transcript extraction
    • remote PDF download
    • URL extraction with Firecrawl, Jina, Crawl4AI 0.9.4 and simple
    • OpenAI Whisper transcription of real MP3 and MP4 files
    • Docling PDF extraction, including the formula and picture enrichments
  • Error paths: a site unreachable from the test host raised the new typed errors, and auto tried every engine before raising.
  • Not verified live: YouTube with a real residential proxy or cookies file, and STT providers other than OpenAI.
  • Security alerts: Dependabot alerts are disabled for this repository, so they weren't checked. pip-audit was run over the locked dependency set instead; see Security above.

The release workflow builds its own artifacts. The published files will be checked from PyPI after publication.

Thanks

  • @kyxap1: configurable audio segment length (#94).
  • @lfnovo: OpenDocument support, YouTube cookies and proxies, typed extraction errors, review follow-ups and release preparation (#95, #96, #97, #99, #100, #101, #102).

Thank you to everyone who reported issues and tested the changes.

Install: pip install --upgrade content-core==2.2.0

Run without installation: uvx --from content-core==2.2.0 content-core --help

With Docling: pip install --upgrade "content-core[docling]==2.2.0"

Full changelog.