Repository navigation
Content Core 2.2.0 adds OpenDocument support, YouTube cookies and proxies, and configurable audio segments. Extraction failures now raise typed errors across the library, CLI and MCP instead of returning empty content.
New capabilities
- Extract OpenDocument files:
.odt,.odsand.odp(#43, #96). Text documents become Markdown with paragraphs, headings, lists and tables. Spreadsheets become one table per sheet (# Sheet: <name>), and presentations become one block per slide (# Slide N), the same structure XLSX and PPTX already use. Detection reads the archive'smimetype, so a wrong or missing extension still works.odfpyis a new core dependency. - YouTube cookies and proxies for transcripts (#46, #95).
youtube_cookies_file(CCORE_YOUTUBE_COOKIES_FILE) loads a Netscapecookies.txt, which unblocks networks YouTube has flagged.youtube_proxy(CCORE_YOUTUBE_PROXY) routes the transcript paths and the title lookup through a proxy; use a residential proxy, because datacenter proxies are blocked. With neither set, behavior is unchanged. Cookie values are never logged. audio_segment_minutes(CCORE_AUDIO_SEGMENT_MINUTES) sets the length of the segments long audio is split into before transcription. The default stays 10 minutes.0sends the file whole, for self-hosted STT endpoints without an upload limit (#94).
⚠️ Behavior changes
Hiding an error is treated as a bug, so these changes ship in 2.x (#60, #97). If your code checked for empty content to detect a failure, catch ContentCoreError (or one of its subclasses) instead. content == "" now only means the source was empty.
-
URL extraction raises. A named
url_enginethat fails raises one of these:NetworkErrorfor connection, timeout or DNS failuresNotFoundErrorfor HTTP 404/410 with thesimpleengineExternalServiceErrorfor engine API and HTTP errorsConfigurationErrorwhencrawl4aiis selected but not installed
autostill falls back through firecrawl → jina → crawl4ai → simple and raises only when every engine fails. A malformed URL raisesInvalidInputError. Thesimpleengine no longer returns error text as page content. -
File and media extraction raise. Corrupted PDF and EPUB files, and text files that can't be read, raise
FileOperationError. ffmpeg/ffprobe failures raiseFileOperationError; video extraction no longer returns empty content. STT provider failures raiseExternalServiceErrorafter retries. -
YouTube raises. When no transcript can be fetched, YouTube extraction raises
NoTranscriptFound. When YouTube blocks or fails the request on both paths, it raisesExternalServiceError, and a URL without a video ID raisesInvalidInputError. -
An explicit STT model is honored or raises. If
audio_provider/audio_modelare set and the model can't be created, Content Core raisesConfigurationError. It no longer silently transcribes with the default model (#99). -
Retries are classified by error type. Configuration, input and file errors are permanent, and network errors are transient. Blocked YouTube requests are retried, so a rotating proxy can get a new IP. Other transcript errors that name a state of the video (unavailable, disabled, age-restricted) are not retried (#99).
Fixes
youtube_proxynow also covers the video title lookup (#99).requestsis declared as a direct dependency instead of arriving transitively (#99).- A failed Crawl4AI crawl (navigation timeout, blocked page) raises
ExternalServiceErrorinstead of crashing with a pydanticValidationError, andautofalls through to the next engine (#101).
Security
- Minimum versions raised to releases without known vulnerabilities:
aiohttp>=3.14.3,pillow>=12.3.0,click>=8.3.3,pydantic-settings>=2.14.2; in the extras,docling>=2.94.0,crawl4ai>=0.9.0andlangchain-core>=1.3.3. Upgrading content-core upgrades these in an existing environment (#102). - Two advisories remain in the locked set:
nltk(no fixed release yet) andtransformers, which thedoclingextra keeps below 5 until docling compatibility is validated (#103).
Validation
- Tests: 526 unit and integration tests passed on the release commit. CI passed on Python 3.10, 3.11 and 3.12.
- Built package: the built wheel passed isolated library and CLI smokes (TXT and PDF). Over MCP stdio, a real
extract_contenttool call returned the fixture text, with fastmcp 4 and mcp 2. Versions and runtime assets were checked. The docling, crawl4ai and langchain extras installed; the LangChainextract_content_toolran with langchain-core 1.6.6. - Live checks, all passed:
- YouTube transcript extraction
- remote PDF download
- URL extraction with Firecrawl, Jina, Crawl4AI 0.9.4 and
simple - OpenAI Whisper transcription of real MP3 and MP4 files
- Docling PDF extraction, including the formula and picture enrichments
- Error paths: a site unreachable from the test host raised the new typed errors, and
autotried every engine before raising. - Not verified live: YouTube with a real residential proxy or cookies file, and STT providers other than OpenAI.
- Security alerts: Dependabot alerts are disabled for this repository, so they weren't checked.
pip-auditwas run over the locked dependency set instead; see Security above.
The release workflow builds its own artifacts. The published files will be checked from PyPI after publication.
Thanks
- @kyxap1: configurable audio segment length (#94).
- @lfnovo: OpenDocument support, YouTube cookies and proxies, typed extraction errors, review follow-ups and release preparation (#95, #96, #97, #99, #100, #101, #102).
Thank you to everyone who reported issues and tested the changes.
Install: pip install --upgrade content-core==2.2.0
Run without installation: uvx --from content-core==2.2.0 content-core --help
With Docling: pip install --upgrade "content-core[docling]==2.2.0"