v0.2.2

What's Changed

Lexidown 0.2.2 introduces an opt-in llm preset for converting scraped HTML into cleaner Markdown for LLM context.

• Add LLM content extraction plugin with preprocessing hook by @EvickaStudio in #7.
• Reduce common page noise using HTML structure and accessibility attributes, without website-specific rules.
• Preserve headings, lists, fenced code blocks, and declared TeX math, with readable fallbacks for complex tables.
• Omit embedded SVG and image data while retaining useful text descriptions.
• Add includeLinkUrls and includeImageUrls options to keep labels while omitting destination URLs.
• Support custom DOM cleanup through an optional preprocess(root) callback.

Usage

The preset is included in the standard Lexidown installation. This example uses the optional requests package to fetch a page:

import requests

from lexidown import TurndownService
from lexidown.plugins.llm import llm

response = requests.get("https://example.com/", timeout=30)
response.raise_for_status()

service = TurndownService(
    {
        "baseUrl": response.url,
        "includeLinkUrls": False,
        "includeImageUrls": False,
    }
).use(llm)

print(service.turndown(response.text))

Both URL options default to True. Setting them to False keeps link text and image descriptions, reducing the amount of text sent to an LLM. Literal URLs inside page text or code remain intact.

Full Changelog: v0.2.1...v0.2.2

v0.2.1

What's Changed

• Enhance performance and update benchmarks by @EvickaStudio in #5
• Enhance performance and update benchmarks for conversion by @EvickaStudio in #6

Full Changelog: v0.2.0...v0.2.1

v0.2.0

What's Changed

• Add fast checks for development branches by @EvickaStudio in #3
• Add bundled GFM and Firecrawl presets by @EvickaStudio in #2
• Add bundled presets and fast checks for development branches by @EvickaStudio in #4

Full Changelog: v0.1.1...v0.2.0

v0.1.1

What's Changed

• Add project badges and bump version by @EvickaStudio in #1

New Contributors

• @EvickaStudio made their first contribution in #1

Full Changelog: v0.1.0...v0.1.1

v0.1.0

First public release of Lexidown: fast native HTML-to-Markdown for Python,
compatible with Turndown 7.2.4.

• Native Cython conversion engine with bundled Lexbor.
• Turndown-compatible options, custom rules, plugins, and keep/remove methods.
• Precompiled wheels for CPython 3.10–3.14 on Linux, macOS, and Windows.
• No runtime Python dependencies.

Install: python -m pip install lexidown==0.1.0

PyPI
Quickstart