Repository navigation
Python
Parsec's Python package is parsec-html, and the module you import is parsec_html. Its API is selectolax's, so moving over is mostly a change of import, and it adds what selectolax can't do: one parsed page shared by many threads, free-threaded Python with the GIL off, and selectors selectolax can't run, such as :lang(), :dir() and :valid.
On this page: Install · A first page · Threads · Moving from selectolax · HTMLParser · Node · Attributes · Errors · Speed · Wheels · Working on it
Note
Parsec isn't released. Wheels go to PyPI with 0.1.0; until then you build from git, which needs Rust 1.88 or newer.
pip install "git+https://github.com/parsec-html/parsec#subdirectory=bindings/python"It runs on CPython 3.9 and later, free-threaded 3.14t and later included (PyO3, which the package is built with, doesn't support free-threaded 3.13). The package ships type stubs, so your editor and mypy know every signature on this page.
Why parsec_html and not parsec? There's already a parsec on PyPI (a parser-combinator library), and two packages that install the same module name overwrite each other. parsec_html matches the name you pip install, so nothing clashes.
from parsec_html import HTMLParser
page = HTMLParser("""
<title>Shop</title>
<ul>
<li class=item data-sku=A1><a href=/a>Kettle</a> <span class=price>£24</span>
<li class=item data-sku=B2><a href=/b>Toaster</a> <span class=price>£31</span>
</ul>
""")
for item in page.css("li.item"):
link = item.css_first("a")
print(item.attrs["data-sku"], link.text(), link.attributes["href"],
item.css_first(".price").text())
page.css_first("li[data-sku=B2]").decompose()
print(page.body.html)A parsed page belongs to no thread. Read it from as many as you like, and edit it from any of them: readers share it, and an edit waits for them, then has it to itself. Parsing and css() let go of the GIL while they work, so on ordinary CPython two threads parsing or querying really do run at once.
On a free-threaded build (3.14t, 3.15t) the module declares that it doesn't need the GIL, so importing it leaves the GIL off:
import sys, threading
from parsec_html import HTMLParser
page = HTMLParser(open("big.html").read())
counts = []
threads = [threading.Thread(target=lambda: counts.append(len(page.css("a")))) for _ in range(8)]
for t in threads: t.start()
for t in threads: t.join()
assert not sys._is_gil_enabled() # on 3.14tselectolax documents its Lexbor parsers as not thread-safe: one parser per thread. Underneath, Parsec's Document is Send and Sync, checked when it compiles; see Thread safety and memory safety.
# from selectolax.lexbor import LexborHTMLParser as HTMLParser
from parsec_html import HTMLParserThese were checked against selectolax 1.0.2 by running the same calls through both. Everything below this table works the way selectolax's does, except where it says so.
| selectolax 1.0.2 | parsec_html | |
|---|---|---|
<a href>'s attributes
|
{'href': None} |
{'href': ''}. Parsec's tree doesn't yet record whether a value was written. |
| A selector that doesn't parse | SelectolaxError |
SelectorError, a ValueError
|
A node after decompose()
|
freed; reading it gives an empty element | still there, detached: readable, and can be inserted again |
attrs.items(), keys(), values()
|
iterators | lists |
iter(), traverse()
|
generators | iterators over a list taken when you call them |
bytes in |
UTF-8, or sniffed with encoding=True
|
UTF-8, bad bytes replaced with U+FFFD; sniffing comes with #5 |
| Class names |
LexborHTMLParser, LexborNode, LexborAttributes
|
HTMLParser, Node, Attributes
|
:lang(), :dir(), :valid, :in-range, :scope
|
SelectolaxError: "Can't parse CSS selector" |
match |
Not there yet: clone(), raw_html, raw_value, html_pretty and inner_html_pretty, select() and its filters, scripts_contain, script_srcs_contain, template_fragments, text_lexbor, mem_id, tag_id, parser.inner_html, the :lexbor-contains() pseudo-class, and create_tag() and extract_html_comment() at module level. Source positions come with #8, and XPath with #31.
HTMLParser(html, is_fragment=False, fragment_tag="div", fragment_namespace="html",
scripting=False, budget=None, max_depth=None, url=None)| Argument | |
|---|---|
html |
str, or bytes read as UTF-8 |
is_fragment |
parse as the contents of a fragment_tag element, as innerHTML does: HTMLParser("<td>1", is_fragment=True, fragment_tag="tr")
|
fragment_namespace |
"html", "svg" or "math" (or their namespace URLs), for the fragment's context element |
scripting |
parse <noscript> as a browser with JavaScript on does |
budget |
give up after this many seconds and raise TimeoutError; see Limits and hostile input
|
max_depth |
keep the tree at most this deep, as Chromium does at 512 |
url |
the page's address, so :target and :local-link match |
| Member | Gives |
|---|---|
root |
the <html> element; a fragment's first node; None for an empty fragment |
head, body
|
those elements, or None (always None for a fragment) |
html |
the whole document as markup |
text(deep=True, separator="", strip=False, skip_empty=False) |
the text, as Node.text
|
css(query) |
every matching element, in document order |
css_first(query, default=None, strict=False) |
the first match, or default; with strict, a ValueError if there's more than one |
css_matches(selector) |
whether anything matches |
any_css_matches(selectors) |
whether any of them matches anything |
tags(name) |
every element of that name |
create_node(tag) |
a new element, not yet in the tree |
merge_text_nodes() |
joins each run of adjacent text nodes into one |
strip_tags(tags, recursive=False) |
removes every element matching each selector in tags, contents and all |
unwrap_tags(tags, delete_empty=False) |
replaces each match with its contents |
A node is its parser plus a position in the tree, so it stays valid after any edit, and two Node objects for the same node are equal and hash the same.
| Member | Gives |
|---|---|
tag |
the element's name (div, foreignObject); -text, -comment, -doctype, -document for those |
id |
the id attribute, or None
|
attributes |
a new dict of the attributes, names as written (xlink:href, viewBox) |
attrs |
the live Attributes mapping |
html |
the node and its contents as markup (outerHTML) |
inner_html |
the contents as markup; set it to replace them, parsed in this element's context |
text(deep=True, separator="", strip=False, skip_empty=False) |
the text nodes' text, joined by separator; deep=False takes only the node's own text children; strip strips each piece; skip_empty drops whitespace-only ones |
text_content |
a text node's text, else None
|
comment_content |
a comment's text, stripped, else None
|
parent, next, prev, first_child, last_child, child
|
the neighbours, text and comments included; None at the edges (a fragment's top-level nodes have no parent) |
iter(include_text=False, skip_empty=False) |
the children: elements only, unless include_text, which adds text and comments |
traverse(include_text=False, skip_empty=False) |
the node and everything under it, in tree order, filtered the same way |
css(query), css_first(...), css_matches(...), any_css_matches(...)
|
as on the parser, but under this node, and counting the node itself, as selectolax does |
decompose(), remove()
|
take the node out of the tree; it stays readable |
unwrap(delete_empty=False) |
replaces the element with its contents; an empty one stays unless delete_empty
|
unwrap_tags(...), strip_tags(...)
|
as on the parser, under this node |
insert_before(value, move=False), insert_after(...), insert_child(...), replace_with(...)
|
put value beside, inside (last) or in place of the node: a str or bytes goes in as text, never parsed; a Node goes in as a copy, from this page or another, or itself with move=True
|
merge_text_nodes() |
as on the parser, under this node |
is_element_node, is_text_node, is_comment_node, is_document_node, is_empty_text_node
|
what kind of node it is |
parser |
the HTMLParser it belongs to |
To insert markup rather than text, parse it and pass the node: div.insert_child(HTMLParser("<b>new</b>", is_fragment=True).root).
node.attrs reads and writes the element's attributes: attrs["class"], attrs["data-x"] = "1" (None sets it empty), del attrs["hidden"], "id" in attrs, len(attrs), iteration over the names, keys(), values(), items(), get(key, default=None), and sget(key, default=""). On HTML elements names are looked up in lowercase, as getAttribute does. A missing key raises KeyError.
| Raised | When |
|---|---|
parsec_html.SelectorError (a ValueError) |
a selector doesn't parse; the message says where |
TimeoutError |
a parse ran past its budget
|
ValueError |
input over 512 MiB; an edit that would put a node inside itself; css_first(strict=True) with more than one match |
TypeError |
html that isn't str or bytes; inserting something that isn't text or a Node
|
bindings/python/bench.py times parsec_html against selectolax's Lexbor backend in one process on the 13 real pages bench/ uses, 24 MB of Wikipedia, MDN, GitHub, the BBC and the rest: parsing, the same 32 selectors the Rust benchmark runs, and a scraper's loop that reads every link's href and text.
Two runs on an Apple M2 Max with CPython 3.14 and selectolax 1.0.2, medians of 20 runs each, at a load of 2.7 to 3.1:
| parsec_html | selectolax | ||
|---|---|---|---|
| Parse all 13 pages | 66.8, 65.6 ms | 105.6, 105.9 ms | 1.6× |
| The 32 selectors over all 13 pages | 283.1, 283.5 ms | 781.4, 793.8 ms | 2.8× |
Every link's href and text |
42.6, 45.0 ms | 50.0, 49.8 ms | 1.1 to 1.2× |
Page by page, parsing is 1.5 to 2.8 times faster (MDN and the one-page HTML spec are the narrowest, near 1.5×, and the New York Times the widest). Selector by selector it's ahead on all 32, from 1.04× on [class] and 1.07× on * to 29× on p:nth-child(2n+1), which costs selectolax 350 ms. The close ones are the selectors that match most of a page: there the time goes on making a Python object for each of 400,000 matches, which both libraries have to do. The match counts agree except on [href], where Lexbor also counts SVG's xlink:href, as the Rust Benchmarks explain.
To run it yourself:
bench/fetch-pages.sh
cd bindings/python && maturin develop --release && pip install selectolax
python bench.pyThe release builds a wheel for each CPython from 3.9 to 3.15 and for free-threaded 3.14t and 3.15t, on Linux (glibc and musl) and macOS, x86-64 and ARM, and Windows x86-64 and ARM. One more wheel uses the stable ABI that 3.15 brought for both builds of CPython (abi3t), so a newer Python than any of those still installs. That one is slower at making Python objects (31 ms against 22 ms for 400,000 nodes), which is why the per-version wheels come first.
cd bindings/python
pip install maturin pytest
maturin develop --release
pytest tests
bench/fetch-pages.sh && pip install selectolax && python bench.pyCI builds and tests the wheel on Linux, macOS and Windows with 3.9, 3.14 and 3.14t, and on Linux with 3.15 and the abi3t wheel on 3.15 and 3.15t.
Parsec is Apache-2.0 and not yet released; these pages describe main. Found something wrong here? Open an issue or ask in Discussions.
Repository · Issues · Discussions · Roadmap · Security policy · Licence