Skip to content
James Conway edited this page Nov 19, 2026 · 4 revisions

Parsec's Python package is parsec-html, and the module you import is parsec_html. Its API is selectolax's, so moving over is mostly a change of import, and it adds what selectolax can't do: one parsed page shared by many threads, free-threaded Python with the GIL off, selectors selectolax can't run, such as :lang(), :dir() and :valid, XPath 1.0 that answers the way lxml's does, where every node was written in your input, and the parse errors.

On this page: Install · A first page · Threads · Moving from selectolax · HTMLParser · Node · Attributes · XPath · Positions · Parse errors · Errors · Speed · Wheels · Working on it

Install

Note

Parsec isn't released. Wheels go to PyPI with 0.1.0; until then you build from git, which needs Rust 1.88 or newer.

pip install "git+https://github.com/parsec-html/parsec#subdirectory=bindings/python"

It runs on CPython 3.9 and later, free-threaded 3.14t and later included (PyO3, which the package is built with, doesn't support free-threaded 3.13). The package ships type stubs, so your editor and mypy know every signature on this page.

Why parsec_html and not parsec? There's already a parsec on PyPI (a parser-combinator library), and two packages that install the same module name overwrite each other. parsec_html matches the name you pip install, so nothing clashes.

A first page

from parsec_html import HTMLParser

page = HTMLParser("""
<title>Shop</title>
<ul>
  <li class=item data-sku=A1><a href=/a>Kettle</a> <span class=price>£24</span>
  <li class=item data-sku=B2><a href=/b>Toaster</a> <span class=price>£31</span>
</ul>
""")

for item in page.css("li.item"):
    link = item.css_first("a")
    print(item.attrs["data-sku"], link.text(), link.attributes["href"],
          item.css_first(".price").text())

page.css_first("li[data-sku=B2]").decompose()
print(page.body.html)

Threads

A parsed page belongs to no thread. Read it from as many as you like, and edit it from any of them: readers share it, and an edit waits for them, then has it to itself. Parsing and css() let go of the GIL while they work, so on ordinary CPython two threads parsing or querying really do run at once.

On a free-threaded build (3.14t, 3.15t) the module declares that it doesn't need the GIL, so importing it leaves the GIL off:

import sys, threading
from parsec_html import HTMLParser

page = HTMLParser(open("big.html").read())
counts = []
threads = [threading.Thread(target=lambda: counts.append(len(page.css("a")))) for _ in range(8)]
for t in threads: t.start()
for t in threads: t.join()
assert not sys._is_gil_enabled()  # on 3.14t

selectolax documents its Lexbor parsers as not thread-safe: one parser per thread. Underneath, Parsec's Document is Send and Sync, checked when it compiles; see Thread safety and memory safety.

Moving from selectolax

# from selectolax.lexbor import LexborHTMLParser as HTMLParser
from parsec_html import HTMLParser

These were checked against selectolax 1.0.2 by running the same calls through both. Everything below this table works the way selectolax's does, except where it says so.

selectolax 1.0.2 parsec_html
<a href>'s attributes {'href': None} {'href': ''}. Parsec's tree doesn't yet record whether a value was written.
A selector that doesn't parse SelectolaxError SelectorError, a ValueError
A node after decompose() freed; reading it gives an empty element still there, detached: readable, and can be inserted again
attrs.items(), keys(), values() iterators lists
iter(), traverse() generators iterators over a list taken when you call them
bytes in UTF-8, or sniffed with encoding=True UTF-8, bad bytes replaced with U+FFFD; sniffing comes with #5
Class names LexborHTMLParser, LexborNode, LexborAttributes HTMLParser, Node, Attributes
:lang(), :dir(), :valid, :in-range, :scope SelectolaxError: "Can't parse CSS selector" match

Not there yet: clone(), raw_html, raw_value, html_pretty and inner_html_pretty, select() and its filters, scripts_contain, script_srcs_contain, template_fragments, text_lexbor, mem_id, tag_id, parser.inner_html, the :lexbor-contains() pseudo-class, and create_tag() and extract_html_comment() at module level.

HTMLParser

HTMLParser(html, is_fragment=False, fragment_tag="div", fragment_namespace="html",
           scripting=False, budget=None, max_depth=None, url=None,
           positions=False, errors=False)
Argument
html str, or bytes read as UTF-8
is_fragment parse as the contents of a fragment_tag element, as innerHTML does: HTMLParser("<td>1", is_fragment=True, fragment_tag="tr")
fragment_namespace "html", "svg" or "math" (or their namespace URLs), for the fragment's context element
scripting parse <noscript> as a browser with JavaScript on does
budget give up after this many seconds and raise TimeoutError; see Limits and hostile input
max_depth keep the tree at most this deep, as Chromium does at 512
url the page's address, so :target and :local-link match
positions record where each node and attribute was written; see Positions
errors keep the parse errors, for errors; see Parse errors
Member Gives
root the <html> element; a fragment's first node; None for an empty fragment
head, body those elements, or None (always None for a fragment)
html the whole document as markup
text(deep=True, separator="", strip=False, skip_empty=False) the text, as Node.text
css(query) every matching element, in document order
css_first(query, default=None, strict=False) the first match, or default; with strict, a ValueError if there's more than one
css_matches(selector) whether anything matches
any_css_matches(selectors) whether any of them matches anything
tags(name) every element of that name
xpath(query, **variables) an XPath 1.0 query from the document node; see XPath
errors the parse errors, a list of ParseError
line_col(offset) the (line, column) of an offset into the input, counted as source_range counts
create_node(tag) a new element, not yet in the tree
merge_text_nodes() joins each run of adjacent text nodes into one
strip_tags(tags, recursive=False) removes every element matching each selector in tags, contents and all
unwrap_tags(tags, delete_empty=False) replaces each match with its contents

Node

A node is its parser plus a position in the tree, so it stays valid after any edit, and two Node objects for the same node are equal and hash the same.

Member Gives
tag the element's name (div, foreignObject); -text, -comment, -doctype, -document for those
id the id attribute, or None
attributes a new dict of the attributes, names as written (xlink:href, viewBox)
attrs the live Attributes mapping
html the node and its contents as markup (outerHTML)
inner_html the contents as markup; set it to replace them, parsed in this element's context
text(deep=True, separator="", strip=False, skip_empty=False) the text nodes' text, joined by separator; deep=False takes only the node's own text children; strip strips each piece; skip_empty drops whitespace-only ones
text_content a text node's text, else None
comment_content a comment's text, stripped, else None
parent, next, prev, first_child, last_child, child the neighbours, text and comments included; None at the edges (a fragment's top-level nodes have no parent)
iter(include_text=False, skip_empty=False) the children: elements only, unless include_text, which adds text and comments
traverse(include_text=False, skip_empty=False) the node and everything under it, in tree order, filtered the same way
css(query), css_first(...), css_matches(...), any_css_matches(...) as on the parser, but under this node, and counting the node itself, as selectolax does
xpath(query, **variables) an XPath query with this node as the context, as lxml's element.xpath()
source_range, end_tag_range, position, attribute_range(name) where the node was written; see Positions
decompose(), remove() take the node out of the tree; it stays readable
unwrap(delete_empty=False) replaces the element with its contents; an empty one stays unless delete_empty
unwrap_tags(...), strip_tags(...) as on the parser, under this node
insert_before(value, move=False), insert_after(...), insert_child(...), replace_with(...) put value beside, inside (last) or in place of the node: a str or bytes goes in as text, never parsed; a Node goes in as a copy, from this page or another, or itself with move=True
merge_text_nodes() as on the parser, under this node
is_element_node, is_text_node, is_comment_node, is_document_node, is_empty_text_node what kind of node it is
parser the HTMLParser it belongs to

To insert markup rather than text, parse it and pass the node: div.insert_child(HTMLParser("<b>new</b>", is_fragment=True).root).

Attributes

node.attrs reads and writes the element's attributes: attrs["class"], attrs["data-x"] = "1" (None sets it empty), del attrs["hidden"], "id" in attrs, len(attrs), iteration over the names, keys(), values(), items(), get(key, default=None), and sget(key, default=""). On HTML elements names are looked up in lowercase, as getAttribute does. A missing key raises KeyError.

XPath

xpath() runs XPath 1.0, on the parser from the document node and on a node from that node, and gives back what lxml gives back, so code written for lxml or parsel ports as it is:

page.xpath("//li[@class='item']/a/text()")    # ['Kettle', 'Toaster']
page.xpath("//li/@data-sku")                   # ['A1', 'B2']
page.xpath("count(//li)")                      # 2.0
page.xpath("boolean(//table)")                 # False
item.xpath("string(span[@class='price'])")     # '£24'
page.xpath("//li[@data-sku=$sku]", sku="B2")   # [<Node li>]
The query gives You get
a node-set a list in document order: elements, comments and the document as Nodes, text nodes and attributes as str
a string str
a number float
a boolean bool

Keyword arguments bind $variables: a str, bytes, int, float or bool. A query that doesn't parse, or uses a $variable you didn't pass, raises XPathError, a ValueError. Names match as lxml matches them in HTML, lowercase and in no namespace: //div finds every div, //DIV none. A <template>'s contents aren't in the tree a query walks. The XPath page has the engine's limits; a query that would run for hours stops with an XPathError.

The first query after the page changes indexes it, once, and later queries reuse the index until the next edit. The index lives on the parser, so threads share it, and a query lets go of the GIL while it runs, as css() does. A node you've removed can still be queried: its subtree is a tree of its own, so //li from it finds the lis under it.

What's different from lxml: no namespaces= (HTML names have no prefixes), no smart_strings (a string is a plain str, without .getparent()), and no compiled etree.XPath object; each call compiles its query, which takes microseconds.

Positions

Parse with positions=True and every node the parser read knows where it was in your input:

src = "<p class=note>Hi</p>\n<div>x</div>"
page = HTMLParser(src, positions=True)
p = page.css_first("p")
p.source_range                 # (0, 14): the start tag, <p class=note>
p.end_tag_range                # (16, 20): </p>
p.attribute_range("class")     # (3, 13)
page.css_first("div").position # (2, 1)
src[slice(*p.source_range)]    # '<p class=note>'
Member Gives
source_range (start, end) of an element's start tag, from < to >, or of a whole text node, comment or doctype
end_tag_range (start, end) of the end tag that closed the element, or None if it had none
attribute_range(name) (start, end) of one attribute, from its name to the end of its value; the name is looked up as attrs looks it up
position the (line, column) of source_range's start
HTMLParser.line_col(offset) the (line, column) of any offset, in the same units

Ranges index the input as you passed it, as a (start, end) pair like re.Match.span() gives, so slicing it gives the source: code point indexes into a str (src[start:end], café and emoji included), byte offsets into bytes (data[start:end], even when some of those bytes weren't UTF-8 and were read as U+FFFD). Lines and columns count from 1, as editors and lxml's sourceline do, and a column counts characters. Each (line, column) is worked out from the input when you ask, so it costs a pass over the text before it.

The parser itself counts UTF-8 bytes. For a str with anything beyond ASCII, the package keeps a copy of the text and, on the first range you read, a table of code points per 64 bytes, so each conversion after that looks at 64 bytes at most. ASCII text and bytes need neither, and nothing is kept without positions=True or errors=True.

Everything is None without positions=True, and for nodes nobody wrote: the <html>, <head>, <body> and <tbody> the parser implied, and anything an edit made. Without it the parse is as fast as ever; with it, each node keeps a few bytes more.

Parse errors

Parse with errors=True and parser.errors lists the parse errors the parser recovered from, in source order. Each is a ParseError (a record, not an exception: the page still parsed):

page = HTMLParser("<p>a\n</b>", errors=True)
for e in page.errors:
    print(e)          # 1:1: expected-doctype-but-got-start-tag
                      # 2:1: unexpected-end-tag
page.errors[1].code, page.errors[1].offset   # ('unexpected-end-tag', 5)

code is the HTML Standard's name for the error (eof-in-tag, duplicate-attribute), or, for one tree construction finds, which the standard leaves unnamed, the name html5lib uses. offset indexes the input as source_range does, so src[e.offset:] starts where the error was found; line and column count from 1. Without errors=True the list is empty. Parsing has the detail.

Errors

Raised When
parsec_html.SelectorError (a ValueError) a selector doesn't parse; the message says where
parsec_html.XPathError (a ValueError) an XPath query doesn't parse, uses a $variable you didn't pass, or runs past the work limit
TimeoutError a parse ran past its budget
ValueError input over 512 MiB; an edit that would put a node inside itself; css_first(strict=True) with more than one match
TypeError html that isn't str or bytes; inserting something that isn't text or a Node; an XPath variable that isn't a string, number or bool

Speed

bindings/python/bench.py times parsec_html against selectolax's Lexbor backend in one process on the 13 real pages bench/ uses, 24 MB of Wikipedia, MDN, GitHub, the BBC and the rest: parsing, the same 32 selectors the Rust benchmark runs, and a scraper's loop that reads every link's href and text.

Two runs on an Apple M2 Max with CPython 3.14 and selectolax 1.0.2, medians of 20 runs each, at a load of 2.7 to 3.1:

parsec_html selectolax
Parse all 13 pages 66.8, 65.6 ms 105.6, 105.9 ms 1.6×
The 32 selectors over all 13 pages 283.1, 283.5 ms 781.4, 793.8 ms 2.8×
Every link's href and text 42.6, 45.0 ms 50.0, 49.8 ms 1.1 to 1.2×

Page by page, parsing is 1.5 to 2.8 times faster (MDN and the one-page HTML spec are the narrowest, near 1.5×, and the New York Times the widest). Selector by selector it's ahead on all 32, from 1.04× on [class] and 1.07× on * to 29× on p:nth-child(2n+1), which costs selectolax 350 ms. The close ones are the selectors that match most of a page: there the time goes on making a Python object for each of 400,000 matches, which both libraries have to do. The match counts agree except on [href], where Lexbor also counts SVG's xlink:href, as the Rust Benchmarks explain.

To run it yourself:

bench/fetch-pages.sh
cd bindings/python && maturin develop --release && pip install selectolax
python bench.py

Wheels

The release builds a wheel for each CPython from 3.9 to 3.15 and for free-threaded 3.14t and 3.15t, on Linux (glibc and musl) and macOS, x86-64 and ARM, and Windows x86-64 and ARM. One more wheel uses the stable ABI that 3.15 brought for both builds of CPython (abi3t), so a newer Python than any of those still installs. That one is slower at making Python objects (31 ms against 22 ms for 400,000 nodes), which is why the per-version wheels come first.

Working on it

cd bindings/python
pip install maturin pytest
maturin develop --release
pytest tests
bench/fetch-pages.sh && pip install selectolax && python bench.py

CI builds and tests the wheel on Linux, macOS and Windows with 3.9, 3.14 and 3.14t, and on Linux with 3.15 and the abi3t wheel on 3.15 and 3.15t.

Clone this wiki locally