Getting started

This page takes you from an empty project to a parsed, queried, edited and re-serialized page. Every example on it compiles and runs as written.

On this page: Add Parsec · Parse a page · Find elements with selectors · Read text and attributes · Walk the tree · Change it · Write it back out · All of it together

Add Parsec

Parsec isn't on crates.io yet, so you take it from git. In your Cargo.toml:

[dependencies]
parsec-html = { git = "https://github.com/parsec-html/parsec" }

Tip

Want builds that don't move under you? Pin a commit.

Put a real hash from the commit list where COMMIT is:

[dependencies]
parsec-html = { git = "https://github.com/parsec-html/parsec", rev = "COMMIT" }

Cargo also records the exact commit in Cargo.lock, so a project with a lock file stays put until you run cargo update.

The package is called parsec-html, but the library inside it is parsec, so your code says use parsec::.... You need Rust 1.88 or newer. Parsec's only dependency is memchr.

Parse a page

parsec::parse takes the page as a &str and a flag for scripting, and gives you a Document:

let doc = parsec::parse("<title>Shop</title><p>Hello", false);
assert_eq!(
    doc.inner_html(doc.root()),
    "<html><head><title>Shop</title></head><body><p>Hello</p></body></html>"
);

The parser fixed up the missing <html>, <head> and <body> and closed the <p>, exactly as a browser would. It never fails on bad markup; the HTML Standard says how to recover from every mistake, and Parsec follows it.

Pass false for scripting unless you want <noscript> handled as a browser with JavaScript on would. Parsing explains the difference.

The input has to be a &str, so if you have bytes you decode them first. Parsec doesn't sniff encodings yet (#5); for pages you know are UTF-8, String::from_utf8_lossy does the job.

Find elements with selectors

Parse a selector once with SelectorList::parse, then run it with select, which returns the matching elements under a node, in document order:

use parsec::css::SelectorList;

let html = r#"
  <ul>
    <li class="book" data-price="12"><a href="/p/1">Dune</a></li>
    <li class="book sold" data-price="9"><a href="/p/2">Emma</a></li>
    <li class="book" data-price="15"><a href="/p/3">Ulysses</a></li>
  </ul>"#;
let doc = parsec::parse(html, false);

let in_stock = SelectorList::parse("li.book:not(.sold) a").unwrap();
let links = in_stock.select(&doc, doc.root());
assert_eq!(links.len(), 2);

select hands back NodeIds, which are plain u32 indexes into the document. You read everything else through the document.

Read text and attributes

use parsec::css::SelectorList;
use parsec::Kind;

let doc = parsec::parse(
    r#"<a href="/p/1" title="Dune, 1965">Dune <b>(paperback)</b></a>"#,
    false,
);
let a = SelectorList::parse("a").unwrap().select(&doc, doc.root())[0];

// Every text node under the element, joined, as textContent gives it.
assert_eq!(doc.text(a), "Dune (paperback)");

// The element's name and namespace.
match doc.kind(a) {
    Kind::Element { name, .. } => assert_eq!(name, "a"),
    _ => unreachable!(),
}

// Attributes come back as (namespace, name, value), in source order.
let href = doc
    .attrs(a)
    .find(|&(_, name, _)| name == "href")
    .map(|(_, _, value)| value);
assert_eq!(href, Some("/p/1"));

There's no get_attribute shortcut today; the find above is the idiom. Character references are already decoded: & in the source reads as &.

Walk the tree

Every node has links to its parent, first and last child, and siblings, and children iterates a node's children in order:

use parsec::Kind;

let doc = parsec::parse("<ul><li>one<li>two<li>three</ul>", false);

// Find the <ul> by walking down from the root.
let mut stack = vec![doc.root()];
let mut items = Vec::new();
while let Some(node) = stack.pop() {
    if let Kind::Element { name: "li", .. } = doc.kind(node) {
        items.push(doc.text(node));
    }
    let mut kids: Vec<_> = doc.children(node).collect();
    kids.reverse();
    stack.extend(kids);
}
assert_eq!(items, ["one", "two", "three"]);

Change it

Edits are methods on Document and take &mut self. New nodes start detached; you put them in place with append or insert_before:

use parsec::css::SelectorList;
use parsec::Ns;

let mut doc = parsec::parse(r#"<ul><li class="old">a</li></ul>"#, false);
let ul = SelectorList::parse("ul").unwrap().select(&doc, doc.root())[0];
let old = SelectorList::parse(".old").unwrap().select(&doc, doc.root())[0];

// A new <li> with text in it, added at the end of the list.
let li = doc.create_element(Ns::Html, "li");
let text = doc.create_text("b & c");
doc.append(li, text).unwrap();
doc.append(ul, li).unwrap();

// Attributes: set replaces one of the same name, remove_attr deletes.
doc.set_attr(li, "class", "new");
assert!(doc.remove_attr(old, "class"));

// Take the old item out of the tree.
doc.remove(old);

assert_eq!(doc.outer_html(ul), r#"<ul><li class="new">b &amp; c</li></ul>"#);

append and insert_before return a Result: they refuse to put a node inside itself, into a node that can't have children, or before a node that isn't a child of the parent. Editing has every function and its rules.

Write it back out

use parsec::css::SelectorList;

let doc = parsec::parse(r#"<!DOCTYPE html><p class=x>1 &lt; 2</p>"#, false);
let p = SelectorList::parse("p").unwrap().select(&doc, doc.root())[0];

// The whole document: serialize the root's children.
assert_eq!(
    doc.inner_html(doc.root()),
    r#"<!DOCTYPE html><html><head></head><body><p class="x">1 &lt; 2</p></body></html>"#
);
// An element with its tags, as outerHTML gives it.
assert_eq!(doc.outer_html(p), r#"<p class="x">1 &lt; 2</p>"#);
// Only what's inside, as innerHTML gives it.
assert_eq!(doc.inner_html(p), "1 &lt; 2");
// The start tag alone.
assert_eq!(doc.html_alone(p), r#"<p class="x">"#);
// The text, with references decoded.
assert_eq!(doc.text(p), "1 < 2");

All of it together

A small program that reads a product list, marks cheap items and prints the result.

The whole program, about 30 lines

use parsec::css::SelectorList;

fn main() -> Result<(), Box<dyn std::error::Error>> {
    let html = r#"<ul id=shop>
      <li><span class=name>Dune</span> <span class=price>12</span></li>
      <li><span class=name>Emma</span> <span class=price>4</span></li>
    </ul>"#;
    let mut doc = parsec::parse(html, false);

    let items = SelectorList::parse("#shop > li")?;
    let price = SelectorList::parse(".price")?;

    for li in items.select(&doc, doc.root()) {
        // Run from the <li>, select only looks at what's inside it.
        let p = price.select(&doc, li)[0];
        let value: u32 = doc.text(p).trim().parse()?;
        if value < 5 {
            doc.set_attr(li, "class", "cheap");
        }
    }

    let cheap = SelectorList::parse("li.cheap .name")?;
    let names: Vec<String> = cheap
        .select(&doc, doc.root())
        .into_iter()
        .map(|n| doc.text(n))
        .collect();
    assert_eq!(names, ["Emma"]);
    println!("{}", doc.inner_html(doc.root()));
    Ok(())
}

Where next

• Parsing for fragments, the time budget and the size limit.
• Selectors for the full list of what matches.
• The document tree for every way to read a node.
• Moving from Lexbor if you're porting code.