The document tree

A parsed page is a Document: every node in one vector, linked by index. You hold NodeIds, which are those indexes, and ask the document about them. This page covers every way to read the tree. Changing it is on Editing, and writing it out on Serialization.

On this page: NodeId · Document · Navigation · kind and Kind · Ns · Attributes · AttrNs · Quirks · Names and tag numbers

The types are defined in crates/parsec/src/dom.rs and re-exported at the crate root: parsec::{Document, NodeId, Kind, Ns, AttrNs, Quirks}.

`NodeId`

pub type NodeId = u32;

A node's index in its document. It's Copy, cheap, and only means something together with the document it came from.

The document node (or fragment node) is always 0, which Document::root() returns.
Ids are never reused. A removed node keeps its id and stays in the arena (see Editing).
Every method that takes a NodeId panics with an index-out-of-bounds error if the id is doc.len() or more. An id from a different document doesn't panic as long as it's in range; it just names some other node. Planned: handles that can tell when their node was removed (#12).

`Document`

pub struct Document { /* private */ }

A parsed document or fragment, or one you built by hand. It owns all of its strings; nothing in it borrows from the input you parsed. It doesn't implement Clone or Debug.

Getting one

parse, parse_with, parse_fragment build one from HTML.
Document::empty() -> Document makes a document with nothing in it but the root Kind::Document node: no source, no <html>. It's there for building a tree by hand with the edit functions.

use parsec::{Document, Kind, Ns};

let mut doc = Document::empty();
assert_eq!(doc.len(), 1);
assert!(doc.is_empty());
assert_eq!(doc.kind(doc.root()), Kind::Document);

let p = doc.create_element(Ns::Html, "p");
doc.append(doc.root(), p).unwrap();
assert_eq!(doc.inner_html(0), "<p></p>");

Whole-document methods

Method	Returns
`root(&self) -> NodeId`	Always `0`: the `Document` node for a parsed page, the `Fragment` node for a fragment.
`quirks(&self) -> Quirks`	The mode the doctype put the document in. Always `Quirks::No` for fragments and `Document::empty()`. See Parsing.
`len(&self) -> usize`	How many nodes are in the arena. That includes the root, nodes you removed, nodes you created and never inserted, each template's contents node, and for fragments the context element and internal `<html>` element. It's the bound for valid ids, not a count of what's in the tree.
`is_empty(&self) -> bool`	`true` when the arena holds only the root (`len() <= 1`). A parsed document is never empty, because the parser always adds `<html>`, `<head>` and `<body>`.

Navigation

All of these return None where the spec's DOM would return null.

Method	Returns
`parent(&self, id) -> Option<NodeId>`	The parent, or `None` for the root, for a detached node, and for a template's contents node.
`first_child(&self, id) -> Option<NodeId>`	The first child.
`last_child(&self, id) -> Option<NodeId>`	The last child.
`prev_sibling(&self, id) -> Option<NodeId>`	The sibling before.
`next_sibling(&self, id) -> Option<NodeId>`	The sibling after.
`children(&self, id) -> impl Iterator<Item = NodeId> + '_`	The children in order. It borrows the document, so collect it first if you want to edit while looping.
`template_contents(&self, id) -> Option<NodeId>`	For an HTML `<template>` element, its contents: a separate `Kind::Fragment` node. `None` for every other node.

Every node can be asked; text, comments and doctypes just have no children. All of them panic on an out-of-range id.

use parsec::css::SelectorList;

let doc = parsec::parse("<ul><li>a</li><li>b</li><li>c</li></ul>", false);
let ul = SelectorList::parse("ul").unwrap().select(&doc, doc.root())[0];

let first = doc.first_child(ul).unwrap();
let last = doc.last_child(ul).unwrap();
assert_eq!(doc.text(first), "a");
assert_eq!(doc.text(last), "c");
assert_eq!(doc.next_sibling(first), doc.prev_sibling(last));
assert_eq!(doc.parent(first), Some(ul));
assert_eq!(doc.children(ul).count(), 3);
assert_eq!(doc.parent(doc.root()), None);

Template contents

The HTML Standard keeps a <template>'s contents out of the main tree, in a document fragment of their own, and Parsec does the same. After a parse, a template element has no children; its contents hang off template_contents. Selectors, children and text don't look inside, as in a browser.

use parsec::css::SelectorList;
use parsec::Kind;

let doc = parsec::parse("<template><p>hidden</p></template>", false);
let t = SelectorList::parse("template").unwrap().select(&doc, doc.root())[0];

assert_eq!(doc.first_child(t), None);
let contents = doc.template_contents(t).unwrap();
assert_eq!(doc.kind(contents), Kind::Fragment);
assert_eq!(doc.inner_html(contents), "<p>hidden</p>");

// Selectors from the root don't see it...
assert!(SelectorList::parse("p").unwrap().select(&doc, doc.root()).is_empty());
// ...but they do from the contents node.
assert_eq!(SelectorList::parse("p").unwrap().select(&doc, contents).len(), 1);

If you'd rather treat template contents as ordinary children, as lxml and Beautiful Soup do, call adopt_template_contents once after parsing.

`kind` and `Kind`

pub fn kind(&self, id: NodeId) -> Kind<'_>

#[derive(Debug, PartialEq, Eq)]
pub enum Kind<'a> {
    Document,
    Fragment,
    Doctype { name: &'a str, public_id: &'a str, system_id: &'a str },
    Element { ns: Ns, name: &'a str },
    Text(&'a str),
    Comment(&'a str),
    ProcessingInstruction { target: &'a str, data: &'a str },
    Cdata(&'a str),
}

What a node is, with its strings borrowed from the document.

Variant	Means
`Document`	The root of a parsed page or of `Document::empty()`.
`Fragment`	The root of a fragment parse, or a template's contents.
`Doctype`	A doctype. The tokenizer lowercases the name. A missing public or system identifier reads as `""`, so you can't tell a missing one from an empty one.
`Element`	An element. `name` is the name as it serializes: lowercase for HTML, with SVG's case put back for the 37 names the spec lists (`clipPath`, `foreignObject`, `linearGradient`...), lowercase for MathML. Elements you create keep the case you gave unless they're HTML.
`Text`	A text node, references decoded. The parser adds text to the node before it where the spec says to, so a run of text is usually one node, but moves such as the adoption agency's can leave two side by side. If you want an element's whole text, use `text`.
`Comment`	A comment's text, without `<!--` and `-->`.
`ProcessingInstruction`	`<?target data?>`. The current HTML Standard parses these in HTML; `<?xml ...?>` and `<?xml-stylesheet ...?>` are still bogus comments. `data` keeps trailing spaces.
`Cdata`	A CDATA section. The HTML parser never makes one (CDATA in SVG or MathML becomes text); only `create_cdata` does.

Kind isn't Clone; match on it where you get it.

use parsec::{Kind, Ns};

let doc = parsec::parse("<!DOCTYPE html><!--hi--><svg><clipPath/></svg><?php x?>", false);
let kinds: Vec<String> = (0..doc.len() as u32)
    .map(|id| match doc.kind(id) {
        Kind::Document => "document".to_string(),
        Kind::Doctype { name, .. } => format!("doctype {name}"),
        Kind::Comment(c) => format!("comment {c}"),
        Kind::Element { ns: Ns::Svg, name } => format!("svg {name}"),
        Kind::Element { name, .. } => format!("html {name}"),
        Kind::ProcessingInstruction { target, data } => format!("pi {target} {data}"),
        Kind::Text(t) => format!("text {t}"),
        Kind::Fragment | Kind::Cdata(_) => unreachable!(),
    })
    .collect();
assert!(kinds.contains(&"svg clipPath".to_string()));
assert!(kinds.contains(&"pi php x".to_string()));

`Ns`

#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum Ns { Html, Svg, MathMl, Xml }

An element's namespace. The HTML parser makes Html, Svg (inside <svg>) and MathMl (inside <math>) elements. Xml is for trees built by hand: an XML element's name is its qualified name exactly as written, it's never treated as an HTML element whatever it's called, and selectors match it differently (see Selectors).

Attributes

Two methods read an element's attributes. Both give (AttrNs, name, value) triples in source order, with values decoded, and both give an empty iterator for anything that isn't an element. They differ in the name.

Method	Name you get
`attrs(&self, id) -> impl Iterator<Item = (AttrNs, &str, &str)> + '_`	The name as stored. For a parsed element that's lowercase, as the tokenizer leaves it. For `xlink:`, `xml:` and `xmlns:` attributes, the prefix is dropped: `xlink:href` reads as `href` with `AttrNs::XLink`.
`attrs_written(&self, id) -> impl Iterator<Item = (AttrNs, &str, &str)> + '_`	The name as the serializer writes it: SVG's case put back (`viewBox`, `gradientUnits`, and the rest of the spec's table), MathML's `definitionURL`, and prefixes kept (`xlink:href`, `xml:lang`, `xmlns:xlink`).

So attrs is what you compare against when you look an attribute up by its lowercase name, and attrs_written is what you show someone.

use parsec::css::SelectorList;
use parsec::AttrNs;

let doc = parsec::parse(r##"<svg viewBox="0 0 1 1"><use xlink:href="#a"/></svg>"##, false);
let svg = SelectorList::parse("svg").unwrap().select(&doc, doc.root())[0];
let use_ = doc.first_child(svg).unwrap();

let stored: Vec<_> = doc.attrs(svg).map(|(_, n, _)| n).collect();
let written: Vec<_> = doc.attrs_written(svg).map(|(_, n, _)| n).collect();
assert_eq!(stored, ["viewbox"]);
assert_eq!(written, ["viewBox"]);

assert_eq!(doc.attrs(use_).next(), Some((AttrNs::XLink, "href", "#a")));
assert_eq!(doc.attrs_written(use_).next(), Some((AttrNs::XLink, "xlink:href", "#a")));

What the parser does to attributes, from the tokenizer:

Names are lowercased.
When a tag repeats a name, the first one wins and the rest are dropped: <a href=x HREF=y> has one href, "x".
Character references in values are decoded, with the spec's rule for legacy references in attributes (&notit; stays as written).
A second <html> or <body> tag adds its attributes to the first one's element, skipping names it already has.

Looking one attribute up is a find:

use parsec::css::SelectorList;
use parsec::AttrNs;

let doc = parsec::parse(r#"<a href="/x" HREF="/y" title="&amp; more">t</a>"#, false);
let a = SelectorList::parse("a").unwrap().select(&doc, doc.root())[0];

let get = |name: &str| {
    doc.attrs(a)
        .find(|&(ns, n, _)| ns == AttrNs::None && n == name)
        .map(|(_, _, v)| v)
};
assert_eq!(get("href"), Some("/x"));
assert_eq!(get("title"), Some("& more"));
assert_eq!(doc.attrs(a).count(), 2);

`AttrNs`

#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum AttrNs { None, XLink, Xml, Xmlns, Host(u32) }

An attribute's namespace, from the spec's "adjust foreign attributes" table. Ordinary attributes are AttrNs::None. On SVG and MathML elements the parser gives these names a namespace: xlink:actuate, xlink:arcrole, xlink:href, xlink:role, xlink:show, xlink:title, xlink:type (XLink); xml:lang, xml:space (Xml); xmlns, xmlns:xlink (Xmlns). On HTML elements every attribute is None, even one spelled xlink:href.

Host(n) is for attributes of hand-built XML trees in a namespace you numbered yourself; the HTML parser never makes it, and attrs doesn't strip its prefix.

`Quirks`

#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum Quirks { No, Limited, Quirks }

The document's mode: no-quirks, limited-quirks or quirks. Parsing says how the doctype picks one and what each changes.

Names and tag numbers

Comparing strings for every element adds up on a big page, so the parser numbers the 124 tag names it treats specially, and you can use those numbers too.

`lower_name`

pub fn lower_name(&self, id: NodeId) -> Option<&str>

An element's name in lowercase, without allocating: SVG's foreignObject gives foreignobject. None for anything that isn't an element. For an Ns::Xml element, the name as written.

One exception: for SVG and MathML elements you created yourself with mixed case, it gives the name as you wrote it, not lowercased. Parsed elements are always lowercase here.

`known_tag` and `tag_number`

pub fn known_tag(&self, id: NodeId) -> Option<u8>
pub fn tag_number(name: &str) -> Option<u8>

known_tag gives the parser's number for the element's tag: the same number for every element with that lowercase name, which is what tag_number returns for the name. It's None for non-elements, for Ns::Xml elements, and for names the parser has no number for.

The 124 known names are the ones the tree builder has rules for: the HTML tags it treats specially (a, table, template, p and so on, plus obsolete ones such as image, keygen and xmp), the root and integration points of SVG (svg, foreignobject, desc, title), and MathML's math, mi, mo, mn, ms, mtext, mglyph, malignmark and annotation-xml. Plenty of real tags aren't there: label, abbr and blink all give None.
The number depends on the name, not the namespace. SVG's <a> and <title> get the same numbers as HTML's.
tag_number wants the lowercase name. tag_number("DIV") is None.
The numbers come from the order of a table in the source. Use them within one build; don't store them.

use parsec::css::SelectorList;

let doc = parsec::parse("<div><blink>x</blink></div>", false);
let div = SelectorList::parse("div").unwrap().select(&doc, doc.root())[0];
let blink = doc.first_child(div).unwrap();

assert_eq!(doc.known_tag(div), parsec::tag_number("div"));
assert_eq!(doc.known_tag(blink), None);
assert_eq!(doc.lower_name(blink), Some("blink"));
assert_eq!(parsec::tag_number("DIV"), None);

`is_html_element`

pub fn is_html_element(name: &str) -> bool

Whether name, in any case, is the name of an HTML element, current or obsolete, or the root of SVG or MathML. div, DIV, marquee, image, svg and math are; rss isn't. It's a fixed list of 145 names, useful for telling a custom or XML tag from an HTML one.

assert!(parsec::is_html_element("DIV"));
assert!(parsec::is_html_element("marquee"));
assert!(!parsec::is_html_element("rss"));

`named_reference`

pub fn named_reference(name: &str) -> Option<&'static str>

What one of HTML's named character references stands for. Give the name without the &, and with or without the ; exactly as the spec's table has it: most names need the ;, and the legacy ones that browsers accept without it are in the table both ways.

assert_eq!(parsec::named_reference("nbsp;"), Some("\u{a0}"));
assert_eq!(parsec::named_reference("nbsp"), Some("\u{a0}"));
assert_eq!(parsec::named_reference("NotEqualTilde;"), Some("\u{2242}\u{338}"));
assert_eq!(parsec::named_reference("&amp;"), None);
assert_eq!(parsec::named_reference("nope;"), None);

The document tree

NodeId

Document

Getting one

Whole-document methods

Navigation

Template contents

kind and Kind

Ns

Attributes

AttrNs

Quirks

Names and tag numbers

lower_name

known_tag and tag_number

is_html_element

named_reference

See also

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Clone this wiki locally

`NodeId`

`Document`

`kind` and `Kind`

`Ns`

`AttrNs`

`Quirks`

`lower_name`

`known_tag` and `tag_number`

`is_html_element`

`named_reference`