Namespace handling

[mattfbacon]

Proper recognition of XML namespaces is important in some contexts. I think this is something the parser should take care of.

[rgrove]

What does "proper recognition" mean to you? Namespace support is a broad concept, many aspects of which are likely out of scope for the goals of this project, but if you can share concrete use cases, that would help me understand what you're looking for. Bonus points for detailed examples of how you think this should work!

[mattfbacon]

I think something like this: Basically, it should be possible to identify elements and attributes from a specific namespace, where a namespace is defined exclusively by its URL (i.e., xmlns attributes are an internal detail of the XML format that should be abstracted out by the parser).

I wrote this for my personal use which AFAICT fulfills that requirement. It uses some helper functions that should be self-explanatory -- they're basically all from Rust.

type XmlNs = string | null;

declare module "@rgrove/parse-xml" {
	interface XmlElement {
		ns?: XmlNs;
		is([ns, name]: readonly [XmlNs, string]): boolean;
	}
}

function tag_namespaces(node: XmlElement, default_ns: XmlNs = null, namespaces: Map<string, string> = new Map()) {
	let namespaces_cloned = false;

	for (const [attr, value] of Object.entries(node.attributes)) {
		const namespace = str_strip_prefix("xmlns:", attr);

		if (namespace === null) {
			continue;
		}

		// Optimistically avoid copying, because nested namespace declarations are uncommon.
		if (!namespaces_cloned) {
			namespaces = new Map(namespaces);
			namespaces_cloned = true;
		}

		namespaces.set(namespace, value);
	}

	// XML allows to set a "default namespace" with a bare `xmlns="<ns url>"` attribute.
	if ("xmlns" in node.attributes) {
		const xmlns_attr = node.attributes["xmlns"];
		default_ns = xmlns_attr;
	}

	const [old_node_ns, new_node_name] = str_split_once(":", node.name) ?? [null, node.name];
	const new_node_ns: XmlNs | undefined = old_node_ns !== null ? namespaces.get(old_node_ns) : default_ns;
	assert(`undefined ns ${dbg(old_node_ns)} used`, new_node_ns !== undefined);

	node.ns = new_node_ns;
	node.name = new_node_name;

	for (const child of node.children) {
		if (!(child instanceof XmlElement)) {
			continue;
		}
		tag_namespaces(child, default_ns, namespaces);
	}
}

XmlElement.prototype.is = function ([ns, name]: readonly [XmlNs, string]) {
	assert("tag_namespaces not run", this.ns !== undefined);
	return this.ns === ns && this.name === name;
}

export function xml_parse(raw: string): XmlDocument {
	const parsed = parseXml(raw);
	for (const child of parsed.children) {
		if (child instanceof XmlElement) {
			tag_namespaces(child);
		}
	}
	return parsed;
}

[rgrove]

Thanks! This is helpful.

It looks like you were able to whip up a working solution for your use case by doing a tree-walking pass on parse-xml's output, which I like because this is exactly where I see parse-xml providing the most value: it handles the low level parsing and other tools handle the rest.

The monkeypatching and type modification are awkward, but could be avoided by defining a subclass of XmlElement that adds namespace functionality. The remaining drawback is the performance/memory impact of needing to walk the tree and construct new nodes, though that's likely to be minor at the relatively small document sizes parse-xml is optimized for.

While I'm still not sure that namespace support is something that belongs in parse-xml's core (especially given the need to ensure strict compliance with at least a meaningful subset of the Namespaces in XML 1.0 spec for any implementation parse-xml would ship), it would be interesting to think about how parse-xml could make it easier/more efficient for this use case to be met by a wrapper library, or even some kind of pluggable extension.

[mattfbacon]

I took a look at that spec. I think the only aspect that my code didn't implement is unsetting of the default namespace with xmlns="", which would need to set default_ns to null instead of "".

Subclasses would be an interesting approach here. I don't tend to use object-oriented programming techniques except when forced to by the language or standard library, so I didn't think of it.

I can understand the desire not to increase the complexity of the core library. I guess it depends on what one considers as the minimal definition of XML. I think that namespaces are inextricably part of XML, because one can for example define two namespaces with the same URL, then two elements or attributes, each using one of the namespaces, and those two elements/attributes would actually be the same, despite having different names when compared verbatim (i.e., for elements, with the name property in this library). In this situation, it is indeed relevant for correctness that namespaces are considered in at least some capacity.

In any case, with this issue/discussion I just wanted to confirm that the feature was intentionally left out rather than just forgotten about or not gotten to yet.

[rgrove]

I'm not intimately familiar with the Namespaces in XML spec, but at a glance I think you're also missing attribute namespacing and various things related to reserved prefixes and validation, but I assume those aren't important for your use case.

To answer your question about whether namespaces were intentionally left out of parse-xml: yes, very intentionally. 🙂

It's understandable that you might expect namespace support to be included because many XML parsers do include it, and because the name "XML" is often used to describe a broad umbrella of related specifications beyond just the core XML spec, but parse-xml is very intentionally a tightly focused implementation of a safe subset of only the Extensible Markup Language (XML) 1.0 (Fifth Edition) specification (because that's all I needed for the use case I built it for).

[mattfbacon]

Ah yes, that's true, I left attributes out.

Good to know regarding the specs, and I guess cool that namespacing can be flexibly added on top.

Thanks for the great work on this library btw, aside from the namespaces it's perfect for my use case ({web,cal,card}dav) too.