Can AngleSharp be used to create a diff of/compare of two HTML fragments?

[egil]

This looks like a great library, that I could potentially use in my Razor Components Testing Library. One of my needs is the ability to intelligently compare to HTML(5) fragments and assert if they are equal. Is that possible with AngleSharp?

Currently I am using XMLUnit for generating the comparison, but since it is XML based it does make certain things harder.

Note: When comparing, I don't want insignificant whitespace to be counted as an error, also, the order of css classes defined in the class="..." attribute should not matter.

[FlorianRappl]

You can do that.

Some remarks:

• Not sure about "insignificant" whitespace; most whitespace ends up in text nodes, but AngleSharp has methods to normalize this (e.g., no matter if newline, tab, multiple whitespace - it all becomes one single whitespace)
• CSS classes could be ordered

In general if making two string comparisons with such requirements the best way would be to implement a custom markup formatter (could be based on one of the existing markup formatters).

Does that help as a starting point or would some more information (which ones) be helpful?

[egil]

Thank you @FlorianRappl for taking the time to answer.

Let me elaborate a bit my needs. Basically, what I need to compare is two different HTML fragments. One is generated based by a Razor component, the other is more or less manually entered by the developer - similar to snapshot testing. This makes a direct string comparison fail most of the time, since there will be indentions, line breaks, and additional spaces, that does not affect the display of the HTML to the user, but breaks comparison.

The approach XMLUnit takes is to compare individual nodes/elements for equality and skip insignificant whitespace while doing so. That gives a very stable comparison, problem is that XMLDiff doesn't understand HTML5, including self-closing-tags, embedded stylesheet, and javascript, etc.

My take on your input is that with AngleSharp, I would basically "normalize" both HTML fragments, and if and only if they contain equal significant content, the two HTML fragments (as strings) will become equal. Is that correct?

[FlorianRappl]

Just to clarify: Razor also is not capable of handling valid HTML5 (as an example just try having a <ul> with <li> that are not closed anywhere; Razor complains the closing tag is missing, which is not by the spec).

Regarding the approach; you can either normalize the string representation of both fragments (this is a powerful technique that allows simple algorithms to follow) or you can compare the tree structure of both (which is what XML unit does). The latter can have some benefits (e.g., abort on first mismatch, improved error output, ...).

Now regarding:

This makes a direct string comparison fail most of the time, since there will be indentions, line breaks, and additional spaces, that does not affect the display of the HTML to the user, but breaks comparison.

Well, as described above whitespace is always significant. They will become text nodes. And they are rendered (only question is how). While this does not affect most layouts, especially in an inline / inline-block content it matters (this is one of the problems when doing HTML minification; whitespace is in general not removed; it is rather normalized such that N spaces become 1 space or newlines, tabs, etc. becoming a space). Here, we already have a solution - the MinifyMarkupFormatter.

But the MinifyMarkupFormatter is not sufficient. It does not touch the attributes. In your case the class attribute would need some normalization, too. Hence the reason I suggest to implement a custom formatter (which could be based on the minify formatter).

For a logic to compare the trees, I'll try to sketch something.

[FlorianRappl]

One addition regarding the whitespace issue.

Let's say in the source (e.g., Razor) you got:

<div><h1 class=""foo bar"">My title</h1><p>Some text</p></div>

but the user enters:

<div>
	<h1 class=""bar foo"">
		My title
	</h1>
	<p>
		Some text
	</p>
</div>

This is not equivalent. In no case. There must be a space in heading and trailing the h1 content (same applies to p).

The rendering may be the same, but to see that there is some difference let's just have

a<h1 class=""bar foo"">
    My title
</h1>b

and let's switch different styling modes for the h1. In block and inline-block the heading and trailing space is ignore, but once you go to inline we have an issue.

Long story short (or why am I writing this): If you have exactly this comparison in mind, then there is no way you can achieve it (as just discarding all heading / trailing whitespace would certainly lead to other errors). So whitespace minification is acceptable, but just removing is not.

[egil]

y: Razor also is not capable of handling valid HTML5 (as an example just try having a <ul> with <li> that are not closed anywhere; Razor complains the closing tag is missing, which is not by the spec).

Razor generally generates HTML5 markup, so it does not have a problem. It is XMLUnit, which expects valid XML (XHTML), that causes problems.

Well, as described above whitespace is always significant. ...

Yes, we agree. But the minifying process should normalize both HTML fragments to remove false positives.

From what I gather, we could take this approach:

1. Normalize using MinifyMarkupFormatter - I assume the minifier also processes stylesheet information in <style> tags and javascript in <script> tags
2. Then walk the two markup trees using a comparer like XMLUnit. XMLUnit makes it possible to hook into the comparison process, such that each node/attribute that is compared, can be done with custom code/hook. That is how I currently deal with css classes in class=".." attributes. I have a custom method that will do the comparison if the attribute is named "class": https://github.com/egil/razor-components-testing-library/blob/10fcfc911b728b367e11e9c47aab41560d2e63ef/src/Diffing/HtmlDifferenceEvaluators.cs#L41-L61

[egil]

Long story short (or why am I writing this): If you have exactly this comparison in mind, then there is no way you can achieve it (as just discarding all heading / trailing whitespace would certainly lead to other errors). So whitespace minification is acceptable, but just removing is not.

Thanks for clarifying. Even if we could make the comparison logic so intelligent that it would know the display setting of an element, it wouldn't make much difference in most cases (in my case at least). The testing logic will test HTML fragments generated by Razor components (used in Blazor apps), that will be part of a large app, and thus, the styling of the individual components is not know at compare time (unless the component author is using inline styles).

[FlorianRappl]

Razor generally generates HTML5 markup, so it does not have a problem.

Sure - the remark was just that Razor cannot handle valid HTML5 input. What it generates is of course valid HTML5.

Regarding the approach: 1. and 2. are two different approaches; Either normalize string representations and make a string comparison or walk both trees.

I still want to provide some example code but I'll need to go now; I should be back later today / tomorrow to craft some first draft.

[egil]

Regarding the approach: 1. and 2. are two different approaches; Either normalize string representations and make a string comparison or walk both trees.

Sorry, I was not clear. I ment we should first (1) normalize and then (2) walk both trees. I.e. combine both approaches to make the comparison as stable as possible. My guess is that it will make it simpler to create a tree walker comparer, since there will be less edge-cases to handle (might be wrong, do not know AngleSharps options here).

I still want to provide some example code but I'll need to go now; I should be back later today / tomorrow to craft some first draft.

Awesome, thank you.

[FlorianRappl]

Here's what I crafted out quickly as a first draft (sorry for length and formatting):

void Main()
{
	var actual = @"<div><h1 class=""foo bar"">My title</h1><p>Some text</p></div>"; // originally from Razor
	var expected = @"<div>
	<h1 class=""bar foo"">
	  My title
	</h1>
	<p>Some text</p>
</div>"; // from user
	AreFragmentsEqual(actual, expected).Dump();
}

// this could be exposed
bool AreFragmentsEqual(string actual, string expected, string root = "div", bool strict = false)
{
	var context = BrowsingContext.New();
	var parser = context.GetService<IHtmlParser>();
	var document = context.OpenNewAsync().Result;
	var rootElement = document.QuerySelector(root) ?? document.CreateElement(root);
	var actualFragment = parser.ParseFragment(actual, rootElement);
	var expectedFragment = parser.ParseFragment(expected, rootElement);
	return AreFragmentsEqual(actualFragment, expectedFragment, strict);
}

// this would be private; for simplicity it modifies the lists - otherwise only work with clones inside
bool AreFragmentsEqual(INodeList actual, INodeList expected, bool strict)
{
	// Remove comments
	RemoveComments(actual);
	RemoveComments(expected);
	
	// Handle whitespace
	if (!strict)
	{
		RemoveWhitespace(actual);
		RemoveWhitespace(expected);
	}

	if (actual.Length != expected.Length)
	{
		actual.ToHtml().Dump();
		expected.ToHtml().Dump();
		return false;
	}

	for (var i = 0; i < actual.Length; i++)
	{
		if (!AreEqual(actual[i], expected[i], strict))
		{
			return false;
		}
	}

	return true;
}

bool AreEqual(INode actual, INode expected, bool strict)
{
	if (actual.NodeType != expected.NodeType)
	{
		return false;
	}
	
	switch (actual.NodeType)
	{
		case NodeType.Element:
			if (!AreEqual((IElement)actual, (IElement)expected, strict))
			{
				return false;
			}
			break;
		case NodeType.Text:
			if (!AreEqual((IText)actual, (IText)expected, strict))
			{
				return false;
			}
			break;
	}

	return AreFragmentsEqual(actual.ChildNodes, expected.ChildNodes, strict);
}

bool AreEqual(IElement actual, IElement expected, bool strict)
{
	var excluded = new[]{ "class" };
	
	var actualAttributes = actual.Attributes.Where(a => !excluded.Contains(a.Name)).OrderBy(a => a.Name).ToArray();
	var expectedAttributes = expected.Attributes.Where(a => !excluded.Contains(a.Name)).OrderBy(a => a.Name).ToArray();
	
	var actualClasses = actual.ClassList.OrderBy(a => a).ToArray();
	var expectedClasses = expected.ClassList.OrderBy(a => a).ToArray();

	if (actualAttributes.Length != expectedAttributes.Length)
	{
		Debug.WriteLine("Elements are different. Expected number of attributes '{0}' to be '{1}'", expectedAttributes.Length, actualAttributes.Length);
		return false;
	}
	else if (actualClasses.Length != expectedClasses.Length)
	{
		Debug.WriteLine("Elements are different. Expected number of classes '{0}' to be '{1}'", expectedClasses.Length, actualClasses.Length);
		return false;
	}

	for (var i = 0; i < actualClasses.Length; i++)
	{
		if (string.CompareOrdinal(actualClasses[i], expectedClasses[i]) != 0)
		{
			Debug.WriteLine("Elements are different. Expected class '{0}' to be '{1}'", expectedClasses[i], actualClasses[i]);
			return false;
		}
	}

	for (var i = 0; i < actualAttributes.Length; i++)
	{
		if (string.CompareOrdinal(actualAttributes[i].Name, expectedAttributes[i].Name) != 0 ||
			string.CompareOrdinal(actualAttributes[i].Value, expectedAttributes[i].Value) != 0)
		{
			return false;
		}
	}

	return true;
}

bool AreEqual(IText actual, IText expected, bool strict)
{
	var actualText = actual.Data;
	var expectedText = expected.Data;

	if (!strict)
	{
		actualText = actualText.Trim();
		expectedText = expectedText.Trim();
	}
	
	if (string.CompareOrdinal(actualText, expectedText) != 0)
	{
		Debug.WriteLine("Texts are different; Expected text '{0}' to be '{1}'", expectedText, actualText);
		return false;
	}
	
	return true;
}

void RemoveComments(INodeList nodes)
{
	for (var i = nodes.Length - 1; i >= 0; i--)
	{
		if (nodes[i].NodeType == NodeType.Comment)
		{
			nodes[i].RemoveFromParent();
		}
	}
}

void RemoveWhitespace(INodeList nodes)
{
	for (var i = nodes.Length - 1; i >= 0; i--)
	{
		if (nodes[i] is IText text && string.IsNullOrWhiteSpace(text.Data))
		{
			text.Remove();
		}
	}
}

Right now strict mode removes all whitespace only nodes. Thus there could be false positives, but its the easiest way to make such a comparison. This could also be how strict vs non-strict could play out. In this version non-strict removes all whitespace only nodes (and trims other text nodes). I can imagine the strict version to still removes / normalize whitespace - just not so aggressively.

For the list of attribute exclusions I just included class. There could be more, which would then either require separate handling (e.g., style could be a candidate) or would actually be disregarded (could be some tag helper artifact).

Hope that helps!

[egil]

Awesome @FlorianRappl - this is a great start and an excellent place where I can continue from. I like how easy the code is to follow, even for somebody who does not know the AngleSharp library or API.

A question. Instead of manually removing whitespace and trimming text nodes, why not use AngleSharp's minification feature first on both fragments (if strick == false) before walking the nodelist/node tree? I imagine the minification feature has been vetted and tested such that it doesn't produce invalid code.

[FlorianRappl]

It does not modify the tree - it just produces a string representation.

That is also why I wrote that these are two separate approaches.

You cannot directly combine them; you can indirectly if you modify the tree upfront, e.g., "normalizing" attributes (order, special cases such as classes), and then perform the serialization. However, note that the examples given in my code would be listed as "false" due to the whitespaces being used for code formatting. The minification just reduces them to a single (normalized) space character. With the shown approach you have the flexibility of describing the options for comparing (right now just a single "strict", but it could be more).

Maybe there is way to combine these approaches (e.g., by exposing the logic to perform some normalization). I don't know yet.

Glad the draft helps so far!

[egil]

OK, I will close the issue as I feel I've gotten all the help I could possibly expect. Thanks a lot! Let me know if you want to get notified when I/if I get something working.