fn:deep-equal: Order of child elements (unordered-elements)

[ndw]

At meeting 024 where PR #320 was accepted, there remained an open question of how best to specify that in some circumstances the comparisons should be made without regard to the order of (some) children.

Can the name of the option be improved?

Should the option support wildcard names?

[ChristianGruen]

Thanks for yesterday’s discussion, which helped me to understand how unordered-elements is supposed to work.

I like @cmsmcq’s suggestion to add wildcard support. The value type could be union(xs:QName, enum('*'). A more powerful solution could be to use the name test syntax and resolve the supplied string dynamically:

deep-equal(
  <a><b/><c/></a>,
  <a xmlns='x'><c/><b/></a>,
  options := map { 'unordered-elements': ('*:a', '*:b') } (: 'a', '*:a', 'a:*', '*' :) 
)

My remaining notes are editorial:

A) The option unordered-elements is defined as follows:

A list of QNames of elements considered to be unordered: that is, their child elements may appear in any order.

I was confused by the reference to “child elements”. I would suggest changing it to something like:

A list of QNames of elements. If an element name matches a QName, its child nodes can be unordered: that is, they may appear in any order.

B) simple types, complex type, variety, type-annotations, type-variety, typed-values, annotated as…

As it’s perfectly possible to implement the function without XML Schema support, I wonder if it’s possible to choose a schema-oblivious wording. In the XQFO spec, the listed terms seem to be exclusively used for fn:deep-equal. I must admit that I would need to read the XML Schema specification in detail to understand the current rules – which I could certainly do (and probably should), but I assume that other readers may have similar experiences. On the other hand, there may be no need for that if we have good test cases and if the option is self-explanatory and intuitive enough.

[michaelhkay]

I am not at all convinced that wildcards would be a useful addition. I think the typical use case for unordered comparison is something like the document at https://learn.microsoft.com/en-us/dotnet/standard/linq/sample-xml-file-customers-orders

Here it seems clear that the contents of the Customers and Orders elements are intrinsically unordered, whereas everything else is ordered. Other elements such as Customer, Order, FullAddress could be compared without order, but since the order of children of these elements is entirely regular, it's more likely that the user would want an ordered comparison in these cases.

I can just about imagine a case for "treat everything as unordered", or perhaps "treat everything except X, Y, Z as unordered". I find it hard to imagine a case where everything in one namespace is ordered, and everything in a different namespace is unordered. If there is, please show me the document.

[ChristianGruen]

I think the quoted document is actually a pretty good example for using a simple wildcard. If someone is only interested in the data, and if the data is well-structured and has no mixed content, there would be no need to expect any node to occur in any particular order. After all, it’s one of the strengths of XPath that the order of an element is irrelevant (//Customer/Phone gives us all phone numbers, no matter where they are located).

Instead of…

map { 'unordered-elements': (
  xs:QName('Customers'), xs:QName('Customer'), xs:QName('Orders'),
  xs:QName('Order'), xs:QName('FullAddress'), xs:QName('ShipInfo')
)}

(: …or something slightly more compact… :)
map { 'unordered-elements':
  ('Customers', 'Customer', 'Orders', 'Order', 'FullAddress', 'ShipInfo') ! xs:QName(.)
}

…one could simply write '*'.

Another use case for wildcards are unordered name/value pairs of JSON objects (are any other unordered input) that is converted to an XML representation. The element order may be completely irrelevant.

I agree, though, that the name test syntax seems a bit overambitious.

[dnovatchev]

With the risk of this becoming tedious:

Will not this and many other problems' discussion benefit if/when we have the Set datatype in place?

The comparison "without regards to order" is nothing else but a set comparison.

Trying to implement the properties and functionality of the set datatype every single time they are needed results in obvious redundancy, waste of resources and failure to reach common language and common understanding.

[michaelhkay]

My objections to introducing a set datatype are that (a) from previous experience, it will take at least 2 years of WG meetings (probably more, since we aren't proposing to have any week-long face-to-face meetings), and (b) that sets depend on a universal definition of equality, and the whole point of this issue is that equality in XML is a very fuzzy concept that needs to be adapted and customised for different user requirements.

[dnovatchev]

My objections to introducing a set datatype are that (a) from previous experience, it will take at least 2 years of WG meetings (probably more, since we aren't proposing to have any week-long face-to-face meetings), and (b) that sets depend on a universal definition of equality, and the whole point of this issue is that equality in XML is a very fuzzy concept that needs to be adapted and customized for different user requirements.

@michaelhkay

(a) The proposal "[XPath]Proposal to introduce the set datatype in XPath 4" was submitted almost 2 1/2 years ago, on 19th December 2020. If we took it seriously, even by your own estimation, it would already have been produced in final form.

(b) Even so, this wouldn't need as many special options as the current fn:deep-equal proposal. If we can do this for fn:deep-equal, we can do it not worse for sets.

Let us do at least some serious and important work, not just find justifications to the contrary.

We often stumble upon real-life use-cases, the latest was today, raised by Martin Holmes in the general channel of the Xml.com slack

Are we here to do something useful, or waste time witlessly?

[ChristianGruen]

@dnovatchev Can we please try to keep the issues clean?

I think this is the wrong place to restart the fundamental discussion on sets. If you want to be constructive, you could present an example how you believe a solution based on sets could possibly look like and contrast it with the proposed unordered-elements option.

We often stumble upon real-life use-cases, the latest was today, raised by Martin Holmes in the general channel of the Xml.com slack

The option discussed here would exactly solve Martin’s problem.

If we took it seriously, even by your own estimation, it would already have been produced in final form.

You don't consider that proposing a feature only takes 5-10% of the time that needs to be invested to finalize the feature, and that someone needs to invest this time. Next, you seem to want to imply that nothing else happened during the initial proposal.

[michaelhkay]

I'm sorry, but I would class that as a suggestion, not as a proposal. Adding sets to the data model is likely to involve adding 60-100 pages of text to the specification, not three paragraphs. It takes us about an hour of CG time to agree each page.

[ndw]

The CG agreed to close this issue without action at meeting 051.