String comparison in deep-equal

[michaelhkay]

The code showing how strings should be compared in deep-equal has gone awry, it doesn't match the prose. In equal-strings(), the lines

let $n1 := if ($options?whitespace = "normalize"))
             then normalize-unicode(?, $options?normalization-form) 
             else identity#1
  let $n2 := if ($options?normalize-space)
             then normalize-space#1 
             else identity#1      

should read:

let $n1 := if ($options?whitespace = 'normalize')
             then normalize-space#1 
             else identity#1    
let $n2 := if ($options?normalization-form))
             then normalize-unicode(?, $options?normalization-form) 
             else identity#1
  

Actually, the whole thing can now be expressed more concisely using fn:chain:

declare function equal-strings(
  $string1   as xs:string,
  $string2   as xs:string, 
  $collation as xs:string,
  $options   as map(*)
) as xs:boolean {
  let $norm := fn:chain(?,
                        (normalize-unicode(?, $options?normalization-form)[$options?whitespace = "normalize"],
                         normalize-space#1[$options?normalize-space]))          
  return compare($norm($string1), $norm($string2), $collation) eq 0    
}

[michaelhkay]

Related: we say

Whitespace-only text nodes are discarded if both the following conditions are true:

1. Either:
   1.1 The option whitespace is set to strip or normalize; or
   1.2 $parent is a schema-validated element node whose type annotation is a complex type with an element-only or empty content model.
2. The text node is not within the scope of an element that has the attribute xml:space="preserve".

I need to check this, but I think that whitespace text nodes will be removed from elements with element-only content model during the schema validation process even if xml:space="preserve" has been set.

[michaelhkay]

Also closely related: the current rules for string comparison under whitespace normalization are non-transitive.

This is because whitespace normalization is not applied to anyURI values.

Currently, if the whitespace='normalize' option is set, then

• string(' a ') equals string('a')
• string('a') equals anyURI('a')
• string(' a ') does not equal anyURI('a')

This makes it very hard to implement unordered comparison (option ordered="false", or the implicit unordered comparison for attribute nodes) because it's not enough to find a matching pair of values from the two sequences and then compare the items that remain; you have to consider the possibility that there might be a different pairing of the two sequences that works.

This is the only case of non-transitivity that I have found, but there may be others. This one is easily fixed by saying that whitespace normalization should be applied when comparing strings and URIs.

(Of course, we depend on the collation being transitive. Perhaps we should state this as an explicit assumption/dependency. I believe that the UCA is designed to be transitive).

[michaelhkay]

The whitespace=normalize option is not working as expected for attribute nodes. This is because by default, the typed-values option is set to true, and the whitespace option does not apply when comparing typed values (as distinct from string values). I think there would be fewer surprises if the rule for attribute nodes were changed from:

F. Either the option typed-value is false, or the typed value of $i1 is deep-equal to the typed value of $i2.
G. Either the option typed-value is true, or the equal-strings function returns true when applied to the string value of $i1 and the string value of $i2.

to:

F. Let T be true if the typed-value option is true and both attributes have a type annotation other than xs:untypedAtomic. Then either T is true and the typed value of $i1 is deep-equal to the typed value of $i2, or T is false and the equal-strings function returns true when applied to the string value of $i1 and the string value of $i2.

This parallels the rule for element nodes, where typed-value comparison takes place only if the actual nodes to be compared are typed.

[ChristianGruen]

@michaelhkay The test suite contains fn:deep-equal tests with the obsolete text-boundaries option. Do you plan to rewrite these tests to use the whitespace option, or should they be dropped?

[ChristianGruen]

Two more questions. The current rules in the spec are as follows:

1. Comment nodes are discarded if the option comments is false.
2. Processing instruction nodes are discarded if the option processing-instructions is false.
3. Adjacent text nodes are merged.
4. Whitespace-only text nodes are discarded if both the following conditions are true […]

I assume that with new rules, K2-SeqDeepEqualFunc-20 and K2-SeqDeepEqualFunc-22 should be rewritten to return true?

Next, is it correct that the following query is expected to return true, because Step 4 (the check for whitespace-only nodes) is defined after Step 3 (the merge of adjacent text nodes)?

deep-equal(
  <x>A<?_?>{ ' ' }<?_?>C</x>,
  <x>A C</x>,
  options := map { 'whitespace': 'strip' }
)

[ChristianGruen]

…and one more (sorry): Isn’t deep-equal-40-whitespace-010 supposed to return true?