Should ??KS flatten the results

[michaelhkay]

Currently the result of ??KS (like ?KS) is flattened. So if you do ??dimensions and the value of each dimensions entry is a sequence of zero or more numbers, the result munges them all together into a single sequence (dropping any empty values in the process).

Should we change this, for example to return a sequence of arrays, or an array of sequences?

This makes life a bit more difficult in the simple case where all the values are singletons -- and notably, when constructing a path such as ??A??B??C -- but it makes it possible to handle the more general case where they aren't all singletons.

[ChristianGruen]

Personally, I prefer the flattened representation, partly because that’s what the flat lookup operator returns, and partly because I like the analogy with path expressions (a descendant step doesn’t reveal which subpaths are empty).

The deep lookup operator is a powerful feature, but I think its use for real structured navigation is necessarily limited.

[dnovatchev]

Definitely produce true results - not mixing them together and destroying their individual characteristics.

Whoever wants a flattened result can always do this. On the other hand, it is impossible to deconstruct the original sequences from their concatenation - an irreversible loss 😢

[michaelhkay]

It's not actually clear what a non-flattened result would look like. If it's not flattened at all, then you get back some kind of projection of the original tree, but it's not clear that would be useful. You could return a depth-2 tree, a sequence of arrays in which each array represents the value of a leaf entry in a map or array of the original tree structure. But I think we probably need to look at this again in conjunction with "zipper" style proposals that return more information about the selected items, including their parentage.

[dnovatchev]

It's not actually clear what a non-flattened result would look like. If it's not flattened at all, then you get back some kind of projection of the original tree, but it's not clear that would be useful. You could return a depth-2 tree, a sequence of arrays in which each array represents the value of a leaf entry in a map or array of the original tree structure. But I think we probably need to look at this again in conjunction with "zipper" style proposals that return more information about the selected items, including their parentage.

Correct.

I imagine that the result could be similar to the result returned by the evaluation of the XPath expression //someElementName - that is a set (unordered in the case of a map) of all elements named "someElementName". We will also need for each returned result also the path that leads to it, because members of a map/array don't know their ancestors.

Thus, this could be a sequence of arrays with size 2, where the 1st member is the exact path to the result and the second member of the array is the (single) result.

[dnovatchev]

To reply to myself:

We will also need for each returned result also the path that leads to it, because members of a map/array don't know their ancestors.

Thus, this could be a sequence of arrays with size 2, where the 1st member is the exact path to the result and the second member of the array is the (single) result.

Or instead of using an array to return each result from the search, we could have a path-value record (similar to the already proposed and in use "key-value record" where the record has two keys named "path" and "value" and the search (single, one of possibly many) result is the value for the "value" key, and the path to reach this search result is the value of the "path" key.

[michaelhkay]

I think the main practical problem with flattening is with empty sequences, which are generated by parse-json() to represent the JSON null value. I'm going to propose adding an option to parse-json() to use an alternative user-supplied representation for null, for use in cases where this causes a problem.

We should perhaps propose some item that the serializer recognises as null so that it round trips though parsing and serialization. Perhaps a magic QName such as Q{http://www.w3.org/2005/xpath-functions}null.

[michaelhkay]

This discussion has led to action on a number of fronts, notably (a) the option in parse-json to use singleton items rather than () to represent null, and (b) selection on pinned arrays which allow information about the hierarchic structure to be preserved. It is therefore time to close the issue.

[ndw]

The CG decided to close this issue without any further action at meeting 069.