Arrays' counterparts for functions on sequences, and vice versa

[dnovatchev]

When we have a proposal for a function f1 that has a sequence-argument, we need to also have (or propose if not-existent) a corresponding function f2 that has an array-argument in place of the f1's sequence argument.

For example:

all($input as item()*, $predicate as function(item()) as xs:boolean)

the above function accepts a sequence as its 1st argument. In this case there is no existing function all for arrays, therefore we will define/propose it together with the above function:

array:all($input as array(*), $predicate as function(item()*) as xs:boolean))

For consistency, clarity and to not confuse the reader of the Spec (for example trying to find why there is no corresponding 2nd function and abuse their imagination) we shall as a rule always provide a pair of such functions: one defined on sequence(s) and one defined on array(s).

Even if we were not going to propose a new function, but an "orphan" such function already exists, we will add its corresponding 2nd function.

[michaelhkay]

I think this is a reasonable principle for "generic" sequence functionality, but I think it's a principle that shouldn't be applied blindly. For example, the use case for the function fn:one-or-more() is to get around the problems caused with strict static typing implementations, which (a) is essentially obsolete because no recent implementations of XQuery do strict static typing, and (b) isn't relevant to arrays anyway, because the type system recognises cardinality categories for sequences but not for arrays.

So for this issue to become actionable, it needs to evolve from a statement of desirable principles to a specific proposal of functions to be added to the specification.

[dnovatchev]

@michaelhkay

but I think it's a principle that shouldn't be applied blindly.

Exactly! This is why the proposal says:

"... we shall as a rule always provide a pair of such functions".

And here "as a rule" means that we will do this in most cases but not always and not blindly.

As for a list of functions, I expect collecting this to be a gradual process and everybody is encouraged to contribute to this list.

For a start, here are 8 incomplete pairs of existing in XPath 3.1 functions (we will want to apply this rule for most newly-proposed functions, too):

array:put()            <==>
array:append($arr, ?)  <==>  insert-before($seq, count($seq) +1, ?)
                       <==>  empty($seq) 
                       <==>  exists($seq)
                       <==>  avg($seq)
                       <==>  max($seq)
                       <==>  min($seq)
                       <==>  sum($seq) 

[ChristianGruen]

Please note that you can already use sum([ 1, 2 ]) (same for avg, max, min).

[dnovatchev]

Good catch, @ChristianGruen.

What about index-of() ?

This:

  let $ar := [1, [2, [3, 4]], [4, [3,5]]]
   return
     index-of($ar, [3])

returns:

3
6

But one would probably need:

[2, 2, 1], [3, 2, 1]

[michaelhkay]

Functions like sum(), avg(), index-of(), distinct-values() that operate on a sequence of atomic values don't need a direct array equivalent because they can already be applied to an array of atomic values, as a result of implicit atomization.

The index-where() function has been proposed as a generalization of index-of(), and for this there is certainly an intuitive array equivalent array:index-where($array, function(item()*) as s:boolean).

Deep search of nested arrays (and maps and sequences) is another matter. Currently we only have map:find(), and that's pretty well useless. That, however, is a separate issue.

[dnovatchev]

@michaelhkay , @ChristianGruen

Functions like sum(), avg(), index-of(), distinct-values() that operate on a sequence of atomic values don't need a direct array equivalent because they can already be applied to an array of atomic values, as a result of implicit atomization.

This looks like another "hidden gem - feature" that needs to be properly revealed/highlighted -- in the description of these functions and not only in the examples to them.

Apart from this, I don't feel that using the current function index-of($someArray, $someValue) produces a valuable and useful result at all.

So yes, we do indeed need deep searching functions.

Shall we create a separate issue for this?

[michaelhkay]

Deep searching is related to deep update, which is related to issue #77 - though that's expressed in terms of enhancements to XQuery Update. Yes, it's probably worth another issue; though I'm not optimistic about coming up with a good design (I've made several attempts).

[ChristianGruen]

This looks like another "hidden gem - feature" that needs to be properly revealed/highlighted -- in the description of these functions and not only in the examples to them.

it's sometimes difficult to judge which features are hidden if you are already used to apply them regularly.

[michaelhkay]

Note also that fn:put() already exists (in the XQuery Update specification) and has a meaning completely unrelated to array:put().

[ChristianGruen]

With #843, I tried to push this further systematically, but we didn’t really succeed. In addition, we have #910, so I propose to close this issue.

[dnovatchev]

I think we already added a few new functions on arrays, thus closing this issue is not "with no action".

[ChristianGruen]

I think we already added a few new functions on arrays, thus closing this issue is not "with no action".

It seems you took this personal, Dimitre. I’m sorry. This was not the intention. If you believe we should revise the existing label set (which I simply took advantage of), we can certainly discuss this separately.

[dnovatchev]

Nothing personal,

We just need to say something like:

"Closing after we introduced some new, needed functions on arrays."

[ndw]

The (intended) meaning of "close with no action" is not to stipulate that the working group has done nothing about the issue in a broader sense, rather it is to say that "nothing in this specific issue, as it is written, in the context of the other work that we have done, is going to be the focus of future work". Or something like that.

We could close an issue with no action because we've decided it isn't a good idea, or because we've decided it isn't practical to implement, or because we've already implemented it all in PRs that were related to other issues.

"Closing with no action" isn't a reflection on the merits of the issue or the person who proposed it, it's a reflection of where a small group of people think time is best spent in order to finish the QT4 work in a practical amount of time. Leaving issues open when the working group doesn't seriously believe that work is necessary to address them in QT 4.0 gives a misleading impression of the amount of work left to do.

Anyone is free to say "no, I don't want to close this one, I still think it's critical for QT 4.0 and I believe I can persuade the rest of the working group that I'm right" or to re-open it (or open a similar issue) later, if other work has made the issue obviously now a good idea or so easy to implement that it would be foolish not to.

Language is hard. Spec-ese is hard. Together they're almost a complete impediment to understanding! 🤣

[ndw]

The CG agreed to close this issue without action at meeting 066