Generalize lookup operator for function items

[ChristianGruen]

The current lookup operator is a specialized expression for maps and arrays. All kinds of data structures can be realized with functions, and maps and arrays are functions as well, so it would be pretty straightforward to extend the lookup operator to arbitrary function items:

Use Cases

Return name elements whose string values contain supplied substrings

declare variable $DOC := <xml>
  <name>Jack Daniels</name>
  <name>Jim Beam</name>
  <name>Johnny Walker</name>
</xml>;

let $names := function($key) {
  $DOC//name[contains(string(), $key)]
}
return $names?('Jack', 'Jim', 'Johnny')

(: result :)
<name>Jack Daniels</name>,
<name>Jim Beam</name>,
<name>Johnny Walker</name>

Return squares of supplied integers

let $square := math:pow(?, 2)
return $square?(1 to 5)

(: result :)
1, 4, 9, 16, 25

Remarks

• XPTY0004 must be raised if the wildcard * is specified as key, and if the input is neither a map nor an array.
• The extension could easily be combined with the extension for sequences (see [XPath] Introduce the lookup operator for sequences #50).

[martin-honnen]

The proposal seems fine. I only wonder why the example uses name[contains(string(), $key)] instead of simply name[contains(., $key)]. Just curious.

[ChristianGruen]

…just an arbitrary example, data(), text(), etc. would work as well (personally, I tend to use text() if I know that a single text node is to be addressed).

[dnovatchev]

This is really nice!

With this proposal implemented, then ? becomes the function-composition operator:

$f ? (1, 2, 3) is $f(1), $f(2), $f(3)

Then:

$g ? $f ? (1, 2, 3) is $g ? ($f(1), $f(2), $f(3)) is $g($f(1)), $g($f(2)), $g($f(3))

Thus:

$g ? $f is exactly the function composition of $f and $g .

We might need to specify that ? is right-associative.

Good result!

[michaelhkay]

I think the "?" operator as currently defined on maps and arrays is left-associative.

let $a := [map{'k':[1,2]}, map{'k':[3,4]}]
return $a ? k ? 2

only works if you interpret it as ($a ? k) ? 2; the reading $a ? (k ? 2) makes no sense.

Nevertheless, I think the interpretation of '?' as a mapped function call makes some sense.

One concern is that the '?' operator (in its general form, with a parenthesised expression on the RHS) atomizes the result of the RHS expression. That makes sense for array and map lookup, but it doesn't obviously make sense for function application.

Interestingly, I don't think it was ever the intention, but the current spec doesn't actually say explicitly that when used in the form $a?(expression), every item in $a must be a map or an array. Rather it says:

If the KeySpecifier is a ParenthesizedExpr, then the expression E?(S) is equivalent to
for $e in E, $s in fn:data(S) return $e($s)

which would work whenever $e is an arity-one function that accepts a single atomic value as an argument.

[dnovatchev]

Interestingly, I don't think it was ever the intention, but the current spec doesn't actually say explicitly that when used in the form $a?(expression), every item in $a must be a map or an array. Rather it says:

If the KeySpecifier is a ParenthesizedExpr, then the expression E?(S) is equivalent to
for $e in E, $s in fn:data(S) return $e($s)

which would work whenever $e is an arity-one function that accepts a single atomic value as an argument.

1. Which "current spec" are you referring to: 4.0 or 3.1 ?

2. Does this mean that $f?($x) is valid but not $f?$x ?

[michaelhkay]

I was referring to the 3.1 spec, but the current 4.0 draft also has this text.

Yes, in 3.1 $f?$x is a syntax error. In 4.0 the grammar allows it, but the semantics proposed in §4.14.3.2 don't seem right.

[michaelhkay]

It would certainly be possible (though clumsy) to change the rule so that atomization of the RHS only happens if the function signature requires an atomic value (which is true, of course, for maps and arrays). But it might be simpler and less error-prone if we allow the lookup operator only when the LHS is a function whose signature requires the (single) argument to be a singleton or optional atomic value.

[dnovatchev]

Interestingly, I don't think it was ever the intention, but the current spec doesn't actually say explicitly that when used in the form $a?(expression), every item in $a must be a map or an array. Rather it says:

If the KeySpecifier is a ParenthesizedExpr, then the expression E?(S) is equivalent to
for $e in E, $s in fn:data(S) return $e($s)

which would work whenever $e is an arity-one function that accepts a single atomic value as an argument.

1. Which "current spec" are you referring to: 4.0 or 3.1 ?
2. Does this mean that $f?($x) is valid but not $f?$x ?

I was referring to the 3.1 spec, but the current 4.0 draft also has this text.

Yes, in 3.1 $f?$x is a syntax error. In 4.0 the grammar allows it, but the semantics proposed in §4.14.3.2 don't seem right.

Quite interestingly, both BaseX 10.4 and Saxon 11.4 do not implement the current (3.1) rules, or is there something I am not understanding correctly?:

[michaelhkay]

The implementors of both Saxon and BaseX presumably read the section heading "The lookup operator for maps and arrays" which is all about maps and arrays, and failed to notice that for one particular case of the syntax, the rule that the LHS must be a map or array has been accidentally omitted.

[dnovatchev]

The implementors of both Saxon and BaseX presumably read the section heading "The lookup operator for maps and arrays" which is all about maps and arrays, and failed to notice that for one particular case of the syntax, the rule that the LHS must be a map or array has been accidentally omitted.

Shall we issue bugs then ? 😄

[dnovatchev]

The implementors of both Saxon and BaseX presumably read the section heading "The lookup operator for maps and arrays" which is all about maps and arrays, and failed to notice that for one particular case of the syntax, the rule that the LHS must be a map or array has been accidentally omitted.

Shall we issue bugs then ? 😄

Also, a test must be added to the test suite asserting that the expressions in the screenshots are evaluated successfully (and the result of the evaluation is 3)

[michaelhkay]

I've added an issue against the 3.1 specs at w3c/qtspecs#46. I think the intent is perfectly clear that the lookup operator is only designed to apply to maps and arrays, but there is a rule missing from the spec.

[ChristianGruen]

I agree that we should maintain the atomization of the RHS result, and not make the behavior dependent on the type of the LHS expression.

[ChristianGruen]

I would still be in favor of generalizing lookups to arbitrary functions – we had a use case for that in the past. I have concerns, however, that the introduction of the deep lookup operator ?? will lead to new questions that are difficult to answer.

If no one objects, I would tend to close this proposal.

[ndw]

The CG agreed to close this issue without action meeting 59.