[XSLT] Use of multiple predicates: order of evaluation

[michaelhkay]

I notice I added an example pattern to the draft XSLT4 spec match=".[. castable as xs:date][xs:date(.) le current-date()]" which is incorrect because processors are allowed to change the order of predicates, so you can't use the first predicate as a guard to stop the second predicate throwing an error. I've seen users fall over this (Saxon does sometimes reorder predicates). My instinct is to ban reordering of predicates; if you want to allow it, you can use the "and" operator. An alternative would be an "and" operator (say "and-also") with explicit ordering semantics, as in XPath 1.0.

[benibela]

I would ban any reordering that would lead to throwing errors

[ChristianGruen]

Banning sounds good.

[yamahito]

I have been thinking a while about this, and agree that banning is correct; consider:

preceding-sibling::*[1][@type='footnote']
preceding-sibling::*[@type='footnote'][1]

Even though no type error would be thrown, reordering of the predicates here significantly changes the meaning and result of the XPath statement.

[michaelhkay]

Obviously you're only allowed to reorder predicates where it doesn't affect the semantics of a non-error query. The debate here is about case where the only effect of reordering is on the detection or non-detection of errors, that is, the scenario within the scope of the infamous "Errors and Optimization" section of the spec.

[dnovatchev]

The debate here is about case where the only effect of reordering is on the detection or non-detection of errors

But we still can have (note the parens below):

(.[. castable as xs:date]) [xs:date(.) le current-date()]

and this enforces the wanted order of evaluation, right?

For me the problem is whether or not there should be "shortcutting". That is, should evaluation continue even after it has been determined that the value of the sub-expression, on which the predicate must be applied, is ()

Not sure if the Spec says anything about shortcutting in this case.

BaseX doesn't raise an error on this:

for $d in 10,
    $sub in $d[. castable as xs:date]
  return
     $sub[xs:date(.) le current-date()]

But throws an error ("[XPTY0004] Cannot convert xs:integer to xs:date : 10") on this:

for $d in  10
  return
    ($d[. castable as xs:date])[xs:date(.) le current-date()]

[ChristianGruen]

BaseX doesn't raise an error on this:

I noticed it also raises an error with the latest release of BaseX (because both queries are now rewritten to the same representation). The error results from a compilation step, which determines that the evaluation of the second predicate will never succeed for the given input.

[michaelhkay]

Parens don't enforce order of evaluation. The infamous rules in "Errors and Optimization" allow all expressions (with a few exceptions such as conditionals) to be rewritten to equivalent expressions without regard to the effect on error behaviour. Stripping parens is an obvious example.

As for your example, type errors are subject to different rules. A processor can report a type error statically for any expression if it can determine that execution of that expression would always fail; it doesn't have to establish that the expression will actually be evaluated.

[dnovatchev]

Parens don't enforce order of evaluation

Then what are they for?

[michaelhkay]

Parens override the default precedence of operators, they don't control order of evaluation. (a+b)+c can be rewritten as a+(b+c) because in the absence of errors the results are the same.

[dnovatchev]

Parens override the default precedence of operators, they don't control order of evaluation.

Yes. Then we probably need a special operator or function that would instruct the XPath processor that a sub-expression should be evaluated lazily. Such as:

.[. castable as xs:date] => lazy( [xs:date($z) le current-date()] )

It seems that (at least in this case) we can use the ! operator to achieve evaluation in the wanted order.
For example, instead of:

for $d in  10
  return
    $d[. castable as xs:date]![xs:date(.) le current-date()]

which produces an error in Saxon but is evaluated successfully in BaseX 9.5

we can have:

for $d in  10
  return
    $d[. castable as xs:date]  !  function($x) {$x[xs:date(.) le current-date()]}

And the latter expression is evaluated without raising an error, both in Saxon and BaseX.

So,

lazy(expression($arg1, $arg2, ..., $argN))

could be an abbreviation for:

! function($arg1, $arg2, ..., $argN) {expression($arg1, $arg2, ..., $argN)}

[ChristianGruen]

But throws an error ("[XPTY0004] Cannot convert xs:integer to xs:date : 10") on this:

The query yielded an error because the input was supplied statically. While that should be a rather uncommon use case, I decided to change the behavior of BaseX: If values are inlined at compile time, no error will be raised anymore, but the predicate expression will be replaced with an fn:error function, which will only be raised later on at runtime if the predicate will actually be evaluated.

[ChristianGruen]

@michaelhkay Thanks for my answer to #78 (comment). I’m continuing in the original issue:

Because some XQuery vendors translate operators such as and/or to other languages such as SQL, and reordering of terms in a WHERE clause to take maximum advantage of indexes is a key feature of SQL optimisers.

I see how this was a substantial argument for XQuery 1.0. I wonder if it’s still relevant for XQuery 4? XQuery 3/3.1 has so many constructs that cannot be expressed with standard relational algebra anymore. Do any SQL implementations for XQuery 3.1 exist at all?

Multiple predicates can easily be rewritten to a sequence of and operands, and vice versa. The same applies to multiple subsequent where clauses, so I always regarded all those as interchangeable:

(: currently equivalent if the tests are not positional :)
SEQ [ TEST1 and TEST2 ]

SEQ [ TEST1 ] [ TEST2 ]

for ITEM in SEQ
where TEST1 and TEST2
return RESULT

for ITEM in SEQ
where TEST1
where TEST2
return RESULT

If we restricted the evaluation order of predicates, those queries would not be equivalent anymore, and potential SQL implementations would also need to suppress these kinds of rewritings in the future.

From the user perspective, I’m pretty sure that the reordering of logical expressions is rather unexpected (as long as a developer doesn’t have a strong database background). I would claim there is already so much code out there that relies on a short-circuit evaluation of logical expressions – even more than for predicates, the syntax of which is not that explicit.