Keyword for functions that can produce multiple bindings

[HolgerKnublauch]

Currently, BIND assignments can produce at most one binding for a single variable.

Many SPARQL implementations have a notion of "special" or "magic" predicates that can be used to bind multiple variables at once, and to produce multiple rows at once. Examples include Jena's built-in magic properties/property functions (e.g. https://jena.apache.org/documentation/query/extension.html#property-functions) and various free-text search implementations that take a string as input and produce all subjects that have values. Further, some implementations even use the SERVICE keyword as a hack, e.g. SERVICE wikibase:label. From our own modest experience there are tons of use cases for such multi-binding functions.

Even if SPARQL 1.2 doesn't include any specific built-in multi-binding functions, it would be great to have a proper syntax and an extension point so that these special functions can be consistently identified, supported by syntax-directed editors etc.

I struggle to find a proper keyword, so as a placeholder I am using ITERATE in this proposal:

SELECT * WHERE { ... ITERATE (ex:myFunction(?inputA, ?inputB, ...) AS ?outputA, ?outputB, ...) . }

The above would produce an iterator of bindings evaluating whatever is implemented by ex:myFunction. The right hand side would have unbound variables that get bound in each iteration. It's similar to BIND otherwise.

As related work, SHACL-AF includes a mechanism to declare new SPARQL functions that wrap ASK or SELECT queries. Obviously this could be extended to turn SELECT queries into reusable "stored procedures" that can then be reused in other queries using the new keyword.

[cygri]

A very common use case for this is splitting a string on a delimiter. In Jena this can be done with the apf:strSplit property function:

?segment apf:strSplit (?s ", ")

This works, but the property function syntax is pretty obscure and is really an abuse of triple pattern syntax.

So let us consider other syntax options. Extending the BIND keyword for this looks much clearer:

BIND (split(?s, ", ") AS ?segment)

The idea is to extend BIND to allow producing multiple values, leading to multiple result solutions per input solution. One could even imagine something like:

BIND (concat("segment ", split(?s, ", ")) AS ?segment)

split returns multiple values, and the concat function would be called for each result, again leading to multiple result solutions per input solution. (It's less clear how that proposal would work so nicely if the output of the function is multiple variables like in Holger's example.)

[rvesse]

+1000

This is one of my big bug bears with the current syntax and the abuse of it by magic properties. You definitely want to list out all the resulting variables so that it is clear what the output columns would be although you probably also want some way to have implicit outputs.

A lot of existing "magic" properties are used to join extra info or filter the existing result set so the input variables are implicitly included in the output variables in many cases

[HolgerKnublauch]

I am aware that some magic properties are supposed to work "in all directions", i.e. regardless of which combination on left/right side is present. I don't think this (complex) contract is often needed in practice, so I'd suggest we focus on a clear separation between input and output.

[jindrichmynarz]

Would allowing BIND to produce multiple bindings (e.g., via a SPARQL extension function) break the current evaluation model?

[cygri]

Would allowing BIND to produce multiple bindings (e.g., via a SPARQL extension function) break the current evaluation model?

In current SPARQL, the argument to BIND is an expression, and expressions return exactly one value (or an error). Extension functions are one kind of expression, so they cannot return multiple values. Allowing multiple return values from an expression would be a change to the evaluation model. It also raises the question what would happen in other contexts where expressions are used, like in FILTER, in SELECT ... (expr AS ?var) ..., or in ORDER BY.

[rvesse]

I think you would definitely need a different keyword to distinguish the existing row-in row-out semantics of BIND from this use case

[cygri]

I think you would definitely need a different keyword to distinguish the existing row-in row-out semantics of BIND from this use case

Why? If the semantics of BIND were changed to multiple-solutions-out-per-solution-in, then the current one-solution-out-per-solution-in semantics would continue to work as an edge case. Expressions that return zero or more results (as all SPARQL 1.1 expressions do) would continue to behave as they do today. So it's a clean extension even without a new keyword.

I'm more worried about other uses of expressions like FILTER, where it's less obvious how to handle expressions with multiple results.

[joernhees]

i agree with the reasoning about reusing BIND (as long as we're not in the projection part anyhow).

Wrt. to tuple asignments: how about something like expr AS (?v1 ?v2) (so additional parenthesis denote an explicit tuple assignment, a AS ?v1 could become syntactic sugar for AS (?v1))?

[dydra]

I'm more worried about other uses of expressions like FILTER, where it's less obvious how to handle expressions with multiple results.

would that not be just a matter of the respective function signatures?
they would need to agree between argument results and function parameters.
if the effective boolean were extended to define a truth value for result sets, that would close it.

[dydra]

allegro provides one example as to how to approach this "signature" aspect of this issue. it permits a list of terms as both subject and object respective a functional predicate term. they call them "magic" properties.

[cygri]

I am losing confidence in my proposal to extend BIND for this. It gets confusing because these solution sequence returning functions are quite different from normal SPARQL functions, and I don't think we want to imply that they can be used anywhere that normal SPARQL functions/expressions can be used.

Then how about extending a different existing keyword?

VALUES ?item OF split(?items, ", ")
VALUES (?ns ?local) OF splitIRI(?s)
VALUES (?col1 ?col2 ?col3) OF my:readCSV(<data.csv>)

So the result is a solution sequence—just like with the current VALUES clause—but the bound values are provided by the function, rather than listed literally in the clause.

[dydra]

how about

{ (?col1 ?col2 ?col3) my:readCSV (<data.csv>) 
... }

?

[cygri]

@dydra Did you read the issue? That is exactly what everyone is doing at the moment, and the point of this issue is that we want to get away from that abuse of triple pattern syntax.