Improve support for handling undefined inside of queries

[tsandall]

It's often necessary to select a nested field out of a document but fallback to a default value if the referenced field is undefined. In the past users could write a helper function to assist with this, however, over time we ended up adding the object.get built-in function to make it easier.

Programming languages don't typically include this type of feature--it's just implemented in libraries. Looking at query languages for hierarchical data:

• jq uses // for it's alternative operator
• XPath 2.0 allows you to construct a sequence. If an expression is undefined, it's not included in the sequence (roughly). So you can say (foo, bar)[1] and if foo is undefined, the return value is bar (XPath indices are 1-based.)

One issue with object.get is that it only works one level at a time. For example:

a := object.get(input, "foo", {})
b := object.get(input, "bar", {})
c := object.get(input, "baz", 7)

You could improve this using something like walk:

c := object.select(input, "foo.bar.baz", 7)

One option would be to include some new operator for providing alternatives, e.g., following jq:

c := input.foo.bar.baz // 7

Some questions:

• Can alternative operator be applied to non-ref terms or expressions? If it's applied to expressions, it's tempting to make it catch false as well.
• What about allowing default to be used against references (similar to with)?

[jaspervdj]

Throwing in my $.50, an alternative operator that can be applied to expressions is very close to an inline-style OR (as opposed to incremental definitions). This would be a useful quality of life improvement for situations where where you need to resort to an extra auxiliary rule now. It also doesn't introduce any new semantics to the language since it can be rewritten as auxiliary rules.

I like //; but I also like || (despite | already being a bit confusing for the parser).

else / else = should also be considered since the discussed semantics are close to what else does already and the keyword is free to be used in this context.

[aholmis]

I think this feature would be great, also for when calling functions that could be undefined.
As I'm from the C# world, I'm used to ?? as
isOk := checkUser(input.user) ?? false

[anderseknert]

With object.get getting extended to support nested keys, I wonder if we're still going to need a special operator for this use case?

[aholmis]

Will that object.get solve my case with functions returning undefined?
Let's say I want result to be true or false in this setting, but sometimes true or undefined as the default behavior is.
result := validate_user(input.user)
Now I've made this as
validate_user(user) { user.age == 10 } else = false
So I'm missing the default undefined behavior. I would like to keep the function as is, but sometimes transform the undefined output.

[anderseknert]

No, object.get won't work on functions directly, but you could assign the result and use that. But yeah, I guess a more generally applicable operator would help with that.

I'm not sure I follow your example completely though. Could you provide a complete one where you show what you want to accomplish.. where that something is currently cumbersome? 🙂

[aholmis]

If you see here: https://play.openpolicyagent.org/p/DTC58WYnFl
I would like to have "aa" and "bb" as true or false, using functions
That doesn't work now, as soon as a function is undefined, the whole rule is undefined.
Sometimes that is useful, and sometimes not.
If you can point me to another solution, I'm happy :)

[anderseknert]

Thanks @aholmis, I think I understand, although I'm not sure I got how an "undefined operator" would help here. Undefined as introduced by unknown attributes in input or data is a good candidate for object.get:

canbe(thing) { thing == 0 }

canbe(input.count)

-->

canbe(thing) { thing == 0 }

canbe(object.get(input, ["count"], -1))

Undefined as introduced by a function that doesn't evaluate can be dealt with either like you did with else, or by adding a mutually exclusive "else" condition to the function:

canbe(thing) = true { thing == 0 }
canbe(thing) = false { thing != 0 }

If we had an "undefined operator" we could do something like this, which would be pretty nice I guess :)

o := {
  "x": f(x) // false
}

[aholmis]

o := {
"x": f(x) // false
}

This is what I would like, because then you don't modify the function itself, you just adapt the usage.

canbe(object.get(input, ["count"], -1))

This would still be undefined and cause the rule to be undefined as well, because the function body does not evaluate to true

[aholmis]

That said, it is not a big problem for us (I just found this issue on github), and the amount of work required to introduce a new operator might be too much?