[ios, macos] Add predicate like expressions to NSExpression

[fabian-guerra]

Fixes #11013

Add the following operators to NSExpression.

• BETWEEN
• IN
• CONTAINS

[1ec5]

It’s unnecessary to provide a custom expression function for the BETWEEN operator. This operator is specific to NSPredicate and can’t be used inside an NSExpression. There’s already a way to use the BETWEEN operator without a format string:

[NSComparisonPredicate predicateWithLeftExpression:… rightExpression:… modifier:NSDirectPredicateModifier type:NSBetweenPredicateOperatorType options:0]

In general, we should be using NSPredicate itself, rather than NSExpression, to represent Boolean-typed mbgl expressions, which is why I didn’t bother to create an aftermarket function for converting to Boolean values in #11472.

[1ec5]

Create an NSComparisonPredicate with NSBetweenPredicateOperatorType rather than an NSExpression.

[1ec5]

Instead of adding a case here, add a case to -[NSComparisonPredicate(MGLAdditions) mgl_jsonExpressionObject] for NSBetweenPredicateOperatorType.

[1ec5]

Good catch.

[1ec5]

This variable name is misleading. The right expression contains the minimum and maximum bounds, so bounds or limits would be a better name.

[1ec5]

BETWEEN, CONTAINS, and IN are most commonly written with aggregate expressions inline, not with arrays as constant values. For example, in the format string a BETWEEN {1, 3}, {1, 3} is an aggregate expression whose collection is an array. It’s important to support aggregate expressions with these three operators, although supporting constant value arrays is also nice to have.

[1ec5]

The following JSON object:

[
  "all",
  ["<=", ["get", "a"], 3],
  ["<=", 1, ["get", "a"]]
]

should convert to a BETWEEN {1, 3}. Instead, this code turns it into 3 BETWEEN {a, a}.

[1ec5]

The following JSON object:

[
  "all",
  [">=", 3, ["get", "a"]],
  ["<=", 1, ["get", "a"]]
]

should convert to a BETWEEN {1, 3}. Instead, this code turns it into 3 BETWEEN {a, a}.

[1ec5]

All the following cases (in pseudocode) should be translated to an expression with the format string a BETWEEN {1, 3} or a BETWEEN {3, 1}:

• a >= 1 AND a <= 3
  • 1 <= a AND a <= 3
    • 1 <= a AND 3 >= a
  • a >= 1 AND 3 >= a
• a <= 3 AND a >= 1
  • 3 >= a AND a >= 1
    • 3 >= a AND 1 <= a
  • a <= 3 AND 1 <= a

[1ec5]

The following JSON objects:

[
  "match",
  ["get", "a"],
  1, true,
  2, true,
  3, true,
  4, true,
  false
]

[
  "match",
  ["get", "a"],
  [1, 2, 3, 4], true,
  false
]

should translate to an NSExpression with the format string a IN {1, 2, 3, 4}. We don’t currently support translating an NSExpression to a match that consolidates cases like that (#11539), but I think it’s possible for us to encounter such an expression here if the style JSON defines it that way.

Not all match expressions can be represented by the IN operator, only expressions that have two unique output values, true and false.

[1ec5]

A has expression checks whether an object has a particular property; it doesn’t check whether an array contains a particular object. We need to convert a match expression to either the CONTAINS or IN operator (pick one), but we shouldn’t convert a has expression to either operator.

[1ec5]

A CONTAINS predicate should be translated into a match expression. For example, an NSExpression with the format string {1, 2, 3, 4} CONTAINS a should produce the following JSON object:

[
  "match",
  ["get", "a"],
  [1, 2, 3, 4], true,
  false
]

[1ec5]

A match expression takes different arguments than an IN predicate. An NSExpression with the format string a IN {1, 2, 3, 4} should produce the following JSON object:

[
  "match",
  ["get", "a"],
  [1, 2, 3, 4], true,
  false
]

[1ec5]

The right expression is more likely to be an aggregate expression than a constant value expression. An expression with the format a BETWEEN {1, 3} will crash here.

[fabian-guerra]

This works for aggregate expressions. In https://github.com/mapbox/mapbox-gl-native/pull/11632/files#diff-6753380c30f27dfd9377e67fca76d44cR581 I added the test.
Wether is a NSConstantValueExpressionType — for this case — or NSAggregateExpressionType the underlaying value is an `NSArray.

[1ec5]

a BETWEEN {1, 3} and a BETWEEN {3, 1} are equivalent, but this code turns the latter into a >= 3 AND a <= 1, which is always false. That said, I don’t think -mgl_filter has ever handled this case correctly either – I just learned today that it’s possible to reverse the bounds this way – so it’d be fine to address the issue separately.

Musing a bit, I can think of two ways to approach the problem, both of them a bit uglier:

• Cover both scenarios explicitly: (a >= 3 AND a <= 1) OR (a <= 3 AND a >= 1)
• Ensure the correct order: a >= min({3, 1}) AND a <= max({3, 1})

[fabian-guerra]

I think the reverse format we can follow up later. I was adding a predicate to ensure the correct order, but evaluating a >= min({3, 1}) AND a <= max({3, 1}) will not work all the time.

[1ec5]

Oh, I meant that we could translate a BETWEEN {b, c} to a >= min({b, c}) AND a <= max({b, c}) then convert to JSON. I don’t see why that wouldn’t work, but it certainly would complicate any attempt to round-trip back to NSPredicate, if we decide later on that that would be desirable.

[fabian-guerra]

What I meant is if you evaluate the predicate a >= min({3, 1}) AND a <= max({3, 1}) to ensure that BETWEEN is in the correct order it won't work since evaluating to a keyPath a does not contain any value at that moment. If we evaluate a constant value like 2 >= min({3, 1}) AND 2 <= max({3, 1}) then it will.

[1ec5]

Couldn’t we do something like this?

[NSPredicate predicateWithFormat:@"%@ >= min(%@) AND %@ <= max(%@)", self.leftExpression, self.rightExpression, self.leftExpression, self.rightExpression];

[1ec5]

It’s unclear to me why the first parameter is named operand and the second is named options. Based on how this method is being used, I think one parameter could be named inputExpression and the other valuesExpression.

Better yet, we can inline this function under the NSInPredicateOperatorType case above. Then the case for NSContainsPredicateOperatorType could recurse, similar to the following:

NSPredicate *inPredicate = [NSComparisonPredicate predicateWithLeftExpression:self.rightExpression
                                                              rightExpression:self.leftExpression
                                                                     modifier:self.modifier
                                                                         type:NSInPredicateOperatorType
                                                                      options:self.options];
return inPredicate.mgl_jsonExpressionObject;

[1ec5]

For an operator like >= or <=, the modifier should be NSDirectPredicateModifier. This SDK’s NSPredicate implementation has so far never supported NSAllPredicateModifier, which is for an expression like a > ALL {1, 2, 3}, which means “a is greater than each of the elements in {1, 2, 3}”.

[1ec5]

The right side of an IN predicate or the left side of a CONTAINS predicate is more likely to be an aggregate expression than a constant array value.

[fabian-guerra]

I know but applies the same as in #11632 (comment)

[1ec5]

This is correct, but I think [elements addObject:@YES] would be clearer.

[fabian-guerra]

This was changed to:

while (id object = optionsEnumerator.nextObject) {
                id option = ((NSExpression *)object).mgl_jsonExpressionObject;
                [elements addObject:option];
                [elements addObject:@YES];
            }
            [elements addObject:@NO];

But for some reason is not refreshed in github.

[1ec5]

Did you mean to check whether leftConditionExpression is non-nil?

[fabian-guerra]

Yes leftConditionExpression

[1ec5]

Starting from here, I think it would be more concise to work with the NSPredicates in subpredicates, then only call +[NSExpression mgl_expressionWithJSONObject:] at the end.

[1ec5]

A while loop with an NSEnumerator is only useful when you need to also work with indices, which isn’t the case here. (And in that case, you could use the more modern -enumerateObjectsAtIndexes:options:usingBlock: anyways.) Replace the while loop with a for-in loop, which takes advantage of fast enumeration.

[1ec5]

From this line onward, there should be no references to jsonObjects. Use subpredicates instead.

/ref #11632 (comment)

[1ec5]

Check whether the first and last subpredicates are both NSComparisonPredicates, then declare leftPredicate and rightPredicate variables of type NSComparisonPredicate.

[1ec5]

Compare leftPredicate.predicateOperatorType == NSGreaterThanOrEqualToPredicateOperatorType instead of comparing strings. I don’t think there would be a significant performance difference either way, but we’d benefit from stronger type checking.

[1ec5]

I think we want ["match", ["get", "a"], 1, true, 2, true, 3, true, 4, true, false] to end up as MGL_MATCH(a, 1, true, 2, true, 3, true, false) == true, so the type should be NSEqualToPredicateOperatorType.

[1ec5]

Add tests that convert match expressions to NSPredicates.

[fabian-guerra]

What is the purpose of this test? comparing directly match expressions to match expression predicates will not work because we add the condition ... == YES

[1ec5]

Right, what I mean is that we should test that behavior, to verify that #11632 (comment) is fixed.

[1ec5]

Looks like you already added these tests, never mind.

[1ec5]

A more concise way to write predicateAfter would be [NSPredicate predicateWithFormat:@"MGL_MATCH(x, 6, YES, 5, YES, 4, YES, 3, YES, NO) == YES"], so a separate -matchPredicateWithFormat: method would be unnecessary.

[1ec5]

This case is written generically enough to be a catch-all for any operator not handled above. So we can have the code below run unconditionally and remove the assertion at the bottom of the method.