Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix array predicates in JSON path filters #4199

Merged

Conversation

kwvanderlinde
Copy link
Collaborator

@kwvanderlinde kwvanderlinde commented Jul 5, 2023

Identify the Bug or Feature request

Fixes #3358

Description of the Change

We now use GSON as the mapping provider in addition to the JSON provider when using JsonPath. This allows array values to be built by JsonPath, where before it was unable to instantiate List.

The only remaining non-GSON representation is JSONSmart, used by JsonPath to parse literals in the path expression. This is hardcoded, and unfortunately results in some inconsistencies noted below.

I also included some cleanup to eliminate JsonParser instances, as those are deprecated and we don't need them anyways.

Possible Drawbacks

There will no doubt be some confusion over the exact behaviour of array predicates. Most predicates should just work as expected (e.g., anyof, equality between numbers, etc), but specifically equality checks on numeric arrays are a problematic case that I don't think we can just fix. Such predicates will require the integral literals to be written with a decimal point so that JSONSmart treats them as doubles to match what the GSON mapping provider produces for the other operand. Using integers, the predicate will never match.

Documentation Notes

Given this data set:

[h: data = json.append("[]",
	json.set("{}","key",1,"array",json.append("",2,3)),
	json.set("{}","key",2,"array",json.append("",1,2,3))
)]

The following expressions will work to filter an array:

[r: json.path.read(data, "\$[?(@.array == [2.0, 3.0])]")]
[r: json.path.read(data, "\$[?(@.key in [2,3])]")]
[r: json.path.read(data, "\$[?(@.key == 1)]")]
[r: json.path.read(data, "\$[?(@.key == 1.0)]")]
[r: json.path.read(data, "\$[?(@.array anyof [2,3])]")]
[r: json.path.read(data, "\$[?(@.array anyof [2.0,3.0])]")]

But this one will not match any elements since the integers inside the array are not considered equal to the floats used internally for the array being matched:

[r: json.path.read(data, "\$[?(@.array == [2, 3])]")]

Release Notes

  • Fixed json.path functions so that array predicates can be used.

This change is Reviewable

The `JsonParser` class no longer needs to be instantiated and should be used purely through the static methods. Most of
our code already does this, but a few places held into `JsonParser` instances for no gain.
We already used the GSON JSON povider, and now we also use GSON from mapping. This gives more consistency in the
representation, which allows certain predicates to work which did not work before.

There is still one incosistency hardcoded in JsonPath which uses JSONSmart to parse literals. Since JSONSmart will try
to represent integral literals as Java `Integer`s, but GSON just represents all numbers as Java `Double`s, we now have
to write our integral literals will decimal points in order for them to work in all circumstances.
@cwisniew cwisniew added this pull request to the merge queue Jul 5, 2023
Merged via the queue into RPTools:develop with commit 816e2a1 Jul 5, 2023
4 checks passed
@cwisniew cwisniew added the bug label Aug 2, 2023
@kwvanderlinde kwvanderlinde deleted the bugfix/3358-json-path-array-filter branch August 31, 2023 02:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
Status: Merged
Development

Successfully merging this pull request may close these issues.

[Bug]: json.path's filter expressions cannot compare arrays
2 participants