ARROW-17483: [Python] Support Expression filters in non-legacy ParquetDataset/read_table

[milesgranger]

No description provided.

[github-actions]

https://issues.apache.org/jira/browse/ARROW-17483

[AlenkaF]

Looks good!

As expressions get supported with this PR I guess expressions with nested fields also work? Could we add that in the tests, something like:

integer_keys = [0, 1, 2, 3, 4]
df = pd.DataFrame({
    'index': np.arange(len(integer_keys)),
    'integers': np.array(integer_keys, dtype='i4'),
    'nested': np.array([{'a': j % 3, 'b': str(j % 3)} for j in range(5)])
}, columns=['index', 'integers', 'nested'])

and add pc.field("nested", "b") == 1 to the fixture?

I haven't tested it, but am curious if this works.

[milesgranger]

Thanks @AlenkaF!
Extended as suggested (it works) with some slight modifications, just to keep the filter comparison operation the same throughout the fixtures if that's acceptable.

[milesgranger]

@jorisvandenbossche, as you're the one who pointed me to this issue, would you like to take a gander?

[jorisvandenbossche]

Thanks! Some minor comments

[jorisvandenbossche]

Thanks!

[ursabot]

Benchmark runs are scheduled for baseline = ef8cb09 and contender = 8ceb3b8. 8ceb3b8 is a master commit associated with this PR. Results will be available as each benchmark for each run completes.
Conbench compare runs links:
[Finished ⬇️0.0% ⬆️0.0%] ec2-t3-xlarge-us-east-2
[Finished ⬇️0.37% ⬆️0.0%] test-mac-arm
[Failed ⬇️4.51% ⬆️0.0%] ursa-i9-9960x
[Finished ⬇️0.46% ⬆️0.11%] ursa-thinkcentre-m75q
Buildkite builds:
[Finished] 8ceb3b8a ec2-t3-xlarge-us-east-2
[Finished] 8ceb3b8a test-mac-arm
[Failed] 8ceb3b8a ursa-i9-9960x
[Finished] 8ceb3b8a ursa-thinkcentre-m75q
[Finished] ef8cb099 ec2-t3-xlarge-us-east-2
[Finished] ef8cb099 test-mac-arm
[Failed] ef8cb099 ursa-i9-9960x
[Finished] ef8cb099 ursa-thinkcentre-m75q
Supported benchmarks:
ec2-t3-xlarge-us-east-2: Supported benchmark langs: Python, R. Runs only benchmarks with cloud = True
test-mac-arm: Supported benchmark langs: C++, Python, R
ursa-i9-9960x: Supported benchmark langs: Python, R, JavaScript
ursa-thinkcentre-m75q: Supported benchmark langs: C++, Java

[ursabot]

['Python', 'R'] benchmarks have high level of regressions.
ursa-i9-9960x