test: add unit test for multiple partition filters on same column

[suryaprasanna]

Describe the issue this Pull Request addresses

Initially, we observed partition filters being dropped in our production systems when multiple conditions were applied to the same partition column (e.g., datestr > '2016-01-01' AND datestr < '2016-12-31'). However, when creating a unit test to reproduce the issue, the filters were correctly preserved without any code changes. Since we could not reliably reproduce the issue in the test environment, this PR adds test coverage to prevent potential regressions.

Summary and Changelog

Adds testMultiplePartitionFiltersPushDown test in TestHoodiePruneFileSourcePartitions to validate that multiple partition filters (IsNotNull, GreaterThan, LessThan) on the same column are preserved during query optimization. Test covers both COW and MOR table types with range queries.

Impact

Adds test coverage to ensure partition filter correctness, helping prevent potential filter loss issues in the future. No user-facing changes.

Risk Level

none - Test-only change, no production code modifications.

Documentation Update

none

Contributor's checklist

• Read through contributor's guide
• Enough context is provided in the sections above
• Adequate tests were added if applicable

[voonhous]

Possible to add an FT for this? Thank you

[nsivabalan]

thanks for the fix.
we need to add tests please.

[suryaprasanna]

@voonhous @nsivabalan
In our systems, we noticed partition filters getting dropped when multiple conditions are added for the same column.
Ex: Where condition like WHERE datestr >= '2025-01-01' and datestr <= '2025-01-31'.
What we noticed is filter dropped one of them,s o we removed the usage of ExpressionSet. But somehow when I created the unit test, it is working without the fix as well.
Not sure, why the error showed up in our systems. Can it because we compiled the code using Java 8 and running with Java 11?

[suryaprasanna]

If the build passes, then I will reword the PR to include only the unit test.

[voonhous]

@voonhous @nsivabalan In our systems, we noticed partition filters getting dropped when multiple conditions are added for the same column. Ex: Where condition like WHERE datestr >= '2025-01-01' and datestr <= '2025-01-31'. What we noticed is filter dropped one of them,s o we removed the usage of ExpressionSet. But somehow when I created the unit test, it is working without the fix as well. Not sure, why the error showed up in our systems. Can it because we compiled the code using Java 8 and running with Java 11?

No idea, i was trying to reproduce this too and couldn't. Which is why i asked for a unit test to be added so i could repro it... Hmmm, nonetheless, we can change this PR to a test PR to add test coverage.

[suryaprasanna]

@voonhous @nsivabalan I reworded the description, can you please review this again?
Thanks!

[nsivabalan]

oh, can you parametrize to run this for both version 6 and 9.

[suryaprasanna]

Yes, made the change.

[hudi-bot]

CI report:

• 41e4e96 Azure: SUCCESS

Bot commands

@hudi-bot supports the following commands:

• @hudi-bot run azure re-run the last Azure build