Enhance Metadata tests to validate col stats #15951
Description
I happened to check for col stats tests in TestHoodieBackedMetadata and did not find any good tests that validates col stats records from MDT and bloom partition records. I do see some validations for log files, but none for base files. we do have one test named testColStatsPrefixLookup which tests col stats. but if we can add a separate validateColStats() method we can call it with validateMetadata() for any tests we have to gain more coverage.
We do a sparate test class TestColumnStatsIndex for testing col stats. But would be good to enhance the validations in TestHoodieBackedMetadata tests.
we should enhance them for sure.
JIRA info
- Link: https://issues.apache.org/jira/browse/HUDI-6206
- Type: Improvement