test(spark): add streaming source and writer support coverage

[yihua]

Describe the issue this Pull Request addresses

The Spark structured-streaming source and sink/writer support in hudi-spark-common had under-covered branches, notably the legacy incremental-relation streaming path in HoodieStreamSourceV1.

Summary and Changelog

• TestStreamingSource: new COW and MOR streaming-read tests that force the legacy incremental relation path in HoodieStreamSourceV1 (streaming read table version 6 with hoodie.file.group.reader.enabled=false), complementing the existing tests that cover the file-group-reader path. Reads are synchronized via processAllAvailable.
• TestStructuredStreaming: new test that injects a failing micro-batch and verifies HoodieStreamingSink swallows it under hoodie.datasource.write.streaming.ignore.failed.batch, completing the query without crashing and writing no commit.
• TestDataSkippingQuery: new column-stats data-skipping test over multiple base files with disjoint value ranges, covering equality, range, IN, and empty-match predicates for COW and MOR, with a data-skipping-off parity check.

Impact

Test-only change. No production code is modified.

Risk Level

none

Documentation Update

none

Contributor's checklist

• Read through contributor's guide
• Enough context is provided in the sections above
• Adequate tests were added if applicable

[hudi-agent]

⚠️ 🤖 This review was generated by an AI agent and may contain mistakes. Please verify any suggestions before applying.

Thanks for the contribution! This PR adds Spark test coverage for the legacy incremental streaming-read path, the HoodieStreamingSink ignore-failed-batch behavior, and column-stats data skipping across multiple base files. No correctness issues found. A few style/readability suggestions in the inline comments. Please take a look, and this should be ready for a Hudi committer or PMC member to take it from here. A couple of small naming nits below — records1/inputDF1 dangling suffixes and a test name capitalisation inconsistency.

cc @yihua

[hudi-agent]

🤖 nit: the test name starts with a capital "Test" while the other tests added in this same PR (e.g. "test cow stream source…") use lowercase — could you change it to "test column stats data skipping…" for consistency?

_{⚠️ AI-generated; verify before applying. React 👍/👎 to flag quality.}

[voonhous]

This is too nit, let's ignore.

[hudi-bot]

CI report:

• 38556e9 Azure: SUCCESS

Bot commands

@hudi-bot supports the following commands:

• @hudi-bot run azure re-run the last Azure build