fix(metadata-table): follow-ups for the zero-size file skip (#18611)

[voonhous]

Describe the issue this Pull Request addresses

Follow-up to #18611, which added the opt-in config hoodie.metadata.skip.zero.size.files.on.initialize (skip zero-size data files while listing the data table during MDT initialization). #19243 was a duplicate submission of the same branch and collected review comments that apply to the merged code; it is now closed. This PR addresses those comments and carries over the E2E test from #19243 that never merged (cherry-picked with authorship preserved, thanks @lokeshj1703 @nada-attia).

Summary and Changelog

• Honor the config in the restore-sync relisting. Previously update(HoodieRestoreMetadata, ...) passed skipZeroSizeFiles=false unconditionally, so a restore would diff the filesystem against MDT, see the files skipped at initialization as FS-only, and re-add them via partitionFilesToAdd -- re-introducing the failure the config exists to avoid (e.g. the column-stats restore build reading a zero-size parquet footer). With the config enabled, restore now skips them too, and zero-size files already tracked in MDT are removed by the restore sync, converging MDT to the configured semantics.
• Document the operational trade-off in the config description: skipped files remain on storage and are not tracked by the metadata table or the cleaner; they must be removed manually, and the metadata validator will report them as inconsistencies.
• Correct the config sinceVersion to 1.3.0 (it first ships in 1.3.0, not 1.2.0).
• Move the metric name into HoodieMetadataMetrics.SKIPPED_ZERO_SIZE_FILES_ON_INITIALIZE_STR alongside the other metric-name constants, and only emit it when files were actually skipped (previously an unconditional 0 was emitted on every listing, including restore).
• Reduce log noise: one aggregated WARN per partition with the skipped count; individual file paths moved to DEBUG.
• Add a Scala E2E test (TestRecordLevelIndex.testUpsertAfterSkippingZeroSizeFileOnInitialize): after a zero-size base file is skipped during MDT bootstrap with global RLI, an upsert of the affected records succeeds, routes them to a new file group (never the zero-size one), and leaves RLI consistent with the data.

Impact

None by default. With the config enabled, restore-sync now treats zero-size files the same way initialization does instead of re-adding them to MDT. The skipped_zero_size_files_on_initialize metric is now only emitted when non-zero.

Risk Level

low. All behavior changes are gated behind the default-off config; the metric/logging changes are observability-only.

Documentation Update

None beyond the updated config description (the website config reference is generated from it).

Contributor's checklist

• Read through contributor's guide
• Enough context is provided in the sections above
• Adequate tests were added if applicable

[hudi-bot]

CI report:

• c2767ce Azure: SUCCESS

Bot commands

@hudi-bot supports the following commands:

• @hudi-bot run azure re-run the last Azure build

[wombatu-kun]

The test-spark-java-tests-part1 (spark3.3) failure here is the known flaky TestSparkFilterHelper.testConvertInExpression (fails with LiveListenerBus is stopped), a pre-existing test-session lifecycle issue unrelated to this PR's changes. The fix is already up and approved in #19356, so re-running that single job should bring CI green.