[core] Fix orphan deletion-vector index entries when full compaction goes through FileRewriteCompactTask

[ZZZxDong]

Purpose

Fixes #8778.

When a full compaction finds a bucket containing only max-level files, CompactStrategy.pickFullCompaction picks the files carrying deletion vectors and builds a CompactUnit with fileRewrite=true, which is executed by FileRewriteCompactTask.

Inside the task the rewriter correctly calls notifyRewriteCompactBefore → dvMaintainer.removeDeletionVectorOf(...) for every rewritten file, so the maintainer's in-memory state is updated. However, unlike MergeTreeCompactTask, FileRewriteCompactTask never received the compactDfSupplier and never called result.setDeletionFile(...), so the updated deletion-vector index was never written or attached to the CompactResult. The commit then contained no index-manifest change for this bucket, and IndexManifestFileHandler kept the previous (now stale) DV entry — an orphan entry pointing to data files that no longer exist.

This was introduced by #5751 which extracted FileRewriteCompactTask from the old code path but dropped the compactDfSupplier handling.

This PR passes compactDfSupplier into FileRewriteCompactTask and calls result.setDeletionFile(compactDfSupplier.get()) at the end of doCompact(), mirroring MergeTreeCompactTask.

Tests

Added DeletionVectorIndexCleanupTest#testFullCompactionCleansDvIndex, which reproduces the write-only + standalone compaction pipeline:

1. write rows, full compact → data at max level
2. write -D rows → L0
3. minor compact → DV generated for the max-level file
4. full compact → data files rewritten; assert no DV index entry points to dead files

The test fails on current master and passes with this fix. Also ran *Compact*Test / DeletionVector*Test / *DeletionVectors*Test in paimon-core (136 tests) — all green.

API and Format

No public API or storage format change. Only metadata hygiene: the consumed DV index entries are now correctly removed by the commit of the full compaction.

Documentation

N/A

[JingsongLi]

+1

[JingsongLi]

Both Flink tests failed, please ensure CI pass.

[ZZZxDong]

The failing job is "UTCase and ITCase Spark 3.x" (scala 2.13; the 2.12 one was cancelled by fail-fast), while this PR only touches paimon-core compaction. The same workflow on master shows 4 failures in the last 8 runs, so it looks flaky. Could you help re-trigger the CI? Thanks!

[ZZZxDong]

@JingsongLi Could you help re-trigger the failed CI job when you have a moment? It appears to be a flaky Spark 3.x test suite unrelated to this change (details above). Thanks!

[JingsongLi]

+1