fix(compaction): avoid stranded remainder fragments

[lance-gatefixer]

Summary

• derive target-scale output counts from surviving rows and balance remainders across outputs
• keep ordinary planner-sized compaction tasks in one fragment without allowing oversized tasks to collapse into one file
• cover both the reported small-fragment layout and an oversized deletion rewrite next to a small fragment

Root cause

The planner already groups adjacent small fragments into tasks with approximately target_rows_per_fragment rows. Execution applied that target again as a hard max_rows_per_file, so each task above the target produced a full fragment followed by a small remainder that could become isolated. Removing the cap solely based on source-fragment count then let an indivisible oversized source collapse into one arbitrarily large output.

The revised executor computes the output count from logical surviving rows and balances the tail across that count. Planner-sized tasks still produce one output, while genuinely oversized tasks remain target-scale.

Validation

• cargo test -p lance --lib dataset::optimize::tests (123 passed)
• cargo fmt --all
• cargo clippy --all --tests --benches -- -D warnings
• make install and make build from python/
• uv run pytest python/tests/test_scalar_index.py::test_zonemap_index_remapping
• uv run make lint

Fixes #8506