[HUDI-8285] implement row writer cluster with fg reader

[jonvex]

Change Logs

Implement row writer clustering by adding fixed file index to spark datasource

Impact

Clustering with full read featureset

Risk level (write none, low medium or high below)

low

Documentation Update

N/A

Contributor's checklist

• Read through contributor's guide
• Change Logs and Impact were stated clearly
• Adequate tests were added if applicable
• CI passed

[hudi-bot]

CI report:

• c1a3e63 UNKNOWN
• af41953 Azure: FAILURE

Bot commands

@hudi-bot supports the following commands:

• @hudi-bot run azure re-run the last Azure build

[danny0405]

what's the relationship with this PR: #12395

[jonvex]

This is a contingency incase that other pr got stuck. Both have the same end goal of clustering with the new fg reader, but different approac. In that pr, we use the fg reader directly in spark clustering to read the filegroups. This pr implements a new index so that we can use spark datasource to read all the filegroups.