[MINOR] hoodie.datasource.write.row.writer.enable should set to be true.

[renshangtao]

Change Logs

Fix the mismatch between the value of hoodie.datasource.write.row.writer.enable and the document.

Impact

If the user does not find this configuration when sorting within the partition, the desired result will not be obtained.

Risk level

low

Documentation Update

No documentation update needed.

Contributor's checklist

• Read through contributor's guide
• Change Logs and Impact were stated clearly
• Adequate tests were added if applicable
• CI passed

[KnightChess]

does it cause error result?

[renshangtao]

does it cause error result?

Yes, when I tested clustering, I found that the files under the same partition after sorting can only ensure internal order of the files, but there is still no order between the files. The location code only found that this configuration and document settings are inconsistent

[KnightChess]

on, I got it, the default value in config is true. But I think it will not lead to the differences of sorting results

[renshangtao]

on, I got it, the default value in config is true. But I think it will not lead to the differences of sorting results

You can test it，if the value is false , it will create a RDDCustomColumnsSortPartitioner who's class description is "
A partitioner that does sorting based on specified column values for each RDD partition."

[bvaradar]

cc @nsivabalan .

Good catch. It looks like we cannot use the ConfigProperty directly due to circular dependency. Can you comb the codebase to see if there are similar cases ?

[nsivabalan]

guess this was intentional. unless user explicitly enables this config, we don't want to enable it for clustering.

[nsivabalan]

lets also consider issues like #8259 before we can make it default.

[bvaradar]

@renshangtao : Can you create a jira and add it to the PR description so that PR validation can succeed.

[hudi-bot]

CI report:

• 6758894 Azure: FAILURE

Bot commands

@hudi-bot supports the following commands:

• @hudi-bot run azure re-run the last Azure build

[xushiyan]

on, I got it, the default value in config is true. But I think it will not lead to the differences of sorting results

You can test it，if the value is false , it will create a RDDCustomColumnsSortPartitioner who's class description is " A partitioner that does sorting based on specified column values for each RDD partition."

@renshangtao Both RDDCustomColumnsSortPartitioner and RowCustomColumnsSortPartitioner should sort globally. If you observe sorting issue, then it's a different bug to be fixed. Flipping this default value here is irrelevant to sorting issue

[xushiyan]

as row-writing for clustering is introduced recently and we want to keep row-writing to false for longer to fully stablize it, will close this for now.