Skip to content

[MINOR] Fix parameter parsing in hudi utilities#7435

Merged
YannByron merged 1 commit intoapache:masterfrom
Zouxxyy:xinyu/fix-util-param
Jan 6, 2023
Merged

[MINOR] Fix parameter parsing in hudi utilities#7435
YannByron merged 1 commit intoapache:masterfrom
Zouxxyy:xinyu/fix-util-param

Conversation

@Zouxxyy
Copy link
Contributor

@Zouxxyy Zouxxyy commented Dec 12, 2022

Change Logs

Some values may contain '=', such as the partition path, when using --hoodie-conf to add configuration, the verification will fail, for example:

spark-submit \
--class org.apache.hudi.utilities.HoodieClusteringJob \
hudi-utilities-bundle_2.11-0.12.0.jar \
--mode scheduleAndExecute \
--base-path xxx \
--table-name xxx \
--spark-memory 16g \
--hoodie-conf hoodie.clustering.plan.partition.filter.mode=SELECTED_PARTITIONS \
--hoodie-conf hoodie.clustering.plan.strategy.cluster.begin.partition=ds=2022-12-03/hour=21 \
--hoodie-conf hoodie.clustering.plan.strategy.cluster.end.partition=ds=2022-12-03/hour=21

begin.partition and end.partition will be verified as failed

Impact

After adding the patch, hudi-utilities supports parsing the value containing '=' when using --hoodie-conf

Risk level (write none, low medium or high below)

Low

Documentation Update

None

Contributor's checklist

  • Read through contributor's guide
  • Change Logs and Impact were stated clearly
  • Adequate tests were added if applicable
  • CI passed

Copy link
Contributor

@danny0405 danny0405 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1, nice catch ~

@Zouxxyy Zouxxyy force-pushed the xinyu/fix-util-param branch from 623c9a6 to 245afd9 Compare January 6, 2023 02:47
@hudi-bot
Copy link
Collaborator

hudi-bot commented Jan 6, 2023

CI report:

Bot commands @hudi-bot supports the following commands:
  • @hudi-bot run azure re-run the last Azure build

@YannByron YannByron merged commit a090ad0 into apache:master Jan 6, 2023
@Zouxxyy Zouxxyy deleted the xinyu/fix-util-param branch January 7, 2023 04:04
fengjian428 pushed a commit to fengjian428/hudi that referenced this pull request Apr 5, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants