Skip to content

[HUDI-6813] Support table name for meta sync in bootstrap#9600

Merged
danny0405 merged 1 commit intoapache:masterfrom
beyond1920:HUDI-6813
Sep 2, 2023
Merged

[HUDI-6813] Support table name for meta sync in bootstrap#9600
danny0405 merged 1 commit intoapache:masterfrom
beyond1920:HUDI-6813

Conversation

@beyond1920
Copy link
Contributor

@beyond1920 beyond1920 commented Sep 1, 2023

Change Logs

Support table name for meta sync in bootstrap.
Using bootstrap procedure to migrate an existing table into a Hudi table.

call run_bootstrap(
  table=> "test_db.test_hudi_table",
  table_type=>"COPY_ON_WRITE",
  bootstrap_path=>"XXX",
  base_path=>"XXX",
  rowKey_field=>"device_id",
  partition_path_field=>"p_date,product",
  key_generator_class=>"org.apache.hudi.keygen.ComplexKeyGenerator",
  enable_hive_sync=>true,
  options => 'hoodie.datasource.hive_sync.mode=hms'
);

After sync to HMS, I get a hudi table which stored as test_db. unknown instead of test_db.test_hudi_table in HMS.
The root cause is not set table configure 'META_SYNC_TABLE_NAME' before start hive sync.
The pr aims to fix this bug.

Impact

NA

Risk level (write none, low medium or high below)

NA

Documentation Update

NA

Contributor's checklist

  • Read through contributor's guide
  • Change Logs and Impact were stated clearly
  • Adequate tests were added if applicable
  • CI passed

@danny0405 danny0405 added priority:blocker Production down; release blocker release-0.14.0 labels Sep 1, 2023
Copy link
Member

@codope codope left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for fixing it. Is this change tested? Maybe we can add a test in TestBootstrapProcedure

@hudi-bot
Copy link
Collaborator

hudi-bot commented Sep 1, 2023

CI report:

Bot commands @hudi-bot supports the following commands:
  • @hudi-bot run azure re-run the last Azure build

@danny0405 danny0405 merged commit 2196694 into apache:master Sep 2, 2023
leosanqing pushed a commit to leosanqing/hudi that referenced this pull request Sep 13, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

priority:blocker Production down; release blocker release-0.14.0

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants