Skip to content

Comments

[HUDI-5957] Fix table props not being properly propagated for HoodieCLIUtils#8243

Merged
XuQianJin-Stars merged 1 commit intoapache:masterfrom
Zouxxyy:xinyu/fix-table-props
Mar 23, 2023
Merged

[HUDI-5957] Fix table props not being properly propagated for HoodieCLIUtils#8243
XuQianJin-Stars merged 1 commit intoapache:masterfrom
Zouxxyy:xinyu/fix-table-props

Conversation

@Zouxxyy
Copy link
Contributor

@Zouxxyy Zouxxyy commented Mar 20, 2023

Change Logs

At present, finalParameters in HoodieCLIUtils.createHoodieClientFromPath not contain table props.

This causes some problems, such as:

Run cluster should fail when using bucket index, but it does not, because it didn't use hoodie.index.type in table props

drop table hudi_cow_test_tbl;
create table hudi_cow_test_tbl (
  id bigint,  
  name string,  
  ts bigint,  
  dt string,  
  hh string
) using hudi
tblproperties (
  type = 'cow',
  primaryKey = 'id',
  preCombineField = 'ts',
  hoodie.index.type = 'BUCKET'
)partitioned by (dt, hh);

insert into hudi_cow_test_tbl values (1, 'a1', 1001, '2021-12-09', '10');
insert into hudi_cow_test_tbl values (2, 'a2', 1001, '2021-12-09', '10');

# this operation should fail, otherwise the table will become unavailable
call run_clustering(table => 'hudi_cow_test_tbl');

insert into hudi_cow_test_tbl values (3, 'a3', 1001, '2021-12-09', '10'); 

Impact

Fix finalParameters in HoodieCLIUtils.createHoodieClientFromPath

Risk level (write none, low medium or high below)

low

Documentation Update

none

Contributor's checklist

  • Read through contributor's guide
  • Change Logs and Impact were stated clearly
  • Adequate tests were added if applicable
  • CI passed

@hudi-bot
Copy link
Collaborator

CI report:

Bot commands @hudi-bot supports the following commands:
  • @hudi-bot run azure re-run the last Azure build

@Zouxxyy
Copy link
Contributor Author

Zouxxyy commented Mar 21, 2023

@XuQianJin-Stars Can you help with a review?

Copy link
Contributor

@XuQianJin-Stars XuQianJin-Stars left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@XuQianJin-Stars XuQianJin-Stars added the area:sql SQL interfaces label Mar 23, 2023
@XuQianJin-Stars XuQianJin-Stars merged commit bf2e64c into apache:master Mar 23, 2023
@Zouxxyy Zouxxyy deleted the xinyu/fix-table-props branch March 23, 2023 06:01
southernriver pushed a commit to southernriver/hudi that referenced this pull request Mar 31, 2023
nsivabalan pushed a commit to nsivabalan/hudi that referenced this pull request Mar 31, 2023
fengjian428 pushed a commit to fengjian428/hudi that referenced this pull request Apr 5, 2023
stayrascal pushed a commit to stayrascal/hudi that referenced this pull request Apr 20, 2023
KnightChess pushed a commit to KnightChess/hudi that referenced this pull request Jan 2, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:sql SQL interfaces engine:spark Spark integration

Projects

Archived in project

Development

Successfully merging this pull request may close these issues.

4 participants