Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FLINK-33566] HBase sql-connector needs overwrite the rowKey #34

Open
wants to merge 3 commits into
base: main
Choose a base branch
from

Conversation

JankoWilliam
Copy link

add 'sink.overwrite-key' option, you can delete this key first before adding new data

Copy link

boring-cyborg bot commented Nov 16, 2023

Thanks for opening this pull request! Please check out our contributing guidelines. (https://flink.apache.org/contributing/how-to-contribute.html)

@JankoWilliam
Copy link
Author

Because the hbase sink table is also structured data with fixed fields, the 'sink. overwrite key' option needs to be used in conjunction with 'sink. ignore full value', otherwise deleting the key before adding it is meaningless.for example:
CREATE TABLE hbase_table_ (
rowkey STRING,
cf ROW<q1 INT,q2 INT,q3 INT,q4 INT>,
PRIMARY KEY (rowkey ) NOT ENFORCED
) WITH (
'connector' = 'my-hbase-2.2',
'table-name' = 'test_table',
'sink.ignore-null-value' = 'true',
'sink.overwrite-key' = 'true',
'zookeeper.quorum' = '127.0.0.1:2181'
);

insert into hbase_table_
select
id AS rowkey ,
ROW(
case when q1 <> '0' then cast(q1 as INT) else null end,
case when q2 <> '0' then cast(q2 as INT) else null end,
case when q3 <> '0' then cast(q3 as INT) else null end,
case when q4 <> '0' then cast(q4 as INT) else null end
) as cf
from kafka_table_

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
1 participant