[SUPPORT] Primary key check in deltastreamer

[WangCHX]

Describe the problem you faced

we accidentally configure wrong primary key in the spark write config, it cause duplicate data. wondering if there is a way to avoid it.

To Reproduce
change the primary config in write config and run the spark job.

Expected behavior
maybe should block the spark job to write data if the primary key config is different from the primary key in the original table.

Environment Description

• Hudi version : 0.11.0

• Spark version : 3.2.1

• Storage (HDFS/S3/GCS..) : GCS

• Running on Docker? (yes/no) : yes. on k8s.

[yihua]

@WangCHX Let me check the code. There should be validation of the write config against the table config to make sure they are consistent.

[WangCHX]

thanks. @yihua

[nsivabalan]

have created a jira for tracking. closing the issue.
thanks for reporting.