Skip to content

[FLINK-40130][connect/paimon] Fix missing primary key type compatibility check of Paimon sink#4471

Merged
lvyanquan merged 1 commit into
apache:masterfrom
haruki-830:FLINK-40130
Jul 13, 2026
Merged

[FLINK-40130][connect/paimon] Fix missing primary key type compatibility check of Paimon sink#4471
lvyanquan merged 1 commit into
apache:masterfrom
haruki-830:FLINK-40130

Conversation

@haruki-830

@haruki-830 haruki-830 commented Jul 13, 2026

Copy link
Copy Markdown
Contributor

Summary

This commit adds primary key type compatibility validation to the Flink CDC pipeline Paimon sink. When writing to an existing Paimon table, the sink now rejects incompatible upstream primary key types before entering the record coercion path, preventing unexpected conversion failures or incorrect behavior.

Key Changes

  1. Primary Key Type Compatibility Validation
  • Added validation in BucketAssignOperator for existing Paimon tables.
  • Checks whether each existing Paimon primary key column is type-compatible with the corresponding upstream primary key column before record coercion.
  • Reports a clear error when the primary key type is incompatible.
  1. Reuse Existing Type Compatibility Logic
  • Made SchemaMergingUtils.isDataTypeCompatible(...) public and reused it directly in the Paimon sink for primary key type checks.
  • Ignores nullability during this check, so validation focuses on type compatibility while nullability remains enforced by the normal write/table validation path.
  1. Test Coverage
  • Added coverage for incompatible primary key types.
  • Added coverage for compatible primary key type changes to ensure valid cases continue to work.

JIRA Reference

https://issues.apache.org/jira/browse/FLINK-40130

@haruki-830 haruki-830 marked this pull request as ready for review July 13, 2026 07:29
@haruki-830

Copy link
Copy Markdown
Contributor Author

Hi @lvyanquan, could you please help review this PR when you have time? Thanks!

@lvyanquan lvyanquan left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1.

@lvyanquan lvyanquan merged commit 36d2121 into apache:master Jul 13, 2026
19 of 20 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants