Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Proposal] Real-time wide table join #197

Closed
xuanyu66 opened this issue May 18, 2022 · 0 comments
Closed

[Proposal] Real-time wide table join #197

xuanyu66 opened this issue May 18, 2022 · 0 comments
Labels
type/proposal proposal for new feature

Comments

@xuanyu66
Copy link
Collaborator

xuanyu66 commented May 18, 2022

Problem

Currently, Flink doesn't support the statement INSERT ... ON DUPLICATE KEY UPDATE.
It is a common requirement that updates some fields based on the primary key in the Materialized View scenario.
image

Solution

  • Use SQL hints to Achieve semantics INSERT ... ON DUPLICATE KEY UPDATE.
  • Prune column to get the needed columns
  • Use InsertOrDuplicateKeyUpdateOutputFormat to generate statement INSERT ... ON DUPLICATE KEY UPDATE and batch execute them in TiDB.

Design

#198

Implement

#196

Test

#196

Doc

#196

@xuanyu66 xuanyu66 added the type/proposal proposal for new feature label May 31, 2022
@xuanyu66 xuanyu66 changed the title [Proposal] Insert on duplicate key update [Proposal] Real-time wide table join Jun 16, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type/proposal proposal for new feature
Projects
None yet
Development

No branches or pull requests

1 participant