Skip to content

Core: remove reference files for BaseEqualityDeltaWriter#4733

Closed
chenjunjiedada wants to merge 1 commit intoapache:masterfrom
chenjunjiedada:remove-ref-data-files
Closed

Core: remove reference files for BaseEqualityDeltaWriter#4733
chenjunjiedada wants to merge 1 commit intoapache:masterfrom
chenjunjiedada:remove-ref-data-files

Conversation

@chenjunjiedada
Copy link
Collaborator

Flink upsert commit doesn't validate the conflict of data files that position deletes referred, thus it doesn't have to keep referred data files of position deletes in memory anymore.

@github-actions github-actions bot added the core label May 9, 2022
@chenjunjiedada chenjunjiedada force-pushed the remove-ref-data-files branch from 139966e to 2d8b120 Compare May 9, 2022 12:12
@github-actions github-actions bot added the data label May 9, 2022
@rdblue
Copy link
Contributor

rdblue commented May 10, 2022

I don't think this is a good idea. While Flink may not need the referenced files set, the writer that you're modifying here lives in core and could be used by other modules. If this were entirely in Flink I think it would be fine, but it probably isn't a good idea to modify this class.

@chenjunjiedada
Copy link
Collaborator Author

Sounds reasonable to me. Let me close this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

Comments