Skip to content

[Flink]support where sql for compaction#3403

Merged
JingsongLi merged 12 commits intoapache:masterfrom
wg1026688210:flink/support_where_sql_for_compaction
Jun 24, 2024
Merged

[Flink]support where sql for compaction#3403
JingsongLi merged 12 commits intoapache:masterfrom
wg1026688210:flink/support_where_sql_for_compaction

Conversation

@wg1026688210
Copy link
Copy Markdown
Contributor

@wg1026688210 wg1026688210 commented May 27, 2024

Purpose

  1. Add a validation that the paritions arg and the where sql of compaction action/procedure should not specify non-partition field.
    CompactActionITCase#testSpecifyNonPartitionField
  2. Add a common utility class for converting SQL statements into predicates.
    SimpleSqlPredicateConvertorTest
  3. Add a support for WHERE clause configuration in compaction action/procedure.
    CompactProcedureITCase/CompactActionITCase

Linked issue: close #3135

Tests

CompactActionITCase#testSpecifyNonPartitionField
SimpleSqlPredicateConvertorTest
CompactProcedureITCase
CompactActionITCase

API and Format

Documentation

@wg1026688210 wg1026688210 force-pushed the flink/support_where_sql_for_compaction branch from 64374a9 to 46ce303 Compare May 27, 2024 09:54
@wg1026688210 wg1026688210 changed the title Flink/support where sql for compaction [Flink]/support where sql for compaction May 27, 2024
@wg1026688210 wg1026688210 changed the title [Flink]/support where sql for compaction [Flink]support where sql for compaction May 27, 2024
@wg1026688210 wg1026688210 force-pushed the flink/support_where_sql_for_compaction branch 14 times, most recently from cfc6c62 to 55a9483 Compare May 29, 2024 11:44
@wg1026688210 wg1026688210 force-pushed the flink/support_where_sql_for_compaction branch 4 times, most recently from b68c2be to 6f3c67f Compare June 3, 2024 08:20
@wg1026688210 wg1026688210 force-pushed the flink/support_where_sql_for_compaction branch from 6f3c67f to 1eb555f Compare June 3, 2024 08:47
Copy link
Copy Markdown
Contributor

@JingsongLi JingsongLi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please update documentation too.

@wg1026688210 wg1026688210 force-pushed the flink/support_where_sql_for_compaction branch 2 times, most recently from 0a8c3b5 to 003b79a Compare June 4, 2024 11:21
@wg1026688210 wg1026688210 force-pushed the flink/support_where_sql_for_compaction branch from 003b79a to 9f2f1e8 Compare June 4, 2024 11:31
@wg1026688210 wg1026688210 requested a review from JingsongLi June 4, 2024 12:36
@wg1026688210
Copy link
Copy Markdown
Contributor Author

Please update documentation too.

done

@JingsongLi
Copy link
Copy Markdown
Contributor

Does flink have calcite dependencies in cluster?

@wg1026688210
Copy link
Copy Markdown
Contributor Author

wg1026688210 commented Jun 20, 2024

Does flink have calcite dependencies in cluster?

done,I access calcite classes from a special classLoader in flink cluster by reflect

@wg1026688210 wg1026688210 force-pushed the flink/support_where_sql_for_compaction branch from 1dae756 to 438f007 Compare June 20, 2024 06:38
Copy link
Copy Markdown
Contributor

@JingsongLi JingsongLi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1

@JingsongLi JingsongLi merged commit c878a72 into apache:master Jun 24, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature] Flink compact procedure supports 'where' filter

2 participants