Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[FLINK-30497][table-api] Introduce TableChange to represent DROP change #21592

Merged
merged 5 commits into from
Jan 5, 2023
Merged

[FLINK-30497][table-api] Introduce TableChange to represent DROP change #21592

merged 5 commits into from
Jan 5, 2023

Conversation

fsk119
Copy link
Member

@fsk119 fsk119 commented Jan 3, 2023

What is the purpose of the change

Introduce TableChange to represent DROP syntax.

Verifying this change

This change is already covered by existing tests, such as SqlToOperationConverterTest.

Does this pull request potentially affect one of the following parts:

  • Dependencies (does it add or upgrade a dependency): (yes / no)
  • The public API, i.e., is any changed class annotated with @Public(Evolving): (yes / no)
  • The serializers: (yes / no / don't know)
  • The runtime per-record code paths (performance sensitive): (yes / no / don't know)
  • Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: (yes / no / don't know)
  • The S3 file system connector: (yes / no / don't know)

Documentation

  • Does this pull request introduce a new feature? (yes / no)
  • If yes, how is the feature documented? (not applicable / docs / JavaDocs / not documented)

@flinkbot
Copy link
Collaborator

flinkbot commented Jan 3, 2023 •
edited

CI report:

Bot commands The @flinkbot bot supports the following commands:
  • @flinkbot run azure re-run the last Azure build

LadyForest
LadyForest reviewed Jan 3, 2023
View reviewed changes
Copy link
Contributor

@LadyForest LadyForest left a comment •
edited

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the contribution, and it looks good in general. I left some comments. Btw, please run mvn spotless:apply to fix the format issue.

flink-table/flink-table-common/src/main/java/org/apache/flink/table/catalog/TableChange.java Outdated Show resolved Hide resolved
...-planner/src/main/java/org/apache/flink/table/planner/expressions/ColumnReferenceFinder.java
spec ->
Stream.concat(
visitor.visit(spec.getWatermarkExpression()).stream(),
Stream.of(spec.getRowtimeAttribute())))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm curious under what condition the watermark expression does not contain the rowtime field?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For example, you can specify WATERMARK FOR rowtime AS col - INTERVAL 5 SECOND. I add a case in the ColumnReferencesFinderTest.

...ner/src/test/java/org/apache/flink/table/planner/operations/SqlToOperationConverterTest.java Outdated Show resolved Hide resolved
...ner/src/test/java/org/apache/flink/table/planner/operations/SqlToOperationConverterTest.java Outdated Show resolved Hide resolved
...ner/src/test/java/org/apache/flink/table/planner/operations/SqlToOperationConverterTest.java Outdated Show resolved Hide resolved
...able-planner/src/main/java/org/apache/flink/table/planner/utils/OperationConverterUtils.java Outdated Show resolved Hide resolved
...le-planner/src/main/java/org/apache/flink/table/planner/operations/AlterSchemaConverter.java Outdated Show resolved Hide resolved
...le-planner/src/main/java/org/apache/flink/table/planner/operations/AlterSchemaConverter.java Outdated
columns.remove(columnName);
columns.add(newName);

primaryKeys.remove(columnName);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think we can directly add newColumnName to primaryKeys.

Suggested change
primaryKeys.remove(columnName);
if (primaryKeys.remove(columnName)) {
primaryKeys.add(newName);
}

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Btw, I don't think we need to update primaryKeys here because buildUpdatedPrimaryKey will compute the primary key name.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK. I remove this.

...le-planner/src/main/java/org/apache/flink/table/planner/operations/AlterSchemaConverter.java Outdated
Comment on lines 831 to 841
columnToDependencies
.getOrDefault(columnName, Collections.emptySet())
.forEach(
referredColumn -> {
columnToReferences.get(referredColumn).remove(columnName);
columnToReferences.get(referredColumn).add(newName);
});
columnToDependencies.put(newName, columnToDependencies.remove(columnName));

columns.remove(columnName);
columns.add(newName);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need to update columnToDependencies and columnToReferences?
ALTER TABLE RENAME is not a bulk operation; there won't exist any further calls to get value from them.

...le-planner/src/main/java/org/apache/flink/table/planner/operations/AlterSchemaConverter.java Outdated Show resolved Hide resolved
fsk119 and others added 3 commits January 4, 2023 10:07
@fsk119 @LadyForest
[fix] Apply suggestions from code review
0b982e2 
Co-authored-by: Jane Chan <55568005+LadyForest@users.noreply.github.com>
@fsk119
[fix] address comments
08d46a9
@fsk119
[fix] improve codes
92d58a3
LadyForest
LadyForest reviewed Jan 4, 2023
View reviewed changes
Copy link
Contributor

@LadyForest LadyForest left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the update! The PR is almost good to merge except for one minor comment.

...planner/src/main/java/org/apache/flink/table/planner/operations/SqlToOperationConverter.java Outdated Show resolved Hide resolved
LadyForest
LadyForest reviewed Jan 4, 2023
View reviewed changes
Copy link
Contributor

@LadyForest LadyForest left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the update! The PR is almost good to merge except for one minor comment.

LadyForest
LadyForest approved these changes Jan 4, 2023
View reviewed changes
Copy link
Contributor

@LadyForest LadyForest left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the update, LGTM 👍

@fsk119 fsk119 merged commit cce059c into apache:master Jan 5, 2023
chucheng92 pushed a commit to chucheng92/flink that referenced this pull request Feb 3, 2023
@fsk119 @chucheng92
[FLINK-30497][table-api] Introduce TableChange to represent DROP change
e73af95 
This closes apache#21592
akkinenivijay pushed a commit to krisnaru/flink that referenced this pull request Feb 11, 2023
@fsk119 @akkinenivijay
[FLINK-30497][table-api] Introduce TableChange to represent DROP change
8ab40e0 
This closes apache#21592
@fsk119 fsk119 deleted the drop branch March 8, 2023 01:23
Tan-JiaLiang pushed a commit to Tan-JiaLiang/flink that referenced this pull request Mar 4, 2024
@fsk119 @Tan-JiaLiang
[FLINK-30497][table-api] Introduce TableChange to represent DROP change
c709d0a 
This closes apache#21592

(cherry picked from commit cce059c)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@LadyForest LadyForest LadyForest approved these changes

Assignees
No one assigned
Labels
component=TableSQL/API
Projects
None yet
Milestone
No milestone
3 participants
@fsk119 @flinkbot @LadyForest