[FLINK-30025][table] Unified the max display column width for SqlClient and Table APi in both Streaming and Batch execMode #21322

JingGe · 2022-11-14T21:49:05Z

What is the purpose of the change

Currently, the default value 30 of DEFAULT_MAX_COLUMN_WIDTH in PrintStyle has been used when table.execute().print() has been called. There is no way to adjust it if user wants to print more content of some columns.

Users want to configure the max display column width in a unified way while:

using CLI
Using Table API
Running job in batch execution mode
Running job in streaming execution mode

For further information, please refer to https://cwiki.apache.org/confluence/display/FLINK/FLIP-279+Unified+the+max+display+column+width+for+SqlClient+and+Table+APi+in+both+Streaming+and+Batch+execMode

Brief change log

Add new TableConfigOption.DISPLAY_MAX_COLUMN_WIDTH with default value 30
Use the config value while executeQueryOperation
TableConfigOption.DISPLAY_MAX_COLUMN_WIDTH will be the fallback key of SqlClientOptions.DISPLAY_MAX_COLUMN_WIDTH
SqlClientOptions.DISPLAY_MAX_COLUMN_WIDTH will used both in batch and in stream mode
deprecate SqlClientOptions.DISPLAY_MAX_COLUMN_WIDTH
update test

Verifying this change

This change added tests and can be verified by executing CliTableauResultViewTest and DisplayMaxColumnWidthSQLExampleTCase.

Does this pull request potentially affect one of the following parts:

Dependencies (does it add or upgrade a dependency): (yes / no)
The public API, i.e., is any changed class annotated with @Public(Evolving): (yes / no) Configurations are changed.
The serializers: (yes / no / don't know)
The runtime per-record code paths (performance sensitive): (yes / no / don't know)
Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: (yes / no / don't know)
The S3 file system connector: (yes / no / don't know)

Documentation

Does this pull request introduce a new feature? (yes / no)

flinkbot · 2022-11-14T22:01:17Z

CI report:

8c6a9ab Azure: SUCCESS

Bot commands

The @flinkbot bot supports the following commands:

@flinkbot run azure re-run the last Azure build

JingGe · 2022-11-15T21:49:15Z

@flinkbot run azure

JingGe · 2022-11-15T22:10:52Z

@flinkbot run azure

fsk119

Thanks for your contribution. I left some comments.

...flink-table-api-java/src/main/java/org/apache/flink/table/api/config/TableConfigOptions.java

...k-table-api-java/src/main/java/org/apache/flink/table/api/internal/TableEnvironmentImpl.java

docs/layouts/shortcodes/generated/table_config_configuration.html

JingGe · 2023-01-27T21:21:12Z

@fsk119 @wuchong would you like to review this PR?

wuchong

I think the change didn't verify impacts to SQL CLI.

Could you add tests in flink-table/flink-sql-client/src/test/resources/sql/select.q to
(1) set table.print.max-column-width=40 should also work
(2) truncating should also work on batch mode.

And please also add tests in flink-table/flink-sql-client/src/test/resources/sql/table.q to verify truncating doesn't affect show create table with table names longer than 30 characters.

...flink-table-api-java/src/main/java/org/apache/flink/table/api/config/TableConfigOptions.java

docs/layouts/shortcodes/generated/sql_client_configuration.html

...le/flink-sql-client/src/main/java/org/apache/flink/table/client/config/SqlClientOptions.java

JingGe · 2023-01-30T22:34:36Z

All tests have been added.

wuchong

Please append fixup commits to address comments. Please DO NOT squash or amend commits. Otherwise, the reviewer doesn't know what was updated and has to review the whole pull request again. You can squash or modify commits after the review pass.

...c/main/java/org/apache/flink/table/examples/java/basics/DisplayMaxColumnWidthSQLExample.java

flink-table/flink-sql-client/src/test/resources/sql/select.q

...le/flink-sql-client/src/main/java/org/apache/flink/table/client/config/SqlClientOptions.java

wuchong

LGMT. I only left some minor comments.
You can rebase and restructure your commits now.

flink-table/flink-table-api-java/src/test/java/org/apache/flink/table/api/TableConfigTest.java

...le/flink-sql-client/src/main/java/org/apache/flink/table/client/config/SqlClientOptions.java

wuchong · 2023-02-02T07:06:58Z

Waiting for branch cut.

JingGe · 2023-02-05T19:39:17Z

@flinkbot run azure

JingGe · 2023-02-05T20:24:16Z

@flinkbot run azure

JingGe · 2023-02-05T23:22:42Z

It's weird. Running the CliClientITCase locally in IDE or via maven has no failures. It seems CI has some special setting that the result of explain changelog_mode, estimated_cost, plan_advice, json_execution_plan select user, product from orders; contains smaller or bigger nodes' id, 1 digit numbers instead of 2. Therefore the calculated print width of the border lines are different.

JingGe · 2023-02-06T11:48:33Z

@flinkbot run azure

JingGe · 2023-02-13T23:24:55Z

Found reason that node ids are two digit numbers running locally but are three digit numbers in the CI env which make the header and footer of the printing result have different lengths.

JingGe · 2023-02-14T00:21:18Z

@flinkbot run azure

…lumn width while printing the query result to the client console.

…n.DISPLAY_MAX_COLUMN_WIDTH

…st SET 'table.display.max-column-width'

…de ids have been removed because the ids might have different length in different envs.

flinkbot added the component=TableSQL/API label Nov 14, 2022

JingGe changed the title ~~[FLINK-30025][table] the max column width is resizable while printing the query result to the client console.~~ [FLINK-30025][table] the max column width should be resizable while printing the query result to the client console. Nov 14, 2022

JingGe force-pushed the hotfix-table-print-max-col-length branch 3 times, most recently from d6777b1 to 8222ff4 Compare November 15, 2022 21:38

fsk119 reviewed Nov 21, 2022

View reviewed changes

JingGe force-pushed the hotfix-table-print-max-col-length branch from 8222ff4 to adb3568 Compare January 10, 2023 22:54

JingGe force-pushed the hotfix-table-print-max-col-length branch 4 times, most recently from d0acd8b to 733eb0c Compare January 20, 2023 03:36

JingGe changed the title ~~[FLINK-30025][table] the max column width should be resizable while printing the query result to the client console.~~ [FLINK-30025][table] Unified the max display column width for SqlClient and Table APi in both Streaming and Batch execMode Jan 20, 2023

JingGe force-pushed the hotfix-table-print-max-col-length branch 4 times, most recently from 6c2fe89 to 97b3b75 Compare January 20, 2023 16:54

wuchong reviewed Jan 29, 2023

View reviewed changes

JingGe force-pushed the hotfix-table-print-max-col-length branch 2 times, most recently from 8f852b0 to 690f194 Compare January 30, 2023 17:54

JingGe force-pushed the hotfix-table-print-max-col-length branch from 690f194 to 2d4e0a0 Compare January 30, 2023 23:57

wuchong reviewed Jan 31, 2023

View reviewed changes

wuchong approved these changes Feb 1, 2023

View reviewed changes

flink-table/flink-table-api-java/src/test/java/org/apache/flink/table/api/TableConfigTest.java Show resolved Hide resolved

...le/flink-sql-client/src/main/java/org/apache/flink/table/client/config/SqlClientOptions.java Outdated Show resolved Hide resolved

JingGe force-pushed the hotfix-table-print-max-col-length branch from 8ae5474 to 9cc74e1 Compare February 1, 2023 18:52

JingGe force-pushed the hotfix-table-print-max-col-length branch from 9cc74e1 to defdde0 Compare February 2, 2023 09:30

JingGe force-pushed the hotfix-table-print-max-col-length branch 3 times, most recently from c5581cb to 8fcdbe5 Compare February 5, 2023 15:13

JingGe force-pushed the hotfix-table-print-max-col-length branch 3 times, most recently from 1ae8cf1 to 7c640ff Compare February 7, 2023 02:28

JingGe force-pushed the hotfix-table-print-max-col-length branch from 84621c4 to 91eaae2 Compare February 14, 2023 06:28

JingGe added 3 commits February 14, 2023 16:46

[FLINK-30025][table] add new table config option to resize the max co…

c180723

…lumn width while printing the query result to the client console.

[FLINK-30025][table] fix print batch result and deprecate ConfigOptio…

5d69b63

…n.DISPLAY_MAX_COLUMN_WIDTH

[FLINK-30025][table][test] split streaming and batch test queries. Te…

8a93dde

…st SET 'table.display.max-column-width'

JingGe force-pushed the hotfix-table-print-max-col-length branch from 91eaae2 to 8b2d903 Compare February 14, 2023 15:46

[FLINK-30025][table][test] remove same amount of "-" and " " after no…

8c6a9ab

…de ids have been removed because the ids might have different length in different envs.

JingGe force-pushed the hotfix-table-print-max-col-length branch from 8b2d903 to 8c6a9ab Compare February 14, 2023 15:47

JingGe merged commit c03efae into apache:master Feb 15, 2023

fsk119 mentioned this pull request Mar 13, 2023

[FLINK-30025][table][test] remove same amount of "-" and " " after no… #22164

Merged

[FLINK-30025][table] Unified the max display column width for SqlClient and Table APi in both Streaming and Batch execMode #21322

[FLINK-30025][table] Unified the max display column width for SqlClient and Table APi in both Streaming and Batch execMode #21322

Uh oh!

Conversation

JingGe commented Nov 14, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What is the purpose of the change

Brief change log

Verifying this change

Does this pull request potentially affect one of the following parts:

Documentation

Uh oh!

flinkbot commented Nov 14, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

CI report:

Uh oh!

JingGe commented Nov 15, 2022

Uh oh!

JingGe commented Nov 15, 2022

Uh oh!

fsk119 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

JingGe commented Jan 27, 2023

Uh oh!

wuchong left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

JingGe commented Jan 30, 2023

Uh oh!

wuchong left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

wuchong left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

wuchong commented Feb 2, 2023

Uh oh!

JingGe commented Feb 5, 2023

Uh oh!

JingGe commented Feb 5, 2023

Uh oh!

JingGe commented Feb 5, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

JingGe commented Feb 6, 2023

Uh oh!

JingGe commented Feb 13, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

JingGe commented Feb 14, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

JingGe commented Nov 14, 2022 •

edited

Loading

flinkbot commented Nov 14, 2022 •

edited

Loading

JingGe commented Feb 5, 2023 •

edited

Loading

JingGe commented Feb 13, 2023 •

edited

Loading