[FLINK-28750][Hive]Add field comment parse for hive ddl

[hehuiyuan]

What is the purpose of the change

The field comment is lost when using hive parser.

"set table.sql-dialect=hive;\n" +
"CREATE TABLE IF NOT EXISTS myhive.dev.shipu3_test_1125 (\n" +
"   `id` int COMMENT 'iadddd',\n" +
"   `cartdid` bigint COMMENT 'aaa',\n" +
"   `customer` string COMMENT 'vvvv',\n" +
"   `product` string COMMENT 'cccc',\n" +
"   `price` double COMMENT '',\n" +
"   `dt` STRING COMMENT ''\n" +
") PARTITIONED BY (dt STRING) STORED AS TEXTFILE TBLPROPERTIES (\n" +
"  'streaming-source.enable' = 'false',\n" +
"  'streaming-source.partition.include' = 'all',\n" +
"  'lookup.join.cache.ttl' = '12 h'\n" +
")"; 

Brief change log

Add field comment for hive ddl

Does this pull request potentially affect one of the following parts:

• Dependencies (does it add or upgrade a dependency): ( no)
• The public API, i.e., is any changed class annotated with @Public(Evolving): (no)
• The serializers: (no )
• The runtime per-record code paths (performance sensitive): ( no )
• Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: (no )
• The S3 file system connector: (no )

Documentation

• Does this pull request introduce a new feature? ( no)
• If yes, how is the feature documented? (not applicable / docs / JavaDocs / not documented)

[hehuiyuan]

Hi @beyond1920 , @luoyuxia , look this when you have time.

[flinkbot]

CI report:

• f107e8a Azure: FAILURE

Bot commands

The @flinkbot bot supports the following commands:

• @flinkbot run azure re-run the last Azure build

[hehuiyuan]

@flinkbot run azure

[luoyuxia]

@hehuiyuan FYI. The flink-sql-parser-hive will be dropped. And all the new features releated to Hive dialect should be added in HiveParser in flink-connector-hive

[luoyuxia]

I have another question. Is it unsupported to add field comment when using HiveDialect?

[hehuiyuan]

@luoyuxia thanks for your reply.
If the ddl have the keyword of comment, why not?

We can do somehing by using the flink hive parser, then I found the field comment is lost.

[hehuiyuan]

@hehuiyuan FYI. The flink-sql-parser-hive will be dropped. And all the new features releated to Hive dialect should be added in HiveParser in flink-connector-hive

Thanks ,got it. flink1.15 have field comment for hive ?

[luoyuxia]

I haven't checked. But I go through the source code, and I think It should have been supported.

[hehuiyuan]

@luoyuxia thanks. It shoud have been supported.

[hehuiyuan]

Hi @luoyuxia , it should match comment fitst, then create SqlTableColumn,

does this code error need to be fixed in 1.14 ?

If not, I will close this.