Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FLINK-12756][hive] migrate HiveCatalog from TypeInformation-based old type system to DataType-based new type system #8639

Closed
wants to merge 2 commits into from

Conversation

bowenli86
Copy link
Member

What is the purpose of the change

This PR migrates HiveCatalog from TypeInformation-based old type system to DataType-based new type system

Note: hive table InputFormat and OutputFormat may also need to migrate similarly. This PR doesn't contain those changes.

Brief change log

  • migrated HiveCatalog from TypeInformation-based old type system to DataType-based new type system
  • updated unit tests

Verifying this change

This change is already covered by existing test

Does this pull request potentially affect one of the following parts:

  • Dependencies (does it add or upgrade a dependency): (no)
  • The public API, i.e., is any changed class annotated with @Public(Evolving): (no)
  • The serializers: (no)
  • The runtime per-record code paths (performance sensitive): (no)
  • Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Yarn/Mesos, ZooKeeper: (no)
  • The S3 file system connector: (no)

Documentation

  • Does this pull request introduce a new feature? (no)
  • If yes, how is the feature documented? (not applicable)

…d type system to DataType-based new type system
@bowenli86
Copy link
Member Author

cc @xuefuz @lirui-apache @zjuwangg

@flinkbot
Copy link
Collaborator

flinkbot commented Jun 5, 2019

Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community
to review your pull request. We will use this comment to track the progress of the review.

Review Progress

  • ❓ 1. The [description] looks good.
  • ❓ 2. There is [consensus] that the contribution should go into to Flink.
  • ❓ 3. Needs [attention] from.
  • ❓ 4. The change fits into the overall [architecture].
  • ❓ 5. Overall code [quality] is good.

Please see the Pull Request Review Guide for a full explanation of the review process.


The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands
The @flinkbot bot supports the following commands:

  • @flinkbot approve description to approve one or more aspects (aspects: description, consensus, architecture and quality)
  • @flinkbot approve all to approve all aspects
  • @flinkbot approve-until architecture to approve everything until architecture
  • @flinkbot attention @username1 [@username2 ..] to require somebody's attention
  • @flinkbot disapprove architecture to remove an approval you gave earlier

*
* <p>The call order of this method determines the order of fields in the schema.
*/
public Builder fields(String[] names, DataType[] dataTypes) {
Copy link
Contributor

@xuefuz xuefuz Jun 5, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It appears to me this builder isn't seem necessary or proper as the constructor does the same thing and probably in a more complete fashion.

Copy link
Member Author

@bowenli86 bowenli86 Jun 6, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the builder is already there, and is not introduced by this PR. I feel it’s good to have this method to allow users batch loading elements with builder, which is required by this PR.

I think whether exposing the class constructor as public is orthogonal

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I meant this builder method is new! However, I didn't realize the constructor is private. With that, it's fine to have this.

Copy link
Contributor

@xuefuz xuefuz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR looks mostly good to me except the introduced builder.

@bowenli86
Copy link
Member Author

Thanks @xuefuz for your review.

Merging

@bowenli86
Copy link
Member Author

Rebased to master before merged

@asfgit asfgit closed this in baaea36 Jun 6, 2019
sjwiesman pushed a commit to sjwiesman/flink that referenced this pull request Jun 26, 2019
…d type system to DataType-based new type system

This PR migrates HiveCatalog from TypeInformation-based old type system to DataType-based new type system

This closes apache#8639.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
4 participants