Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[schema] Introduce schema builder to build schema. #3682

Merged
merged 1 commit into from
Feb 26, 2019

Conversation

sijie
Copy link
Member

@sijie sijie commented Feb 25, 2019

Motivation

Currently we are supporting POJO based schema in java clients.
POJO schema is only useful when the POJO is predefined. However
in applications like a CDC pipeline, POJO is no predefined, there
is no other way to define a schema.

Since we are using avro schema for schema management, this PR
is proposing a simple schema builder wrapper on avro schema builder.

Modifications

Introduce schema builder to build a record schema.

NOTES

Currently we only support primitives in defining fields in a record schema in this PR.
We will add nested types in future PRs.

Verifying this change

This change added tests and can be verified as follows:

  • Add schema builder test to verify the schema build via builder is same as building from POJO

Does this pull request potentially affect one of the following parts:

If yes was chosen, please highlight the changes

  • Dependencies (does it add or upgrade a dependency): (no)
  • The public API: (yes)
  • The schema: (yes)
  • The default values of configurations: (no)
  • The wire protocol: (no)
  • The rest endpoints: (no)
  • The admin cli options: (no)
  • Anything that affects deployment: (no)

Documentation

  • Does this pull request introduce a new feature? (yes)
  • If yes, how is the feature documented? (not documented)
  • If a feature is not applicable for documentation, explain why?
    • the feature is partially completed. documenting a partially completed feature will confuse people
  • If a feature is not documented yet in this PR, please create a followup issue for adding the documentation

*Motivation*

Currently we are supporting POJO based schema in java clients.
POJO schema is only useful when the POJO is predefined. However
in applications like a CDC pipeline, POJO is no predefined, there
is no other way to define a schema.

Since we are using avro schema for schema management, this PR
is proposing a simple schema builder wrapper on avro schema builder.

*Modifications*

Introduce schema builder to build a record schema.

*NOTES*

Currently we only support primitives in defining fields in a record schema in this PR.
We will add nested types in future PRs.
@sijie sijie added this to the 2.4.0 milestone Feb 25, 2019
@sijie sijie self-assigned this Feb 25, 2019
@sijie
Copy link
Member Author

sijie commented Feb 26, 2019

run java8 tests

@jiazhai
Copy link
Member

jiazhai commented Feb 26, 2019

rerun java8 tests

Copy link
Contributor

@codelipenghui codelipenghui left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm +1

@jiazhai
Copy link
Member

jiazhai commented Feb 26, 2019

rerun java8 tests

@sijie sijie merged commit d46474b into apache:master Feb 26, 2019
@sijie sijie deleted the schema_builder_impl branch February 26, 2019 12:03
@merlimat merlimat modified the milestones: 2.4.0, 2.3.1 Mar 29, 2019
merlimat pushed a commit that referenced this pull request Mar 29, 2019
*Motivation*

Currently we are supporting POJO based schema in java clients.
POJO schema is only useful when the POJO is predefined. However
in applications like a CDC pipeline, POJO is no predefined, there
is no other way to define a schema.

Since we are using avro schema for schema management, this PR
is proposing a simple schema builder wrapper on avro schema builder.

*Modifications*

Introduce schema builder to build a record schema.

*NOTES*

Currently we only support primitives in defining fields in a record schema in this PR.
We will add nested types in future PRs.
@merlimat
Copy link
Contributor

merlimat commented Apr 1, 2019

Merged in 2.3.1 at
f458ec3

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants