Support passing schema definition for JSON and AVRO schemas #3766
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
*Motivation* Currently AVRO and Schema generated schemas from POJO directly. Sometime people would like to use pre-generated/defined schemas, so allow passing in schema definitions would clear the confusions on parsing schemas from POJO. *Modifications* - Abstract a common base class `StructSchema` for AVRO/PROTOBUF/JSON - Standarize on using avro schema for defining schema (we already did that. this change only makes it clearer) - Add methods to pass schema definition for JSON and AVRO schemas *NOTES* We don't support passing schema definition for PROTOBUF. since we only supported generated messages as POJO class for protobuf schema, and we generate schema definition from the generated messages. it doesn't make sense to pass in a different schema definition.
sijie
added
type/feature
|
run java8 tests |
1 similar comment
|
run java8 tests |
jiazhai
approved these changes
Mar 14, 2019
|
@jiazhai I was waiting for #3752 to be merged first, since #3752 will introduce |
|
Pushing to 2.3.1 since #3752 was marked fro 2.3.1 and it now depends on this one |
merlimat
pushed a commit
that referenced
this pull request
Mar 29, 2019
* Support passing schema definition for JSON and AVRO schemas *Motivation* Currently AVRO and Schema generated schemas from POJO directly. Sometime people would like to use pre-generated/defined schemas, so allow passing in schema definitions would clear the confusions on parsing schemas from POJO. *Modifications* - Abstract a common base class `StructSchema` for AVRO/PROTOBUF/JSON - Standarize on using avro schema for defining schema (we already did that. this change only makes it clearer) - Add methods to pass schema definition for JSON and AVRO schemas *NOTES* We don't support passing schema definition for PROTOBUF. since we only supported generated messages as POJO class for protobuf schema, and we generate schema definition from the generated messages. it doesn't make sense to pass in a different schema definition. * Add missing license header
|
Merged in 2.3.1 at |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Motivation
Currently AVRO and Schema generated schemas from POJO directly.
Sometime people would like to use pre-generated/defined schemas,
so allow passing in schema definitions would clear the confusions
on parsing schemas from POJO.
Modifications
StructSchemafor AVRO/PROTOBUF/JSONNOTES
We don't support passing schema definition for PROTOBUF. since we only supported generated messages as POJO
class for protobuf schema, and we generate schema definition from the generated messages. it doesn't make sense
to pass in a different schema definition.
Related Issues: #3752 #3741
Does this pull request potentially affect one of the following parts:
If
yeswas chosen, please highlight the changesDocumentation