-
Notifications
You must be signed in to change notification settings - Fork 2.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[HUDI-6240] Adding default value as CORRECTED for rebase modes in write and read for avro #8764
Conversation
@@ -54,7 +54,7 @@ private[sql] class AvroSerializer(rootCatalystType: DataType, | |||
def this(rootCatalystType: DataType, rootAvroType: Schema, nullable: Boolean) = { | |||
this(rootCatalystType, rootAvroType, nullable, | |||
LegacyBehaviorPolicy.withName(SQLConf.get.getConf( | |||
SQLConf.LEGACY_AVRO_REBASE_MODE_IN_WRITE))) | |||
SQLConf.LEGACY_AVRO_REBASE_MODE_IN_WRITE, "CORRECTED"))) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit: CORRECTED
to static final variable?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
fixed using LegacyBehaviorPolicy.CORRECTED.toString
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
…te and read for avro (apache#8764) (apache#294) Co-authored-by: Sivabalan Narayanan <n.siva.b@gmail.com>
Change Logs
Adding default value as "CORRECTED" for rebase modes in write and read for avro, to be used when encountering timestamps older than 1970.
Impact
Will automatically work out of the box, unless user prefers to override them.
Risk level (write none, low medium or high below)
low.
Documentation Update
Describe any necessary documentation update if there is any new feature, config, or user-facing change
ticket number here and follow the instruction to make
changes to the website.
Contributor's checklist