[SPARK-33571][SQL][DOCS][3.0] Add a ref to INT96 config from the doc for spark.sql.legacy.parquet.datetimeRebaseModeInWrite/Read
#30604
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…spark.sql.legacy.parquet.datetimeRebaseModeInWrite/Read` For the SQL configs `spark.sql.legacy.parquet.datetimeRebaseModeInWrite` and `spark.sql.legacy.parquet.datetimeRebaseModeInRead`, improve their descriptions by: 1. Explicitly document on which parquet types, those configs influence on 2. Refer to corresponding configs for `INT96` To avoid user confusions like reposted in SPARK-33571, and make the config description more precise. No By running `./dev/scalastyle`. Closes apache#30596 from MaxGekk/clarify-rebase-docs. Authored-by: Max Gekk <max.gekk@gmail.com> Signed-off-by: HyukjinKwon <gurwls223@apache.org> (cherry picked from commit 94c144b) Signed-off-by: Max Gekk <max.gekk@gmail.com>
HyukjinKwon
approved these changes
Dec 4, 2020
|
Kubernetes integration test starting |
|
Kubernetes integration test status failure |
|
Test build #132214 has finished for PR 30604 at commit
|
|
Test build #132232 has finished for PR 30604 at commit
|
dongjoon-hyun
approved these changes
Dec 4, 2020
There was a problem hiding this comment.
+1, LGTM. Thank you, @MaxGekk and @HyukjinKwon .
Merged to branch-3.0.
dongjoon-hyun
pushed a commit
that referenced
this pull request
Dec 4, 2020
…for `spark.sql.legacy.parquet.datetimeRebaseModeInWrite/Read` ### What changes were proposed in this pull request? For the SQL configs `spark.sql.legacy.parquet.datetimeRebaseModeInWrite` and `spark.sql.legacy.parquet.datetimeRebaseModeInRead`, improve their descriptions by: 1. Explicitly document on which parquet types, those configs influence on 2. Refer to corresponding configs for `INT96` ### Why are the changes needed? To avoid user confusions like reposted in SPARK-33571, and make the config description more precise. ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? By running `./dev/scalastyle`. Closes #30604 from MaxGekk/clarify-rebase-docs-3.0. Authored-by: Max Gekk <max.gekk@gmail.com> Signed-off-by: Dongjoon Hyun <dongjoon@apache.org>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What changes were proposed in this pull request?
For the SQL configs
spark.sql.legacy.parquet.datetimeRebaseModeInWriteandspark.sql.legacy.parquet.datetimeRebaseModeInRead, improve their descriptions by:INT96Why are the changes needed?
To avoid user confusions like reposted in SPARK-33571, and make the config description more precise.
Does this PR introduce any user-facing change?
No
How was this patch tested?
By running
./dev/scalastyle.