-
Notifications
You must be signed in to change notification settings - Fork 28k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[SPARK-2386] [SQL] RowWriteSupport should use the exact types to cast. #1315
Conversation
Merged build triggered. |
Merged build started. |
Merged build triggered. |
Merged build started. |
Added a test for this issue. |
Merged build finished. All automated tests passed. |
All automated tests passed. |
Merged build finished. All automated tests passed. |
All automated tests passed. |
When execute `saveAsParquetFile` with non-primitive type, `RowWriteSupport` uses wrong type `Int` for `ByteType` and `ShortType`. Author: Takuya UESHIN <ueshin@happy-camper.st> Closes #1315 from ueshin/issues/SPARK-2386 and squashes the following commits: 20d89ec [Takuya UESHIN] Use None instead of null. bd88741 [Takuya UESHIN] Add a test. 323d1d2 [Takuya UESHIN] Modify RowWriteSupport to use the exact types to cast. (cherry picked from commit 4deeed1) Signed-off-by: Michael Armbrust <michael@databricks.com>
Thanks, merged into master and 1.0. |
When execute `saveAsParquetFile` with non-primitive type, `RowWriteSupport` uses wrong type `Int` for `ByteType` and `ShortType`. Author: Takuya UESHIN <ueshin@happy-camper.st> Closes apache#1315 from ueshin/issues/SPARK-2386 and squashes the following commits: 20d89ec [Takuya UESHIN] Use None instead of null. bd88741 [Takuya UESHIN] Add a test. 323d1d2 [Takuya UESHIN] Modify RowWriteSupport to use the exact types to cast.
…he#1315) ### What changes were proposed in this pull request? This PR aims to add a new executor roll policy, FAILED_TASKS. ### Why are the changes needed? This is helpful to clean up a faulty executor first. ### Does this PR introduce _any_ user-facing change? Yes, but this is a new feature. ### How was this patch tested? Pass the CIs. Closes apache#34938 from williamhyun/SPARK-37679. Authored-by: William Hyun <william@apache.org> Signed-off-by: Dongjoon Hyun <dongjoon@apache.org> (cherry picked from commit 897f056) Signed-off-by: Dongjoon Hyun <dongjoon@apache.org> Co-authored-by: William Hyun <william@apache.org>
When execute
saveAsParquetFile
with non-primitive type,RowWriteSupport
uses wrong typeInt
forByteType
andShortType
.