-
Notifications
You must be signed in to change notification settings - Fork 28.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[SPARK-6505][SQL]Remove the reflection call in HiveFunctionWrapper #5660
Conversation
Can one of the admins verify this patch? |
ok to test |
Test build #30848 has started for PR 5660 at commit |
Would like to add some background about this change for future reference: Spark SQL repackaged Hive jars to remove unnecessary shading and dependencies. One of them is Kryo. Hive shades Kryo to package |
import java.io.{OutputStream, InputStream} | ||
|
||
import com.esotericsoftware.kryo.io.Input | ||
import com.esotericsoftware.kryo.io.Output |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Move imports to the header of the file.
add to whitelist |
Test build #30849 has started for PR 5660 at commit |
Test build #30848 has finished for PR 5660 at commit
|
Test PASSed. |
Test build #30849 has finished for PR 5660 at commit
|
Test PASSed. |
Test build #30892 has started for PR 5660 at commit |
Test build #30892 has finished for PR 5660 at commit
|
Test PASSed. |
can this patch be merged? :) |
Thanks for working on this! I'm merging this to master. |
@baishuo what's your Apache JIRA username? I can assign you to the JIRA to credit you. |
hi @srowen , my jira name is also baishuo |
according liancheng‘s comment in https://issues.apache.org/jira/browse/SPARK-6505, this patch remove the reflection call in HiveFunctionWrapper, and implement the functions named "deserializeObjectByKryo" and "serializeObjectByKryo" according the functions with the save name in org.apache.hadoop.hive.ql.exec.Utilities.java Author: baishuo <vc_java@hotmail.com> Closes apache#5660 from baishuo/SPARK-6505-20150423 and squashes the following commits: ae61ec4 [baishuo] modify code style 78d9fa3 [baishuo] modify code style 0b522a7 [baishuo] modify code style a5ff9c7 [baishuo] Remove the reflection call in HiveFunctionWrapper
according liancheng‘s comment in https://issues.apache.org/jira/browse/SPARK-6505, this patch remove the reflection call in HiveFunctionWrapper, and implement the functions named "deserializeObjectByKryo" and "serializeObjectByKryo" according the functions with the save name in org.apache.hadoop.hive.ql.exec.Utilities.java Author: baishuo <vc_java@hotmail.com> Closes apache#5660 from baishuo/SPARK-6505-20150423 and squashes the following commits: ae61ec4 [baishuo] modify code style 78d9fa3 [baishuo] modify code style 0b522a7 [baishuo] modify code style a5ff9c7 [baishuo] Remove the reflection call in HiveFunctionWrapper
according @liancheng‘s comment in https://issues.apache.org/jira/browse/SPARK-6505, this patch remove the reflection call in HiveFunctionWrapper, and implement the functions named "deserializeObjectByKryo" and "serializeObjectByKryo" according the functions with the save name in
org.apache.hadoop.hive.ql.exec.Utilities.java