Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

SPARK-11716 - UDFRegistration just drops the input type when re-creat… #9739

Closed
wants to merge 2 commits into from
Closed

SPARK-11716 - UDFRegistration just drops the input type when re-creat… #9739

wants to merge 2 commits into from

Conversation

jbonofre
Copy link
Member

…ing the UserDefinedFunction

@const-ae
Copy link

Hi, I guess you should also update the script in the comments that generates the file.

@jbonofre
Copy link
Member Author

Good point, let me update the PR accordingly. Thanks.

@jbonofre
Copy link
Member Author

PR rebased and updated according to @Artjom-Metro comment.

@jbonofre
Copy link
Member Author

Rebase

@yhuai
Copy link
Contributor

yhuai commented Nov 18, 2015

Can you change the title format to [SPARK-11716] [SQL] ...?

@yhuai
Copy link
Contributor

yhuai commented Nov 18, 2015

ok to test.

@yhuai
Copy link
Contributor

yhuai commented Nov 18, 2015

Can you add a test?

@yhuai
Copy link
Contributor

yhuai commented Nov 18, 2015

So, the problem is that when we use apply method on this UDF, we will not be able to pass in the type and thus, we fail to do type casting, right? It is not the case for using expr because the udf we get from the function registry has the input type. It will be great to update the description to describe the problem.

@SparkQA
Copy link

SparkQA commented Nov 19, 2015

Test build #46257 has finished for PR 9739 at commit 3ebf487.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@jbonofre
Copy link
Member Author

@yhuai Thanks for your feedbacks. Let me update the PR accordingly.

@yhuai yhuai mentioned this pull request Nov 20, 2015
Closed
asfgit pushed a commit that referenced this pull request Nov 20, 2015
@jbonofre @marmbrus
[SPARK-11716][SQL] UDFRegistration just drops the input type when re-…
b9b0e17 
…creating the UserDefinedFunction

https://issues.apache.org/jira/browse/SPARK-11716

This is one is #9739 and a regression test. When commit it, please make sure the author is jbonofre.

You can find the original PR at #9739

closes #9739

Author: Jean-Baptiste Onofré <jbonofre@apache.org>
Author: Yin Huai <yhuai@databricks.com>

Closes #9868 from yhuai/SPARK-11716.

(cherry picked from commit 03ba56d)
Signed-off-by: Michael Armbrust <michael@databricks.com>
@asfgit asfgit closed this in 03ba56d Nov 20, 2015
@jbonofre
Copy link
Member Author

Awesome, thanks a lot @yhuai !

@yhuai
Copy link
Contributor

yhuai commented Nov 21, 2015

no problem! Thank you for your fix!

kiszk pushed a commit to kiszk/spark-gpu that referenced this pull request Dec 26, 2015
@jbonofre @kiszk
[SPARK-11716][SQL] UDFRegistration just drops the input type when re-…
d4760d5 
…creating the UserDefinedFunction

https://issues.apache.org/jira/browse/SPARK-11716

This is one is #9739 and a regression test. When commit it, please make sure the author is jbonofre.

You can find the original PR at apache/spark#9739

closes #9739

Author: Jean-Baptiste Onofré <jbonofre@apache.org>
Author: Yin Huai <yhuai@databricks.com>

Closes #9868 from yhuai/SPARK-11716.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
4 participants
@jbonofre @const-ae @yhuai @SparkQA