Skip to content

Commit

Permalink
[SPARK-5193][SQL] Reconcile Java and Scala UDFRegistration.
Browse files Browse the repository at this point in the history
1. Removed UDFRegistration as a mixin in SQLContext and made it a field ("udf").
2. For Java UDFs, renamed dataType to returnType, and made that the 2nd argument since the UDF itself can be long.
3. Added all Java UDF registration methods to Scala's UDFRegistration.
4. Documentation
  • Loading branch information
rxin committed Jan 15, 2015
1 parent 4b325c7 commit 032f006
Show file tree
Hide file tree
Showing 6 changed files with 383 additions and 44 deletions.
28 changes: 27 additions & 1 deletion sql/core/src/main/scala/org/apache/spark/sql/SQLContext.scala
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,6 @@ class SQLContext(@transient val sparkContext: SparkContext)
extends org.apache.spark.Logging
with CacheManager
with ExpressionConversions
with UDFRegistration
with Serializable {

self =>
Expand Down Expand Up @@ -338,6 +337,33 @@ class SQLContext(@transient val sparkContext: SparkContext)
*/
val experimental: ExperimentalMethods = new ExperimentalMethods(this)

/**
* A collection of methods for registering user-defined functions (UDF).
*
* The following example registers a Scala closure as UDF:
* {{{
* sqlContext.udf.register("myUdf", (arg1: Int, arg2: String) => arg2 + arg1)
* }}}
*
* The following example registers a UDF in Java:
* {{{
* sqlContext.udf().register("myUDF", DataTypes.StringType,
* new UDF2<Integer, String, String>() {
* @Override
* public String call(Integer arg1, String arg2) {
* return arg2 + arg1;
* }
* });
* }}}
*
* Or, to use Java 8 lambda syntax:
* {{{
* sqlContext.udf().register("myUDF", DataTypes.StringType,
* (Integer arg1, String arg2) -> arg2 + arg1));
* }}}
*/
val udf: UDFRegistration = new UDFRegistration(this)

protected[sql] class SparkPlanner extends SparkStrategies {
val sparkContext: SparkContext = self.sparkContext

Expand Down
Loading

0 comments on commit 032f006

Please sign in to comment.