Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[SPARK-5274][SQL] Reconcile Java and Scala UDFRegistration. #4056

Closed
wants to merge 5 commits into from
Closed

[SPARK-5274][SQL] Reconcile Java and Scala UDFRegistration. #4056

wants to merge 5 commits into from

Conversation

rxin
Copy link
Contributor

@rxin rxin commented Jan 15, 2015

As part of SPARK-5193:

  1. Removed UDFRegistration as a mixin in SQLContext and made it a field ("udf").
  2. For Java UDFs, renamed dataType to returnType.
  3. For Scala UDFs, added type tags.
  4. Added all Java UDF registration methods to Scala's UDFRegistration.
  5. Documentation

@rxin
[SPARK-5193][SQL] Reconcile Java and Scala UDFRegistration.
032f006 
1. Removed UDFRegistration as a mixin in SQLContext and made it a field ("udf").
2. For Java UDFs, renamed dataType to returnType, and made that the 2nd argument since the UDF itself can be long.
3. Added all Java UDF registration methods to Scala's UDFRegistration.
4. Documentation
@SparkQA
Copy link

SparkQA commented Jan 15, 2015

Test build #25594 has started for PR 4056 at commit 032f006.

  • This patch merges cleanly.

@SparkQA
Copy link

SparkQA commented Jan 15, 2015

Test build #25594 has finished for PR 4056 at commit 032f006.

  • This patch fails PySpark unit tests.
  • This patch merges cleanly.
  • This patch adds the following public classes (experimental):
    • class UDFRegistration (sqlContext: SQLContext) extends org.apache.spark.Logging

@AmplabJenkins
Copy link

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/25594/
Test FAILed.

@SparkQA
Copy link

SparkQA commented Jan 15, 2015

Test build #25615 has started for PR 4056 at commit 5f00c45.

  • This patch merges cleanly.

@rxin rxin changed the title [SPARK-5193][SQL] Reconcile Java and Scala UDFRegistration. [SPARK-5274][SQL] Reconcile Java and Scala UDFRegistration. Jan 15, 2015
@SparkQA
Copy link

SparkQA commented Jan 15, 2015

Test build #25615 has finished for PR 4056 at commit 5f00c45.

  • This patch fails Scala style tests.
  • This patch merges cleanly.
  • This patch adds the following public classes (experimental):
    • class UDFRegistration (sqlContext: SQLContext) extends org.apache.spark.Logging

@AmplabJenkins
Copy link

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/25615/
Test FAILed.

@SparkQA
Copy link

SparkQA commented Jan 15, 2015

Test build #25616 has started for PR 4056 at commit 675a3c9.

  • This patch merges cleanly.

@SparkQA
Copy link

SparkQA commented Jan 15, 2015

Test build #25618 has started for PR 4056 at commit ae9c556.

  • This patch merges cleanly.

@SparkQA
Copy link

SparkQA commented Jan 15, 2015

Test build #25616 has finished for PR 4056 at commit 675a3c9.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds the following public classes (experimental):
    • class UDFRegistration (sqlContext: SQLContext) extends org.apache.spark.Logging

@AmplabJenkins
Copy link

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/25616/
Test PASSed.

@rxin
Copy link
Contributor Author

rxin commented Jan 16, 2015

Merging in master.

@asfgit asfgit closed this in 1881431 Jan 16, 2015
@SparkQA
Copy link

SparkQA commented Jan 16, 2015

Test build #25618 has finished for PR 4056 at commit ae9c556.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds the following public classes (experimental):
    • class UDFRegistration (sqlContext: SQLContext) extends org.apache.spark.Logging

@AmplabJenkins
Copy link

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/25618/
Test PASSed.

@rxin rxin mentioned this pull request Jan 16, 2015
Closed
asfgit pushed a commit that referenced this pull request Jan 17, 2015
@rxin
[SPARK-5193][SQL] Remove Spark SQL Java-specific API.
61b427d 
After the following patches, the main (Scala) API is now usable for Java users directly.

#4056
#4054
#4049
#4030
#3965
#3958

Author: Reynold Xin <rxin@databricks.com>

Closes #4065 from rxin/sql-java-api and squashes the following commits:

b1fd860 [Reynold Xin] Fix Mima
6d86578 [Reynold Xin] Ok one more attempt in fixing Python...
e8f1455 [Reynold Xin] Fix Python again...
3e53f91 [Reynold Xin] Fixed Python.
83735da [Reynold Xin] Fix BigDecimal test.
e9f1de3 [Reynold Xin] Use scala BigDecimal.
500d2c4 [Reynold Xin] Fix Decimal.
ba3bfa2 [Reynold Xin] Updated javadoc for RowFactory.
c4ae1c5 [Reynold Xin] [SPARK-5193][SQL] Remove Spark SQL Java-specific API.
andmarios pushed a commit to andmarios/spark-examples that referenced this pull request Mar 20, 2015
@rxin
[SPARK-5193][SQL] Remove Spark SQL Java-specific API.
3364cc8 
After the following patches, the main (Scala) API is now usable for Java users directly.

apache/spark#4056
apache/spark#4054
apache/spark#4049
apache/spark#4030
apache/spark#3965
apache/spark#3958

Author: Reynold Xin <rxin@databricks.com>

Closes #4065 from rxin/sql-java-api and squashes the following commits:

b1fd860 [Reynold Xin] Fix Mima
6d86578 [Reynold Xin] Ok one more attempt in fixing Python...
e8f1455 [Reynold Xin] Fix Python again...
3e53f91 [Reynold Xin] Fixed Python.
83735da [Reynold Xin] Fix BigDecimal test.
e9f1de3 [Reynold Xin] Use scala BigDecimal.
500d2c4 [Reynold Xin] Fix Decimal.
ba3bfa2 [Reynold Xin] Updated javadoc for RowFactory.
c4ae1c5 [Reynold Xin] [SPARK-5193][SQL] Remove Spark SQL Java-specific API.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
3 participants
@rxin @SparkQA @AmplabJenkins