[SPARK-5123][SQL] Reconcile Java/Scala API for data types. #3958
Conversation
|
Test build #25272 has started for PR 3958 at commit
|
|
Test build #25272 has finished for PR 3958 at commit
|
|
Test FAILed. |
| trait PrimitiveType extends DataType { | ||
| override def isPrimitive = true | ||
| } | ||
|
|
||
|
|
|
Okay, the other PR is in, if you fix the merge conflicts this LGTM. We also need to figure out how to publish scala/java docs here. |
…ve the Java-specific API).
rxin
force-pushed
the
SPARK-5123-datatype-2
branch
from
b4f9649
to
66505cc
Compare
|
I just rebased. Let's see ... |
|
Test build #25487 has started for PR 3958 at commit
|
|
Test build #25487 has finished for PR 3958 at commit
|
|
Test FAILed. |
|
Jenkins, retest this please. |
|
Test build #25492 has started for PR 3958 at commit
|
|
You might want to copy the commit message over from the other PR. |
|
done - thanks for reminding, josh. |
rxin
changed the title
|
Ok merging in master. Thanks for looking at this. |
|
Test build #25492 has finished for PR 3958 at commit
|
|
Test PASSed. |
After the following patches, the main (Scala) API is now usable for Java users directly. #4056 #4054 #4049 #4030 #3965 #3958 Author: Reynold Xin <rxin@databricks.com> Closes #4065 from rxin/sql-java-api and squashes the following commits: b1fd860 [Reynold Xin] Fix Mima 6d86578 [Reynold Xin] Ok one more attempt in fixing Python... e8f1455 [Reynold Xin] Fix Python again... 3e53f91 [Reynold Xin] Fixed Python. 83735da [Reynold Xin] Fix BigDecimal test. e9f1de3 [Reynold Xin] Use scala BigDecimal. 500d2c4 [Reynold Xin] Fix Decimal. ba3bfa2 [Reynold Xin] Updated javadoc for RowFactory. c4ae1c5 [Reynold Xin] [SPARK-5193][SQL] Remove Spark SQL Java-specific API.
After the following patches, the main (Scala) API is now usable for Java users directly. apache/spark#4056 apache/spark#4054 apache/spark#4049 apache/spark#4030 apache/spark#3965 apache/spark#3958 Author: Reynold Xin <rxin@databricks.com> Closes #4065 from rxin/sql-java-api and squashes the following commits: b1fd860 [Reynold Xin] Fix Mima 6d86578 [Reynold Xin] Ok one more attempt in fixing Python... e8f1455 [Reynold Xin] Fix Python again... 3e53f91 [Reynold Xin] Fixed Python. 83735da [Reynold Xin] Fix BigDecimal test. e9f1de3 [Reynold Xin] Use scala BigDecimal. 500d2c4 [Reynold Xin] Fix Decimal. ba3bfa2 [Reynold Xin] Updated javadoc for RowFactory. c4ae1c5 [Reynold Xin] [SPARK-5193][SQL] Remove Spark SQL Java-specific API.
Having two versions of the data type APIs (one for Java, one for Scala) requires downstream libraries to also have two versions of the APIs if the library wants to support both Java and Scala. I took a look at the Scala version of the data type APIs - it can actually work out pretty well for Java out of the box.
As part of the PR, I created a sql.types package and moved all type definitions there. I then removed the Java specific data type API along with a lot of the conversion code.
This subsumes #3925