SPARK-1456 Remove view bounds on Ordered in favor of a context bound on Ordering. #410
Conversation
|
Merged build triggered. |
|
Merged build started. |
| @@ -89,12 +89,14 @@ class HashPartitioner(partitions: Int) extends Partitioner { | |||
| * A [[org.apache.spark.Partitioner]] that partitions sortable records by range into roughly | |||
| * equal ranges. The ranges are determined by sampling the content of the RDD passed in. | |||
| */ | |||
| class RangePartitioner[K <% Ordered[K]: ClassTag, V]( | |||
| class RangePartitioner[K : Ordering : ClassTag, V]( | |||
There was a problem hiding this comment.
Can the user still pass in an Ordering if they want a specific Ordering?
There was a problem hiding this comment.
Yes, when de-sugared there is an implicit Ordering -- that's why Michael can recover and bind it explicitly at line 98. And yes, it can be overridden in the normal way.
|
Merged build finished. All automated tests passed. |
|
All automated tests passed. |
| @@ -27,12 +27,14 @@ import org.apache.spark.{Logging, RangePartitioner} | |||
| * use these functions. They will work with any key type that has a `scala.math.Ordered` | |||
| * implementation. | |||
| */ | |||
| class OrderedRDDFunctions[K <% Ordered[K]: ClassTag, | |||
| class OrderedRDDFunctions[K : Ordering : ClassTag, | |||
There was a problem hiding this comment.
@marmbrus Do you mind updating the scaladoc above since now users can pass their own orderings in addition to an Ordered type? It might be nice to just give a one-line example of how they can define a custom ordering as well.
|
Merged build triggered. |
|
Merged build started. |
|
Merged build finished. All automated tests passed. |
|
All automated tests passed. |
| * } | ||
| * | ||
| * // Sort by key, using the above case insensitive ordering. | ||
| * rdd.sortByKey |
There was a problem hiding this comment.
a nit: put parenthesis around sortByKey
|
We should also update the Java API to create Ordering directly (instead of Ordered). It still compiles now, but kind of strange to have Ordered in the Java API that gets converted into Ordering. |
|
Merged build triggered. |
|
Merged build started. |
|
Merged build finished. All automated tests passed. |
|
All automated tests passed. |
|
lgtm |
|
merged. |
…on Ordering. This doesn't require creating new Ordering objects per row. Additionally, [view bounds are going to be deprecated](https://issues.scala-lang.org/browse/SI-7629), so we should get rid of them while APIs are still flexible. Author: Michael Armbrust <michael@databricks.com> Closes #410 from marmbrus/viewBounds and squashes the following commits: c574221 [Michael Armbrust] fix example. 812008e [Michael Armbrust] Update Java API. 1b9b85c [Michael Armbrust] Update scala doc. 35798a8 [Michael Armbrust] Remove view bounds on Ordered in favor of a context bound on Ordering. (cherry picked from commit c399baa) Signed-off-by: Reynold Xin <rxin@apache.org>
Updated JavaStreamingContext to make scaladoc compile. `sbt/sbt doc` used to fail. This fixed it.
…on Ordering. This doesn't require creating new Ordering objects per row. Additionally, [view bounds are going to be deprecated](https://issues.scala-lang.org/browse/SI-7629), so we should get rid of them while APIs are still flexible. Author: Michael Armbrust <michael@databricks.com> Closes apache#410 from marmbrus/viewBounds and squashes the following commits: c574221 [Michael Armbrust] fix example. 812008e [Michael Armbrust] Update Java API. 1b9b85c [Michael Armbrust] Update scala doc. 35798a8 [Michael Armbrust] Remove view bounds on Ordered in favor of a context bound on Ordering.
Do refactor for Ruby install
This doesn't require creating new Ordering objects per row. Additionally, view bounds are going to be deprecated, so we should get rid of them while APIs are still flexible.