Skip to content

Commit

Permalink
[SPARK-12298][SQL] Fix infinite loop in DataFrame.sortWithinPartitions
Browse files Browse the repository at this point in the history
Modifies the String overload to call the Column overload and ensures this is called in a test.

Author: Ankur Dave <ankurdave@gmail.com>

Closes #10271 from ankurdave/SPARK-12298.
  • Loading branch information
ankurdave authored and yhuai committed Dec 12, 2015
1 parent a0ff6d1 commit 1e799d6
Show file tree
Hide file tree
Showing 2 changed files with 3 additions and 3 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -609,7 +609,7 @@ class DataFrame private[sql](
*/
@scala.annotation.varargs
def sortWithinPartitions(sortCol: String, sortCols: String*): DataFrame = {
sortWithinPartitions(sortCol, sortCols : _*)
sortWithinPartitions((sortCol +: sortCols).map(Column(_)) : _*)
}

/**
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -1090,8 +1090,8 @@ class DataFrameSuite extends QueryTest with SharedSQLContext {
}

// Distribute into one partition and order by. This partition should contain all the values.
val df6 = data.repartition(1, $"a").sortWithinPartitions($"b".asc)
// Walk each partition and verify that it is sorted descending and not globally sorted.
val df6 = data.repartition(1, $"a").sortWithinPartitions("b")
// Walk each partition and verify that it is sorted ascending and not globally sorted.
df6.rdd.foreachPartition { p =>
var previousValue: Int = -1
var allSequential: Boolean = true
Expand Down

0 comments on commit 1e799d6

Please sign in to comment.