Skip to content

Commit

Permalink
[SPARK-7317] [Shuffle] Expose shuffle handle
Browse files Browse the repository at this point in the history
Details in JIRA, in a nut-shell, all machinary for custom RDD's to leverage spark shuffle directly (without exposing impl details of shuffle) exists - except for this small piece.

Exposing this will allow for custom dependencies to get a handle to ShuffleHandle - which they can then leverage on reduce side.

Author: Mridul Muralidharan <mridulm@yahoo-inc.com>

Closes apache#5857 from mridulm/expose_shuffle_handle and squashes the following commits:

d8b6bd4 [Mridul Muralidharan] Expose ShuffleHandle
  • Loading branch information
Mridul Muralidharan authored and rxin committed May 2, 2015
1 parent 38d4e9e commit b79aeb9
Showing 1 changed file with 4 additions and 1 deletion.
Original file line number Diff line number Diff line change
Expand Up @@ -17,9 +17,12 @@

package org.apache.spark.shuffle

import org.apache.spark.annotation.DeveloperApi

/**
* An opaque handle to a shuffle, used by a ShuffleManager to pass information about it to tasks.
*
* @param shuffleId ID of the shuffle
*/
private[spark] abstract class ShuffleHandle(val shuffleId: Int) extends Serializable {}
@DeveloperApi
abstract class ShuffleHandle(val shuffleId: Int) extends Serializable {}

0 comments on commit b79aeb9

Please sign in to comment.