[SPARK-12615] Remove some deprecated APIs in RDD/SparkContext #10569
Conversation
| @@ -144,9 +131,6 @@ abstract class TaskContext extends Serializable { | |||
| */ | |||
| def attemptNumber(): Int | |||
|
|
|||
| @deprecated("use attemptNumber", "1.3.0") | |||
| def attemptId(): Long | |||
There was a problem hiding this comment.
I'm so happy to see this go... the old name was the subject of much confusion.
rxin
changed the title
|
Test build #48641 has finished for PR 10569 at commit
|
|
+1. On this note, I am about to submit a change that fixes about 75% of the remaining build warnings, most due to deprecated API usages. Not sure whether it's easier to merge that first since it will remove usages of these methods in the tests and examples that I think are otherwise going to fail. |
|
Test build #2313 has started for PR 10569 at commit |
|
Test build #2314 has finished for PR 10569 at commit
|
|
Test build #2316 has finished for PR 10569 at commit
|
| @@ -87,7 +87,7 @@ class HiveSparkSubmitSuite | |||
| runSparkSubmit(args) | |||
| } | |||
|
|
|||
| test("SPARK-8489: MissingRequirementError during reflection") { | |||
| ignore("SPARK-8489: MissingRequirementError during reflection") { | |||
There was a problem hiding this comment.
@andrewor14 this test case is failing due to binary compatibility.
There was a problem hiding this comment.
Let's file a followup JIRA for this one.
There was a problem hiding this comment.
|
Test build #48733 has finished for PR 10569 at commit
|
|
Test build #48735 has finished for PR 10569 at commit
|
rxin
changed the title
| @@ -34,10 +34,6 @@ case class Aggregator[K, V, C] ( | |||
| mergeValue: (C, V) => C, | |||
| mergeCombiners: (C, C) => C) { | |||
|
|
|||
| @deprecated("use combineValuesByKey with TaskContext argument", "0.9.0") | |||
| def combineValuesByKey(iter: Iterator[_ <: Product2[K, V]]): Iterator[(K, C)] = | |||
| combineValuesByKey(iter, null) | |||
There was a problem hiding this comment.
Ha, the old code here doesn't even make sense: why not pass TaskContext.get() instead of null? Glad to see this gone.
|
LGTM. I'm going to merge this now in order to avoid merge conflicts. Let's file a followup to deal with #10569 (diff), though. |
I looked at each case individually and it looks like they can all be removed. The only one that I had to think twice was toArray (I even thought about un-deprecating it, until I realized it was a problem in Java to have toArray returning java.util.List).