[SPARK-3027] TaskContext: tighten visibility and provide Java friendly callback API by rxin · Pull Request #1938 · apache/spark

rxin · 2014-08-14T06:17:00Z

Note this also passes the TaskContext itself to the TaskCompletionListener. In the future we can mark TaskContext with the exception object if exception occurs during task execution.

…endly callback API

SparkQA · 2014-08-14T06:19:52Z

QA tests have started for PR 1938. This patch merges cleanly.
View progress: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/18524/consoleFull

SparkQA · 2014-08-14T06:20:00Z

QA results for PR 1938:
- This patch FAILED unit tests.
- This patch merges cleanly
- This patch adds the following public classes (experimental):
trait TaskCompletionListener extends EventListener {

For more information see test ouptut:
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/18524/consoleFull

SparkQA · 2014-08-14T06:24:52Z

QA tests have started for PR 1938. This patch merges cleanly.
View progress: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/18525/consoleFull

SparkQA · 2014-08-14T07:12:09Z

QA results for PR 1938:
- This patch FAILED unit tests.
- This patch merges cleanly
- This patch adds the following public classes (experimental):
trait TaskCompletionListener extends EventListener {

For more information see test ouptut:
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/18525/consoleFull

rxin · 2014-08-14T07:16:11Z

Jenkins, retest this please.

SparkQA · 2014-08-14T07:19:57Z

QA tests have started for PR 1938. This patch merges cleanly.
View progress: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/18529/consoleFull

SparkQA · 2014-08-14T08:07:15Z

QA results for PR 1938:
- This patch FAILED unit tests.
- This patch merges cleanly
- This patch adds the following public classes (experimental):
trait TaskCompletionListener extends EventListener {

For more information see test ouptut:
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/18529/consoleFull

rxin · 2014-08-14T08:32:17Z

Jenkins, retest this please.

SparkQA · 2014-08-14T08:34:53Z

QA tests have started for PR 1938. This patch merges cleanly.
View progress: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/18532/consoleFull

SparkQA · 2014-08-14T09:21:36Z

QA results for PR 1938:
- This patch FAILED unit tests.
- This patch merges cleanly
- This patch adds the following public classes (experimental):
trait TaskCompletionListener extends EventListener {

For more information see test ouptut:
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/18532/consoleFull

rxin · 2014-08-14T18:23:12Z

Jenkins, retest this please.

rxin · 2014-08-14T18:23:24Z

None of the previous test failures are caused by this PR ...

JoshRosen · 2014-08-14T20:49:22Z

Jenkins, retest this please.

JoshRosen · 2014-08-14T21:03:57Z

core/src/main/scala/org/apache/spark/TaskContext.scala

I guess another option would be an implicit conversion from TaskContext => Unit to TaskCompletionListener, but that would require the user to import the implicits (maybe they could be included in the SparkContext implicits, though).

yea, implicits are evil.

SparkQA · 2014-08-14T21:10:29Z

QA tests have started for PR 1938. This patch merges cleanly.
View progress: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/18568/consoleFull

JoshRosen · 2014-08-14T21:12:56Z

Now that you've added a Java implementation of the new listener interface, this looks good to me.

rxin · 2014-08-14T23:30:56Z

Jenkins, retest this please.

rxin · 2014-08-15T00:37:12Z

Jenkins, retest this please.

SparkQA · 2014-08-15T00:40:03Z

QA tests have started for PR 1938. This patch merges cleanly.
View progress: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/18585/consoleFull

SparkQA · 2014-08-15T01:35:48Z

QA results for PR 1938:
- This patch PASSES unit tests.
- This patch merges cleanly
- This patch adds the following public classes (experimental):
trait TaskCompletionListener extends EventListener {

For more information see test ouptut:
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/18585/consoleFull

rxin · 2014-08-15T01:37:13Z

Thanks. Merging this in master & branch-1.1.

…y callback API Note this also passes the TaskContext itself to the TaskCompletionListener. In the future we can mark TaskContext with the exception object if exception occurs during task execution. Author: Reynold Xin <rxin@apache.org> Closes #1938 from rxin/TaskContext and squashes the following commits: 145de43 [Reynold Xin] Added JavaTaskCompletionListenerImpl for Java API friendly guarantee. f435ea5 [Reynold Xin] Added license header for TaskCompletionListener. dc4ed27 [Reynold Xin] [SPARK-3027] TaskContext: tighten the visibility and provide Java friendly callback API (cherry picked from commit 655699f) Signed-off-by: Reynold Xin <rxin@apache.org>

…y callback API Note this also passes the TaskContext itself to the TaskCompletionListener. In the future we can mark TaskContext with the exception object if exception occurs during task execution. Author: Reynold Xin <rxin@apache.org> Closes apache#1938 from rxin/TaskContext and squashes the following commits: 145de43 [Reynold Xin] Added JavaTaskCompletionListenerImpl for Java API friendly guarantee. f435ea5 [Reynold Xin] Added license header for TaskCompletionListener. dc4ed27 [Reynold Xin] [SPARK-3027] TaskContext: tighten the visibility and provide Java friendly callback API

[SPARK-3027] TaskContext: tighten the visibility and provide Java fri…

dc4ed27

…endly callback API

Added license header for TaskCompletionListener.

f435ea5

rxin changed the title ~~[SPARK-3027] TaskContext: tighten the visibility and provide Java friendly callback API~~ [SPARK-3027] TaskContext: tighten visibility and provide Java friendly callback API Aug 14, 2014

Added JavaTaskCompletionListenerImpl for Java API friendly guarantee.

145de43

JoshRosen reviewed Aug 14, 2014
View reviewed changes

asfgit closed this in 655699f Aug 15, 2014

Conversation

rxin commented Aug 14, 2014

Uh oh!

SparkQA commented Aug 14, 2014

Uh oh!

SparkQA commented Aug 14, 2014

Uh oh!

SparkQA commented Aug 14, 2014

Uh oh!

SparkQA commented Aug 14, 2014

Uh oh!

rxin commented Aug 14, 2014

Uh oh!

SparkQA commented Aug 14, 2014

Uh oh!

SparkQA commented Aug 14, 2014

Uh oh!

rxin commented Aug 14, 2014

Uh oh!

SparkQA commented Aug 14, 2014

Uh oh!

SparkQA commented Aug 14, 2014

Uh oh!

rxin commented Aug 14, 2014

Uh oh!

rxin commented Aug 14, 2014

Uh oh!

JoshRosen commented Aug 14, 2014

Uh oh!

JoshRosen Aug 14, 2014

Choose a reason for hiding this comment

Uh oh!

rxin Aug 14, 2014

Choose a reason for hiding this comment

Uh oh!

SparkQA commented Aug 14, 2014

Uh oh!

JoshRosen commented Aug 14, 2014

Uh oh!

rxin commented Aug 14, 2014

Uh oh!

rxin commented Aug 15, 2014

Uh oh!

SparkQA commented Aug 15, 2014

Uh oh!

SparkQA commented Aug 15, 2014

Uh oh!

rxin commented Aug 15, 2014

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants