Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[SPARK-10369][Streaming]Don't remove ReceiverTrackingInfo when deregisterReceivering since we may reuse it later #8538

Closed
wants to merge 1 commit into from
Closed

[SPARK-10369][Streaming]Don't remove ReceiverTrackingInfo when deregisterReceivering since we may reuse it later #8538

wants to merge 1 commit into from

Conversation

zsxwing
Copy link
Member

@zsxwing zsxwing commented Aug 31, 2015

deregisterReceiver should not remove ReceiverTrackingInfo. Otherwise, it will throw java.util.NoSuchElementException: key not found when restarting it.

@SparkQA
Copy link

SparkQA commented Aug 31, 2015

Test build #41831 has finished for PR 8538 at commit 51c52d4.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

tdas
tdas reviewed Aug 31, 2015
View reviewed changes
streaming/src/test/scala/org/apache/spark/streaming/scheduler/ReceiverTrackerSuite.scala
}
})
val input = ssc.receiverStream(new StoppableReceiver)
val output = new TestOutputStream(input)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

input.register is sufficient

@tdas
Copy link
Contributor

tdas commented Aug 31, 2015

Ignore the nit and merging this to master and 1.5. Thanks @zsxwing for finding and catching this.

asfgit pushed a commit that referenced this pull request Aug 31, 2015
@zsxwing @tdas
[SPARK-10369] [STREAMING] Don't remove ReceiverTrackingInfo when dere…
33ce274 
…gisterReceivering since we may reuse it later

`deregisterReceiver` should not remove `ReceiverTrackingInfo`. Otherwise, it will throw `java.util.NoSuchElementException: key not found` when restarting it.

Author: zsxwing <zsxwing@gmail.com>

Closes #8538 from zsxwing/SPARK-10369.

(cherry picked from commit 4a5fe09)
Signed-off-by: Tathagata Das <tathagata.das1565@gmail.com>
@asfgit asfgit closed this in 4a5fe09 Aug 31, 2015
@zsxwing zsxwing deleted the SPARK-10369 branch September 1, 2015 01:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
3 participants
@zsxwing @SparkQA @tdas