Join GitHub today
GitHub is home to over 50 million developers working together to host and review code, manage projects, and build software together.
Sign up[SPARK-16599][CORE] java.util.NoSuchElementException: None.get at at org.apache.spark.storage.BlockInfoManager.releaseAllLocksForTask #17290
Conversation
|
CC @JoshRosen |
SparkQA
commented
Mar 14, 2017
|
Test build #74523 has finished for PR 17290 at commit
|
SparkQA
commented
Mar 14, 2017
|
Test build #74526 has finished for PR 17290 at commit
|
|
I'd like to merge this as a step towards possibly resolving the problem or uncovering its root. |
|
Merged to master. I don't believe this necessarily resolves the JIRA. |
| @@ -340,7 +340,7 @@ private[storage] class BlockInfoManager extends Logging { | |||
| val blocksWithReleasedLocks = mutable.ArrayBuffer[BlockId]() | |||
|
|
|||
| val readLocks = synchronized { | |||
| readLocksByTask.remove(taskAttemptId).get | |||
| readLocksByTask.remove(taskAttemptId).getOrElse(ImmutableMultiset.of[BlockId]()) | |||
This comment has been minimized.
This comment has been minimized.
mridulm
Mar 18, 2017
Contributor
Missed this PR earlier .. sorry about that.
Since this is an unexpected scenario (it really must not happen according to current code !), we should probably have logged the stack trace and warning message in the logs to help debug.
It silently ignores a fairly bad bug otherwise.
|
That's fine I can add a warning. I don't know if it is a bug situation but it sure could be. |
|
As found in #17338, I am afraid this one caused the following test failure.
|
|
After removing the changes in this PR, the test case passed in my local environment. I think we need to revert it back. |
|
Without code changes, https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/74836/ still can pass after triggering a retest. I think it is not related to this PR. |
|
Hm, that concerns me @gatorsmile . Without this change I might expect that it would have also failed, just in the same way the JIRA describes. I could add a warning for the case where there is no read lock acquired here. Is your gut feeling that this is risky? I can back it out too. |
|
I agree with @srowen I dont see how this change affects the test. |
aphasingnirvana
commented
Oct 13, 2017
|
@srowen I believe the bug still persists. Shouldn't we reopen it? |
|
I don't think this change was the ultimate fix, and it caused another problem, so no I don't think this PR should be reopened. |
aphasingnirvana
commented
Oct 17, 2017
|
@srowen So could I open another issue altogether? |
|
No, https://issues.apache.org/jira/browse/SPARK-16599 is still open |
srowen commentedMar 14, 2017
What changes were proposed in this pull request?
Avoid None.get exception in (rare?) case that no readLocks exist
Note that while this would resolve the immediate cause of the exception, it's not clear it is the root problem.
How was this patch tested?
Existing tests