[SPARK-33198][CORE] getMigrationBlocks should not fail at missing files#30110
[SPARK-33198][CORE] getMigrationBlocks should not fail at missing files#30110dongjoon-hyun wants to merge 3 commits intoapache:masterfrom dongjoon-hyun:SPARK-33198
Conversation
core/src/main/scala/org/apache/spark/shuffle/IndexShuffleBlockResolver.scala
Outdated
Show resolved
Hide resolved
|
Thank you for review and approval, @holdenk ! |
|
Kubernetes integration test starting |
|
Kubernetes integration test status failure |
|
Kubernetes integration test starting |
|
Kubernetes integration test status success |
|
GitHub Action and K8s IT passed. Merged to master. |
|
Test build #130053 has finished for PR 30110 at commit
|
|
Test build #130054 has finished for PR 30110 at commit
|
|
Test build #130055 has finished for PR 30110 at commit
|
### What changes were proposed in this pull request? This PR aims to fix `getMigrationBlocks` error handling and to add test coverage. 1. `getMigrationBlocks` should not fail at indexFile only case. 2. `assert` causes `java.lang.AssertionError` which is not an `Exception`. ### Why are the changes needed? To handle the exception correctly. ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? Pass the CI with the newly added test case. Closes apache#30110 from dongjoon-hyun/SPARK-33198. Authored-by: Dongjoon Hyun <dhyun@apple.com> Signed-off-by: Dongjoon Hyun <dhyun@apple.com>
What changes were proposed in this pull request?
This PR aims to fix
getMigrationBlockserror handling and to add test coverage.getMigrationBlocksshould not fail at indexFile only case.assertcausesjava.lang.AssertionErrorwhich is not anException.Why are the changes needed?
To handle the exception correctly.
Does this PR introduce any user-facing change?
No.
How was this patch tested?
Pass the CI with the newly added test case.