Please sign in to comment.
[SPARK-23121][CORE] Fix for ui becoming unaccessible for long running…
… streaming apps ## What changes were proposed in this pull request? The allJobs and the job pages attempt to use stage attempt and DAG visualization from the store, but for long running jobs they are not guaranteed to be retained, leading to exceptions when these pages are rendered. To fix it `store.lastStageAttempt(stageId)` and `store.operationGraphForJob(jobId)` are wrapped in `store.asOption` and default values are used if the info is missing. ## How was this patch tested? Manual testing of the UI, also using the test command reported in SPARK-23121: ./bin/spark-submit --class org.apache.spark.examples.streaming.HdfsWordCount ./examples/jars/spark-examples_2.11-2.4.0-SNAPSHOT.jar /spark Closes #20287 Author: Sandor Murakozi <firstname.lastname@example.org> Closes #20330 from smurakozi/SPARK-23121. (cherry picked from commit 446948a) Signed-off-by: Marcelo Vanzin <email@example.com>
- Loading branch information...
Showing with 27 additions and 16 deletions.