-
Notifications
You must be signed in to change notification settings - Fork 28.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[SPARK-6860][Streaming][WebUI] Fix the possible inconsistency of StreamingPage #5470
Conversation
…obProgressListener when creating the content
That makes some sense. It depends on the methods that mutate the listener also being synchronized on the object. It looks like they are, at a glance, but have you looked into that a bit? |
Test build #30076 has finished for PR 5470 at commit
|
Thank you for reminding me. Some public methods of |
Test build #30081 has finished for PR 5470 at commit
|
LGTM. The first commit fixes a problem that would indeed in rare cases cause the page to display an inconsistent view of the state, which is probably minor but still bad. The second synchronizes access to mutable data structures that were inconsistently synchronized. |
@srowen, do you think if this one should be merged to branch 1.3? |
I would back-port if anyone felt strongly about it; it seems like a theoretical and cosmetic problem, so didn't seem high priority. |
Fair. If necessary, I would help send a back-port pull request. |
Meh, I'll just back port it. It's low risk. |
…amingPage Because `StreamingPage.render` doesn't hold the `listener` lock when generating the content, the different parts of content may have some inconsistent values if `listener` updates its status at the same time. And it will confuse people. This PR added `listener.synchronized` to make sure we have a consistent view of StreamingJobProgressListener when creating the content. Author: zsxwing <zsxwing@gmail.com> Closes #5470 from zsxwing/SPARK-6860 and squashes the following commits: cec6f92 [zsxwing] Add missing 'synchronized' in StreamingJobProgressListener 7182498 [zsxwing] Add synchronized to make sure we have a consistent view of StreamingJobProgressListener when creating the content
Because
StreamingPage.render
doesn't hold thelistener
lock when generating the content, the different parts of content may have some inconsistent values iflistener
updates its status at the same time. And it will confuse people.This PR added
listener.synchronized
to make sure we have a consistent view of StreamingJobProgressListener when creating the content.