[SPARK-25720][WEBUI] Support auto refresh page for the WEBUI

[shahidki31]

What changes were proposed in this pull request?

Currently spark webui doesn't have an option of auto refresh page. Because of this, user has to reload
every time to see the updates in the page.

For SQL page and Streaming page, currently we are passing the refresh interval, but it is not implemented in the 'headerSparkPage' method. This PR, added the refresh functionality to the webui.

How was this patch tested?

Manually verified.
Every 5 seconds the SQL page is refreshing.

[shahidki31]

cc @srowen @cloud-fan . Kindly review.

[AmplabJenkins]

Can one of the admins verify this patch?

[srowen]

Shouldn't it go in commonHeaderNodes?

Looks like this was added waaay back in JoshRosen@6aa08c3 by @tdas but never used. Only a few call sites set it. The problem I can see is that those hard-code it to 5 second refreshes.

I think we'd have to introduce a new parameter to control this, like spark.ui.refreshInterval or something, and then stop passing it into this method and just read it in this code somewhere?

[shahidki31]

Thank you @srowen for the comment. Yes, we should not hard-code the refresh interval and we let the user to enable the parameter. I will update the code accordingly.

[felixcheung]

can this be passed in as a param in the url?

[gengliangwang]

BTW there is another approach: #21512

[shahidki31]

@gengliangwang Sorry, I didn't see the PR before opening this PR. Yes, that PR also for auto refreshing the webui.

I have taken the patch and checked the functionality, and it seems fine. Below is the screen shot of the ui.
Except the alignment of the tabs, functionality seems good.

After the patch

Before the patch

[shahidki31]

I am closing the PR, since already one PR is there for the webui auto refresh. Thanks.