[SPARK-31004][WEBUI][SS] Show message for empty Streaming Queries instead of empty timelines and histograms. #27755

sarutak · 2020-03-02T03:17:33Z

What changes were proposed in this pull request?

StreamingQueryStatisticsPage shows a message "No visualization information available because there is no batches" instead of showing empty timelines and histograms for empty streaming queries.

[Before this change applied]

[After this change applied]

Why are the changes needed?

Empty charts are ugly and a little bit confusing.
It's better to clearly say "No visualization information available".

Also, this change fixes a JS error shown in the capture above.
This error occurs because drawTimeline in streaming-page.js is called even though formattedDate will be undefined for empty streaming queries.

Does this PR introduce any user-facing change?

Yes. screen captures are shown above.

How was this patch tested?

Manually tested by creating an empty streaming query like as follows.

val df = spark.readStream.format("socket").options(Map("host"->"<non-existing hostname>", "port"->"...")).load
df.writeStream.format("console").start

This streaming query will fail because of non-existing hostname and has no batches.

HeartSaVioR · 2020-03-02T06:58:37Z

sql/core/src/main/scala/org/apache/spark/sql/streaming/ui/StreamingQueryStatisticsPage.scala

-    // scalastyle:on
+    } else {
+      <div id="empty-streaming-query-message">
+        <b>No visualization information available because there is no batch.</b>


there is no batch - this might be slightly misleading if the query was running with some batches and stopped, and restarted with checkpoint. I can't think of good alternative though as of now...

So how about just simply saying "No visualization information available" ?

I'm OK with that - maybe hearing more voices gives more options, hopefully.

SparkQA · 2020-03-02T07:59:06Z

Test build #119148 has finished for PR 27755 at commit 0044aae.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

SparkQA · 2020-03-02T20:35:58Z

Test build #119177 has finished for PR 27755 at commit abe8679.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

HeartSaVioR · 2020-03-02T23:21:17Z

sql/core/src/main/scala/org/apache/spark/sql/streaming/ui/StreamingQueryStatisticsPage.scala

      <table id="stat-table" class="table table-bordered" style="width: auto">
        <thead>
          <tr>
            <th style="width: 160px;"></th>
            <th style="width: 492px;">Timelines</th>
-            <th style="width: 350px;">Histograms</th></tr>
+            <th style="width: 350px;">Histograms</th>
+          </tr>


Nice catch!

HeartSaVioR

LGTM. We may be able to still open to hear the opinions on the "message" but IMHO that can be changed as follow-up minor PR as well.

sarutak · 2020-03-06T16:33:55Z

cc: @zsxwing @gengliangwang

planga82

Thanks for the work, I like the message in that way.

gengliangwang · 2020-03-09T04:22:25Z

@sarutak Great catch!

gengliangwang · 2020-03-13T03:59:38Z

retest this please.

SparkQA · 2020-03-13T07:05:02Z

Test build #119741 has finished for PR 27755 at commit abe8679.

This patch fails due to an unknown error code, -9.
This patch merges cleanly.
This patch adds no public classes.

gengliangwang · 2020-03-13T07:18:25Z

retest this please.

zsxwing · 2020-03-13T07:22:18Z

LGTM

SparkQA · 2020-03-13T11:10:37Z

Test build #119747 has finished for PR 27755 at commit abe8679.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

gengliangwang · 2020-03-13T19:58:09Z

Thanks, merging to master/3.0

…tead of empty timelines and histograms ### What changes were proposed in this pull request? `StreamingQueryStatisticsPage` shows a message "No visualization information available because there is no batches" instead of showing empty timelines and histograms for empty streaming queries. [Before this change applied] ![before-fix-for-empty-streaming-query](https://user-images.githubusercontent.com/4736016/75642391-b32e1d80-5c7e-11ea-9c07-e2f0f1b5b4f9.png) [After this change applied] ![after-fix-for-empty-streaming-query2](https://user-images.githubusercontent.com/4736016/75694583-1904be80-5cec-11ea-9b13-dc7078775188.png) ### Why are the changes needed? Empty charts are ugly and a little bit confusing. It's better to clearly say "No visualization information available". Also, this change fixes a JS error shown in the capture above. This error occurs because `drawTimeline` in `streaming-page.js` is called even though `formattedDate` will be `undefined` for empty streaming queries. ### Does this PR introduce any user-facing change? Yes. screen captures are shown above. ### How was this patch tested? Manually tested by creating an empty streaming query like as follows. ``` val df = spark.readStream.format("socket").options(Map("host"->"<non-existing hostname>", "port"->"...")).load df.writeStream.format("console").start ``` This streaming query will fail because of `non-existing hostname` and has no batches. Closes #27755 from sarutak/fix-for-empty-batches. Authored-by: Kousuke Saruta <sarutak@oss.nttdata.com> Signed-off-by: Gengliang Wang <gengliang.wang@databricks.com> (cherry picked from commit 6809815) Signed-off-by: Gengliang Wang <gengliang.wang@databricks.com>

…tead of empty timelines and histograms ### What changes were proposed in this pull request? `StreamingQueryStatisticsPage` shows a message "No visualization information available because there is no batches" instead of showing empty timelines and histograms for empty streaming queries. [Before this change applied] ![before-fix-for-empty-streaming-query](https://user-images.githubusercontent.com/4736016/75642391-b32e1d80-5c7e-11ea-9c07-e2f0f1b5b4f9.png) [After this change applied] ![after-fix-for-empty-streaming-query2](https://user-images.githubusercontent.com/4736016/75694583-1904be80-5cec-11ea-9b13-dc7078775188.png) ### Why are the changes needed? Empty charts are ugly and a little bit confusing. It's better to clearly say "No visualization information available". Also, this change fixes a JS error shown in the capture above. This error occurs because `drawTimeline` in `streaming-page.js` is called even though `formattedDate` will be `undefined` for empty streaming queries. ### Does this PR introduce any user-facing change? Yes. screen captures are shown above. ### How was this patch tested? Manually tested by creating an empty streaming query like as follows. ``` val df = spark.readStream.format("socket").options(Map("host"->"<non-existing hostname>", "port"->"...")).load df.writeStream.format("console").start ``` This streaming query will fail because of `non-existing hostname` and has no batches. Closes apache#27755 from sarutak/fix-for-empty-batches. Authored-by: Kousuke Saruta <sarutak@oss.nttdata.com> Signed-off-by: Gengliang Wang <gengliang.wang@databricks.com>

Add message for empty streaming query instead of drawing empty table.

0044aae

HeartSaVioR reviewed Mar 2, 2020

View reviewed changes

Tweak the message

abe8679

HeartSaVioR reviewed Mar 2, 2020

View reviewed changes

HeartSaVioR approved these changes Mar 2, 2020

View reviewed changes

planga82 approved these changes Mar 8, 2020

View reviewed changes

gengliangwang closed this in 6809815 Mar 13, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[SPARK-31004][WEBUI][SS] Show message for empty Streaming Queries instead of empty timelines and histograms. #27755

[SPARK-31004][WEBUI][SS] Show message for empty Streaming Queries instead of empty timelines and histograms. #27755

sarutak commented Mar 2, 2020 •

edited

HeartSaVioR Mar 2, 2020

sarutak Mar 2, 2020

HeartSaVioR Mar 2, 2020

SparkQA commented Mar 2, 2020

SparkQA commented Mar 2, 2020

HeartSaVioR Mar 2, 2020

HeartSaVioR left a comment

sarutak commented Mar 6, 2020

planga82 left a comment

gengliangwang commented Mar 9, 2020 •

edited

gengliangwang commented Mar 13, 2020

SparkQA commented Mar 13, 2020

gengliangwang commented Mar 13, 2020

zsxwing commented Mar 13, 2020

SparkQA commented Mar 13, 2020

gengliangwang commented Mar 13, 2020

[SPARK-31004][WEBUI][SS] Show message for empty Streaming Queries instead of empty timelines and histograms. #27755

[SPARK-31004][WEBUI][SS] Show message for empty Streaming Queries instead of empty timelines and histograms. #27755

Conversation

sarutak commented Mar 2, 2020 • edited

What changes were proposed in this pull request?

Why are the changes needed?

Does this PR introduce any user-facing change?

How was this patch tested?

HeartSaVioR Mar 2, 2020

Choose a reason for hiding this comment

sarutak Mar 2, 2020

Choose a reason for hiding this comment

HeartSaVioR Mar 2, 2020

Choose a reason for hiding this comment

SparkQA commented Mar 2, 2020

SparkQA commented Mar 2, 2020

HeartSaVioR Mar 2, 2020

Choose a reason for hiding this comment

HeartSaVioR left a comment

Choose a reason for hiding this comment

sarutak commented Mar 6, 2020

planga82 left a comment

Choose a reason for hiding this comment

gengliangwang commented Mar 9, 2020 • edited

gengliangwang commented Mar 13, 2020

SparkQA commented Mar 13, 2020

gengliangwang commented Mar 13, 2020

zsxwing commented Mar 13, 2020

SparkQA commented Mar 13, 2020

gengliangwang commented Mar 13, 2020

sarutak commented Mar 2, 2020 •

edited

gengliangwang commented Mar 9, 2020 •

edited