Improvements and fixes to the Host Overview dashboard #5340

tsg · 2017-10-05T22:05:10Z

Use beat.name instead of beat.hostname in visualizations. See 6.0 Metricbeat dashboards switched name key? #5276 for
the motivation
Add a "tip" widget that tells user how they can select another host.
Automatically set a search pattern of the form beat.name: "HOSTNAME" where
HOSTNAME is the Beat name (hostname) that uploads the dashboards.

The way the last point works is that I saved the dashbaord using this filter:
beat.name:"CHANGEME_HOSTNAME". The kibana loader code does a string replacement
and replaces CHANGEME_HOSTNAME with the actual hostname. The disadvantage of this
approach (vs doing JSON parsing) is that we must remember to always save that
dashboard with the CHANGEME_HOSTNAME wildcard in place. I have added a unit test
to the system module that checks for that, so it should be relatively hard for
us to forget.

Closes #5276.

* Use `beat.name` instead of `beat.hostname` in visualizations. See elastic#5276 for the motivation * Add a "tip" widget that tells user how they can select another host. * Automatically set a search pattern of the form `beat.name: "HOSTNAME"` where HOSTNAME is the Beat name (hostname) that uploads the dashboards. The way the last point works is that I saved the dashbaord using this filter: `beat.name:"CHANGEME_HOSTNAME"`. The kibana loader code does a string replacement and replaces `CHANGEME_HOSTNAME` with the actual hostname. The disadvantage of this approach (vs doing JSON parsing) is that we must remember to always save that dashboard with the `CHANGEME_HOSTNAME` wildcard in place. I have added a unit test to the system module that checks for that, so it should be relatively hard for us to forget. Closes elastic#5276.

tsg · 2017-10-05T22:10:02Z

ruflin

LGTM. Few minor comment, but ok for me to merge as is as I think it's all intentional.

ruflin · 2017-10-06T11:22:49Z

CHANGELOG.asciidoc

@@ -47,6 +47,8 @@ https://github.com/elastic/beats/compare/v6.0.0-beta2...master[Check the HEAD di
 *Metricbeat*

 - Change field type of http header from nested to object {pull}5258[5258]
+- Use `beat.name` instead of `beat.hostname` in the Host Overview dashboard. {pull}5340[5340]
+- Auto-select a hostname (based on the host on which the Beat is running) in the Host Overview dashboard. {pull}5340[5340]


That is more a feature then a bug fix?

Depends on how you consider it. It's fixing the empty dashboard if no host is selected.

I'll move it to Added, I think it makes more sense there.

ruflin · 2017-10-06T11:24:39Z

metricbeat/module/system/_meta/kibana/default/dashboard/Metricbeat-containers-overview.json

@@ -13,7 +13,7 @@
      },
      "id": "Container-CPU-usage",
      "type": "visualization",
-      "version": 2
+      "version": 1


What do these version changes mean?

I think they can be ignored, it seems that the version is incremented everytime you save a dashboard in Kibana, but I don't think they have a relevance beyond that.

yes, we can ignore that for now.

ruflin · 2017-10-06T11:25:41Z

metricbeat/module/system/_meta/kibana/default/dashboard/Metricbeat-system-overview.json

@@ -67,13 +67,13 @@
          "searchSourceJSON": "{\"index\":\"metricbeat-*\",\"query\":{\"query\":{\"query_string\":{\"query\":\"*\",\"analyze_wildcard\":true,\"default_field\":\"*\"}},\"language\":\"lucene\"},\"filter\":[]}"
        },
        "title": "Hosts histogram by CPU usage [Metricbeat System]",
-        "uiStateJSON": "{\"vis\":{\"defaultColors\":{\"0% - 5%\":\"rgb(247,252,245)\",\"5% - 10%\":\"rgb(199,233,192)\",\"10% - 15%\":\"rgb(116,196,118)\",\"15% - 20%\":\"rgb(35,139,69)\"}}}",
+        "uiStateJSON": "{\"vis\":{\"defaultColors\":{\"0% - 20%\":\"rgb(247,252,245)\",\"20% - 40%\":\"rgb(199,233,192)\",\"40% - 60%\":\"rgb(116,196,118)\",\"60% - 80%\":\"rgb(35,139,69)\"}}}",


Did you change these on purpose?

Hmm, I didn't change those, they might be dynamic? I'm going to revert the changes just to be sure.

ruflin · 2017-10-06T11:26:20Z

metricbeat/module/system/_meta/kibana/default/dashboard/Metricbeat-system-overview.json

        },
        "optionsJSON": "{\"darkTheme\":false}",
        "panelsJSON": "[{\"col\":1,\"id\":\"System-Navigation\",\"panelIndex\":9,\"row\":1,\"size_x\":12,\"size_y\":1,\"type\":\"visualization\"},{\"col\":1,\"id\":\"c6f2ffd0-4d17-11e7-a196-69b9a7a020a9\",\"panelIndex\":11,\"row\":2,\"size_x\":2,\"size_y\":2,\"type\":\"visualization\"},{\"col\":7,\"id\":\"fe064790-1b1f-11e7-bec4-a5e9ec5cab8b\",\"panelIndex\":12,\"row\":4,\"size_x\":6,\"size_y\":5,\"type\":\"visualization\"},{\"col\":1,\"id\":\"855899e0-1b1c-11e7-b09e-037021c4f8df\",\"panelIndex\":13,\"row\":4,\"size_x\":6,\"size_y\":5,\"type\":\"visualization\"},{\"col\":1,\"id\":\"7cdb1330-4d1a-11e7-a196-69b9a7a020a9\",\"panelIndex\":14,\"row\":9,\"size_x\":12,\"size_y\":6,\"type\":\"visualization\"},{\"col\":9,\"id\":\"522ee670-1b92-11e7-bec4-a5e9ec5cab8b\",\"panelIndex\":16,\"row\":2,\"size_x\":2,\"size_y\":2,\"type\":\"visualization\"},{\"col\":11,\"id\":\"1aae9140-1b93-11e7-8ada-3df93aab833e\",\"panelIndex\":17,\"row\":2,\"size_x\":2,\"size_y\":2,\"type\":\"visualization\"},{\"col\":7,\"id\":\"825fdb80-4d1d-11e7-b5f2-2b7c1895bf32\",\"panelIndex\":18,\"row\":2,\"size_x\":2,\"size_y\":2,\"type\":\"visualization\"},{\"col\":5,\"id\":\"d3166e80-1b91-11e7-bec4-a5e9ec5cab8b\",\"panelIndex\":19,\"row\":2,\"size_x\":2,\"size_y\":2,\"type\":\"visualization\"},{\"col\":3,\"id\":\"83e12df0-1b91-11e7-bec4-a5e9ec5cab8b\",\"panelIndex\":20,\"row\":2,\"size_x\":2,\"size_y\":2,\"type\":\"visualization\"}]",
        "timeRestore": false,
        "title": "[Metricbeat System] Overview",
-        "uiStateJSON": "{\"P-11\":{\"vis\":{\"defaultColors\":{\"0 - 100\":\"rgb(0,104,55)\"}}},\"P-12\":{\"vis\":{\"defaultColors\":{\"0 - 100\":\"rgb(0,104,55)\"}}},\"P-14\":{\"vis\":{\"defaultColors\":{\"0% - 8.75%\":\"rgb(247,252,245)\",\"17.5% - 26.25%\":\"rgb(116,196,118)\",\"26.25% - 35%\":\"rgb(35,139,69)\",\"8.75% - 17.5%\":\"rgb(199,233,192)\"}}},\"P-16\":{\"vis\":{\"defaultColors\":{\"0 - 100\":\"rgb(0,104,55)\"}}},\"P-2\":{\"vis\":{\"defaultColors\":{\"0 - 100\":\"rgb(0,104,55)\"}}},\"P-3\":{\"vis\":{\"defaultColors\":{\"0 - 100\":\"rgb(0,104,55)\"}}}}",
+        "uiStateJSON": "{\"P-11\":{\"vis\":{\"defaultColors\":{\"0 - 100\":\"rgb(0,104,55)\"}}},\"P-12\":{\"vis\":{\"defaultColors\":{\"0 - 100\":\"rgb(0,104,55)\"}}},\"P-14\":{\"vis\":{\"defaultColors\":{\"0% - 15%\":\"rgb(247,252,245)\",\"15% - 30%\":\"rgb(199,233,192)\",\"30% - 45%\":\"rgb(116,196,118)\",\"45% - 60%\":\"rgb(35,139,69)\"}}},\"P-16\":{\"vis\":{\"defaultColors\":{\"0 - 100\":\"rgb(0,104,55)\"}}},\"P-2\":{\"vis\":{\"defaultColors\":{\"0 - 100\":\"rgb(0,104,55)\"}}},\"P-3\":{\"vis\":{\"defaultColors\":{\"0 - 100\":\"rgb(0,104,55)\"}}}}",


Also here the presentages seem to be different. I think I like the new ones better but curious if it is intentional?

I'm leaving this one like that, I think it's fairly irrelevant and Kibana adjusted that uiStateJSON based on the data.

ruflin · 2017-10-06T11:26:40Z

metricbeat/module/system/util_test.go

+	if err != nil {
+		t.Fatalf("Error reading file %s: %v", dashPath, err)
+	}
+	if !bytes.Contains(contents, []byte("CHANGEME_HOSTNAME")) {


ruflin · 2017-10-06T11:27:57Z

@tsg Should the commits be squashed?

monicasarbu · 2017-10-06T11:53:23Z

LGTM

tsg · 2017-10-06T11:57:27Z

Yes, please squash it.

ruflin · 2017-10-06T12:00:07Z

WFG

* Use `beat.name` instead of `beat.hostname` in visualizations. See elastic#5276 for the motivation * Add a "tip" widget that tells user how they can select another host. * Automatically set a search pattern of the form `beat.name: "HOSTNAME"` where HOSTNAME is the Beat name (hostname) that uploads the dashboards. The way the last point works is that I saved the dashbaord using this filter: `beat.name:"CHANGEME_HOSTNAME"`. The kibana loader code does a string replacement and replaces `CHANGEME_HOSTNAME` with the actual hostname. The disadvantage of this approach (vs doing JSON parsing) is that we must remember to always save that dashboard with the `CHANGEME_HOSTNAME` wildcard in place. I have added a unit test to the system module that checks for that, so it should be relatively hard for us to forget. Closes elastic#5276. (cherry picked from commit 4778c51)

* Use `beat.name` instead of `beat.hostname` in visualizations. See #5276 for the motivation * Add a "tip" widget that tells user how they can select another host. * Automatically set a search pattern of the form `beat.name: "HOSTNAME"` where HOSTNAME is the Beat name (hostname) that uploads the dashboards. The way the last point works is that I saved the dashbaord using this filter: `beat.name:"CHANGEME_HOSTNAME"`. The kibana loader code does a string replacement and replaces `CHANGEME_HOSTNAME` with the actual hostname. The disadvantage of this approach (vs doing JSON parsing) is that we must remember to always save that dashboard with the `CHANGEME_HOSTNAME` wildcard in place. I have added a unit test to the system module that checks for that, so it should be relatively hard for us to forget. Closes #5276. (cherry picked from commit 4778c51)

…lastic#5351) * Use `beat.name` instead of `beat.hostname` in visualizations. See elastic#5276 for the motivation * Add a "tip" widget that tells user how they can select another host. * Automatically set a search pattern of the form `beat.name: "HOSTNAME"` where HOSTNAME is the Beat name (hostname) that uploads the dashboards. The way the last point works is that I saved the dashbaord using this filter: `beat.name:"CHANGEME_HOSTNAME"`. The kibana loader code does a string replacement and replaces `CHANGEME_HOSTNAME` with the actual hostname. The disadvantage of this approach (vs doing JSON parsing) is that we must remember to always save that dashboard with the `CHANGEME_HOSTNAME` wildcard in place. I have added a unit test to the system module that checks for that, so it should be relatively hard for us to forget. Closes elastic#5276. (cherry picked from commit 8b6191e)

tsg added needs_backport PR is waiting to be backported to other branches. review v6.0.0-rc2 :Dashbaords labels Oct 5, 2017

tsg force-pushed the host_dashboard_selected_host branch from 881bd1d to c8e3879 Compare October 5, 2017 22:08

tsg force-pushed the host_dashboard_selected_host branch from c8e3879 to 4e253f5 Compare October 5, 2017 22:09

more beat.hostname replacements

b04c5a8

ruflin approved these changes Oct 6, 2017

View reviewed changes

changelog update + revert a uiState change

e372d93

ruflin merged commit 4778c51 into elastic:master Oct 6, 2017

tsg mentioned this pull request Oct 6, 2017

Cherry-pick #5340 to 6.0: Improvements and fixes to the Host Overview dashboard #5351

Merged

tsg removed the needs_backport PR is waiting to be backported to other branches. label Oct 6, 2017

tsg mentioned this pull request Oct 23, 2017

Fix empty dashboard due to empty query #5283

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Improvements and fixes to the Host Overview dashboard #5340

Improvements and fixes to the Host Overview dashboard #5340

tsg commented Oct 5, 2017

tsg commented Oct 5, 2017

ruflin left a comment

ruflin Oct 6, 2017

monicasarbu Oct 6, 2017 •

edited

tsg Oct 6, 2017

ruflin Oct 6, 2017

tsg Oct 6, 2017

monicasarbu Oct 6, 2017

ruflin Oct 6, 2017

tsg Oct 6, 2017

ruflin Oct 6, 2017

tsg Oct 6, 2017

ruflin Oct 6, 2017

ruflin commented Oct 6, 2017

monicasarbu commented Oct 6, 2017

tsg commented Oct 6, 2017

ruflin commented Oct 6, 2017

Improvements and fixes to the Host Overview dashboard #5340

Improvements and fixes to the Host Overview dashboard #5340

Conversation

tsg commented Oct 5, 2017

tsg commented Oct 5, 2017

ruflin left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

monicasarbu Oct 6, 2017 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ruflin commented Oct 6, 2017

monicasarbu commented Oct 6, 2017

tsg commented Oct 6, 2017

ruflin commented Oct 6, 2017

monicasarbu Oct 6, 2017 •

edited