Index node.id in elasticsearch/node metricset (elastic#9209)

* Index node.id in elasticsearch/node metricset * Adding CHANGELOG entries * Fixing PR number in CHANGELOG entry * Adding breaking change entry for node.name field correction (cherry picked from commit 355d08d)
ycombinator · Nov 29, 2018 · 581f203 · 581f203
1 parent 87ce779
commit 581f203
Show file tree

Hide file tree

Showing 3 changed files with 128 additions and 8 deletions.
diff --git a/CHANGELOG.asciidoc b/CHANGELOG.asciidoc
@@ -62,6 +62,7 @@ https://github.com/elastic/beats/compare/v6.5.0...6.x[Check the HEAD diff]
 - Fix race condition when enriching events with kubernetes metadata. {issue}9055[9055] {issue}9067[9067]
 - Fix panic on docker healthcheck collection on dockers without healthchecks. {pull}9171[9171]
 - Fix issue with not collecting Elasticsearch cross-cluster replication stats correctly. {pull}9179[9179]
+- The `node.name` field in the `elasticsearch/node` metricset now correctly reports the Elasticsarch node name. Previously this field was incorrectly reporting the node ID instead. {pull}9209[9209]
 
 *Packetbeat*
 
@@ -129,6 +130,125 @@ https://github.com/elastic/beats/compare/v6.5.0...6.x[Check the HEAD diff]
 
 ////////////////////////////////////////////////////////////
 
+[[release-notes-7.0.0-alpha1]]
+=== Beats version 7.0.0-alpha1
+https://github.com/elastic/beats/compare/v6.5.0...v7.0.0-alpha1[View commits]
+
+==== Breaking changes
+
+*Affecting all Beats*
+
+- Dissect syntax change, use * instead of ? when working with field reference. {issue}8054[8054]
+- Removed dashboards and index patterns generation for Kibana 5. {pull}8927[8927]
+
+*Auditbeat*
+
+- Use `initial_scan` action for new paths. {pull}7954[7954]
+- Rename beat.name to agent.type, beat.hostname to agent.hostname, beat.version to agent.version.
+- Rename `source.hostname` to `source.domain` in the auditd module. {pull}9027[9027]
+- Remove warning for deprecated option: "filters". {pull}9002[9002]
+
+*Filebeat*
+
+- Rename `fileset.name` to `event.name`. {pull}8879[8879]
+- Rename `fileset.module` to `event.module`. {pull}8879[8879]
+- Rename source to log.file.path and log.source.ip {pull}8902[8902]
+- Remove the deprecated `prospector(s)` option in the configuration use `input(s)` instead. {pull}8909[8909]
+- Rename `offset` to `log.offset`. {pull}8923[8923]
+- Rename `source_ecs` to `source` in the Filebeat Suricata module. {pull}8983[8983]
+- Remove warnings for deprecated options: "spool_size", "publish_async", "idle_timeout". {pull}9002[9002]
+- Rename many `system.syslog.*` fields to map to ECS. {pull}9135[9135]
+- Rename many `iis.access.*` fields to map to ECS. {pull}9084[9084]
+- IIS module's user agent string is no longer encoded (`+` replaced with spaces). {pull}9084[9084]
+- Rename many `haproxy.*` fields to map to ECS. {pull}9117[9117]
+- Rename many `nginx.access.*` fields to map to ECS. {pull}9081[9081]
+- Rename many `system.auth.*` fields to map to ECS. {pull}9138[9138]
+
+*Metricbeat*
+
+- Remove warning for deprecated option: "filters". {pull}9002[9002]
+
+*Functionbeat*
+
+- Function concurrency is now set to 5 instead of unreserved. {pull}8992[8992]
+
+==== Bugfixes
+
+*Affecting all Beats*
+
+- Fixed `-d` CLI flag by trimming spaces from selectors. {pull}7864[7864]
+- Fixed Support `add_docker_metadata` in Windows by identifying systems' path separator. {issue}7797[7797]
+- Do not panic when no tokenizer string is configured for a dissect processor. {issue}8895[8895]
+- Start autodiscover consumers before producers. {pull}7926[7926]
+- Propagate Sync error when running SafeFileRotate. {pull}9069[9069]
+
+*Filebeat*
+
+- Fixed a memory leak when harvesters are closed. {pull}7820[7820]
+- Fix improperly set config for CRI Flag in Docker Input {pull}8899[8899]
+- Just enabling the `elasticsearch` fileset and starting Filebeat no longer causes an error. {pull}8891[8891]
+- Fix macOS default log path for elasticsearch module based on homebrew paths. {pul}8939[8939]
+
+*Heartbeat*
+
+- Heartbeat now always downloads the entire body of HTTP endpoints, even if no checks against the body content are declared. This fixes an issue where timing metrics would be incorrect in scenarios where the body wasn't used since the connection would be closed soon after the headers were sent, but before the entire body was. {pull}8894[8894]
+
+*Metricbeat*
+
+- Fix golang.heap.gc.cpu_fraction type from long to float in Golang module. {pull}7789[7789]
+- Add missing namespace field in http server metricset {pull}7890[7890]
+- Fix race condition when enriching events with kubernetes metadata. {issue}9055[9055] {issue}9067[9067]
+
+*Packetbeat*
+
+- Fixed the mysql missing transactions if monitoring a connection from the start. {pull}8173[8173]
+
+
+==== Added
+
+*Affecting all Beats*
+
+- Add field `host.os.kernel` to the add_host_metadata processor and to the
+  internal monitoring data. {issue}7807[7807]
+- Add debug check to logp.Logger {pull}7965[7965]
+- Count HTTP 429 responses in the elasticsearch output {pull}8056[8056]
+- Allow Bus to buffer events in case listeners are not configured. {pull}8527[8527]
+- Dissect will now flag event on parsing error. {pull}8751[8751]
+- add_cloud_metadata initialization is performed asynchronously to avoid delays on startup. {pull}8845[8845]
+- Added the `redirect_stderr` option that allows panics to be logged to log files. {pull}8430[8430]
+
+*Filebeat*
+
+- Make inputsource generic taking bufio.SplitFunc as input {pull}7746[7746]
+- Add custom unpack to log hints config to avoid env resolution {pull}7710[7710]
+- Make docker input check if container strings are empty {pull}7960[7960]
+- Keep unparsed user agent information in user_agent.original. {pull}8537[8537]
+- Allow to force CRI format parsing for better performance {pull}8424[8424]
+
+*Heartbeat*
+
+- Add automatic config file reloading. {pull}8023[8023]
+
+*Journalbeat*
+
+- Add the ability to check against JSON HTTP bodies with conditions. {pull}8667[8667]
+
+*Metricbeat*
+
+- Add metrics about cache size to memcached module {pull}7740[7740]
+- Add experimental socket summary metricset to system module {pull}6782[6782]
+- Collect custom cluster `display_name` in `elasticsearch/cluster_stats` metricset. {pull}8445[8445]
+- Test etcd module with etcd 3.3. {pull}9068[9068]
+- All `elasticsearch` metricsets now have module-level `cluster.id` and `cluster.name` fields. {pull}8770[8770] {pull}8771[8771] {pull}9164[9164] {pull}9165[9165] {pull}9166[9166] {pull}9168[9168]
+- All `elasticsearch` node-level metricsets now have `node.id` and `node.name` fields. {pull}9168[9168] {pull}9209[9209]
+
+*Packetbeat*
+
+- Add support to decode HTTP bodies compressed with `gzip` and `deflate`. {pull}7915[7915]
+- Added support to calculate certificates' fingerprints (MD5, SHA-1, SHA-256). {issue}8180[8180]
+- Support new TLS version negotiation introduced in TLS 1.3. {issue}8647[8647].
+
+
 [[release-notes-6.5.0]]
 === Beats version 6.5.0
 https://github.com/elastic/beats/compare/v6.4.0...v6.5.0[View commits]

diff --git a/metricbeat/module/elasticsearch/node/_meta/data.json b/metricbeat/module/elasticsearch/node/_meta/data.json
@@ -1,15 +1,16 @@
 {
     "@timestamp": "2017-10-12T08:05:34.853Z",
-    "beat": {
+    "agent": {
         "hostname": "host.example.com",
         "name": "host.example.com"
     },
     "elasticsearch": {
         "cluster": {
-            "id": "91RpCx2xSQ21pVPTZfDK0Q",
-            "name": "elasticsearch"
+            "id": "wafoCXEDTrGxpYViNueSaA",
+            "name": "es1"
         },
         "node": {
+            "id": "v5gHTHqKSRa4bZ9vbyDy7g",
             "jvm": {
                 "memory": {
                     "heap": {
@@ -31,11 +32,11 @@
                 },
                 "version": "11.0.1"
             },
-            "name": "DSiWcTyeThWtUXLB9J0BMw",
+            "name": "es1_1",
             "process": {
                 "mlockall": false
             },
-            "version": "7.0.0-alpha1"
+            "version": "7.0.0"
         }
     },
     "metricset": {

diff --git a/metricbeat/module/elasticsearch/node/data.go b/metricbeat/module/elasticsearch/node/data.go
@@ -75,7 +75,7 @@ func eventsMapping(r mb.ReporterV2, info elasticsearch.Info, content []byte) err
 	}
 
 	var errs multierror.Errors
-	for name, node := range nodesStruct.Nodes {
+	for id, node := range nodesStruct.Nodes {
 		event := mb.Event{}
 
 		event.RootFields = common.MapStr{}
@@ -93,8 +93,7 @@ func eventsMapping(r mb.ReporterV2, info elasticsearch.Info, content []byte) err
 			continue
 		}
 
-		// Write name here as full name only available as key
-		event.MetricSetFields["name"] = name
+		event.MetricSetFields["id"] = id
 
 		r.Event(event)
 	}