Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Metricbeat host namespace causes mapping conflict when used with Logstash #7050

Closed
tsg opened this issue May 9, 2018 · 0 comments
Closed

Comments

@tsg
Copy link
Contributor

tsg commented May 9, 2018

Here is what's happening:

  • the add_host_metadata processor creates a host object. The new processor is disabled by default, but the fields are present in the Beats templates.
  • The Logstash beats input adds a host (string) if host is not found in the original object. Code is here
  • since the Beats processor is not enabled by default, the host sent from Logstash to Elasticsearch is a string. This conflicts with the object mapping that we have in the Metricbeat template.

The result of the above is that a stock MB -> LS -> ES pipeline doesn't work :(

tsg added a commit to tsg/beats that referenced this issue May 9, 2018
As a solution for elastic#7050, we adding a `host.name` field to
all events. This is duplicate information from `beat.name`,
but is used to avoid the mapping conflict and to slowly
introduce the "host as an object" approach.

To remove the duplication, you can remove `beat.name` like this:

    processors:
      - drop_fields.fields: ["beat.name"]

Closes elastic#7050.
ruflin pushed a commit that referenced this issue May 10, 2018
As a solution for #7050, we're adding a `host.name` field to
all events. This is duplicate information from `beat.name`,
but is used to avoid the mapping conflict and to slowly
introduce the "host as an object" approach.

To remove the duplication, you can remove `beat.name` like this:

    processors:
      - drop_fields.fields: ["beat.name"]

Closes #7050.
tsg added a commit to tsg/beats that referenced this issue May 10, 2018
As a solution for elastic#7050, we're adding a `host.name` field to
all events. This is duplicate information from `beat.name`,
but is used to avoid the mapping conflict and to slowly
introduce the "host as an object" approach.

To remove the duplication, you can remove `beat.name` like this:

    processors:
      - drop_fields.fields: ["beat.name"]

Closes elastic#7050.

(cherry picked from commit 15d9539)
monicasarbu pushed a commit that referenced this issue May 10, 2018
* Add host.name in the events (#7051)

As a solution for #7050, we're adding a `host.name` field to
all events. This is duplicate information from `beat.name`,
but is used to avoid the mapping conflict and to slowly
introduce the "host as an object" approach.

To remove the duplication, you can remove `beat.name` like this:

    processors:
      - drop_fields.fields: ["beat.name"]

Closes #7050.

(cherry picked from commit 15d9539)

* changelog cleanup
stevea78 pushed a commit to stevea78/beats that referenced this issue May 20, 2018
As a solution for elastic#7050, we're adding a `host.name` field to
all events. This is duplicate information from `beat.name`,
but is used to avoid the mapping conflict and to slowly
introduce the "host as an object" approach.

To remove the duplication, you can remove `beat.name` like this:

    processors:
      - drop_fields.fields: ["beat.name"]

Closes elastic#7050.
stevea78 pushed a commit to stevea78/beats that referenced this issue May 20, 2018
As a solution for elastic#7050, we're adding a `host.name` field to
all events. This is duplicate information from `beat.name`,
but is used to avoid the mapping conflict and to slowly
introduce the "host as an object" approach.

To remove the duplication, you can remove `beat.name` like this:

    processors:
      - drop_fields.fields: ["beat.name"]

Closes elastic#7050.
leweafan pushed a commit to leweafan/beats that referenced this issue Apr 28, 2023
…#7068)

* Add host.name in the events (elastic#7051)

As a solution for elastic#7050, we're adding a `host.name` field to
all events. This is duplicate information from `beat.name`,
but is used to avoid the mapping conflict and to slowly
introduce the "host as an object" approach.

To remove the duplication, you can remove `beat.name` like this:

    processors:
      - drop_fields.fields: ["beat.name"]

Closes elastic#7050.

(cherry picked from commit 1889f9c)

* changelog cleanup
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant