New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix MongoDB dbstats fields mapping #4025

Merged
merged 1 commit into from Apr 21, 2017

Conversation

Projects
None yet
4 participants
@exekias
Member

exekias commented Apr 17, 2017

Fix mongodb dbstats mapping to avoid errors like this:

WARN Can not index event (status=400): {"type":"mapper_parsing_exception","reason":"object mapping for [mongodb.dbstats.avg_obj_size] tried to parse field [avg_obj_size] as object, but found a concrete value"}

closes #3989

@ruflin

The dbstats metricset is still experimental. We should change the fields in the mapping and not the documentation. Means we should add .bytes etc. as it is according to our convention. This is a BC break but with experimental metricsets that is fine in my opinion.

@exekias exekias added in progress and removed in progress labels Apr 17, 2017

"db": c.Str("db"),
"collections": c.Int("collections"),
"objects": c.Int("objects"),
"avg_obj_size.bytes": c.Int("avgObjSize"),

This comment has been minimized.

@ruflin

ruflin Apr 18, 2017

Collaborator

This will make it incompatible with elasticsearch 2.x. You must use s.Object{ ... }  to add the nesting.

This comment has been minimized.

@exekias

exekias Apr 18, 2017

Member

Ah, didn't know that one, thanks!

@monicasarbu

This comment has been minimized.

Contributor

monicasarbu commented Apr 19, 2017

@exekias Some tests are failing. Can you please have a look?

"num": c.Int("num"),
"size": c.Int("size"),
"num": c.Int("num"),
"size.bytes": c.Int("size"),

This comment has been minimized.

@ruflin

ruflin Apr 20, 2017

Collaborator

Still one with .bytes

@@ -30,13 +30,13 @@ func TestFetch(t *testing.T) {
objects := event["objects"].(int64)
assert.True(t, objects > 0)
avgObjSize := event["avg_obj_size"].(int64)
avgObjSize := event["avg_obj_size.bytes"].(int64)

This comment has been minimized.

@ruflin

ruflin Apr 20, 2017

Collaborator

I assume that is not going to work, as it is a nested object.

@@ -12,15 +12,15 @@
},
"mongodb": {
"dbstats": {
"avg_obj_size": 59,
"avg_obj_size.bytes": 59,

This comment has been minimized.

@ruflin

ruflin Apr 20, 2017

Collaborator

You should update this one with make generation-json.

@ruflin ruflin merged commit fb92dee into elastic:master Apr 21, 2017

6 checks passed

CLA Commit author is a member of Elasticsearch
Details
codecov/patch Coverage not affected when comparing fedbab7...6b0b077
Details
codecov/project 63.93% (-0.01%) compared to fedbab7
Details
continuous-integration/appveyor/pr AppVeyor build succeeded
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details
default Build finished.
Details
@ashwinboinala

This comment has been minimized.

ashwinboinala commented May 4, 2017

@ruflin How do I make changes to my existing metric beat installation, please advise.

@ruflin

This comment has been minimized.

Collaborator

ruflin commented May 9, 2017

@ashwinboinala There are two options. Either you adjust the mapping manually and overwrite the one you have or you switch to the nightly build found here and let metricbeat overwrite your template: https://beats-nightlies.s3.amazonaws.com/index.html?prefix=metricbeat/

@exekias Should we backport this to 5.x?

@exekias

This comment has been minimized.

Member

exekias commented May 9, 2017

@ruflin uhm probably yes, let me open a PR

@ashwinboinala

This comment has been minimized.

ashwinboinala commented May 10, 2017

@ruflin thanks for your response.
I found another issue with mongo module, below is my connection string.
hosts: ["mongodb://username:password@servername:27017/dbname"]
In the dashboard, I see hostname as username:password which is metric.hostname
Is it expected behavior or am I doing anything wrong, below is my metricbeat config.
Please advise

  • module: mongodb
    metricsets: ["dbstats", "status"]
    enabled: true
    period: 10s
    hosts: ["mongodb://username:password@servername:27017/dbname"]

-Ashwin

@ruflin

This comment has been minimized.

Collaborator

ruflin commented May 11, 2017

@ashwinboinala This is not expected. It sounds like something does not work well with the host parser. Could you open a new Github issue for it?

athom added a commit to athom/beats that referenced this pull request Jan 25, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment