[receiver/mongodbreceiver] Added new mongodb metrics to achieve parity with Telegraf

[chan-tim-sumo]

Description

• Added new mongodb metrics to achieve parity with Telegraf.

  mongodb.queries_per_sec
  mongodb.inserts_per_sec
  mongodb.commands_per_sec
  mongodb.getmores_per_sec
  mongodb.deletes_per_sec
  mongodb.updates_per_sec
  mongodb.flushes_per_sec
  mongodb.active.writes
  mongodb.active.reads
  mongodb.wtcache.bytes.read
  mongodb.cache.dirty.percent
  mongodb.cache.used.percent
  mongodb.page_faults

Link to tracking issue

• https://sumologic.atlassian.net/browse/OSC-901

Testing

Screenshot examples:

• mongodb.inserts_per_sec
• mongodb.queries_per_sec

Documentation

• https://github.com/influxdata/telegraf/tree/master/plugins/inputs/mongodb
• https://www.mongodb.com/docs/manual/reference/command/serverStatus/#opcounters

[chan-tim-sumo]

actively fixing the build errors and merge conflicts :)

[chan-tim-sumo]

only failing check is lychee:

# Summary

| Status        | Count |
|---------------|-------|
| 🔍 Total      | 0     |
| ✅ Successful | 0     |
| ⏳ Timeouts   | 0     |
| 🔀 Redirected | 0     |
| 👻 Excluded   | 0     |
| ❓ Unknown    | 0     |
| 🚫 Errors     | 0     |
No links were found. This usually indicates a configuration error.
If this was expected, set 'failIfEmpty: false' in the args.

not really sure how to fix this, doesn't seem like anything is wrong? this check isn't required to pass to merge this PR.

[chan-tim-sumo]

@schmikei saw that you're the code owner for this repo. can you take a look when you're free?

as well as a 2nd pr related to this (broken into 2 parts): #37517

thanks!

[schmikei]

Overall looks good, thank you for your contribution! I think I want to deliberate on some metric naming and potentially consolidate the command rate metrics into a single one.

I also am getting torn between having some version checks for reading from WiredTiger as if my memory recollects it was added to the summaryStats in 4.4

https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/15ab1613406a841bfdb4c7a969291807a08d8f7f/receiver/mongodbreceiver/scraper.go#L52

[schmikei]

It feels like that mongodb.commands.rate could better consolidate metrics here and combine via a metric attribute

• insert
• getmore
• delete
• update
• find

Pretty much the commands in this table I think could be consolidated into a single rate metric with unit {query}/s: https://www.mongodb.com/docs/manual/reference/command/#query-and-write-operation-commands

[chan-tim-sumo]

yeah this is a good idea, the reason why I didn't go with this approach was because this is to match with telegraf metrics which has specific (naming?) rate metrics like:

queries per sec
inserts per sec
etc

while putting this into consolidate metric where I'm thinking something like mongodb.operation.rate similar to mongodb.operation.count and while the metric shows up, it'll be under an operation instead of its own metric (if that makes sense? including an screenshot here

and I'm not exactly sure if telegraf will know the difference between:

 1. mongodb.operation.rate with operation=query or operation=insert
 2. mongodb.queries.rate

just to be safe, I went with mongodb.queries_per_sec (or after your suggestion mongodb.queries.rate).

let me know what you think!

[github-actions]

This PR was marked stale due to lack of activity. It will be closed in 14 days.

[dmitryax]

Why do we need to calculate the rate in this receiver for already emitted set of metrics? Calculating rate from cumulative seems like something that can easily be done on any metrics backend AFAIK. Even if it's needed to be done in the collector, why not make a generic solution for that?

I believe it must be even possible already by applying:

1. Cumulative to delta processor
2. Scaling the values deviding them by the collection interval using the scale_metric function in transform processor

[chan-tim-sumo]

@schmikei if you can take a look at this PR again would appreciate it! the failures seems to be from the kafkatopicsobserver, not related to my change 🤔

[chan-tim-sumo]

Why do we need to calculate the rate in this receiver for already emitted set of metrics? Calculating rate from cumulative seems like something that can easily be done on any metrics backend AFAIK. Even if it's needed to be done in the collector, why not make a generic solution for that?

I believe it must be even possible already by applying:

1. Cumulative to delta processor
2. Scaling the values deviding them by the collection interval using the scale_metric function in transform processor

whoops just saw this, we commented at the same time yesterday 😆

the reason I was thinking of calculating the rate in this receiver was because while there are already a set of emitted metrics which I believe to be mongodb.operation.count which stores the total count for specific/all the operations. so was thinking of adding rate metrics to be in line with telegraf metrics

oh wow hm, didn't know that there was a cumulativetodeltaprocessor. not too familiar with this, will look into this today!

[chan-tim-sumo]

taking a look at the readme

As a result, the cumulativetodelta processor may not work as expected
if used in a deployment of multiple collectors. When using this processor 
it is best for the data source to being sending data to a single collector.

not sure if this is a viable solution since this is best for a single collector 🤔

[chan-tim-sumo]

bump @schmikei

[chan-tim-sumo]

@atoulme can you take a look when you're free? thanks!

[atoulme]

I can take a look, but please resolve conflicts first.

[schmikei]

I think this generally looks good, the only concern I have is around how useful these rate metrics are going to be reported as the collector process if restarted we might get gaps in data depending on the scrape_interval which is why I think we generally want to avoid adding rate calcs in receivers, but I don't feel like blocking your efforts for too much longer.

I see the value in these metrics and think its a good change, I don't have full approver status here so would love to get someone more familiar with the data model if they have a different perspective on it.

[chan-tim-sumo]

I think this generally looks good, the only concern I have is around how useful these rate metrics are going to be reported as the collector process if restarted we might get gaps in data depending on the scrape_interval which is why I think we generally want to avoid adding rate calcs in receivers, but I don't feel like blocking your efforts for too much longer.

I see the value in these metrics and think its a good change, I don't have full approver status here so would love to get someone more familiar with the data model if they have a different perspective on it.

Also open to other comments as well from someone have a different perspective. If people wouldn't want to use these metrics, they can set it to false right? and thanks for taking a look Keith, appreciate it!