Skip to content

Conversation

@timvaillancourt
Copy link

@timvaillancourt timvaillancourt commented Aug 18, 2017

On many systems I've seen the mongodb_exporter use very high CPU % under PMM 1.2.x. This PR aims to reduce some of the work the mgo driver is doing and reduce the amount of connections/log-noise from reconnecting each scrape.

  1. Disable cursor pre-fetching (not needed on simple exporter queries - wastes resources fetching 50 docs we won't use).
  2. Limit connection pool size to 2. All other request will wait for the pool instead of creating more connections. Each additional mongodb connection adds CPU due to background pings, etc so it's best to restrict to a small number in this simple exporter.
  3. Re-use database connection to MongoDB. Currently we reconnect to the DB for every single scrape - this causes MongoDB to have to create a new connection (OS thread), setup buffers etc and log a lot of noise to the logfiles. This change caches the *mgo.Session handle and re-validates it each time it is requested in the code. This results in a single mongodb connection for the runtime of the exporter and a reconnect if/when the ping to the cached handle fails. A .Close() method was added to cleanly close the cached handle at end of program.

…l size and re-use database connection to MongoDB
@timvaillancourt timvaillancourt requested a review from AlekSi August 18, 2017 15:15
@AlekSi AlekSi self-assigned this Aug 18, 2017
@AlekSi
Copy link
Contributor

AlekSi commented Aug 18, 2017

May you please a PR against develop branch?

@timvaillancourt
Copy link
Author

Closing for #79

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants