Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Metrics related to persistent store operations #123

Closed
amitsaha opened this issue Dec 29, 2020 · 7 comments
Closed

Metrics related to persistent store operations #123

amitsaha opened this issue Dec 29, 2020 · 7 comments

Comments

@amitsaha
Copy link

amitsaha commented Dec 29, 2020

I currently don't see any metrics related to persistent store operations. For example:

  • an error counter (for init and upsert operations)
  • a latency metric (for init and upsert operations)

Perhaps others:

  • data store connection pool size

Is this something that you folks have considered and have it in your radar?

We are able to work on a PR as well if you are in agreement with the idea.

@eli-darkly
Copy link
Contributor

It's an interesting idea, but I don't think it's feasible to try to design it in a single PR in this repository. All of the persistent store functionality comes from the Go SDK, and some of the metrics you're talking about could only be reported from within the database-specific store implementation component (e.g. there's no such thing as a universal connection pool that Relay could observe in any generic way) so this would probably require changes in the SDK component APIs.

@amitsaha
Copy link
Author

amitsaha commented Dec 30, 2020

I am missing something perhaps here - but ignoring the pool metrics (which I presume is a SDK implementation detail), can we not perform the metric calculation in the relay_feature_store.go file? Consider the Upsert() method which currently looks as:

sw.loggers.Debugf(`Received feature flag update: %s (version %d)`, key, item.Version)
updated, err := sw.store.Upsert(kind, key, item)

What if we surrounded the call to Upsert() with a timer for latency and a counter for non-nil errors?

@eli-darkly
Copy link
Contributor

I am missing something perhaps here - but ignoring the pool metrics (which I presume is a SDK implementation detail)

That's exactly the thing I just mentioned above:

some of the metrics you're talking about could only be reported from within the database-specific store implementation component (e.g. there's no such thing as a universal connection pool that Relay could observe in any generic way)

So, if you want connection pool information, then this cannot be done only with Relay changes. If you don't want connection pool information after all, then that's another matter, and Relay changes might be sufficient.

@amitsaha
Copy link
Author

So, if you want connection pool information, then this cannot be done only with Relay changes. If you don't want connection pool information after all, then that's another matter, and Relay changes might be sufficient.

Sounds 👍 , I will take a stab at the persistent store operations metrics first and follow up separately related to the pool metrics in the Go SDK project.

@eli-darkly
Copy link
Contributor

You will not be able to add connection pool metrics in the Go SDK project; I think you may have underestimated the scope of such a change. Connection pooling is part of the individual database integrations— as I said, there is no universal concept of a connection pool that the SDK can deal with overall. That is part of the third-party Redis/Consul/DynamoDB clients that we use. We would have to 1. determine what (if any) pool-related metrics are actually available to us from those clients, 2. come up with a standardized way to represent them in the SDK, 3. design a new component interface in the SDK (since the existing PersistentDataStore interface has no method for this), and 4. update each of the integration libraries to implement that interface.

We're happy to note it as a feature request, but I think this one will be more feasible for us to implement in-house. At the very least, I strongly recommend that you look in detail at how persistent data stores are implemented in the SDK before pursuing this.

@eli-darkly
Copy link
Contributor

I'm also not clear on what kind of metrics you're looking for in regard to connection pools. If you want to know how many connections are being made to your database instance, I would think it'd be easier to monitor that on the server side.

@amitsaha
Copy link
Author

I'm also not clear on what kind of metrics you're looking for in regard to connection pools. If you want to know how many connections are being made to your database instance, I would think it'd be easier to monitor that on the server side.

That's a good point. I guess we can do that too and from what you mention in the previous comment, might be an easier starting point.

LaunchDarklyCI pushed a commit that referenced this issue Jan 21, 2021
(v6 - #2) update Go SDK to latest 5.0.0 beta release
@cwaldren-ld cwaldren-ld closed this as not planned Won't fix, can't repro, duplicate, stale Aug 8, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants