Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Cassandra driver Actuator metrics #21624

Closed

Conversation

emerkle826
Copy link

Motivation:

The DataStax Java driver for Cassandra can be configured to expose driver session operational metrics. Currently these metrics are not available through Spring Boot Actuator.

Modifications:

This commit adds support for Cassandra driver metrics through Spring Boot Actuator by binding the native driver metrics to Micrometer metrics.

Result:

Users are now able to access Cassandra driver enabled metrics through Actuator .

@spring-projects-issues spring-projects-issues added the status: waiting-for-triage An issue we've not yet triaged label May 29, 2020
@@ -0,0 +1,58 @@
/*
* Copyright 2012-2019 the original author or authors.
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Any reason why the copyright ends at 2019 instead of 2020?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oversight, from a copy-paste

@emerkle826 emerkle826 force-pushed the cassandra-metrics branch 2 times, most recently from d1dcbb0 to 35e5acf Compare May 29, 2020 20:17
@philwebb philwebb added the for: team-attention An issue we'd like other members of the team to review label May 29, 2020
return CassandraMetricsBinder.SESSION_PREFIX + "." + CQL_SESSION_NAME + "." + metric;
}

private void assertGuageMetric(MeterRegistry registry, String metric, double expectedValue) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Small typo here: Guage -> Gauge.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks for catching that!

@philwebb philwebb added type: enhancement A general enhancement and removed for: team-attention An issue we'd like other members of the team to review status: waiting-for-triage An issue we've not yet triaged labels Jun 11, 2020
@philwebb philwebb added this to the 2.4.x milestone Jun 11, 2020
Copy link
Member

@shakuzen shakuzen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Converting from Dropwizard metrics to Micrometer metrics tends to be lossy and have an impedance mismatch. Furthermore, if a MeterBinder is the solution that ends up being used, I'm not sure it belongs in Spring Boot as there is nothing specific to Spring or Spring Boot in it - either the Cassandra driver or Micrometer would seem more appropriate. Spring Boot could make use of that, then. That said, rather than mapping to non-idiomatic Micrometer metrics, it would be best if the Cassandra driver either:

  • supported Micrometer directly as an option. Or
  • had hooks so that Micrometer could instrument the metrics natively, without passing through Dropwizard metrics.

See micrometer-metrics/micrometer#1397 and micrometer-metrics/micrometer#1359

/**
* Description for micrometer metrics.
*/
private static final String DESCRIPTION = "Please see reference.conf for more information.";
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure what reference.conf is but this does not seem like a useful description, in my opinion. Descriptions may be published to metric backends, which almost certainly won't have access to the mentioned reference.conf. Even accessing the Actuator endpoint does not mean that person has access to this reference.conf file.


TIMER_METRICS.forEach((n, supplier) -> {
String nameTimerMetric = String.format("%s.%s", name, n);
FunctionCounter.builder(nameTimerMetric, timer, (t) -> supplier.apply(timer).doubleValue()).tags(this.tags)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Counters, including FunctionCounter, in Micrometer are expected to be monotonically increasing, which is not the case with these rates, I believe. If you were to convert these, they would be gauges in Micrometer.

String nameTimerMetric = String.format("%s.%s", name, n);
FunctionCounter.builder(nameTimerMetric, timer, (t) -> supplier.apply(timer).doubleValue()).tags(this.tags)
// all rates are calculated per second
.baseUnit(TimeUnit.SECONDS.name()).description(DESCRIPTION).register(meterRegistry);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The rate is per second; not counting seconds, which is what this ends up seeming like - a count of seconds.

final Snapshot snapshot = timer.getSnapshot();
TIMER_SNAPSHOT_METRICS.forEach((n, supplier) -> {
String nameTimerMetric = String.format("%s.%s", name, n);
FunctionCounter.builder(nameTimerMetric, timer, (t) -> supplier.apply(snapshot).longValue()).tags(this.tags)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Again, not monotonically increasing (except for max) so should not be a FunctionCounter.

/**
* Prefix for all Driver configuration parameters.
*/
public static final String SESSION_PREFIX = "spring.data.cassandra.driver.session";
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are there metrics not related to sessions available?

@snicoll snicoll added the status: waiting-for-feedback We need additional information before we can continue label Jun 15, 2020
@emerkle826
Copy link
Author

Converting from Dropwizard metrics to Micrometer metrics tends to be lossy and have an impedance mismatch. Furthermore, if a MeterBinder is the solution that ends up being used, I'm not sure it belongs in Spring Boot as there is nothing specific to Spring or Spring Boot in it - either the Cassandra driver or Micrometer would seem more appropriate. Spring Boot could make use of that, then. That said, rather than mapping to non-idiomatic Micrometer metrics, it would be best if the Cassandra driver either:

* supported Micrometer directly as an option. Or

* had hooks so that Micrometer could instrument the metrics natively, without passing through Dropwizard metrics.

See micrometer-metrics/micrometer#1397 and micrometer-metrics/micrometer#1359

Thank you for the review, you have valid points about this sort of metrics binding not being specific to Spring. We (the driver team at Datastax) have discussed it further and have decided to try the approach of providing different metrics bindings for the Cassandra driver (ticket is here).

I'll convert this to a draft PR and revisit it once that work is completed.

@spring-projects-issues spring-projects-issues added status: feedback-provided Feedback has been provided and removed status: waiting-for-feedback We need additional information before we can continue labels Jun 15, 2020
@emerkle826 emerkle826 marked this pull request as draft June 15, 2020 20:12
@snicoll snicoll added status: blocked An issue that's blocked on an external project change and removed status: feedback-provided Feedback has been provided labels Jun 16, 2020
@snicoll
Copy link
Member

snicoll commented Jun 16, 2020

@emerkle826 thanks for the follow up!

@snicoll
Copy link
Member

snicoll commented Jul 8, 2020

@emerkle826 does this PR still make sense? I understand some major changes are pending in the driver so I am wondering if you would reuse this PR to update us. Thank you.

@emerkle826
Copy link
Author

@snicoll I think this PR will become obsolete once we make changes on the driver side. I'll close it. If a change still needs to happen in Spring Boot after the driver changes, I'll open a new PR as it will most likely not resemble this one much at all.

@emerkle826 emerkle826 closed this Jul 8, 2020
@snicoll
Copy link
Member

snicoll commented Jul 8, 2020

Thanks for confirming. Let us know if we can help!

@snicoll snicoll removed status: blocked An issue that's blocked on an external project change type: enhancement A general enhancement labels Jul 8, 2020
@snicoll snicoll removed this from the 2.4.x milestone Jul 8, 2020
@snicoll snicoll added the status: superseded An issue that has been superseded by another label Jul 8, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
status: superseded An issue that has been superseded by another
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

7 participants