Disable reporting spans for actuator endpoints

[mariusgiger]

Hi guys,

Reposting this here.

When using Opentracing with Jaeger for Spring Cloud Gateway with actuator endpoints enabled (e.g. /actuator/health for K8s readiness probe), all calls to the actuator endpoints produce a span. Is there a way to disable this behaviour?

Config:

        <dependency>
            <groupId>io.opentracing.contrib</groupId>
            <artifactId>opentracing-spring-jaeger-cloud-starter</artifactId>
            <version>2.0.0</version>
        </dependency>

        <dependency>
            <groupId>io.opentracing.contrib</groupId>
            <artifactId>opentracing-spring-cloud-starter</artifactId>
            <version>0.3.2</version>
        </dependency>

Cheers,
Marius

[pavolloffay]

I think this should not happen, the helthcheck probes should not be traced.

@ask4gilles could you please comment on this one? It will also transfer it to web repository.

[geoand]

Endpoints can be manually excluded as well, see: https://github.com/opentracing-contrib/java-spring-web/blob/master/opentracing-spring-web-starter/src/test/resources/application-test.yml#L22

But definitely the actuator endpoints should not be taken into account

[mariusgiger]

Thanks @pavolloffay and @geoand for the replies.

Per default, the actuator health endpoint is already ignored in the skip pattern, which is fine. However, the queries for the healthiness of a database (e.g. Mysql or Mongo) produce a span, which are invoked as part of the request to /actuator/health.

[geoand]

@mariusgiger do you know which URLs are used for querying the healthiness of the database?

[mariusgiger]

@geoand these spans get produced when calling /actuator/health.

[geoand]

OK, thanks.

@ask4gilles any idea off the top of your head?

[ask4gilles]

Hi @mariusgiger @geoand and @pavolloffay

All actuator endpoints are skipped by default.

Per default, the actuator health endpoint is already ignored in the skip pattern, which is fine. However, the queries for the healthiness of a database (e.g. Mysql or Mongo) produce a span, which are invoked as part of the request to /actuator/health.

From what I understand from the above, this could be caused by the underlying select queries targeting the database(s) in the health actuator itself. I think you could verify it easily by either disabling the DB health check inside the actuator (management.health.db.enabled=false) or disabling the SQL tracing.
If this is the issue, the sanity SQL query should probably be excluded from the SQL tracing.

If this does not solve your issue, can you please provide a minimal sample project in order to reproduce?

(BTW, this one opentracing-contrib/java-spring-jaeger#63 (comment) can be closed)

[mariusgiger]

Hi @ask4gilles,

Indeed, there are no more spans if the healthcheck for the database is disabled.

i.e. in my case:

management:
  health:
    db:
      enabled: false
    mongo:
      enabled: false

Most probably this also happens for other dependencies for healthchecks that support tracing.

Is it possible to disable tracing for a package resp. set of classes? If that is the case, either the concrete classes could be disabled (e.g. DataSourceHealthIndicator) or the base classes (e.g. AbstractHealthIndicator and AbstractReactiveHealthIndicator) or the whole package org.springframework.boot.actuate.

[ask4gilles]

@mariusgiger thanks for the feedback.
If you want to keep your DB health checks and avoid them to be traced, you could:

1. If you are not interested in JDBC tracing: opentracing.spring.cloud.jdbc.enabled=false
2. If you want to keep the JDBC tracing but exclude the health query, add it to this property opentracing.spring.cloud.jdbc.ignoreStatements

I don't see the equivalent properties for Mongo DB in the opentracing-spring-cloud-mongo-starter, but apparently there is also a way, see here: https://github.com/opentracing-contrib/java-mongo-driver#exclude-commands-from-tracing

Regarding the other types of health checks in the link and a possible global exclusion for them I don't think it's foreseen but I may be wrong here (@pavolloffay @geoand ?) I think that it would be more on case by case basis.

To summarize, this is not something related to the tracing of the actuators endpoints by themselves (and the title of this issue should be modified). This is also not related to this repo but to starters provided by opentracing-spring-cloud. So it could be moved there as an enhancement request.

[geoand]

Thanks for checking @ask4gilles!

[mariusgiger]

Thank you @ask4gilles for the response! This resolves the issue for me.

[ask4gilles]

@mariusgiger Awesome, glad it helped!

[ask4gilles]

@geoand this one can be closed 😉

[geoand]

👍

[ahmedju]

Please any update for this issue ?

[EricBuist]

The actuator/health endpoint is traced even without a DB, and I cannot figure out how to turn this off. Looking at generated span, this is coming from io.opentelemetry.spring-webmvc-6.0. I never imported that dependency, I searched and cannot find any clue how to configure that. Any idea?