Update metrics documentation to note the benefits of using a MeterBinder when registering a metric that relies on other beans

[waschmittel]

We use Micrometer in our Spring Boot applications.

When shutting down our applications we sometimes get WARN log messages with Exceptions that say:

Failed to apply the value function for the gauge (...)

All of these value functions fail because some part of the application is already shut down. For example, we have a metric that's based on a Spring Data Repository, so we get

org.springframework.beans.factory.BeanCreationNotAllowedException: Error creating bean with name 'entityManagerFactory': Singleton bean creation not allowed while singletons of this factory are in destruction (Do not request a bean from a BeanFactory in a destroy method implementation!)

I assume this would be avoidable if publishing of Metrics is stopped by calling MeterRegistry.clear() and PushMeterRegistry.stop() when the shutdown begins.

(We are using Datadog, which apparently implements PushMeterRegistry)

[mbhave]

This was fixed in #12006 but #12121 removed the need to declare the destroy method explicitly. The tests written in #12006 are still green so I would think the stop methods are called when necessary.

@waschmittel Could you provide a small sample with steps on how to reproduce the issue?

[spring-projects-issues]

If you would like us to look at this issue, please provide the requested information. If the information is not provided within the next 7 days this issue will be closed.

[waschmittel]

Tried to reproduce ... the first thing Datadog's PushMeterRegistry does when close() is called, is calling publish().

So this seems to be a Datadog bug after all.

[waschmittel]

Oh wait ... PushMeterRegistry is a part of micrometer (in io.micrometer.core.instrument.push)

So PushMeterRegistry's close() looks like this:

    @Override
    public void close() {
        if (config.enabled()) {
            publishSafely();
        }
        stop();
        super.close();
    }

And publishSafely looks like this:

    private void publishSafely() {
        try {
            publish();
        } catch (Throwable e) {
            logger.warn("Unexpected exception thrown while publishing metrics for " + this.getClass().getSimpleName(), e);
        }
    }

So Micrometer apparently expects metrics to still be extractable when close() is called.

So I guess Spring should handle a PushMeterRegistry differently from a Non-Push MeterRegistry - Although I assume it would be difficult to call close() while the context is still fully available.

So I can can probably my metrics to handle the Exceptions that occur while the context is already being closed, but I would prefer a generic solution for this issue, of course.

[snicoll]

So Micrometer apparently expects metrics to still be extractable when close() is called.

@waschmittel I think the intention is to let the registry publish metrics "safely" (as the method name states) before closing it. The problem is that other parts of the infrastructure may have already been closed when the registry itself is closed. Perhaps the error message is misleading and can be refined (but that would have to be dealt in Micrometer itself).

Paging @shakuzen for insights.

[waschmittel]

Or Spring could do something to make config.enabled() false before calling close() (if possible - I have no idea).
Although I think this is surprising behavior by Micrometer to do a last publish when closing.

[snicoll]

Or Spring could do something to make config.enabled() false before calling close()

Why would we do that? I'd rather understand the intention of Micrometer rather than trying to force it to do something different. Doing something different may be the right call after all but there is no reason it should be specific to a spring app.