Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Current step data is not published on shutdown when publishing is in progress #3846

Closed
lenin-jaganathan opened this issue May 18, 2023 · 2 comments
Labels
bug A general bug module: micrometer-core An issue that is related to our core module
Milestone

Comments

@lenin-jaganathan
Copy link
Contributor

Describe the bug
#1882 was fixed by introducing the StepMeter Contract and doing a closing rollover. This fixed the original issue but #3750 introduced a polling thread that rolls the step meters at the start of the step. These 2 fixes combined produce a grey area where publishing is happening at the middle of the step and the registry closes. In this scenario, the data accumulated for the current step will be lost and will be never reported.

Environment

  • Micrometer version - 1.11.0
  • Micrometer registry - StepMeterRegistry

To Reproduce
How to reproduce the bug:
Use any Step registry for testing,

  • start the registry and create an increment counter every second.
  • close the registry exactly during publishing.

If the step is 60 seconds and the counter is incremented every second, it will report a value of 60 for every step. If the registry publishes at the 30th second and closes also happens at the 30th second, then publishing will be skipped as there is already publishing that is in progress. Also, the closingRollOver will be skipped as the data is already getting published. The publish will report a value of 60 (the value for the last step) and the data for the current step will never be published.

Expected behavior
Data for the current step should be published on close and publish happening simultaneously.

@jonatan-ivanov
Copy link
Member

Is this the same thing we are discussing here: #3832?

@lenin-jaganathan
Copy link
Contributor Author

I believe they are related.

@lenin-jaganathan lenin-jaganathan changed the title Publishing on shutdown doesn't work when publishing is in progress Publishing on shutdown causes dataloss for unpublished steps. May 30, 2023
@lenin-jaganathan lenin-jaganathan changed the title Publishing on shutdown causes dataloss for unpublished steps. Publishing on shutdown doesn't work when publishing is in progress May 30, 2023
@shakuzen shakuzen added bug A general bug module: micrometer-core An issue that is related to our core module and removed waiting-for-triage labels May 31, 2023
@shakuzen shakuzen added this to the 1.9.x milestone May 31, 2023
@shakuzen shakuzen changed the title Publishing on shutdown doesn't work when publishing is in progress Current step data is not published on shutdown when publishing is in progress May 31, 2023
@shakuzen shakuzen modified the milestones: 1.9.x, 1.11.x May 31, 2023
shakuzen added a commit to shakuzen/micrometer that referenced this issue Nov 6, 2023
If a scheduled publish is in progress when `close` is called, `StepMeterRegistry` was not performing the closing rollover and was not doing a subsequent publish of the final partial step.

Wait for the in-progress publish to finish before the closing rollover is called, which is followed by a final publish of the partial step.

Fixes micrometer-metricsgh-3846
@shakuzen shakuzen modified the milestones: 1.11.x, 1.11.6 Nov 8, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug A general bug module: micrometer-core An issue that is related to our core module
Projects
None yet
Development

No branches or pull requests

3 participants