Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for Prometheus 1.x #4846

Merged
merged 4 commits into from
Mar 11, 2024

Conversation

jonatan-ivanov
Copy link
Member

@jonatan-ivanov jonatan-ivanov commented Mar 8, 2024

Upgrades to the 1.x versions of the Prometheus client artifacts in micrometer-registry-prometheus. This is a breaking change compared to the code that was there before. This is unavoidable due to the breaking changes in the Prometheus client from 0.x to 1.x. This moves the previous code from micrometer-registry-prometheus to a new module micrometer-registry-prometheus-simpleclient that is deprecated but available to give users a backward compatible option in case they need to make changes to use the new Prometheus client versions.

The base package used in micrometer-registry-prometheus was changed to io.micrometer.prometheusmetrics to differentiate it from the prior package now used in micrometer-registry-prometheus-simpleclient. This avoids split packages and allows both modules to be used in the same application even.


Known issues in the micrometer-registry-prometheus module after this PR:

  • Exemplars and Native Histograms do not work.
  • VictoriaMetrics histograms are not supported

Connected issues/PRs:

@jonatan-ivanov jonatan-ivanov added enhancement A general enhancement registry: prometheus A Prometheus Registry related issue labels Mar 8, 2024
@jonatan-ivanov jonatan-ivanov added this to the 1.13.0-M2 milestone Mar 8, 2024
libs.prometheusMetricsExpositionFormats,
libs.prometheusMetricsTracerCommon,
libs.prometheusSimpleClient,
libs.prometheusSimpleClientPushgateway,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This can be dealt with separate from this PR.
I think with the version catalog, we should move away from putting dependencies in here to be managed globally, especially when they're not transitive dependencies. I've been slowly removing them from here when doing related changes to dependencies that don't need to be here. We should use the version catalog reference in each build.gradle file instead, and then there's no need for them to be here.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should be able to remove things other than the PLATFORM_BOMS and the guava hacks from this file if we use the catalog references everywhere.
I created an issue for this: #4849


@Override
public List<MetricFamilySamples> describe() {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the equivalent of this in 1.x is implementing getPrometheusNames. I don't think it's critical but it may help optimize calls to scrape with a name filter. We can follow-up on this post-merge.

static class Family {

final Type type;
static class Family<T extends DataPointSnapshot> {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it'd be good to move away from the child/family naming to move things closer to the naming in the 1.x versions. Since these are all internal implementation details, we can change this later so no need to change anything before merging.

@shakuzen
Copy link
Member

I updated the base package in micrometer-registry-prometheus to avoid a split package. It is now io.micrometer.prometheusmetrics.

When doing this, the osgi test started failing because the IDE updated the imports to the new package, whereas before two classes were there with the same package, and apparently the -simpleclient version was being used before and passing. I'm not sure exactly why it is failing, but we'll have to note it as another known issue to be fixed later.

Copy link
Member

@shakuzen shakuzen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't we mark public classes in micrometer-registry-prometheus-simpleclient as @Deprecated with a deprecation message to migrate to micrometer-registry-prometheus?

@jonatan-ivanov jonatan-ivanov added the release notes Noteworthy change to call out in the release notes label Mar 11, 2024
@jonatan-ivanov jonatan-ivanov changed the title Prometheus 1.x Add support for Prometheus 1.x Mar 11, 2024
We rename micrometer-registry-prometheus to
micrometer-registry-prometheus-simpleclient so that we can add the new
Prometheus client to micrometer-registry-prometheus.
This changeset adds back micrometer-registry-prometheus as it was
before the rename in the previous commit. This makes it possible
to add the new Prometheus client.
jonatan-ivanov and others added 2 commits March 11, 2024 14:41
Dependencies were upgraded to the 1.x versions of the Prometheus client
artifacts in micrometer-registry-prometheus.
This is a breaking change compared to the code that was there before.
This is unavoidable due to the breaking changes
in the Prometheus client from 0.x to 1.x.
The previous code from micrometer-registry-prometheus was moved to
a new module micrometer-registry-prometheus-simpleclient that
is deprecated but available to give users a backward compatible option
in case they need to make changes to use the new Prometheus client.

The base package used in micrometer-registry-prometheus was changed to
io.micrometer.prometheusmetrics to differentiate it from the
prior package now used in micrometer-registry-prometheus-simpleclient.
This avoids split packages and allows both modules to be used
in the same application even.

Tests were adapted to the new API and differences in the scrape output.
Tests involving exemplars are commented out until
exemplar support is added.

PrometheusHistogram's cumulative buckets were converted to delta in the
registry. The new Prometheus client wants delta buckets which is weird
since it converts them back to cumulative internally.

PrometheusNamingConvention behavior was changed, it does not append
"_total" to counters anymore. The new Prometheus client validates
Counter names and check if they end with "_total".
If they don't, it appends "_total". If they do, it throws an exception:
java.lang.IllegalArgumentException: 'cnt_total': Illegal metric name...
Because of this behavior, if we want to use PrometheusNamingConvention,
we need to modify it so that it does not append "_total".

Known issues in the micrometer-registry-prometheus module after this PR:
- Exemplars and Native Histograms do not work.
- VictoriaMetrics histograms are not supported.
- The OSGi test was failing when the new Prometheus client was used, so
it was updated to use the -simpleclient module for now.

See micrometer-metricsgh-3987
See micrometer-metricsgh-4406

Co-authored-by: Tommy Ludwig <8924140+shakuzen@users.noreply.github.com>
@jonatan-ivanov
Copy link
Member Author

jonatan-ivanov commented Mar 11, 2024

I made some changes before merge:

  • Rebased the branch on main and fixed the merge conflicts (cloudwatch removal)
  • Removed HistogramFlavor
  • Replaced usage of PrometheusNaming to PrometheusNamingConvention
  • Deprecated classes in prometheus-simpleclient
  • Squashed (no diff) intermediate commits together and only kept those separate that are necessary to follow the history

@jonatan-ivanov jonatan-ivanov merged commit d651867 into micrometer-metrics:main Mar 11, 2024
6 checks passed
@jonatan-ivanov jonatan-ivanov deleted the prometheus-1.x branch March 11, 2024 22:14
* must match the regex [a-zA-Z_:][a-zA-Z0-9_:]*
*/
@Override
public String name(String name, Meter.Type type, @Nullable String baseUnit) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we can delegate to PrometheusNaming in the implementation of our default NamingConvention. The important point was for the configured NamingConvention to be used instead of hard-coding using PrometheusNaming in PrometheusMeterRegistry and ignoring what the user configured.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can utilize PrometheusNaming but we still need to keep our custom logic in the naming convention where we append units and other things.

izeye added a commit to izeye/micrometer that referenced this pull request May 25, 2024
@izeye izeye mentioned this pull request May 25, 2024
jonatan-ivanov pushed a commit to izeye/micrometer that referenced this pull request Jun 17, 2024
jonatan-ivanov pushed a commit that referenced this pull request Jun 17, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement A general enhancement registry: prometheus A Prometheus Registry related issue release notes Noteworthy change to call out in the release notes
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants