Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[improve] PIP 342: Support OpenTelemetry metrics in Pulsar client #22178

Merged
merged 10 commits into from Mar 22, 2024

Conversation

merlimat
Copy link
Contributor

@merlimat merlimat commented Mar 3, 2024

Fixes #xyz

Main Issue: #xyz

PIP: #xyz

Motivation

Modifications

Verifying this change

  • Make sure that the change passes the CI checks.

(Please pick either of the following options)

This change is a trivial rework / code cleanup without any test coverage.

(or)

This change is already covered by existing tests, such as (please describe tests).

(or)

This change added tests and can be verified as follows:

(example:)

  • Added integration tests for end-to-end deployment with large payloads (10MB)
  • Extended integration test for recovery after broker failure

Does this pull request potentially affect one of the following parts:

If the box was checked, please highlight the changes

  • Dependencies (add or upgrade a dependency)
  • The public API
  • The schema
  • The default values of configurations
  • The threading model
  • The binary protocol
  • The REST endpoints
  • The admin CLI options
  • The metrics
  • Anything that affects deployment

Documentation

  • doc
  • doc-required
  • doc-not-needed
  • doc-complete

Matching PR in forked repository

PR in forked repository:

@merlimat merlimat added type/PIP release/important-notice The changes which are important should be mentioned in the release note labels Mar 3, 2024
@merlimat merlimat added this to the 3.3.0 milestone Mar 3, 2024
@merlimat merlimat self-assigned this Mar 3, 2024
@github-actions github-actions bot added the doc-required Your PR changes impact docs and you will update later. label Mar 3, 2024
pip/pip-342 OTel client metrics support.md Outdated Show resolved Hide resolved
pip/pip-342 OTel client metrics support.md Show resolved Hide resolved
pip/pip-342 OTel client metrics support.md Show resolved Hide resolved
pip/pip-342 OTel client metrics support.md Show resolved Hide resolved
pip/pip-342 OTel client metrics support.md Outdated Show resolved Hide resolved
pip/pip-342 OTel client metrics support.md Outdated Show resolved Hide resolved
pip/pip-342 OTel client metrics support.md Outdated Show resolved Hide resolved
pip/pip-342 OTel client metrics support.md Outdated Show resolved Hide resolved
pip/pip-342 OTel client metrics support.md Outdated Show resolved Hide resolved
pip/pip-342 OTel client metrics support.md Show resolved Hide resolved
Copy link
Contributor

@asafm asafm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

pip/pip-342 OTel client metrics support.md Show resolved Hide resolved
@merlimat merlimat merged commit 41e515c into apache:master Mar 22, 2024
20 checks passed
@merlimat merlimat deleted the client-otel-pip branch March 22, 2024 16:17
| `pulsar.client.consumer.closed` | Counter | sessions | `pulsar.tenant`, `pulsar.namespace`, [`pulsar.topic`], [`pulsar.partition`], `pulsar.subscription` | The number of consumer sessions closed |
| `pulsar.client.consumer.message.received.count` | Counter | messages | `pulsar.tenant`, `pulsar.namespace`, [`pulsar.topic`], [`pulsar.partition`], `pulsar.subscription` | The number of messages explicitly received by the consumer application |
| `pulsar.client.consumer.message.received.size` | Counter | bytes | `pulsar.tenant`, `pulsar.namespace`, [`pulsar.topic`], [`pulsar.partition`], `pulsar.subscription` | The number of bytes explicitly received by the consumer application |
| `pulsar.client.consumer.receive_queue.count` | UpDownCounter | messages | `pulsar.tenant`, `pulsar.namespace`, [`pulsar.topic`], [`pulsar.partition`], `pulsar.subscription` | The number of messages currently sitting in the consumer receive queue |
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I found that the . will be replaced with _ in the Prometheus exporter. I have uploaded the integration test result with Prometheus exporter open-telemetry/opentelemetry-java-instrumentation#10849

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it better to use pulsar.client.consumer.receiver.queue.usage? and we can also consider to expose pulsar.client.consumer.receiver.queue.limit.

| `pulsar.client.consumer.message.received.size` | Counter | bytes | `pulsar.tenant`, `pulsar.namespace`, [`pulsar.topic`], [`pulsar.partition`], `pulsar.subscription` | The number of bytes explicitly received by the consumer application |
| `pulsar.client.consumer.receive_queue.count` | UpDownCounter | messages | `pulsar.tenant`, `pulsar.namespace`, [`pulsar.topic`], [`pulsar.partition`], `pulsar.subscription` | The number of messages currently sitting in the consumer receive queue |
| `pulsar.client.consumer.receive_queue.size` | UpDownCounter | bytes | `pulsar.tenant`, `pulsar.namespace`, [`pulsar.topic`], [`pulsar.partition`], `pulsar.subscription` | The total size in bytes of messages currently sitting in the consumer receive queue |
| `pulsar.client.consumer.message.ack` | Counter | messages | `pulsar.tenant`, `pulsar.namespace`, [`pulsar.topic`], [`pulsar.partition`], `pulsar.subscription` | The number of acknowledged messages |
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
| `pulsar.client.consumer.message.ack` | Counter | messages | `pulsar.tenant`, `pulsar.namespace`, [`pulsar.topic`], [`pulsar.partition`], `pulsar.subscription` | The number of acknowledged messages |
| `pulsar.client.consumer.message.ack.count` | Counter | messages | `pulsar.tenant`, `pulsar.namespace`, [`pulsar.topic`], [`pulsar.partition`], `pulsar.subscription` | The number of acknowledged messages |

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
doc-required Your PR changes impact docs and you will update later. release/important-notice The changes which are important should be mentioned in the release note type/PIP
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

6 participants