Skip to content

Broker Cache Allocations Metrics Always 0 #16587

@michaeljmarshall

Description

@michaeljmarshall

Describe the bug
The allocator-stats for the broker's entry cache appear to be broken because they are always 0, even when the cache is in use.

To Reproduce
Steps to reproduce the behavior:

  1. Build apache/pulsar master.
  2. Run bin/pulsar standalone
  3. Run bin/pulsar-perf consume -r 99 test
  4. Run bin/pulsar-perf produce -r 100 test
  5. Run bin/pulsar-admin broker-stats allocator-stats ml-cache (I omitted this result because it is too verbose. The allocations are zero every time though and the result aligns with the prometheus metrics.)
  6. You can also observe the broker's allocations related prometheus metrics
  7. Note that while there is an active producer and consumer, I can see that there are cache hits in the prometheus metrics.

Expected behavior
The broker cache metrics for allocations should produce the real cache size.

** Metrics**
Metrics containing ml_cache:

# TYPE pulsar_ml_cache_evictions gauge
pulsar_ml_cache_evictions{cluster="standalone"} 0 1657749677931
# TYPE pulsar_ml_cache_hits_rate gauge
pulsar_ml_cache_hits_rate{cluster="standalone"} 8.367224481632109 1657749677931
# TYPE pulsar_ml_cache_hits_throughput gauge
pulsar_ml_cache_hits_throughput{cluster="standalone"} 9056.903793586238 1657749677931
# TYPE pulsar_ml_cache_misses_rate gauge
pulsar_ml_cache_misses_rate{cluster="standalone"} 349.73998266551104 1657749677931
# TYPE pulsar_ml_cache_misses_throughput gauge
pulsar_ml_cache_misses_throughput{cluster="standalone"} 377094.08960597374 1657749677931
# TYPE pulsar_ml_cache_pool_active_allocations gauge
pulsar_ml_cache_pool_active_allocations{cluster="standalone"} 0 1657749677931
# TYPE pulsar_ml_cache_pool_active_allocations_huge gauge
pulsar_ml_cache_pool_active_allocations_huge{cluster="standalone"} 0 1657749677931
# TYPE pulsar_ml_cache_pool_active_allocations_normal gauge
pulsar_ml_cache_pool_active_allocations_normal{cluster="standalone"} 0 1657749677931
# TYPE pulsar_ml_cache_pool_active_allocations_small gauge
pulsar_ml_cache_pool_active_allocations_small{cluster="standalone"} 0 1657749677931
# TYPE pulsar_ml_cache_pool_allocated gauge
pulsar_ml_cache_pool_allocated{cluster="standalone"} 0 1657749677931
# TYPE pulsar_ml_cache_pool_used gauge
pulsar_ml_cache_pool_used{cluster="standalone"} 0 1657749677931
# TYPE pulsar_ml_cache_used_size gauge
pulsar_ml_cache_used_size{cluster="standalone"} 1077 1657749677931

Metadata

Metadata

Assignees

No one assigned

    Labels

    type/bugThe PR fixed a bug or issue reported a bug

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions