Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support ObjectName patterns in excludeObjectNameAttributes #897

Open
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

KestasG
Copy link

@KestasG KestasG commented Dec 5, 2023

Working on local project encountered performance issues with bean scraping. While analyzing found out that there are lot of unused attributes being tested against rules and taking significant amount of time. It is impossible to list all objects in excludeObjectNameAttributes as there are thousands of them, so decided to add pattern support for it. This allowed to reduce scraping time from 3.9 sec to 1.9 sec.

Signed-off-by: kgedminas <kgedminas@eisgroup.com>
@KestasG KestasG force-pushed the exclude_obj_properties_wildcard branch from ec528ab to d91e138 Compare December 5, 2023 07:40
@dhoard
Copy link
Collaborator

dhoard commented Dec 8, 2023

@KestasG thanks for the PR!

Can you provide an example exporter YAML configuration showing your usage?
Can you implement an integration test?

@KestasG
Copy link
Author

KestasG commented Dec 8, 2023

I've started with

excludeObjectNameAttributes:
  "xxx.stream-consumers:*":
    - "RateUnit"
    - "MeanRate"
    - "FifteenMinuteRate"
    - "OneMinuteRate"
    - "FiveMinuteRate"
    - "DurationUnit"  
  "kafka.consumer:type=consumer-fetch-manager-metrics,client-id=*":
    - "fetch-size-max"
    - "records-consumed-total"
    - "fetch-rate"
    - "records-perd-request-avg"
    - "bytes-consumed-total"
    - "records-lead-min"
    - "fetch-latency-avg"
    - "bytes-consumed-rate"
    - "fetch-throttle-time-avg"
    - "fetch-latency-max"
    - "fetch-throttle-time-max"
    - "records-consumed-rate"
    - "fetch-total"    

and later realized that simpler definition works as well:

excludeObjectNameAttributes:
  "*:*":
    - "RateUnit"
    - "MeanRate"
    - "FifteenMinuteRate"
    - "OneMinuteRate"
    - "FiveMinuteRate"
    - "DurationUnit"
  "kafka.consumer:type=consumer-fetch-manager-metrics,client-id=*":
    - "fetch-size-max"
    - "records-consumed-total"
    - "fetch-rate"
    - "records-perd-request-avg"
    - "bytes-consumed-total"
    - "records-lead-min"
    - "fetch-latency-avg"
    - "bytes-consumed-rate"
    - "fetch-throttle-time-avg"
    - "fetch-latency-max"
    - "fetch-throttle-time-max"
    - "records-consumed-rate"
    - "fetch-total"    

Will try to get the test next week.

Added integration test

Signed-off-by: Kestutis <kgedminas@eisgroup.com>
@KestasG
Copy link
Author

KestasG commented Dec 11, 2023

Added integration test. While writing a test realized that it makes little sense to keep old implementation, checking object names by exact match, so removed it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants