add python changefinder collector #10672
Conversation
andrewm4894
requested review from
Ferroin,
ilyam8,
jacekkolasa and
joelhans
as code owners
|
adding image here to link to in readme
|
github-actions
bot
added
area/build
|
This pull request introduces 1 alert when merging 0625002 into e1f5a8d - view on LGTM.com new alerts:
|
|
This pull request has been mentioned on Netdata Community. There might be relevant details there: https://community.netdata.cloud/t/changefinder-collector-feedback/972/1 |
|
I have a feeling that every collector you add does Is there a chance that it could be one omega ML thing with different components (changefinder, etc.)? So we do only one query 🤔 Are there any other problems with that approach? |
|
This pull request introduces 1 alert when merging 44d31e7 into 790af96 - view on LGTM.com new alerts:
|
|
This pull request introduces 1 alert when merging d8cd5c7 into 0a47bbd - view on LGTM.com new alerts:
|
My understanding from talking to @mfundul a while back was the /allmetrics does not quite go all the way to the database since its sort of cached somewhere and so a lot cheaper than doing a /data type call for each chart. But yes - getting the data into python is probably the biggest challenge with all of this. Interesting enough i think having an efficient and portable/flexible sort of storage that would give more options is part of the reasoning behind influx iox would be cool if there was a way for me to just go and grab the data from arrow and not have to go near the rest api and query engine. Or some sort of python package that could interface more efficiently with the raw data. I was also thinking of some sort of ml redis queue or something where 'hot' data (in terms of likely to be needed by some ml stuff) could be sent and gotten at quickly if/as needed. But again i'm kind of at the edge of my comfort zone in if such ideas make sense or not. For now trying to squeeze as much as possible out of the REST API as its all i can do. |
|
This pull request introduces 1 alert when merging 3c59422 into a96fec9 - view on LGTM.com new alerts:
|
|
This pull request introduces 1 alert when merging 8b1df29 into 7827c6f - view on LGTM.com new alerts:
|
- adds python 'changefinder' based collector for online changepoint detection.
- add a step to remove any chart dimensions not found in `data`. Idea being to maintain as accurate a list of chart dimensions and stop it from growing massive with lots of stale dimensions.
andrewm4894
force-pushed
the
changefinder-python-collector
branch
from
7237a9b
to
084047e
Compare
|
@ilyam8 @thiagoftsm hey - i was hoping to try get this reviewed this week or next if possible. |
Co-authored-by: Ilya Mashchenko <ilya@netdata.cloud>
…drewm4894/netdata into changefinder-python-collector
Co-authored-by: Vladimir Kobal <vlad@prokk.net>
Co-authored-by: Vladimir Kobal <vlad@prokk.net>
|
Approving under the assumption that
was done with latest changes |
going to spin up a vm with latest changes and run that for a few hours to be sure. |
…t set. similar to how we do in the alarms collector.
|
i was missing a small check in |
…or this collector so as to collect it in one place.
|
just retesting - will re-request review once ready. |
|
working now, my apologies. |
Summary
Component Name
collectors/python.d/changefinder
Test Plan
Tested locally on development nodes running for 24-48 hours to find useful examples of changefinder finding actually useful changes and validate runtime performance characteristics.
Additional Information
Much more detail in the README of the collector.