Skip to content

Optimised report loading for anomaly operator #897

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 8 commits into from
Jul 10, 2024
Merged

Conversation

govarsha
Copy link
Contributor

@govarsha govarsha commented Jul 3, 2024

Made following changes to optimise report loading for anomaly detector:

  1. If non-anomolous data points > 1000, it downsamples them to 1000. I chose the threshold as 1000 because visually a plot with more than 1000 datapoints is too crowded.
  2. All anomolous data points are included
  3. In the report, we show the whole dataset which leads to very large file size. Hence, we are showing 1000 data points at max here as well.
  4. The optimisation can be turned off by passing optimize_report = False in spec

Results:
I have used cpu_utilization_asg_misconfiguration.csv from NAB which has 18051 data points.

Report without optimisation:
Loading time - 15 seconds
Size - 6.2MB
Screenshot 2024-07-03 at 12 27 33 PM
Screenshot 2024-07-03 at 12 27 42 PM

Report with optimisation:
Loading time - 2/3 seconds
Size - 411KB

Screenshot 2024-07-03 at 12 27 04 PM

Screenshot 2024-07-03 at 12 27 18 PM

@govarsha govarsha requested review from prasankh, codeloop and ahosler July 3, 2024 07:12
@oracle-contributor-agreement oracle-contributor-agreement bot added the OCA Verified All contributors have signed the Oracle Contributor Agreement. label Jul 3, 2024
@govarsha govarsha requested a review from ahosler July 3, 2024 08:51
ahosler
ahosler previously approved these changes Jul 3, 2024
Copy link
Member

@ahosler ahosler left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks great, thanks!

mrDzurb
mrDzurb previously approved these changes Jul 3, 2024
@govarsha govarsha dismissed stale reviews from mrDzurb, VipulMascarenhas, and ahosler via f94fd9d July 8, 2024 04:00
@govarsha govarsha requested review from mrDzurb and ahosler July 8, 2024 04:06
@ahosler ahosler enabled auto-merge July 8, 2024 08:14
@ahosler ahosler merged commit 32d031e into main Jul 10, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
OCA Verified All contributors have signed the Oracle Contributor Agreement.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants