Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add a filter to anomaly detection #110

Closed
wants to merge 0 commits into from
Closed

Conversation

bachng2017
Copy link
Contributor

@bachng2017 bachng2017 commented Mar 21, 2023

What

add some code to solve this issue re-data/re-data#392

How

We've tried to minimized the impact by just adding some code to utilize the anomaly_detector. The filter basically is based on re_data_z_score.time_window_end

Sample configuration that ignore anomaly for emailevent on Monday and Tuesday. Because only time_window_end is available for now in re_data_z_score, we shift a day for it.

      - name: emailevents
        description: email events
        config:
          re_data_monitored: true
          re_data_anomaly_detector: {'name': 'z_score', 'threshold': 3, 'filter': 'date_format(date_add(''day'',-1,z.time_window_end),''%a'') not in (''Mon'',''Tue'')'}
          re_data_time_filter: date_add('day',2,parse_datetime(cast(dt as varchar),'yyyyMMdd'))

Notes: If this approach is acceptable, adding time_window_start to re_data_z_score might be helpful.

@bachng2017 bachng2017 changed the title add filter to anomaly detection https://github.com/re-data/re-data/is… add a filter to anomaly detection re_data/#392 Mar 21, 2023
@bachng2017 bachng2017 changed the title add a filter to anomaly detection re_data/#392 add a filter to anomaly detection r/#392 Mar 21, 2023
@bachng2017 bachng2017 changed the title add a filter to anomaly detection r/#392 add a filter to anomaly detection Mar 21, 2023
@bachng2017 bachng2017 closed this Dec 9, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

1 participant