Ele 85 group alerts by table by RoiTabach · Pull Request #698 · elementary-data/elementary

RoiTabach · 2023-02-22T15:32:04Z

see ELE-85 for full description.
Also added rate-limit for the slack API client.

… case

# Conflicts: # elementary/monitor/api/alerts/alert_filters.py # elementary/monitor/data_monitoring/data_monitoring_alerts.py # elementary/monitor/fetchers/alerts/alerts.py

linear · 2023-02-22T15:32:12Z

ELE-85 Group alerts by table

https://www.notion.so/elementary-data/Grouped-Alerts-b77b4afbc8064b4cadabc432aa657afb

github-actions · 2023-02-22T15:32:22Z

👋 @RoiTabach
Thank you for raising your pull request.
Please make sure to add tests and document all user-facing changes.
You can do this by editing the docs files in this pull request.

RoiTabach · 2023-02-22T16:00:31Z

Unit tests added:

test_slack_alerts_group_by.py::test_grouping_logic[one_warning_goes_to_one_warning]
test_slack_alerts_group_by.py::test_grouping_logic[one_fail_group_by_all_channel_selection_is_default]
test_slack_alerts_group_by.py::test_grouping_logic[one_fail_one_warn_same_model_group_by_table_groups_them_together]
test_slack_alerts_group_by.py::test_grouping_logic[one_fail_one_warn_same_table_one_other_table_group_by_table_groups_them_to_2_groups]
test_slack_alerts_group_by.py::test_grouping_logic[two_alerts_on_model_1_two_alerts_on_model_2_default_grouping_is_by_table_by_one_alert_has_group_by_alert]
test_slack_alerts_group_by.py::test_grouping_logic[default_grouping_all_and_overrides_existing_by_alert_and_by_table_to_2_out_of_3_of_model_2_s_alerts]
test_slack_alerts_group_by.py::test_alert_group_construction[single_alert_no_channel_goes_to_default_channel]
test_slack_alerts_group_by.py::test_alert_group_construction[single_alert_with_non_default_channel_goes_to_non_default_channel]
test_slack_alerts_group_by.py::test_alert_group_construction[group_by_table_forces_use_of_the_model_channel]
test_slack_alerts_group_by.py::test_alert_group_construction[group_by_all_forces_use_of_the_default_channel]
test_slack_alerts_group_by.py::test_alert_group_construction[owners_are_deduplicated]
test_slack_alerts_group_by.py::test_alert_group_construction[tags_are_deduplicated]
test_slack_alerts_group_by.py::test_alert_group_construction[errors_warnings_and_fails_are_routed_properly]

Update send-slack-alerts.mdx

…ists of strings afterwards (WIP)

IDoneShaveIt

Left some comments.
Most if not all are small.

IDoneShaveIt · 2023-02-28T13:41:32Z

+    def _number_of_failed_block(self):
+        # small_red_triangle: Falied: 36    |    :Warning: Warning: 3    |    :exclamation: Errors: 1
+        fields = []
+        all_components = list(self._components_to_alerts.items())


Co-authored-by: IDoneShaveIt <48473443+IDoneShaveIt@users.noreply.github.com>

RoiTabach added 12 commits February 13, 2023 12:29

fix typo chennel --> channel

345f3fd

prettify _validate_preview_blocks function

5214d80

run pre-commit on all files

97d2a7b

working version only default config possible

309f865

WIP on passing config + business logic to handle it

052a67c

WIP: works on by_alert with some added business logic for the grouped…

72c0b5a

… case

add rate limits on slack API calls

ee98673

WIP - working version and some of Idan's CR

9912373

remove +0200 from timestamps

e923ae0

Merge branch 'master' into ele-85-group-alerts-by-table

cc135d3

# Conflicts: # elementary/monitor/api/alerts/alert_filters.py # elementary/monitor/data_monitoring/data_monitoring_alerts.py # elementary/monitor/fetchers/alerts/alerts.py

refactor to Declerative Slack Message Builder

b5a8c75

add unit tests + fix bugs

7143278

RoiTabach requested a review from IDoneShaveIt February 22, 2023 15:32

RoiTabach and others added 13 commits February 23, 2023 09:04

precommit hook

bbbb76e

extract to files - precommit hook now passing

4f80f1d

add pytest-parametrization requirement

b3089c8

WIP Docs improvments

9da7818

Update send-slack-alerts.mdx

6654716

Merge pull request #709 from elementary-data/Hadarsagiv-grouped_alerts

3e4f813

Update send-slack-alerts.mdx

Update send-slack-alerts.mdx

7a4b1f5

Update slack.mdx

1d0ee55

Merge branch 'master' into ele-85-group-alerts-by-table

ba2d1d9

fix emoji for group of alerts

feeb223

--group by_alert --> --group-by alert

a62b250

missed one more place

3fd7a42

pre-commit fix

8e39a92

RoiTabach commented Feb 26, 2023

View reviewed changes

Comment thread elementary/monitor/data_monitoring/data_monitoring_alerts.py Outdated

RoiTabach added 16 commits February 26, 2023 14:35

Idan CR: pydantic dataclasses + seperate Schema file

b6cb9ff

Idan CR: attn --> attention

8bfceb0

Idan CR: t --> grouping_type

2b4933f

Idan CR: al --> alert

989f15a

Idan CR: al --> alert (more)

5f86613

Idan CR: add typing

560d784

Idan CR: remove local-dev-related-code

489d6bc

Idan CR: remove black magic (WIP)

1673daf

Idan CR: normalize alerts' subs,owners,tags, once and treat them as l…

2863bd4

…ists of strings afterwards (WIP)

Idan CR: bugfixes. tests are passing

be37cfb

Or CR: Remove "by all". WIP.

ec2eb4f

Or CR: Remove "by all". Tests are passing.

fda76c4

Or CR: Cosmetics, add env banner. WIP

21b6873

Or CR: add env : tests are passing

d6097c9

Or CR: db name, schema name in banner + title. tests passing.

eaf3a21

Actually update as sent

0197b80

IDoneShaveIt reviewed Feb 28, 2023

View reviewed changes

RoiTabach added 3 commits February 28, 2023 14:29

WIP cosmetics

cb349d8

Typing bug

d219df9

remove count of alerts

a2e77d8

IDoneShaveIt reviewed Feb 28, 2023

View reviewed changes

sum lists util

7ae99dc

RoiTabach commented Feb 28, 2023

View reviewed changes

Comment thread elementary/monitor/alerts/group_of_alerts.py Outdated

RoiTabach and others added 5 commits February 28, 2023 16:34

Merge branch 'master' into ele-85-group-alerts-by-table

28a3822

Idan CR: consice_name, unroll loops. Pre-commit hooks

d97a056

Update elementary/monitor/alerts/group_of_alerts.py

d5c8fad

Co-authored-by: IDoneShaveIt <48473443+IDoneShaveIt@users.noreply.github.com>

Update elementary/monitor/alerts/group_of_alerts.py

01fee30

Co-authored-by: IDoneShaveIt <48473443+IDoneShaveIt@users.noreply.github.com>

Idan CR: ordered Dict

4ba9efd

RoiTabach merged commit fa97805 into master Feb 28, 2023

RoiTabach deleted the ele-85-group-alerts-by-table branch February 28, 2023 15:42

Conversation

RoiTabach commented Feb 22, 2023

Uh oh!

linear Bot commented Feb 22, 2023

Uh oh!

github-actions Bot commented Feb 22, 2023

Uh oh!

RoiTabach commented Feb 22, 2023

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

IDoneShaveIt left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

IDoneShaveIt Feb 28, 2023

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants