Add basic monitoring #60
Conversation
Deploying with
|
| Latest commit: |
6762d20
|
| Status: | ✅ Deploy successful! |
| Preview URL: | https://9cdff5ab.ask-astro.pages.dev |
| Branch Preview URL: | https://monitoring-dag.ask-astro.pages.dev |
pankajastro
force-pushed
the
monitoring_dag
branch
from
bbeba3b
to
3986261
Compare
|
|
||
|
|
||
| @task(trigger_rule=TriggerRule.ALL_DONE) | ||
| def check_weaviate_status(): |
There was a problem hiding this comment.
what are we trying to check here?
There was a problem hiding this comment.
- weaviate class exist if not then the task fail
- print the number of record in weaviate class
There was a problem hiding this comment.
We should just add what we are monitoring in weaviate as doctoring.
There was a problem hiding this comment.
@pankajastro we need the following changes in the DAGs:
- On a regular basis we can post the status on slack.
- We should run the DAGs like very 10 minutes or so and check if anything is down if it is we should post a status on Slack (and maybe email folks).
Something like this incident can happen any time.
As per the current state of PR, it would post on Slack and we can add an env variable for schedule intervals for example |
As discussed in call:
|
|
@sunank200 I have tested this you can check the sample message in the PR description |
|
@jedcunningham requesting your feedback on this! |
|
|
||
|
|
||
| @task(trigger_rule=TriggerRule.ALL_DONE) | ||
| def check_weaviate_status(): |
There was a problem hiding this comment.
We should just add what we are monitoring in weaviate as doctoring.
closes: #39
an example post success and fail service status
