Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Post error logs to Slack #1564

Closed
1 of 2 tasks
calremmel opened this issue Feb 16, 2021 · 2 comments
Closed
1 of 2 tasks

Post error logs to Slack #1564

calremmel opened this issue Feb 16, 2021 · 2 comments
Assignees
@iamleeg
Labels
Importer P1: Launch blocker Needs fixing before we launch, schedule some time to investigate & fix
Projects
Global.health BCH/NE/Oxford Kanban Board
Milestone
Koa

Comments

@calremmel
Copy link
Contributor

calremmel commented Feb 16, 2021 •
edited by abhidg

The error log reporting function that relays ingestion function errors to Slack is currently broken. Presently, adding parser logs as triggers is a time-consuming manual process, and hard to maintain. Additionally, it only covers a few different cases of errors.

Proposed Solution

  • Wait until AWS Batch Migration is complete
  • Edit the lambda function to point to the Batch log group, which really will make things much simpler
@calremmel calremmel added P2: Nice to have This would be nice to have, if we have time we will fix it, if not good to launch and fix later when Importer labels Feb 16, 2021
@calremmel calremmel self-assigned this Feb 16, 2021
@calremmel
Copy link
Contributor Author

This should now wait on AWS Batch migration

@calremmel calremmel added this to the Post Launch milestone Mar 2, 2021
@joe-brilliant joe-brilliant added P1: Launch blocker Needs fixing before we launch, schedule some time to investigate & fix and removed P2: Nice to have This would be nice to have, if we have time we will fix it, if not good to launch and fix later when labels Mar 4, 2021
@calremmel calremmel removed their assignment Mar 5, 2021
@abhidg abhidg mentioned this issue Apr 19, 2021
Closed
13 tasks
@abhidg abhidg added P2: Nice to have This would be nice to have, if we have time we will fix it, if not good to launch and fix later when and removed P1: Launch blocker Needs fixing before we launch, schedule some time to investigate & fix labels Apr 20, 2021
@abhidg abhidg removed this from the Post Launch (Zoe) milestone Apr 20, 2021
@abhidg abhidg self-assigned this Jun 15, 2021
@abhidg abhidg added this to the Proxy milestone Sep 2, 2021
@joe-brilliant joe-brilliant modified the milestones: Proxy, Lewis Dec 7, 2021
@abhidg abhidg changed the title Rework error logs to slack function Post error logs to Slack Dec 15, 2021
@joe-brilliant joe-brilliant modified the milestones: Lewis, Holding Bin Feb 1, 2022
@abhidg abhidg modified the milestones: Holding Bin, Leonidas Feb 2, 2022
@abhidg abhidg added P1: Launch blocker Needs fixing before we launch, schedule some time to investigate & fix and removed P2: Nice to have This would be nice to have, if we have time we will fix it, if not good to launch and fix later when labels Feb 2, 2022
@abhidg
Copy link
Contributor

abhidg commented Feb 2, 2022

Raising to P1 to get more visibility on ingestion failures. Currently there's an error log posted to Slack via a e-mail hook but that is a JSON dump and not very readable

@iamleeg iamleeg self-assigned this Feb 28, 2022
iamleeg added a commit that referenced this issue Feb 28, 2022
@iamleeg
Add the lambda handler #1564
5d11956
iamleeg added a commit that referenced this issue Feb 28, 2022
@iamleeg
Rewrite script to avoid dependency on lambda #1564
765e24b
iamleeg added a commit that referenced this issue Feb 28, 2022
@iamleeg
Improve formatting of messages #1564
f0e5404 
Handle next token correctly

Handle slack rate limiting us
iamleeg added a commit that referenced this issue Feb 28, 2022
@iamleeg
Document the new script #1564
ce8a89a
iamleeg added a commit that referenced this issue Mar 1, 2022
@iamleeg
fix installation of poetry on python 3.10 #1564
9b9f8a5
iamleeg added a commit that referenced this issue Mar 2, 2022
@iamleeg
Collect logstream details from args to support submission-time defini…
b420fe7 
…tion #1564
iamleeg added a commit that referenced this issue Mar 2, 2022
@iamleeg
Pin the installed version of poetry #1564
c4e2138
iamleeg added a commit that referenced this issue Mar 2, 2022
@iamleeg
workflow to publish the error reporter to ECR #1564
17a350f
iamleeg added a commit that referenced this issue Mar 2, 2022
@iamleeg
Add the lambda handler #1564
914dd24
iamleeg added a commit that referenced this issue Mar 2, 2022
@iamleeg
Rewrite script to avoid dependency on lambda #1564
3311ffc
iamleeg added a commit that referenced this issue Mar 2, 2022
@iamleeg
Improve formatting of messages #1564
98e138a 
Handle next token correctly

Handle slack rate limiting us
iamleeg added a commit that referenced this issue Mar 2, 2022
@iamleeg
Document the new script #1564
01581ac
iamleeg added a commit that referenced this issue Mar 2, 2022
@iamleeg
Build Docker image for script #1564
c73ac28
iamleeg added a commit that referenced this issue Mar 2, 2022
@iamleeg
fix installation of poetry on python 3.10 #1564
273e108
iamleeg added a commit that referenced this issue Mar 2, 2022
@iamleeg
Collect logstream details from args to support submission-time defini…
f9b9549 
…tion #1564
iamleeg added a commit that referenced this issue Mar 2, 2022
@iamleeg
Pin the installed version of poetry #1564
1090d12
iamleeg added a commit that referenced this issue Mar 2, 2022
@iamleeg
workflow to publish the error reporter to ECR #1564
f4ada3b
@abhidg abhidg removed their assignment Mar 3, 2022
iamleeg added a commit that referenced this issue Mar 4, 2022
@iamleeg
push the correct image to ECR #1564
c7e564c
iamleeg added a commit that referenced this issue Mar 4, 2022
@iamleeg
push the correct image to ECR #1564
63226f0
iamleeg added a commit that referenced this issue Mar 7, 2022
@iamleeg
Configure AWS region from env #1564
19ca793 
And define envs in Docker/CD action
iamleeg added a commit that referenced this issue Mar 7, 2022
@iamleeg
Apparently putting the webhook in github is bad #1564
ce84a1c
iamleeg added a commit that referenced this issue Mar 7, 2022
@iamleeg
Give the GH secret a better name #1564
78bffc4
iamleeg added a commit that referenced this issue Mar 7, 2022
@iamleeg
Configure AWS region from env #1564
ced0396 
And define envs in Docker/CD action
iamleeg added a commit that referenced this issue Mar 7, 2022
@iamleeg
Apparently putting the webhook in github is bad #1564
1144704
iamleeg added a commit that referenced this issue Mar 7, 2022
@iamleeg
Give the GH secret a better name #1564
1ed1f8d
iamleeg added a commit that referenced this issue Mar 7, 2022
@iamleeg
Give values to env variables #1564
d1eae24
iamleeg added a commit that referenced this issue Mar 7, 2022
@iamleeg
🤦🏻 #1564
82b8f53
iamleeg added a commit that referenced this issue Mar 7, 2022
@iamleeg
🤦🏻 #1564
b707862
iamleeg added a commit that referenced this issue Mar 8, 2022
@iamleeg
Decomplect the slack logging by putting it directly into the ingestio…
b4df202 
…n job. #1564
iamleeg added a commit that referenced this issue Mar 8, 2022
@iamleeg
fix flake8-reported missing imports #1564
173109f
iamleeg added a commit that referenced this issue Mar 8, 2022
@iamleeg
Decomplect the slack logging by putting it directly into the ingestio…
196f129 
…n job. #1564
iamleeg added a commit that referenced this issue Mar 8, 2022
@iamleeg
fix flake8-reported missing imports #1564
a186510
@iamleeg iamleeg closed this as completed Mar 8, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@iamleeg iamleeg

Labels
Importer P1: Launch blocker Needs fixing before we launch, schedule some time to investigate & fix
Projects
No open projects
Global.health BCH/NE/Oxford Kanban Board
  
Blocked/Icebox
Milestone
Koa
Development

No branches or pull requests
4 participants
@abhidg @iamleeg @calremmel @joe-brilliant