Add dotd flaky test tracking reminders #58811
Conversation
aws/ci_build
Outdated
| @@ -160,6 +160,8 @@ def main | |||
|
|
|||
| # Schedule a reminder for the dotd to check Zendesk in 2 hours | |||
| Slack.remind(Slack.user_id(DevelopersTopic.dotd), Time.now.to_i + 7200, 'Reminder: check <https://codeorg.zendesk.com/agent/filters/44863373|Zendesk>') | |||
| # Remind dotd to update the flaky test tracker on DTP as a once-ish a day reminder. | |||
| Slack.remind(Slack.user_id(DevelopersTopic.dotd), Time.now.to_i, 'Please update the <https://docs.google.com/spreadsheets/d/1DD6ebHM-2fsiMxxYCzhJYd1DzA69Z20qrCItaER9vto|Flaky Test Tracker>') | |||
There was a problem hiding this comment.
Is this a one time reminder, or a recurring reminder?
There was a problem hiding this comment.
It will remind each DTP
There was a problem hiding this comment.
That was the best mechanism I could find. Ideally it would be once a day around 9:00 am PT, but I couldn't find a way to do that
There was a problem hiding this comment.
to do this once every morning, you could add this to bin/cron/update_dotd
There was a problem hiding this comment.
or create a new cronjob, if that's not too heavy-handed
There was a problem hiding this comment.
Perfect! Updated to put into update_dotd instead.
There was a problem hiding this comment.
Oh, I guess the ZenDesk reminder works the same way. I was worried this might create a daily reminder, such that on the third day, the current and two prior DOTD's would get reminders.
bin/cron/update_dotd
Outdated
| # Remind dotd to update the flaky test tracker at 8am PST. | ||
| Slack.remind(Slack.user_id(DevelopersTopic.dotd), Time.now.to_i + 3600, 'Please update the <https://docs.google.com/spreadsheets/d/1DD6ebHM-2fsiMxxYCzhJYd1DzA69Z20qrCItaER9vto|Flaky Test Tracker>') |
There was a problem hiding this comment.
maybe update_dotd was not an ideal suggestion, because it gets run twice, to avoid needing to be updated twice a year for timezone changes:
as a result, I would expect this reminder to get posted twice.
my recommendation would be to expand the unless new_topic == current_topic condition above and include this reminder within that block, as a hacky way of making this only get run once
There was a problem hiding this comment.
my other comment here, perhaps more important, is that this message seems a bit un-actionable. What are you expecting the dotd to do in response to this message? I am wondering if the message on each DTT failure is perhaps the most timely and effective way to prompt the dotd.
There was a problem hiding this comment.
@davidsbailey I changed the mechanics to tag the DOTD and print out the message when a test build fails. I also added a bit more context to the message.
I know this will tag the DOTD twice on a build failure, but I think that will be helpful temporarily to make sure that developers see the new message.
Adds two reminders:
When a DTT fails, it posts a new message with a link to the flaky test tracker
Every day at 8:00 AM PT, the DOTD will get a DM from the DOTD app
Links
Testing story
Deployment strategy
Follow-up work
Privacy
Security
Caching
PR Checklist: