title | date | tags | draft | summary | images | ||||
---|---|---|---|---|---|---|---|---|---|
Setting up slack notification of detecting flaky tests using CircleCI(Orb) |
2022-08-29 |
|
false |
This blog describes how to set up slack notification of detecting flaky tests using CircleCI(Orb) |
This blog describes how to set up slack notification of detecting flaky tests using CircleCI(Orb)
In 2021, CircleCI released Test Insights Dashboard.
You can see flaky tests in Test Insights Dashboard so that your team can improve development performance by finding and fixing them quickly.
For more information, please refer to the blog post and documentation
Usually, you should regularly check Test Insights Dashboard if there are flaky tests.
To find and fix flaky tests faster, I created a CircleCI Orb(Config package) to notify them via slack.
To use this orb, you will need to generate a CircleCI token and Slack token.
Since this orb utilizes Slack Orb, please refer to the below document to generate a Slack token.
CircleCI token is used to get flaky tests via API.
Please make sure to put these environment variables before moving to the next step.
CIRCLE_TOKEN
SLACK_ACCESS_TOKEN
SLACK_DEFAULT_CHANNEL
(Which slack channel to post)
Before using this orb, please make sure that your organization allows uncertified Orb.
Next, create such a CircleCI config file using this orb.
version: '2.1'
orbs:
flaky-tests-notify: tadashi/flaky-tests-notify@1.0.0
parameters:
notify_flaky_tests:
type: boolean
default: false
workflows:
notify_flaky_tests:
jobs:
- flaky-tests-notify/notify
when: << pipeline.parameters.notify_flaky_tests >>
This notify_flaky_tests
workflow does not trigger until notify_flaky_tests
parameter(true) is passed.
We will pass this parameter using Scheduled Pipeline.
To run this job regularly and notify flaky tests via Slack, we will use Scheduled Pipelines.
I added the parameter notify_flaky_tests
so that this scheduled pipeline can trigger the notify_flaky_tests
workflow.
If you configured Scheduled Pipelines correctly, CircleCI will trigger pipelines based on its settings.
If there are flaky tests, you will see a slack notification as below.