chore: allure test reporting #1668
Conversation
size-limit report 📦
|
fbarbu15
added
the
E:End-to-end testing
…re/test-reports
@danisharora099 |
| @@ -6,6 +6,7 @@ import { getPublicKey } from "./crypto/index.js"; | |||
| import { createDecoder, createEncoder } from "./ecies.js"; | |||
|
|
|||
| describe("Ecies Encryption", function () { | |||
| this.timeout(20000); | |||
There was a problem hiding this comment.
With this PR, since I had to replace the mocha config in all packages, I activated parallel runs in the CI for all packages.
And while parallel runs, test are taking longer to execute.
I saw that tests failing in the CI with the test timing out with the default 2000 ms timeout so I've increased it similar with other tests.
There was a problem hiding this comment.
Are these timeouts related to nwaku-jswaku communication or it's random?
There was a problem hiding this comment.
I believe that it's related to either how mocha implements concurrency or by the fact that because we run multiple containers at the same time when running tests in parallel, things start to slow down.
Github actions machine is also pretty low on resources so it's kinda expected to see such timeouts when running tests in parallel.
Those tests run in less than 1 second in serial mode that's why I don't think it's related to waku at all
| if: always() && env.ALLURE_REPORTS == 'true' | ||
| run: node ci/mergeAllureResults.cjs | ||
|
|
||
| - name: Get allure history |
There was a problem hiding this comment.
Since reports are in the separate repo now - do you think it will slow down if we will continue doing so for long period of time?
Also, as I understand we keep history from the test run in nwaku, do we differentiate between tests run in js-waku or nwaku?
There was a problem hiding this comment.
This steps runs only when the reusable test workflow ./.github/workflows/test-node.yml is called with allure_reports: true
And so far I only activated this for the main node job.
I think it will never take more than few seconds because I've setup a retention period of 30 reports
Problem
Test results were hard to follow in the console output.
Also we had no history to check flaky tests
Solution
Added allure test reports that are deployed with github pages: https://waku-org.github.io/allure-jswaku/5676/#
Notes