-
Notifications
You must be signed in to change notification settings - Fork 149
Open
Labels
CIIssues tied with CI/CD workflows.Issues tied with CI/CD workflows.DXDev experienceDev experienceinternalInternal or build-related changesInternal or build-related changes
Description
Version(s)
v1.3.2 (although I assure you it'll be relevant for a long while)
General context
Our testing jobs have gotten awful long & flaky, while we keep on adding more tests. The time has come for us to deal with this, as false-positive failures on the CI are now a daily occurence, I've begun to expect a restart or 2 on every PR or release, and this isn't going anywhere good.
Describe what needs to be done and why
Here's what I'd suggest:
- add the
--offlineflag in all integration tests which don't need to check online activity, to eliminate unnecessary checking of dependencies and other slowdowns/timeouts due to connection quality on the CI. - verify if some tests haven't gone deprecated and could potentially be dropped/updated/optimised.
- eliminate inevitable duplicate tests.
- go through past test reports, identify the flakiest of tests and try to make them pass consistently.
- (optional, as this a tall order) move some of the integration tests to unit tests, where and if applicable
- The
generate-windows-launcherjob fails intermittently oncsdownload timeout #2930 - Migrate from the custom M1 runner to the official Github one #2936
- (after Migrate from the custom M1 runner to the official Github one #2936 is done) split the test suites we run on M1 similar to other platforms and run them for all workflows, rather than just on the
mainbranch.
Metadata
Metadata
Assignees
Labels
CIIssues tied with CI/CD workflows.Issues tied with CI/CD workflows.DXDev experienceDev experienceinternalInternal or build-related changesInternal or build-related changes
Type
Projects
Status
In progress