Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Disable flaky FE tests #2743

Merged
merged 3 commits into from
Dec 16, 2022
Merged

Disable flaky FE tests #2743

merged 3 commits into from
Dec 16, 2022

Conversation

KaiserKarel
Copy link
Contributor

No description provided.

@KaiserKarel
Copy link
Contributor Author

Task linked: CU-8676y2c83 Disable flaky UI tests

@vercel
Copy link

vercel bot commented Dec 16, 2022

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Updated
pablo-nightly ✅ Ready (Inspect) Visit Preview Dec 16, 2022 at 8:55PM (UTC)
picasso-nightly ❌ Failed (Inspect) Dec 16, 2022 at 8:55PM (UTC)

@KaiserKarel KaiserKarel changed the title Disable flaky tests Disable flaky FE tests Dec 16, 2022
@github-actions
Copy link

github-actions bot commented Dec 16, 2022

Nix commands for this PR

NOTE: You can also run our Nix commands in Docker. See the bottom of this comment.

Make sure you have setup the Composable community cache:

(you only need to run it once on your machine)

nix-shell -p cachix --command "cachix use composable-community"

Show all possible apps, shells and packages:

nix flake show "github:ComposableFi/composable/fe3eaad61745910e758c3b930a88fbea102d7d07 --allow-import-from-derivation

Run the Composable node alone:

nix run "github:ComposableFi/composable/fe3eaad61745910e758c3b930a88fbea102d7d07#composable-node" -L

Spin up a local devnet:

nix run "github:ComposableFi/composable/fe3eaad61745910e758c3b930a88fbea102d7d07#devnet" -L --option sandbox relaxed --show-trace

Spin up a local XCVM devnet:

nix run "github:ComposableFi/composable/fe3eaad61745910e758c3b930a88fbea102d7d07#devnet-xcvm" -L

View the docs:

nix run ".#docs-server"

Run this without Nix in Docker.

docker run --rm -v /var/run/docker.sock:/var/run/docker.sock -v nix:/nix -it nixos/nix bash -c "nix-env -iA nixpkgs.cachix && cachix use composable-community && nix run github:ComposableFi/Composable/fe3eaad61745910e758c3b930a88fbea102d7d07#devnet-dali -L --extra-experimental-features nix-command --extra-experimental-features flakes"

NOTE: You can swap devnet-dali in the command above with any Nix package
For more info on how to use Nix, check out our Nix docs
Note that the initial build may take about one hour if it has not been cached by our CI yet. Once it is cached, builds should take about one minute. We currently do not provide build caches for ARM machines such as M1 Macs, but building on ARM is supported.

@github-actions
Copy link

github-actions bot commented Dec 16, 2022

Visit the preview URL for this PR (updated for commit fe3eaad):

https://composable-docs--false-bvmz1swb.web.app

(expires Fri, 23 Dec 2022 21:14:57 GMT)

🔥 via Firebase Hosting GitHub Action 🌎

Sign: 9f02f9c6590528fd01c58c1497c91fed08e5194c

@dzmitry-lahoda
Copy link
Contributor

dzmitry-lahoda commented Dec 16, 2022

@KaiserKarel may be better https://jestjs.io/docs/cli#--testpathignorepatternsregexarray and no modification of target code a lot?

Via CLI can easy wire #flaky or flaky in CI to try run (by using cli params). With config file modification approach will need to patch file. Seem CLI easier for temporal off changes.

CI """owner""" should be able to disable without interfering a lot with coders imho.

@vercel vercel bot temporarily deployed to Preview – picasso-nightly December 16, 2022 20:17 Inactive
@vercel vercel bot temporarily deployed to Preview – pablo-nightly December 16, 2022 20:22 Inactive
@github-actions
Copy link

github-actions bot commented Dec 16, 2022

Visit the preview URL for this PR (updated for commit fe3eaad):

https://picasso-f0d16--pr2743-disable-flaky-ui-tes-7aiax6vg.web.app

(expires Fri, 23 Dec 2022 21:26:24 GMT)

🔥 via Firebase Hosting GitHub Action 🌎

Sign: d4e030ec330173845b1768cd217beee9791d20f1

@github-actions
Copy link

github-actions bot commented Dec 16, 2022

Visit the preview URL for this PR (updated for commit fe3eaad):

https://pablo-afa5d--pr2743-disable-flaky-ui-tes-1uj94jd7.web.app

(expires Fri, 23 Dec 2022 21:26:39 GMT)

🔥 via Firebase Hosting GitHub Action 🌎

Sign: 8ef316967fc7aa470dc109d617260c43c009e4e3

@vercel vercel bot temporarily deployed to Preview – picasso-nightly December 16, 2022 20:34 Inactive
@vercel vercel bot temporarily deployed to Preview – pablo-nightly December 16, 2022 20:40 Inactive
@KaiserKarel KaiserKarel force-pushed the Disable-flaky-UI-tests-(CU-8676y2c83) branch from 2629a4c to 2e5f760 Compare December 16, 2022 20:46
@vercel vercel bot temporarily deployed to Preview – picasso-nightly December 16, 2022 20:49 Inactive
@KaiserKarel
Copy link
Contributor Author

@KaiserKarel may be better https://jestjs.io/docs/cli#--testpathignorepatternsregexarray and no modification of target code a lot?

Via CLI can easy wire #flaky or flaky in CI to try run (by using cli params). With config file modification approach will need to patch file. Seem CLI easier for temporal off changes.

CI """owner""" should be able to disable without interfering a lot with coders imho.

I can move the ignore to CI too; but would prefer to enable strict testing of FE asap, because right now more bad tests are getting into the codebase.

@vercel vercel bot temporarily deployed to Preview December 16, 2022 20:55 Inactive
Copy link
Contributor

@dzmitry-lahoda dzmitry-lahoda left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please add tests to effects gate. because it is very annoying to see failed red CI jobs and wait until all jobs need to retry one failed. so at least there will be urge to fix tests and we can ask for that.

because also you disabled some flaky, but who knows if all. so we would be in same situation after pr merged, unless it is effects gate.

so either we disable and allow to enable using label/tag or if enabled by default - put into gate.
no gated red fails are 100% ignored.

@KaiserKarel
Copy link
Contributor Author

KaiserKarel commented Dec 16, 2022

please add tests to effects gate. because it is very annoying to see failed red CI jobs and wait until all jobs need to retry one failed. so at least there will be urge to fix tests and we can ask for that.

because also you disabled some flaky, but who knows if all. so we would be in same situation after pr merged, unless it is effects gate.

so either we disable and allow to enable using label/tag or if enabled by default - put into gate. no gated red fails are 100% ignored.

👍

We'll be prioritizing fixing these tests after release 3, so only expect them to be disabled until the 22nd.

@KaiserKarel KaiserKarel requested review from a team December 16, 2022 21:12
@mergify mergify bot merged commit b4eef6c into main Dec 16, 2022
@mergify mergify bot deleted the Disable-flaky-UI-tests-(CU-8676y2c83) branch December 16, 2022 21:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants