refactor(swamp/tests)skip tests #2802
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Codecov Report
@@ Coverage Diff @@
## main #2802 +/- ##
==========================================
- Coverage 51.81% 51.80% -0.01%
==========================================
Files 161 161
Lines 10827 10827
==========================================
- Hits 5610 5609 -1
- Misses 4721 4723 +2
+ Partials 496 495 -1 |
vgonkivs
force-pushed
the
skip_flaky_tests
branch
2 times, most recently
from
73dce97
to
bdbada0
Compare
|
|
|
with |
|
|
|
|
|
vgonkivs
force-pushed
the
skip_flaky_tests
branch
from
0919f51
to
b8743e0
Compare
vgonkivs
force-pushed
the
skip_flaky_tests
branch
from
6e41402
to
fb74eb2
Compare
|
@vgonkivs going to close this one and pick up where you got to in another branch with current set of work on trying to clean up flakiness etc |
Merged
ramin
added a commit
that referenced
this pull request
Jan 23, 2024
<!-- Thank you for submitting a pull request! Please make sure you have reviewed our contributors guide before submitting your first PR. Please ensure you've addressed or included references to any related issues. Tips: - Use keywords like "closes" or "fixes" followed by an issue number to automatically close related issues when the PR is merged (e.g., "closes #123" or "fixes #123"). - Describe the changes made in the PR. - Ensure the PR has one of the required tags (kind:fix, kind:misc, kind:break!, kind:refactor, kind:feat, kind:deps, kind:docs, kind:ci, kind:chore, kind:testing) --> Building off @vgonkivs's [work to skip tests](#2802), this attempts to get a baseline "green" ci for us. There are still a couple of tests that VERY intermittently flake in CI, but this way they will stand out WHEN they happen and we can track down vs entire thing being red always. This does quite a few things - introduces build tags on swamp tests in each named file to allow us to run parts of the swamp/integration tests independently (ie: `go test ./... -tags=blob` - adds an `integration` tag to allow running all still - splits the integration tests into it's own workflow file (`integration-tests.yml`) which is now triggered from `go-ci.yml` - splits each swamp/integration tests to run as its own job so we can see which are failing/flakey more explicitly - utilizes go's -short test flag to Skip a few swamp/integration tests that are consistently failing. Current we are skipping `TestFullReconstructFromFulls`, `TestFullReconstructFromLights`, `TestSyncStartStopLightWithBridge` which is less than we were originally skipping in #2802 - plugs in our verbose/debug stuff to integration tests in addition to unit which we had before Unit tests - splits some of the unit tests that were "race flakey" into `*_no_race_test.go` and adds `!race` tag to some others to get to pass consistently when running unit tests with -race flag - macos-latest unit tests still fail on race test ONLY in GitHub actions CI 🤷♂️ Next Steps - create issues for each short run / skipped - create issues for any tests that fail race that are NOT the race issue on upstream cosmos-sdk - create issue for macos-latest in GitHub race fail - create issue for macos-latest intermittent fail I think we let this run for a while, then once we see it be consistently green for a bit and isolate any more flakes that pop up, we can toggle on more branch requirements, then in fixing the above issues, we can be green. [Being green is not easy](https://www.youtube.com/watch?v=51BQfPeSK8k).
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Overview
Checklist