Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: verify flaky tests #9050

Merged

Conversation

ericzzzzzzz
Copy link
Contributor

@ericzzzzzzz ericzzzzzzz commented Aug 28, 2023

Description

  • We've been seeing some verify and custom actions related flaky tests on Kokoro tests
  • The cause is that some integration tests contaminate each other, and this is more likely to happen on gcp tests, as we're running tests against shared remote clusters concurrently-- 4 worker nodes running tests against one shared remote cluster, adding namespace for integration tests should provide the isolation but namespace flag is not added in verify and exec command. enable namespace flag for command verify and exec #9051
  • Github CI should provide better isolation , each worker runs tests against its dedicated minikube -- not 100% sure , but we should also isolate these tests by providing unique names as job names so when skaffold run query to watch resources it can watch the correct resource.

@ericzzzzzzz ericzzzzzzz changed the title Fix verify flaky tests fix: verify flaky tests Aug 28, 2023
@codecov
Copy link

codecov bot commented Aug 28, 2023

Codecov Report

Merging #9050 (4ed037e) into main (290280e) will decrease coverage by 7.01%.
Report is 1011 commits behind head on main.
The diff coverage is 49.90%.

@@            Coverage Diff             @@
##             main    #9050      +/-   ##
==========================================
- Coverage   70.48%   63.47%   -7.01%     
==========================================
  Files         515      624     +109     
  Lines       23150    32024    +8874     
==========================================
+ Hits        16317    20327    +4010     
- Misses       5776    10162    +4386     
- Partials     1057     1535     +478     
Files Changed Coverage Δ
cmd/skaffold/app/cmd/completion.go 13.04% <0.00%> (-1.25%) ⬇️
cmd/skaffold/app/cmd/config/list.go 65.21% <ø> (ø)
cmd/skaffold/app/cmd/config/set.go 88.72% <ø> (ø)
cmd/skaffold/app/cmd/config/util.go 54.28% <ø> (ø)
cmd/skaffold/app/cmd/credits.go 100.00% <ø> (ø)
cmd/skaffold/app/cmd/credits/export.go 0.00% <0.00%> (ø)
cmd/skaffold/app/cmd/deploy.go 40.90% <0.00%> (-12.94%) ⬇️
cmd/skaffold/app/cmd/generate_pipeline.go 60.00% <ø> (ø)
cmd/skaffold/app/cmd/inspect_modules.go 65.00% <ø> (ø)
cmd/skaffold/app/cmd/inspect_profiles.go 66.66% <ø> (ø)
... and 40 more

... and 419 files with indirect coverage changes

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

@pull-request-size pull-request-size bot added size/M and removed size/S labels Aug 28, 2023
@ericzzzzzzz ericzzzzzzz marked this pull request as ready for review August 28, 2023 20:04
@ericzzzzzzz ericzzzzzzz merged commit a66f3fa into GoogleContainerTools:main Aug 29, 2023
15 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants