Skip to content

*: fix test commands racing#4513

Merged
KaloyanTanev merged 5 commits into
mainfrom
kalo/fix-mev-test-indeterminism
May 8, 2026
Merged

*: fix test commands racing#4513
KaloyanTanev merged 5 commits into
mainfrom
kalo/fix-mev-test-indeterminism

Conversation

@KaloyanTanev
Copy link
Copy Markdown
Collaborator

@KaloyanTanev KaloyanTanev commented May 7, 2026

Simplify test commands. This should fix the flaky tests.

category: bug
ticket: none

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR simplifies the “single target” test execution flow across multiple cmd/test* commands to reduce flakiness caused by timeout/close races, and expands internal tests to assert that timeouts still produce per-test results.

Changes:

  • Refactors per-target runners to range over a results channel (buffered) and relies on the producer to close it.
  • Updates producers to emit timeout failures for any remaining queued tests when ctx is cancelled.
  • Extends internal tests to expect full result sets on timeout and adds stress-style timeout tests to prevent regressions.

Reviewed changes

Copilot reviewed 10 out of 10 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
cmd/testvalidator.go Refactors validator single-target execution to drain results until producer closes; producer emits remaining-timeout results.
cmd/testvalidator_internal_test.go Updates expected timeout results and adds a test ensuring timeouts always return non-empty results.
cmd/testpeers.go Refactors peer/self/relay single-target execution similarly; producer emits remaining-timeout results.
cmd/testpeers_internal_test.go Updates expected timeout results for additional testcases and loosens an internal output-consumption assertion.
cmd/testmev.go Refactors MEV per-endpoint execution to drain until close; producer emits remaining-timeout results.
cmd/testmev_internal_test.go Updates expected timeout results and adds a test ensuring each endpoint always has results on timeout.
cmd/testinfra.go Refactors infra single-target execution to drain until close; producer emits remaining-timeout results.
cmd/testinfra_internal_test.go Updates expected timeout results and adds a test ensuring local always has results on timeout.
cmd/testbeacon.go Refactors beacon per-endpoint execution to drain until close; producer emits remaining-timeout results.
cmd/testbeacon_internal_test.go Updates expected timeout results and adds a test ensuring each endpoint always has results on timeout.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread cmd/testpeers.go
Comment thread cmd/testbeacon.go
@codecov
Copy link
Copy Markdown

codecov Bot commented May 7, 2026

Codecov Report

❌ Patch coverage is 80.85106% with 18 lines in your changes missing coverage. Please review.
✅ Project coverage is 57.03%. Comparing base (eba0524) to head (8b29989).
⚠️ Report is 1 commits behind head on main.

Files with missing lines Patch % Lines
cmd/testpeers.go 70.73% 9 Missing and 3 partials ⚠️
cmd/testbeacon.go 64.70% 6 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #4513      +/-   ##
==========================================
- Coverage   57.10%   57.03%   -0.07%     
==========================================
  Files         245      245              
  Lines       32939    32857      -82     
==========================================
- Hits        18809    18740      -69     
+ Misses      11761    11746      -15     
- Partials     2369     2371       +2     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 10 out of 10 changed files in this pull request and generated 4 comments.

Comment thread cmd/testpeers.go
Comment thread cmd/testpeers.go
Comment thread cmd/testpeers.go
Comment thread cmd/testpeers_internal_test.go
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 10 out of 10 changed files in this pull request and generated 10 comments.

Comment thread cmd/testvalidator.go
Comment thread cmd/testmev.go Outdated
Comment thread cmd/testinfra.go Outdated
Comment thread cmd/testbeacon.go
Comment thread cmd/testpeers.go Outdated
Comment thread cmd/testpeers.go Outdated
Comment thread cmd/testpeers.go Outdated
Comment thread cmd/testpeers.go Outdated
Comment thread cmd/testpeers.go Outdated
Comment thread cmd/testpeers.go Outdated
@sonarqubecloud
Copy link
Copy Markdown

sonarqubecloud Bot commented May 7, 2026

@KaloyanTanev KaloyanTanev merged commit 0c7da58 into main May 8, 2026
10 of 12 checks passed
@KaloyanTanev KaloyanTanev deleted the kalo/fix-mev-test-indeterminism branch May 8, 2026 11:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants