Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

documenting reason for skipping test TestConsensus_LeaderForView (after current epoch) #1342

Merged
merged 3 commits into from Sep 27, 2021

Conversation

AlexHentschel
Copy link
Member

@AlexHentschel AlexHentschel commented Sep 23, 2021

This PR covers the quarantined test (see Notion doc Flaky and Skipped Tests - Quarantine for details):

t.Run("after current epoch", func(t *testing.T) {
t.SkipNow()
// get leader for view in next epoch when it is not set up yet
_, err := committee.LeaderForView(250)
assert.Error(t, err)
assert.True(t, errors.Is(err, protocol.ErrNextEpochNotSetup))
})

While the test is skipped for a valid reason (it tests the behaviour of the mature protocol, which is different than the current implementation due to a temporary fallback), but this reasoning is not documented in the code.

This PR adds a comment explaining the reason for skipping the test.

Copy link
Member

@jordanschalm jordanschalm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you 🙏

Copy link
Contributor

@huitseeker huitseeker left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This may not be within scope of this PR, but is there a way that :

  • the presence or absence of EECC
  • the presence of absence of a test targeting EECC
    ... in the source could be maintained in sync using a build tag, rather than a skipped test?

@codecov-commenter
Copy link

Codecov Report

Merging #1342 (76ca57a) into master (2a21b49) will decrease coverage by 0.03%.
The diff coverage is n/a.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #1342      +/-   ##
==========================================
- Coverage   56.29%   56.26%   -0.04%     
==========================================
  Files         502      502              
  Lines       31208    31208              
==========================================
- Hits        17570    17560      -10     
- Misses      11263    11275      +12     
+ Partials     2375     2373       -2     
Flag Coverage Δ
unittests 56.26% <ø> (-0.04%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
...sus/approvals/assignment_collector_statemachine.go 42.30% <0.00%> (-9.62%) ⬇️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 2a21b49...76ca57a. Read the comment docs.

@gomisha
Copy link
Contributor

gomisha commented Sep 27, 2021

thank you for clarifying why the test needs to be skipped and updating quarantine list with this PR - much clearer.

@AlexHentschel AlexHentschel merged commit 2ba7bcb into master Sep 27, 2021
@AlexHentschel AlexHentschel deleted the alex/reason_for_skipping_test branch September 27, 2021 19:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants