Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

test: chain: unit tests for the syncer & sync manager #8072

Merged
merged 4 commits into from
Mar 2, 2022

Conversation

TheDivic
Copy link
Contributor

@TheDivic TheDivic commented Feb 10, 2022

Added a few tests for some uncovered methods of the syncer and sync manager. Deleted some dead code.

Proposed Changes

  • Linter reports sync.go:getLatestBeaconEntry to be dead code. I also checked for any references (didn't have to look far since it's a private method) and there aren't any, so I deleted it.
  • Wrote some unit tests for the syncer similar to the ones that were already there (in sync.go) but I covered the last few methods that didn't have any unit tests.
  • Wrote a basic "lifecycle" test (create->insert->pop) for the sync manager bucket set. I know testing private functions is not the best practice but this one implements some important syncer behavior (primarily choosing what to pop).

Additional Info

  • Was sync.go:getLatestBeaconEntry left there on purpose? ⚠️

Checklist

Before you mark the PR ready for review, please make sure that:

  • All commits have a clear commit message.
  • The PR title is in the form of of <PR type>: <area>: <change being made>
    • example: fix: mempool: Introduce a cache for valid signatures
    • PR type: fix, feat, INTERFACE BREAKING CHANGE, CONSENSUS BREAKING, build, chore, ci, docs,perf, refactor, revert, style, test
    • area: api, chain, state, vm, data transfer, market, mempool, message, block production, multisig, networking, paychan, proving, sealing, wallet, deps
  • This PR has tests for new functionality or change in behaviour
  • If new user-facing features are introduced, clear usage guidelines and / or documentation updates should be included in https://lotus.filecoin.io or Discussion Tutorials.
  • CI is green

Added a few tests for some uncovered methods of the syncer and sync
manager. Deleted some dead code (sync.go:getLatestBeaconEntry).
Add annotations for the test crawler for the lotus.systemtestmatrix.com
dashboard.
@TheDivic
Copy link
Contributor Author

@ZenGround0 @magik6k Hey guy I have two test suites that are completely unrelated (or at least I think so) to my changes that are failing: test-itest-ccupgrade and test-itest-paych_api.
Are these tests flaky or I produced some unintended side effects?

@TheDivic TheDivic marked this pull request as ready for review February 11, 2022 14:52
@TheDivic TheDivic requested a review from a team as a code owner February 11, 2022 14:52
@ZenGround0
Copy link
Contributor

ZenGround0 commented Feb 14, 2022

@TheDivic thanks for the heads up. Looks like they are flaky. itest-ccupgrade should be salvageable, I'll look into it. paych_api I have less confidence in quick turn around but I'll also look.

--edit--
ok I have reason to believe this will fix both: #8088
less sure about ccupgrade but either way this is an improvement

Copy link
Contributor

@ZenGround0 ZenGround0 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. We should wait for @arajasek or @magik6k to sign off on removal of getLatestBeaconEntry.

@@ -1244,25 +1244,3 @@ func (syncer *Syncer) CheckBadBlockCache(blk cid.Cid) (string, bool) {
bbr, ok := syncer.bad.Has(blk)
return bbr.String(), ok
}

func (syncer *Syncer) getLatestBeaconEntry(ctx context.Context, ts *types.TipSet) (*types.BeaconEntry, error) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@arajasek you seem to be the last person who touched this (+ our resident DRAND expert). Can you confirm that we don't want this anymore?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ping @arajasek

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry about the delay here -- Yes, this can definitely be removed.

ts2 := mock.TipSet(mock.MkBlock(ts1, 1, 0))
bucket1 := newSyncTargetBucket(ts1, ts2)
bucketSet := syncBucketSet{buckets: []*syncTargetBucket{bucket1}}
fmt.Println("bucketSet: ", bucketSet.String())
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: these print statements can go

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed.

require.NoError(tu.t, err)
require.Equal(tu.t, "", reason, "block is still bad after manually unmarking")

err = tu.nds[client].SyncUnmarkAllBad(tu.ctx)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm new to the lotus syncer so I'm curious: why do we need the explicit SyncUnmarkBad on the head if we are unmarking all here?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe it's done intentionally, so as to check that both methods of unmarking a bad block are working as intended. In this way, if a change is commited to either SyncUnmarkBad or SyncUnmarkAllBad, the test will cover them.

While this test strays into the "integration" category, I think it's a valid choice.

@codecov
Copy link

codecov bot commented Feb 22, 2022

Codecov Report

Merging #8072 (0a03ec5) into master (5d416de) will increase coverage by 0.10%.
The diff coverage is n/a.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #8072      +/-   ##
==========================================
+ Coverage   39.91%   40.02%   +0.10%     
==========================================
  Files         666      666              
  Lines       72554    72541      -13     
==========================================
+ Hits        28963    29035      +72     
+ Misses      38553    38461      -92     
- Partials     5038     5045       +7     
Impacted Files Coverage Δ
chain/sync.go 70.52% <ø> (+5.52%) ⬆️
markets/loggers/loggers.go 88.88% <0.00%> (-11.12%) ⬇️
chain/actors/builtin/miner/diff.go 48.52% <0.00%> (-10.30%) ⬇️
cli/util.go 70.83% <0.00%> (-8.34%) ⬇️
chain/exchange/peer_tracker.go 66.66% <0.00%> (-4.31%) ⬇️
storage/wdpost_sched.go 77.45% <0.00%> (-3.93%) ⬇️
chain/exchange/client.go 49.15% <0.00%> (-3.82%) ⬇️
miner/miner.go 55.73% <0.00%> (-2.30%) ⬇️
blockstore/autobatch.go 56.30% <0.00%> (-1.69%) ⬇️
chain/vm/vm.go 61.63% <0.00%> (-1.07%) ⬇️
... and 18 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 5d416de...0a03ec5. Read the comment docs.

Copy link
Contributor

@arajasek arajasek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the contribution!

@arajasek arajasek merged commit 9c22065 into master Mar 2, 2022
@arajasek arajasek deleted the bloxico/syncer-tests branch March 2, 2022 00:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants