Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

eth: Backfill blocks in batches #2489

Merged
merged 4 commits into from
Jul 11, 2022

Conversation

leszko
Copy link
Contributor

@leszko leszko commented Jul 1, 2022

What does this pull request do? Explain your changes. (required)
Optimize the number of ETH RPC request by first backfilling blocks in batches during the block sync. In result, only the latest block is polled separately instead of polling all missing blocks one-by-one.

This also fixes the Livepeer downtime issue in the case when Livepeer cannot catch up with the blocks and therefore can't transcode.

Specific updates (required)

  • In syncToLatestBlock(), execute first block backfilling and then poll block only for the latest one
  • Update unit tests

How did you test each of these updates (required)

  • Deployed an orchestrator and checked that it can transcode the video
  • Check that block backfilling catches all interesting information (e.g. DepositFund, etc.)

Does this pull request close any open issues?

fix #2480

Checklist:

@leszko leszko force-pushed the rafal/backfill-blocks-in-batches branch from da825dc to 349b497 Compare July 1, 2022 13:56
@leszko leszko force-pushed the rafal/backfill-blocks-in-batches branch from 21af295 to 596bea4 Compare July 5, 2022 09:42
@leszko leszko marked this pull request as ready for review July 5, 2022 09:49
@codecov
Copy link

codecov bot commented Jul 5, 2022

Codecov Report

Merging #2489 (a68c4ef) into master (3230eb1) will decrease coverage by 0.02841%.
The diff coverage is 28.57143%.

Impacted file tree graph

@@                 Coverage Diff                 @@
##              master       #2489         +/-   ##
===================================================
- Coverage   54.73000%   54.70159%   -0.02841%     
===================================================
  Files             95          95                 
  Lines          19852       19855          +3     
===================================================
- Hits           10865       10861          -4     
- Misses          8383        8390          +7     
  Partials         604         604                 
Impacted Files Coverage Δ
cmd/livepeer/starter/starter.go 2.29665% <0.00000%> (-0.00661%) ⬇️
eth/blockwatch/block_watcher.go 51.54867% <36.36364%> (-0.88496%) ⬇️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 3230eb1...a68c4ef. Read the comment docs.

@leszko leszko requested review from thomshutt and red-0ne July 5, 2022 09:54
@red-0ne
Copy link
Contributor

red-0ne commented Jul 6, 2022

This also fixes the Livepeer downtime issue in the case when Livepeer cannot catch up with the blocks and therefore can't transcode.

Is this solved by the fact that sync is faster because of fewer RPC calls?

Copy link
Contributor

@red-0ne red-0ne left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Have a couple of questions. Other than that, it looks fine for me.

eth/blockwatch/block_watcher.go Show resolved Hide resolved
@leszko
Copy link
Contributor Author

leszko commented Jul 6, 2022

This also fixes the Livepeer downtime issue in the case when Livepeer cannot catch up with the blocks and therefore can't transcode.

Is this solved by the fact that sync is faster because of fewer RPC calls?

Right, it's solved, because in many cases Livepeer will never catch up with blocks polling them one by one, so it's effectively down. Using backfilling, it will most likely catch with the missing blocks.

@leszko leszko requested a review from red-0ne July 6, 2022 12:07
Copy link
Contributor

@red-0ne red-0ne left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me.

@leszko leszko merged commit 159deaa into livepeer:master Jul 11, 2022
@leszko leszko deleted the rafal/backfill-blocks-in-batches branch July 11, 2022 10:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Make it easier to keep nodes synced on block info
3 participants