Skip to content

Commit

Permalink
Speeds up synchronisation of the blocks for the fuel-core-sync serv…
Browse files Browse the repository at this point in the history
…ice (#1916)

The change speeds up synchronisation and removes noise logs when the P2P
request crashes.

The synchronization speed improvement is done by:
- Making all requests asynchronous. Before, the first request to fetch
the blocked header was synchronous and required awaiting it because of
`and_then`.
- The process of block fetching is separated from block execution.
Before, we fetched X blocks and waited for their execution before
requesting new X blocks. The fetching is a new parallel task that uses
the `batch_sender` variable for backpressure. The task fetches blocks in
parallel and sends them into a channel with buffer `1`. The executor's
task is to receive the batch from the channel and execute it, allowing
the request for a new batch to be made in parallel.

If the execution speed is faster than fetching, we request more than one
batch in parallel, up to the buffer limit. Tests shows that.

The change also handles cases better when a P2P request fails without a
known `PeerId`.

Without change, I was syncing 2M blocks for 4 hours. With this change,
it takes me 20 minutes.

## Checklist
- [x] New behavior is reflected in tests

### Before requesting review
- [x] I have reviewed the code myself
  • Loading branch information
xgreenx authored Jun 4, 2024
1 parent 0a1d591 commit 5536f5e
Show file tree
Hide file tree
Showing 9 changed files with 307 additions and 153 deletions.
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ and this project adheres to [Semantic Versioning](http://semver.org/).
## [Unreleased]

### Changed
- [#1916](https://github.com/FuelLabs/fuel-core/pull/1916): Speed up synchronisation of the blocks for the `fuel-core-sync` service.

#### Breaking

Expand Down
1 change: 1 addition & 0 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions crates/services/sync/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@ futures = { workspace = true }
mockall = { workspace = true, optional = true }
rand = { workspace = true }
tokio = { workspace = true, features = ["full"] }
tokio-stream = { workspace = true }
tracing = { workspace = true }

[dev-dependencies]
Expand Down
Loading

0 comments on commit 5536f5e

Please sign in to comment.