Skip to content
This repository has been archived by the owner on Sep 26, 2019. It is now read-only.

[PAN-2427] Prep chain downloader for branch by abstraction #1194

Merged
merged 12 commits into from Apr 2, 2019

Conversation

ajsutton
Copy link
Contributor

PR description

Refactor to make ChainDownloader an interface so that we can implement alternate chain download implementations. The primary benefit is allow us to use branch-by-abstraction to develop a pipeline based chain download in smaller steps rather than having to wholesale replace the chain download mechanism.

Also makes the abstractions more consistent between fast and full sync by introducing FullSynchronizer which bundles together the full sync chain download, block propagation manager and trailing peers config so DefaultSynchronizer is more clearly just switching between fast and full sync. It also let the full sync chain downloader adhere to the ChainDownload api and not need to expose the trailing peer config as well.

this.protocolContext = protocolContext;
this.syncState = syncState;

this.blockPropagationManager =
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the terminology here is getting a bit fuzzy. I think the "FullSynchronizer" is really a "FullDownloaderFactory" (similarly "FastSynchronizer" -> "FastDownloaderFactory"). And I think that the blockPropagationManager should belong to the Synchronizer. In my mind, the synchronizer encapsulates all things related to staying up-to-date with the network: tracking peer stats, managing propagated blocks, downloading from the network. And full / fast modes are only concerned with how we download data from the network.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep, I can see that working. Done.

…astSynchronizer and FullSynchronizer to FastDownloaderFactory and FullDownloaderFactory.

fastSynchronizer =
FastSynchronizer.create(
this.fastSynchronizer =
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fastSynchronizer -> fastDownloaderFactory

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

}

public void start() {
chainDownloader.start();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To follow-through on the factory change, should this class now create() a Downloader that has a start() method? Just seems weird to call Factory.start().

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can't really because there are additional methods the "factories" need to expose - trailing peers for full sync and deleting the state for fast.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe these are FastDownloadManager and FullDownloadManager? Manager is a bit of a weasel word but I'm not seeing a better layout that would give clearer responsibilities at the moment.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you could get away with a Downloader that just exposes start and calculateTrailingPeerRequirements, then does any cleanup before the start future completes. But feel free to just rename if you think that makes more sense.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure we've got this structure quite right yet but I think I've found a reasonable landing point. Fast sync has a FastSyncDownloadFactory which returns a FastSyncDownload - full sync is much simpler so doesn't need the factory and just creates a FullSyncDownload directly.

End result is that DefaultSynchronizer winds up with a Optional<FastSyncDownload> and a FullSyncDownload both of which have start and calculateTrailingPeerRequirements.

There's more tidy-up that could be done in this area but I'm hesitant to get too distracted.

}

public void start() {
chainDownloader.start();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you could get away with a Downloader that just exposes start and calculateTrailingPeerRequirements, then does any cleanup before the start future completes. But feel free to just rename if you think that makes more sense.

ajsutton and others added 2 commits April 2, 2019 12:29
@ajsutton ajsutton merged commit bf9d3b4 into PegaSysEng:master Apr 2, 2019
@ajsutton ajsutton deleted the chain-downloader-abstraction branch April 2, 2019 02:48
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
2 participants