[Sync] Import task

[Voxelot]

The import task will check for work to do.

• If there is none it will wait to be notified about new work.
• If there is some it will run the import stream potentially getting back pressure from sections of the stream. In other words the import task will not take on new work until the current work is complete

Task Loop

flowchart TD
	I(Import Loop) -.-> |Check if work to do| ST{{State}}
	ST -.-> |Yes| I
	ST -.-> |No| N
	I --> |range: i..=j| IS[import stream]
	IS --> N[wait to be notified]
	N --> I

Loading

Import Stream

• get sealed header associated with a block height
• validate the sealed metadata with the consensus module
• fetch all transactions associated with the block header
• send full sealed block to the importer

If any of these stages fail, drop the stream and reset the shared state.

Stream example diagram

	
flowchart TD
	R[range_iterator] --> |sub range m..=n|GH[get_header_range]
	GH --> |get_header future| BP1
	subgraph BP1[Back pressure if MAX_GET_HEADER_REQUESTS is reached ]
		B1[stream::buffered MAX_GET_HEADER_REQUESTS]
	end
	B1 --> FL[stream::flattern]
	FL --> |Header| GT[get_transactions]
	GT --> |get_transactions future| BP2
	subgraph BP2[Back pressure if MAX_GET_TXNS_REQUESTS is reached ]
		B2[stream::buffered MAX_GET_TXNS_REQUESTS]
	end
	B2 --> |Block| BP3
	subgraph BP3[Back pressure while execute is running]
		SB[tokio::spawn_blocking]
		SB --> E[Execute]
	end

Loading

Ports

APIs that the import task will need from other services

trait P2P {
    /// Get the sealed block associated with the provided block height
    async fn get_sealed_block_header(height: BlockHeight) -> Result<(Sealed<Header>, PeerId)>;
    /// Get all transactions associated with a block id from a specific peer
    async fn get_block_transactions(block_id: BlockId, peer_id: PeerId) -> Result<Vec<Transaction>>;
}

trait Importer {
    /// Execute the provided block in verification mode and permanently commit the state if it's valid
    async fn execute_and_commit(&self, block: SealedBlock) -> Result<()>;
}

trait Consensus {
    /// Given a previous block header, validate the next header follows the consensus rules
    async fn validate_next_header(previous_header: Sealed<Header>, new_header: Sealed<Header>);
}