Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Evaluate difficulty of threading through request/response pattern from networking to rest of the system [ Timeboxed ] #4561

Closed
SaiProServ opened this issue Feb 24, 2024 · 1 comment
Assignees

Comments

@SaiProServ
Copy link
Collaborator

Adding the first purely request-based message handling to the fetcher should lay out a blueprint how other components can be integrated post 1.6.

A follow-up issue should be created once this discovery is complete.

@marc-casperlabs
Copy link
Collaborator

Seems to be fairly feasible with a small bit of cleanup, especially for components like the fetcher. Backpressure propagation would be as follows:

  1. Upstream components (mainly the synchronizer) will be asked to have a defined limit on the number of parallel requests they make. They need not worry about timeouts, since the fetcher can now guarantee a response (albeit an error one).
  2. Fetcher itself knows the capacity of each peer (via chainspec configuration and can balance accordingly), keeping an internal, unlimited queue (which is low on memory usage, since it only stores IDs).
  3. Networking does no longer buffer any message beyond the buffers built into the transport itself (juliet).
  4. Proper request-response handling is used (see sample code), resulting in .respond() to be called to answer requests coming, and requests being made are made solely through EffectBuilder::make_network_request.

Essentially this results in a respnse or error being associated with every request made, all enforced by the networking layer. Fetcher and higher up components need not concern themselves with these.

See the 4561-request-response-eval branch for some sample code.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants