Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Upload syncs to closest node #48

Closed
5 of 7 tasks
Eknir opened this issue Apr 1, 2020 · 0 comments · Fixed by #115
Closed
5 of 7 tasks

Upload syncs to closest node #48

Eknir opened this issue Apr 1, 2020 · 0 comments · Fixed by #115
Assignees

Comments

@Eknir
Copy link
Contributor

Eknir commented Apr 1, 2020

Epic

Story

As a user of Swarm, I want that my chunk is stored at the node whose address is closest to the address of the chunk, such that anybody who requests the chunk afterwards, knows where to find this chunk.

Background

Full connectivity

This epic can be fulfilled on a full-connectivity network. The node who uploads a chunk looks in his address book for the node whose address is closest to the address of the chunk. In a full connectivity network, this happens to be the node who is closest to the address of the chunk. In a partial connectivity network (i.e. Kademlia), this may or may not be the node who is closest to the address of the chunk. From the perspective of the node, however, this doesn't matter. He just forwards the push sync request to the connected node who is closest to the address of the chunk.

Note on pricing

For this epic, it will be sufficient to upload the chunk to the closest connected node. However, in the future, there might be a possibility where the closest connected node is not the node to whom we should forward the push-sync, if there is a node who is further away, but cheaper.

See (picture from BOS):
image

Acceptance criteria

  • In a network with full connectivity, 100 random chunks are uploaded, and every chunk is stored only at the node whose address is closest to the chunks address

Dependencies

  • Modify topology driver interface to have a SyncPeer method
  • Modify full topology driver implementation to return closest peer for SyncPeer
  • Create protocol boilerplate
  • Protocol sinks chunks from two sources - push sync localstore subscription and chunks coming from the network on the same protocol

Protocol test vectors:

  • make sure chunk gets picked up from localstore push index and gets sent to the closest peer
  • forwarding test - incoming message with chunk, check if there's a closer peer (there is, or maybe there is not) - forward to that peer
  • no forwarding test - check that node does not forward when it is the closest (handle toplogy.ErrWantSelf)
@Eknir Eknir added the Epic label Apr 1, 2020
@Eknir Eknir added this to the Sprint 1 milestone Apr 14, 2020
@acud acud changed the title Upload to closest node Upload syncs to closest node Apr 15, 2020
acud added a commit that referenced this issue Apr 17, 2020
* add SyncPeer functionality to topology driver to return closest peer to a chunk when picking a peer to send a chunk to
* Import XOR-Distance functionality
acud added a commit that referenced this issue Apr 21, 2020
* netstore: introduce netstore
@acud acud closed this as completed in #115 Apr 28, 2020
acud added a commit that referenced this issue Apr 28, 2020
* pushsync: initial pushsync implementation

Co-authored-by: Zahoor Mohamed <zahoor@ethswarm.org>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants