Skip to content
This repository has been archived by the owner on Aug 2, 2021. It is now read-only.

network: Add Capabilities to Kademlia database #1713

Merged
merged 36 commits into from
Sep 19, 2019
Merged

network: Add Capabilities to Kademlia database #1713

merged 36 commits into from
Sep 19, 2019

Conversation

nolash
Copy link
Contributor

@nolash nolash commented Aug 30, 2019 •
edited
Loading

This PR makes the Capabilities bitvector of each peer record stored in kademlia available to client code. It introduces the possibility of adding separate indices as pots for the kademlia, which will be added to and removed with calls to On and Off if the corresponding capability filter matches the entry being manipulated.

The capabilities are embedded in the BzzAddr object, which makes it convenient to transmit them through the kademlia aswell as the handshake message.

It also contains the following changes:

  • Omit self-connect in cmd/swarm/run_test.go
  • Add constructor for BzzAddr to avoid null capabilities.
  • Explicit rename of BzzAddr related methods to include "Bzz" in name
  • Custom RLP deserialization of peerMsg and BzzAddr due to failure of default operation to handle the capabilities array.
  • Enforce deterministic BzzAddr and enode.ID relation in test peer generation in network.
  • Remove some moot vars in Kademlia methods.
  • Remove obsolete lightnode test
  • Add comments to previously uncommented code from ENR/enode refactor and Capabilities implementation

Protocol versions are bumped to bzz:13 (since nerds after all greet any superstitions with contempt) and hive:11

@nolash nolash self-assigned this Aug 30, 2019
@nolash nolash force-pushed the adaptive-kademlia-2 branch 4 times, most recently from 0ca9e16 to bbc310d Compare September 13, 2019 11:00
zelig
zelig approved these changes Sep 16, 2019
View reviewed changes
network/kademlia_test.go Outdated Show resolved Hide resolved
kortatu
kortatu reviewed Sep 16, 2019
View reviewed changes
Copy link
Contributor

@kortatu kortatu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, except for the LightNode field in BzzPeer, that I think we don't need anymore.

network/kademlia.go Show resolved Hide resolved
network/kademlia.go Show resolved Hide resolved
network/kademlia.go Outdated Show resolved Hide resolved
network/kademlia.go Show resolved Hide resolved
network/kademlia_test.go Show resolved Hide resolved
network/protocol.go Outdated Show resolved Hide resolved
@nolash nolash requested a review from zelig September 17, 2019 08:34
zelig
zelig approved these changes Sep 17, 2019
View reviewed changes
Copy link
Member

@zelig zelig left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we get rid of custom serialisations?

network/discovery.go Show resolved Hide resolved
network/network.go Show resolved Hide resolved
network/network.go Outdated Show resolved Hide resolved
network/network_test.go Show resolved Hide resolved
@nolash
Copy link
Contributor Author

nolash commented Sep 18, 2019

@zelig I intend to make another pass later on the capability package to make the bitflag for the wire directly available behind a method call. I wish I left it the way it was originally, really...

@kortatu kortatu mentioned this pull request Sep 18, 2019
Merged
@nolash
network: Temporary workaround for contingent list in BzzAddr deserialize
84222da
@nolash
network: BzzAddr deserialize fail on empty object
70cb0cd
@nolash
network, cmd: WIP debugging
afc70b0
@nolash
network: Add deserialize in peersMsg to elimiate list error
b209db4
@nolash
network: Re-instate serialize capabilites to bytes in BzzAddr
b8149bb
@nolash
pot: Revert inadvertent rename of RandomAddress*
bd59236
@nolash
network/stream: Use new BzzAddr constructor in stream tests
060331b
@nolash
pot: Remove wrong rename for random address gen in pot test
3d631e9
@nolash
network: Remove comments, reinstate skipped hive test
3d32505
@nolash
network: Add test for kademlia capability indices
83cca5e
@nolash
network: Add test for connections capabilities index filtering
4cc3da1
@nolash
network: Add comments to capabilities index test
bedced0
@nolash
newtork: Add comments, stub for capabilities index removal
0355266
@nolash
network: Complete remove from index method and corresponding test
297eae3
@nolash
network: Speling
a9b5fe8
@nolash
network: Same structure of detect conn peer in add to index as remove
a876276
@nolash
network: Bump protocol versions
82b065d
@nolash
network: Update test protocol version in handshake/bzz protocol tests
2790fdf
@nolash
network: Remove lightnode from BzzPeer and cryptic nil in eachAddr
7d05df6
@nolash
network: Speling
61bf42c
@nolash
network: Remove retrieval light node test
7f24059
@nolash
network: Split up capability indices test
cb71212
@nolash
network: Add test for deserialization result for bzzaddr
f3cdb0d
@nolash nolash merged commit 4137999 into ethersphere:master Sep 19, 2019
@nolash nolash deleted the adaptive-kademlia-2 branch September 19, 2019 06:41
@skylenet skylenet added this to the 0.5.0 milestone Sep 19, 2019
janos added a commit that referenced this pull request Sep 23, 2019
@janos
Revert "network: Add Capabilities to Kademlia database (#1713)"
3bec9e8 
This reverts commit 4137999.
chadsr added a commit to chadsr/swarm that referenced this pull request Sep 23, 2019
@chadsr
Merge branch 'master' of github.com:ethersphere/swarm
febd040 
* 'master' of github.com:ethersphere/swarm: (32 commits)
  network/stream: refactor cursors tests (ethersphere#1786)
  network: Add capabilities if peer from store does not have it (ethersphere#1791)
  Swap logger (ethersphere#1754)
  network: Add capability filtered depth calculation (ethersphere#1787)
  travis: remove go1.12 job (ethersphere#1784)
  cmd/swarm: correct bzznetworkid flag description (ethersphere#1761)
  network, pss: Capability in pss (ethersphere#1764)
  network/stream: handle nil peer in TestNodesExchangeCorrectBinIndexes (ethersphere#1779)
  protocols, retrieval: swap-enabled messages implement Price (ethersphere#1771)
  cmd/swarm-smoke: fix waitToPushSynced connection closing (ethersphere#1781)
  cmd/swarm: simplify testCluster.StartNewNodes (ethersphere#1777)
  build: increase golangci-lint deadline (ethersphere#1778)
  docker: ignore build/bin when copying files (ethersphere#1780)
  swap: fix and rename Peer.getLastSentCumulativePayout (ethersphere#1769)
  network/stream: more resilient TestNodesCorrectBinsDynamic (ethersphere#1776)
  network: Add Capabilities to Kademlia database (ethersphere#1713)
  network: add own address to KademliaInfo (ethersphere#1775)
  pss: Refactor. Step 2. Refactor forward cache (ethersphere#1742)
  all: configurable payment/disconnect thresholds (ethersphere#1729)
  network/stream/v2: more resilient TestNodesExchangeCorrectBinIndexes (ethersphere#1760)
  ...
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Reviewers

@kortatu kortatu kortatu approved these changes

@zelig zelig zelig approved these changes

@jpeletier jpeletier Awaiting requested review from jpeletier

Assignees

@nolash nolash

Labels
enhancement kademlia ready for another review
Projects
None yet
Milestone
0.5.0
Development

Successfully merging this pull request may close these issues.

None yet
4 participants
@nolash @kortatu @zelig @skylenet