network: Add adaptive capabilities message #1619

nolash · 2019-07-29T17:29:15Z

This PR is the implementation of part of https://github.com/nolash/SWIPs/blob/SWIP-ADAPTIVE-NODE-PROTOCOL/SWIPs/swip-adaptive-node-protocol.md

introduce capabilities types
remove the lightnode field in handshake, replace with capabilities
test that capabilities are set as expected through the provided calls

(API will be added in separate PR)

#obsoletes #1529

nolash · 2019-07-30T19:04:39Z

@zelig

If we want these finegrained caps, it is because they can be directly referenced in conditionals at the appropriate place they are relevant, i.e. have an interface where i can write if peer.Can(caps.Retrieve) or something, oder?

Yeah API is next PR.

if caps are a fixed list you enlist in protocol, then caps could just be sequential (iota) and be represented as a boolarray and serialised as bytes interpreted as your favourite bitvector :)

probably we want a "FromBools" or/and "FromBytes" in the API?

If caps are meant to be extensible i dont see how you take care of id clashes.

By convention was the idea.

zelig

the complexity and indirection here is too much for my taste.
do we have use cases of anything beyond a fix set of boolean flags?
I would prefer introducing simple flags first as and when they are needed with a functionality and only then see what if any abstraction needs to emerge, no?

network/adaptive.go

nolash · 2019-08-05T06:49:37Z

do we have use cases of anything beyond a fix set of boolean flags?

As I said before in the last PR, the whole adaptive concept with Status suggests that capabilities will change. This provides a way to change them. Incremental small changes.

Also, the idea is that modules (pss, swap, ...) define and set their own capabilities. That would require an API.

So if we assume that it's not only "setting static bit flags," is it still too complex?

acud

@nolash LGTM.
A few minor comments/questions in the body of the PR but this is a major improvement over the previous PR. No blockers from my side.
I have a separate question though about the Capabilities``&|``Capability types - there seems to be no methods to check whether a certain capability actually exists within the data structure.
Example: I am a full node and I want to know if the other node is not just a light node, but if it also possesses capabilities of stream protocol, or what not. I am not sure if this is intentional but I think we will need this pretty soon. For your consideration.
Peace

network/adaptive.go

network/adaptive_test.go

network/protocol.go

nolash · 2019-08-05T09:59:10Z

there seems to be no methods to check whether a certain capability actually exists within the data structure.

Plan is to add API in which this is possible. But that's for the next PR.

Actually I was thinking probably we shouldn't even have sync as a flag in the bzz core, it should be controlled by the streaming package?

janos · 2019-08-06T09:30:03Z

Imported but not used package https://travis-ci.org/ethersphere/swarm/jobs/567808411.

nolash · 2019-08-06T16:17:51Z

Imported but not used package https://travis-ci.org/ethersphere/swarm/jobs/567808411.

@janos fixed thanks

network/adaptive.go

nolash · 2019-08-15T16:15:32Z

@janos thanks changed

janos

Thanks @nolash. LGTM.

* 'master' of github.com:ethersphere/swarm: chunk, storage: chunk.Store multiple chunk Put (ethersphere#1681) storage: fix pyramid chunker and hasherstore possible deadlocks (ethersphere#1679) pss: Use distance to determine single guaranteed recipient (ethersphere#1672) storage: fix possible hasherstore deadlock on waitC channel (ethersphere#1674) network: Add adaptive capabilities message (ethersphere#1619) p2p/protocols, p2p/testing; conditional propagation of context (ethersphere#1648) api/http: remove unnecessary conversion (ethersphere#1673) storage: fix LazyChunkReader.join potential deadlock (ethersphere#1670) HTTP API support for pinning contents (ethersphere#1658) pot: Add Distance methods with tests (ethersphere#1621) README.md: Update Vendored Dependencies section (ethersphere#1667) network, p2p, vendor: move vendored p2p/testing under swarm (ethersphere#1647) build, vendor: use go modules for vendoring (ethersphere#1532)

nolash added 30 commits July 29, 2019 17:25

network: Add Capability and Capabilities types

6529934

network: Tests pass (but cleanup needed)

6f76c21

network: Remove useless pointers

95931c1

network: Add missing file, extend cap's test, reset altered tests

532a630

network: Add check and test for 'invalid' capabilities

01a2e69

network: Move capabilities to BzzAddr

551169c

network: Remove redundant Capabilities member on BzzPeer

7af6307

network: Add API and test, move adaptive tests to sep file

511b1a1

network: Conceal capability type

5286a73

network: Change Capabilities to struct, intro change chan

b3420aa

network: Add event channel subscribe

273397c

network: Add cleanup on service stop

bba0cd3

network: Add test for notify in API test

776e895

network: Add rpc module Subscribe

27ed8b0

network: Reinstate internal channel notify test

1ea689d

network: Extract API to separate file

29ed01b

network: Remove API files (will be PRd separately)

2f1cf1b

network: Add comments to code

c26a387

pss: Remove stray alteration in pss

ff9ec84

network: Speling

6d9259a

network: Bump protocol version, as rebase updates crossed last one

4e674f7

network: Delint

a7cde4c

network: Make Capabilities in HandshakeMsg serializable

d9c79dc

network: CapabilitiesMsg add get() instead of fromMsg()

1ac7367

network: WIP Simplify capabilities objects

eb5efd0

network: WIP Rehabilitate tests after simplification

32db8eb

network: WIP revert to rlp encode and export cap/caps fields

fb9c2e9

network: Fixed full and light comparisons

967ba7c

network: WIP custom RLP decoder for Capabilities collection

45f8efc

network: Capabilities RLP Decoder now re-populates id to array idx map

86140a5

zelig reviewed Aug 3, 2019

View reviewed changes

network/adaptive.go Show resolved Hide resolved

acud previously approved these changes Aug 5, 2019

View reviewed changes

network/adaptive.go Outdated Show resolved Hide resolved

network/adaptive.go Outdated Show resolved Hide resolved

network/adaptive.go Outdated Show resolved Hide resolved

network/adaptive_test.go Outdated Show resolved Hide resolved

network/protocol.go Show resolved Hide resolved

network: Add comment on cap id index + remove leftover debug

eaba70f

nolash dismissed acud’s stale review via eaba70f August 5, 2019 10:03

nolash added 2 commits August 6, 2019 13:29

network: Remove unused import

637178c

network: Remove comment

8c8cdbe

acud approved these changes Aug 8, 2019

View reviewed changes

zelig approved these changes Aug 14, 2019

View reviewed changes

network: Remove really annoying stdout line

5e10954

zelig suggested changes Aug 14, 2019

View reviewed changes

network/adaptive.go Outdated Show resolved Hide resolved

network/adaptive.go Outdated Show resolved Hide resolved

network: Add pointer receivers due to allow mutex reference passing

9c6d4a5

janos reviewed Aug 15, 2019

View reviewed changes

network/adaptive.go Outdated Show resolved Hide resolved

network/adaptive.go Outdated Show resolved Hide resolved

network: Rename Id->ID, pointer rcv in String()

69b436a

nolash added merge-imminent and removed merge-imminent ready for review labels Aug 15, 2019

nolash requested a review from zelig August 15, 2019 16:14

janos approved these changes Aug 15, 2019

View reviewed changes

zelig approved these changes Aug 16, 2019

View reviewed changes

zelig merged commit c535b27 into ethersphere:master Aug 16, 2019

skylenet added this to the 0.5.0 milestone Aug 16, 2019

nolash deleted the adaptive-capabilities-msg-nonotify branch August 16, 2019 09:34

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

network: Add adaptive capabilities message #1619

network: Add adaptive capabilities message #1619

nolash commented Jul 29, 2019 •

edited

Loading

nolash commented Jul 30, 2019

zelig left a comment

nolash commented Aug 5, 2019

acud left a comment

nolash commented Aug 5, 2019

janos commented Aug 6, 2019

nolash commented Aug 6, 2019

nolash commented Aug 15, 2019

janos left a comment

network: Add adaptive capabilities message #1619

network: Add adaptive capabilities message #1619

Conversation

nolash commented Jul 29, 2019 • edited Loading

nolash commented Jul 30, 2019

zelig left a comment

Choose a reason for hiding this comment

nolash commented Aug 5, 2019

acud left a comment

Choose a reason for hiding this comment

nolash commented Aug 5, 2019

janos commented Aug 6, 2019

nolash commented Aug 6, 2019

nolash commented Aug 15, 2019

janos left a comment

Choose a reason for hiding this comment

nolash commented Jul 29, 2019 •

edited

Loading