33/WAKU-DISCV5: Add first raw version #487

kaiserd · 2022-02-11T20:02:02Z

This PR addresses #486 introducing a first version of an RFC specifying a WAKU2 ambient discovery protocol based on discv5.
In its current state, it is meant as a basis for discussion.

oskarth

Nice to see this coming along!

First glance with misc admin stuff, will have a look at details later

oskarth · 2022-02-14T04:27:45Z

content/docs/rfcs/32/README.md

@@ -0,0 +1,102 @@
+---
+slug: 32


32 taken here #482

oskarth · 2022-02-14T04:28:01Z

content/docs/rfcs/32/README.md

+
+# References
+
+


CC0 copyright section

content/docs/rfcs/32/README.md

kdeme · 2022-02-14T09:06:44Z

content/docs/rfcs/32/README.md

+`32/WAKU2-DISCV5` spans a discovery network isolated from the Etherium Discovery v5 network.
+This separation allows for efficient queries.
+Using a dedicated Waku2 discovery network, Waku2 nodes can query this discovery network for a random set of nodes and directly use these randomly distributed nodes as bootstrap into the Waku2 network.
+If Waku2 would use the Etherium discovery v5 network a retrieved set of random nodes is not guaranteed to contain a Waku2 node leading to a needle-in-the-haystack problem.


is not guaranteed to contain a Waku2 node`

I get what you are trying to say here, but perhaps it should be formulated differently. The way it is written now implies that with a "dedicated Waku2 discovery network" one will always find only Waku nodes, which can not really be enforced. Any node could run on this new network if they adjust the protocol.

yes! I will address that.

@kdeme wdyt about the updated version?

kdeme · 2022-02-14T09:08:31Z

content/docs/rfcs/32/README.md

+* can be forked followed by changing the `protocol-id` string [link to go-discv5 once upstream]
+
+
+# Security Considerations


Perhaps something should be mentioned that due to running on a different network, (initial) small amount of nodes on the network will cause it to be at bigger risk for the attacks listed below.

Yes. I will add that.

@kdeme I added a frist basic explanation. Wdyt? I will elaborate on this later. I did not metion the problem of the (initially) small network size yet (todo).

kaiserd · 2022-02-14T09:50:36Z

Thank you for the feedback :). I will apply it.

jm-clius

This already lgtm as a raw spec. I've left some comments below. Feel free to mark PR as "ready for review" when you feel it is.

jm-clius · 2022-02-14T14:26:59Z

content/docs/rfcs/33/README.md

+* hard coded bootstrap nodes
+* `DNS discovery`
+* `peer-exchange` protocol
+* `33/WAKU2-DISCV5` (specified in this document)


Wondering if a list like this is not better suited to the "Discovery" section in RFC 10/WAKU2? You could then just reference that section here to make the some point. For example, we don't really support "peer-exchange" yet, but will hopefully soon ™. Since the list of supported discovery protocols change frequently, we'll have to update this in several places.

agreed. Maybe we could leave it here as long as the draft is in raw state, and once it advances we could update 10/WAKU and substitue a reference for this list. Wdyt?

content/docs/rfcs/33/README.md

jm-clius · 2022-02-14T14:30:27Z

content/docs/rfcs/33/README.md

+### w.r.t. Ethereum Discovery v5
+`33/WAKU2-DISCV5` spans a discovery network isolated from the Ethereum Discovery v5 network.


Suggested change

### w.r.t. Ethereum Discovery v5

`33/WAKU2-DISCV5` spans a discovery network isolated from the Ethereum Discovery v5 network.

### w.r.t. Ethereum Discovery v5

`33/WAKU2-DISCV5` spans a discovery network isolated from the Ethereum Discovery v5 network.

content/docs/rfcs/33/README.md

staheri14

Nice one! please see my comments :)

staheri14 · 2022-02-22T23:04:35Z

content/docs/rfcs/33/README.md

+The purpose of ambient node discovery within [10/WAKU2](/specs/10) is discovering Waku2 nodes in a decentralized way.
+The unique selling point of `33/WAKU2-DISCV5` is its holistic view of the network, which allows avoiding hotspots and allows merging the network after a split.
+While the other methods provide either a fixed or local set of nodes, `33/WAKU2-DISCV5` can provide a random sample of Waku2 nodes.
+Future iterations of this document will add the possibility of efficiently discovering Waku2 nodes that have certain capabilities, e.g. holding messages of a certain time frame during which the querying node was offine.


Suggested change

Future iterations of this document will add the possibility of efficiently discovering Waku2 nodes that have certain capabilities, e.g. holding messages of a certain time frame during which the querying node was offine.

Future iterations of this document will add the possibility of efficiently discovering Waku2 nodes that have certain capabilities, e.g. holding messages of a certain time frame during which the querying node was offline.

staheri14 · 2022-02-22T23:05:03Z

content/docs/rfcs/33/README.md

+
+* hard coded bootstrap nodes
+* `DNS discovery`
+* `peer-exchange` protocol


Any ref for this?

staheri14 · 2022-02-22T23:07:28Z

content/docs/rfcs/33/README.md

+
+`33/WAKU2-DISCV5` spans an overlay network separate from the [GossipSub](https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/README.md) network [11/WAKU2-RELAY](/specs/11) builds on.
+Being a P2P network on its own, it also depends on bootstrap nodes.
+The advantage of having a separate discovery network is reducing load on the bootstrap nodes as the actual work is done by randomly discovered nodes, which in turn increases decentralization.


Can you please break this sentence into shorter sentences, it is a bit hard to understand.

staheri14 · 2022-02-22T23:16:11Z

content/docs/rfcs/33/README.md

+
+## Security Implications of a Separate Discovery Network
+
+A dedicated Waku discovery network can be subject to eclipse attacks if not properly secured.


Would be good to briefly explain 1) the eclipse attack and how it harms the network and 2)why separating the waku discovery network from Eth discv5 make it vulnerable to this attack.

staheri14 · 2022-02-22T23:21:36Z

content/docs/rfcs/33/README.md

+### w.r.t. Waku2 Relay Network
+
+`33/WAKU2-DISCV5` spans an overlay network separate from the [GossipSub](https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/README.md) network [11/WAKU2-RELAY](/specs/11) builds on.
+Being a P2P network on its own, it also depends on bootstrap nodes.


Since we still need bootstrap nodes for a separate discovery network, wouldn't it be somewhat centralized? (similar to the discovery method#1 you listed above). How about a hybrid approach where some of the nodes in the waku discovery network also take part in the Eth discv5 net hence be discoverable there as well?

kaiserd · 2022-02-25T10:20:18Z

@staheri14 Thank you very much for the feedback. Because good and important comments in the discussion here. I am thinking about possible hybrid solutions, as you also pointed out in your feedback.

Imho, the best "hybrid" would be:

integrating efficient topic/capability discovery into Ethereum discv5
Waku being part of the Ethereum discv5 (not having a separate network at all)
making the waku-capability discoverable as a discv5 topic/capability
additionally, using random-walk discovery, as a fallback for mitigating eclipse attacks.

This would allow us integrating the benefits of having a separate discovery network into the existing Etherium discv5 network.

If the Ethereum discv5 community does not like this, another possible hybrid would be:

using a separate discv5 network as described in the PR
additionally allowing stronger nodes to participate in the Ethereum discv5 network and integrating discovered nodes into the waku-exclusive routing tables.

(Adaptive nodes can support

no discv5
separate discv5
both separate and Ethereum discv5 )

These are my next steps:

finish implementing and testing a first working Waku discv5 integration based on this PR
- this allows early dogfooding and "advertising" this feature to Desktop
~~refine the first hybrid idea I described and~~ post an issue on the Ethereum discv5 Github to start a discussion (I already talked about this with @kdeme)
apply your feedback and adapting this PR

oskarth · 2022-02-28T04:29:11Z

Agree re raw spec comment above. What's blocking from this being ready to review and mergeable?

32 has been taken #482

Co-authored-by: Hanno Cornelius <68783915+jm-clius@users.noreply.github.com>

kaiserd · 2022-03-03T15:12:46Z

I applied feedback and also reworked the raw RFC based on new results and insights. Imo, it is ready to be merged as a raw RFC after another review phase.

jm-clius · 2022-03-03T16:42:14Z

content/docs/rfcs/33/README.md

+
+This version of `33/WAKU2-DISCV5` has a focus on timely deployment of an efficient discovery method for [10/WAKU2](/specs/10).
+Establishing a separate discovery network is in line with this focus.
+However, we are aware of potential resilience problems (see section on security considerations) and are [discussing](https://forum.vac.dev/t/waku-v2-discv5-roadmap-discussion/121/8)


I like that this is specified.

jm-clius

A very good read and clear spec for waku2 discv5. Thanks!
My only comment would be that, since we have a reference implementation already and have demonstrated that this works, the status could likely be draft iso raw already. See https://rfc.vac.dev/spec/1/.

@oskarth, wdyt?

content/docs/rfcs/33/README.md

Co-authored-by: Hanno Cornelius <68783915+jm-clius@users.noreply.github.com>

oskarth · 2022-03-07T03:38:02Z

My only comment would be that, since we have a reference implementation already and have demonstrated that this works, the status could likely be draft iso raw already. See https://rfc.vac.dev/spec/1/.

Agree

oskarth

Nice one!

oskarth · 2022-03-07T03:39:31Z

content/docs/rfcs/33/README.md

+
+Properly protecting against eclipse attacks is challenging and raises research questions that we will address in future stages of our discv5 roadmap.
+
+# References


Can add links referenced throughout text here

content/docs/rfcs/33/README.md

Co-authored-by: Franck R. <franck@status.im>

* master: RFC16: add version call (#505) fix(noise): update RFC to implementation (#508) fixup: 37/WAKU2-NOISE fix images paths (#506) New RFC: 37/WAKU2-NOISE-SESSIONS (#504) 36/WAKU2-BINDINGS-API (#501) docs(16/WAKU2-RPC): add ENR to waku info (#502) Adding 35/WAKU2-NOISE to menu (#500) add RFC33 to index (#499) feat: 32/RLN raw spec New RFC: 35/WAKU2-NOISE (#496) Update on the rln registration figure to match the current spec (#497) 33/WAKU-DISCV5: Add first raw version (#487) Add pubsubTopic field to index (#492) Fix markdown links (#493) Categorize 22 & 31 (#490) Changed PB Timestamp index to 10 (#491) 13/14/16/21: Change in timestamp format (#483) add: RFC31 copyright statement (#489) 17/WAKU-RLN-RELAY: Revise spec for its draft version (#484)

kaiserd requested review from D4nte, richard-ramos, oskarth, s1fr0, staheri14 and jm-clius February 11, 2022 20:02

oskarth added this to New in vac-research Feb 11, 2022

oskarth moved this from New to Review/QA in vac-research Feb 11, 2022

kaiserd marked this pull request as draft February 11, 2022 20:06

oskarth changed the title ~~add: first raw version of 32/WAKU-DISCV5~~ 33/WAKU-DISCV5: Add first raw version Feb 14, 2022

oskarth reviewed Feb 14, 2022

View reviewed changes

kdeme reviewed Feb 14, 2022

View reviewed changes

content/docs/rfcs/32/README.md Outdated Show resolved Hide resolved

kdeme reviewed Feb 14, 2022

View reviewed changes

content/docs/rfcs/32/README.md Outdated Show resolved Hide resolved

kdeme reviewed Feb 14, 2022

View reviewed changes

content/docs/rfcs/32/README.md Outdated Show resolved Hide resolved

kdeme reviewed Feb 14, 2022

View reviewed changes

content/docs/rfcs/32/README.md Outdated Show resolved Hide resolved

kdeme reviewed Feb 14, 2022

View reviewed changes

kaiserd mentioned this pull request Feb 14, 2022

Add/rfc31 copyright #489

Merged

jm-clius reviewed Feb 14, 2022

View reviewed changes

kaiserd mentioned this pull request Feb 22, 2022

discv5 stage1 beta test waku-org/nwaku#862

Merged

staheri14 reviewed Feb 22, 2022

View reviewed changes

kaiserd force-pushed the add/waku-discv5-rfc branch from bd1fb30 to efd28b7 Compare February 25, 2022 10:23

kaiserd added 5 commits March 3, 2022 09:52

add: first raw version of 32/WAKU-DISCV5

b5d5286

fix: title

e8bf036

fix: rename slug 32->33

cf6cd1e

32 has been taken #482

fix: Etherium -> Ethereum 😅

9f164fe

add: copyright section

4565baf

kaiserd and others added 6 commits March 3, 2022 09:52

Update content/docs/rfcs/33/README.md

59c1fe8

Co-authored-by: Hanno Cornelius <68783915+jm-clius@users.noreply.github.com>

Update content/docs/rfcs/33/README.md

79a4f75

Co-authored-by: Hanno Cornelius <68783915+jm-clius@users.noreply.github.com>

feedback @kdeme

4571573

feedback @kdeme

c00bcca

feedback @staheri14: typo fix

d551cef

rework RFC & address feedback

6d7d257

kaiserd force-pushed the add/waku-discv5-rfc branch from efd28b7 to 6d7d257 Compare March 3, 2022 14:53

minor

2beaab6

kaiserd marked this pull request as ready for review March 3, 2022 15:12

kaiserd requested review from oskarth, staheri14, kdeme and jm-clius March 3, 2022 15:12

jm-clius reviewed Mar 3, 2022

View reviewed changes

jm-clius approved these changes Mar 3, 2022

View reviewed changes

content/docs/rfcs/33/README.md Outdated Show resolved Hide resolved

content/docs/rfcs/33/README.md Outdated Show resolved Hide resolved

kaiserd and others added 2 commits March 3, 2022 18:35

Update content/docs/rfcs/33/README.md

e3f55b8

Co-authored-by: Hanno Cornelius <68783915+jm-clius@users.noreply.github.com>

Update content/docs/rfcs/33/README.md

4d80705

Co-authored-by: Hanno Cornelius <68783915+jm-clius@users.noreply.github.com>

kaiserd mentioned this pull request Mar 4, 2022

New RFC: specify Waku2 ambient peer discovery protocol based on discv5 #486

Closed

1 task

oskarth approved these changes Mar 7, 2022

View reviewed changes

D4nte reviewed Mar 7, 2022

View reviewed changes

content/docs/rfcs/33/README.md Outdated Show resolved Hide resolved

kaiserd and others added 2 commits March 7, 2022 12:19

Update content/docs/rfcs/33/README.md

290b1e7

Co-authored-by: Franck R. <franck@status.im>

add: address final feedback

c113395

kaiserd merged commit 6498d92 into master Mar 7, 2022

vac-research automation moved this from Review/QA to Done Mar 7, 2022

kaiserd deleted the add/waku-discv5-rfc branch March 7, 2022 11:44

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

33/WAKU-DISCV5: Add first raw version #487

33/WAKU-DISCV5: Add first raw version #487

kaiserd commented Feb 11, 2022 •

edited

oskarth left a comment

oskarth Feb 14, 2022

kaiserd Feb 14, 2022

oskarth Feb 14, 2022

kaiserd Feb 14, 2022

kdeme Feb 14, 2022 •

edited

kaiserd Feb 14, 2022

kaiserd Feb 14, 2022

kdeme Feb 14, 2022

kaiserd Feb 14, 2022

kaiserd Feb 14, 2022 •

edited

kaiserd commented Feb 14, 2022

jm-clius left a comment

jm-clius Feb 14, 2022

kaiserd Feb 14, 2022

jm-clius Feb 14, 2022

staheri14 left a comment

staheri14 Feb 22, 2022

staheri14 Feb 22, 2022

staheri14 Feb 22, 2022

staheri14 Feb 22, 2022

staheri14 Feb 22, 2022

kaiserd commented Feb 25, 2022 •

edited

oskarth commented Feb 28, 2022

kaiserd commented Mar 3, 2022

jm-clius Mar 3, 2022 •

edited

jm-clius left a comment

oskarth commented Mar 7, 2022

oskarth left a comment

oskarth Mar 7, 2022

kaiserd Mar 7, 2022

		* can be forked followed by changing the `protocol-id` string [link to go-discv5 once upstream]


		# Security Considerations

		### w.r.t. Ethereum Discovery v5
		`33/WAKU2-DISCV5` spans a discovery network isolated from the Ethereum Discovery v5 network.

	Future iterations of this document will add the possibility of efficiently discovering Waku2 nodes that have certain capabilities, e.g. holding messages of a certain time frame during which the querying node was offine.
	Future iterations of this document will add the possibility of efficiently discovering Waku2 nodes that have certain capabilities, e.g. holding messages of a certain time frame during which the querying node was offline.


		## Security Implications of a Separate Discovery Network

		A dedicated Waku discovery network can be subject to eclipse attacks if not properly secured.


		Properly protecting against eclipse attacks is challenging and raises research questions that we will address in future stages of our discv5 roadmap.

		# References

		@@ -0,0 +1,102 @@
		---
		slug: 32

33/WAKU-DISCV5: Add first raw version #487

33/WAKU-DISCV5: Add first raw version #487

Conversation

kaiserd commented Feb 11, 2022 • edited

oskarth left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

kdeme Feb 14, 2022 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

kaiserd Feb 14, 2022 • edited

Choose a reason for hiding this comment

kaiserd commented Feb 14, 2022

jm-clius left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

staheri14 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

kaiserd commented Feb 25, 2022 • edited

oskarth commented Feb 28, 2022

kaiserd commented Mar 3, 2022

jm-clius Mar 3, 2022 • edited

Choose a reason for hiding this comment

jm-clius left a comment

Choose a reason for hiding this comment

oskarth commented Mar 7, 2022

oskarth left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

kaiserd commented Feb 11, 2022 •

edited

kdeme Feb 14, 2022 •

edited

kaiserd Feb 14, 2022 •

edited

kaiserd commented Feb 25, 2022 •

edited

jm-clius Mar 3, 2022 •

edited