publish fleet ENRs and entree:// URL

[kaiserd]

Now that nwaku supports discv5 (v0.8) and soon discv5 DNS bootstrap (v0.9), it would be nice to have ENRs and an entree:// URL for the fleet published in the fleets json.

cc @richard-ramos @jm-clius

[jakubgs]

I don't think that makes sense. That JSON is a way to get addresses of our nodes for baking into our apps at build time.
It's not for publishing things for public use, and especially not things that are supposed to replace use of fleets.status.im.

Also, I'm not even sure how I would fit that into the script that generates the JSON, since it bases this on Consul data, and DNS entries for ENR don't fit with any specific host in any given fleet. Not sure how you imagine this would work.

[kaiserd]

Agreed, in that case entree:// URL would not fit in fleets.status.im.
But, adding the ENR for each fleet node would still be helpful, imo. This would be static bootstrap information and not aim to replace the use of fleets.status.im. Wdyt?

Regarding the entree://, there should be a place where nodes can get the URL from. An option would be loading it from a config file shipped with nwaku, or as part of the docker config.

[jakubgs]

I see, individual ENRs make more sense indeed. I can look into that. Is there an API call that can get me the ENR?

About the entree entries, when you say URL you mean DNS entry, rigth? Like test.waku.nodes.status.im:

 > dig +short TXT test.waku.nodes.status.im 
"enrtree-root:v1 e=SLJBDRNPBTM7X5RSO4WH25IDJY l=FDXN3SN67NA5DKA4J2GOK7BVQI seq=1 sig=zQ0_i4YO91B6wJ9doGoOLKb-V-stM8RdZAmnqSFtwKA8NAx6IbK-k-rRR3vrC5xzDDEFDUYCpC5--OvhKHgr8gA"

Or something else?

Those should probably come baked into nim-waku node binaries. For example, we have the DNS discovery flags:

 --dns-discovery           Enable discovering nodes via DNS [=false].
 --dns-discovery-url       URL for DNS node list in format 'enrtree://<key>@<fqdn>'.
 --dns-discovery-name-server  DNS name server IPs to query. Argument may be repeated.
                               [=@[ValidIpAddress.init("1.1.1.1"), ValidIpAddress.init("1.0.0.1")]].

Wouldn't it make sense to provide enrtree://ABC123@prod.waku.nodes.status.im(or I guess in the future that would be something different that corresponds to infra-status fleet, as infra-nim-waku was always intended more for development) as the default for --dns-discovery-url if --dns-discovery=true is set? @jm-clius thoughts?

[jakubgs]

With get_waku_v2_debug_v1_info I can get multiaddresses for the node. Could we add(if there already isn't one) a call the would return the node ENR address?

[kaiserd]

With entree:// URL, I mean an URL like enrtree://ABC123@prod.waku.nodes.status.im, not the DNS entries.
This URL encodes the public key the node needs to verify the DNS entries (EIP-1459) and the domain name the node has to query. It contains all information a node needs for retrieving ENRs via DNS.

Yes. Imo, we could set the production fleet entree:// URL as the default --dns-discovery-url in the future.
For now, it would be good if we just had a place where the test-fleet entree:// URL is published. Wdyt?

[kaiserd]

With get_waku_v2_debug_v1_info I can get multiaddresses for the node. Could we add(if there already isn't one) a call the would return the node ENR address?

I will check that.

[kaiserd]

I opend an issue and a PR to add the ENR to the info call:

• JSON RPC: add ENR URI to debug_v1_info waku-org/nwaku#917
• feat(jsonrpc): add ENR to debug_v1_info call waku-org/nwaku#918

[jakubgs]

Looks like this is already available in production:

 > a all -o -a '/docker/nim-waku-v2/rpc.sh get_waku_v2_debug_v1_info | jq .result.enrUri'
node-01.do-ams3.wakuv2.prod | CHANGED | rc=0 | (stdout) "enr:-Mi4QLr6Xdnrzp3imebRPr8LXuUKWablTwnXLzxGi5Q47-lYMwf4zPeGcfLy9IXXdT-PwXXv96eBS8ZunmaFlUKJ81cBgmlkgnY0gmlwhLymh5GKbXVsdGlhZGRyc7EALzYobm9kZS0wMS5kby1hbXMzLndha3V2Mi5wcm9kLnN0YXR1c2ltLm5ldAYfQN4DiXNlY3AyNTZrMaEDbl1X_zJIw3EAJGtmHMVn4Z2xhpSoUaP5ElsHKCv7hlWDdGNwgnZfhXdha3UyDw"
node-01.gc-us-central1-a.wakuv2.prod | CHANGED | rc=0 | (stdout) "enr:-NK4QHk-qAIQIDDjQmtKYZJmYikTgechWNeT9Tl-tCaKsImVL-_i4p54tYUW8KBjA4F3T97A5F6Ecq8brC4vhKaXRQ4BgmlkgnY0gmlwhCJ5ZGyKbXVsdGlhZGRyc7g6ADg2MW5vZGUtMDEuZ2MtdXMtY2VudHJhbDEtYS53YWt1djIucHJvZC5zdGF0dXNpbS5uZXQGH0DeA4lzZWNwMjU2azGhA_30kHgQqfXZRioa4J_u5asgXTJ5iw_8w3lEICH4TFu_g3RjcIJ2X4V3YWt1Mg8"
node-01.ac-cn-hongkong-c.wakuv2.prod | CHANGED | rc=0 | (stdout) "enr:-NK4QGWuLwNtQFSGUx_r_1ORLdZXCnSKX-dD1bO2QUGvz7LceTW4I4iBxrstbk7_9a7BdY5kbXr-YlhySHkEMkvNPqoBgmlkgnY0gmlwhAjS3ueKbXVsdGlhZGRyc7g6ADg2MW5vZGUtMDEuYWMtY24taG9uZ2tvbmctYy53YWt1djIucHJvZC5zdGF0dXNpbS5uZXQGH0DeA4lzZWNwMjU2azGhAo0C-VvfgHiXrxZi3umDiooXMGY9FvYj5_d1Q4EeS7eyg3RjcIJ2X4V3YWt1Mg8"

I can now try including this data into output of https://fleets.status.im/.

Although I wouldn't want that JSON file to just grow endlessly with more and more addresses for the same nodes.

[jakubgs]

Done:

 > c fleets.status.im | jq '.fleets."status.test"."enr/p2p/waku"'    
{
  "node-01.ac-cn-hongkong-c.status.test": "enr:-JK4QAi2yenhIbflVpUtkLDAM7yR5qeinZSQ_TNxfAWxdffkQWlstXyspEUlapiLl-S_MTPyp5V1uiV14ATjcVU_iLoBgmlkgnY0gmlwhC_y6SSJc2VjcDI1NmsxoQJkb7I6j5X1-zjieMeBdxgDqQWuMuDUEqDR419lyKNZb4N0Y3CCdl-DdWRwgiMohXdha3UyDw",
  "node-01.do-ams3.status.test": "enr:-JK4QIGyJyjSlponBZqe9XZ75b5ePSU9fWw5UtxEF1vsYXTLYkTa4psnUxJPmOcJGVt9XCYpC2h2kNoowl1t5KzuW6wBgmlkgnY0gmlwhEDhUe2Jc2VjcDI1NmsxoQIE3HkSvDkl2zD64j2kGMH9XZHpuu-5VYT4wnBgJApW_oN0Y3CCdl-DdWRwgiMohXdha3UyDw",
  "node-01.gc-us-central1-a.status.test": "enr:-JK4QM2ylZVUhVPqXrqhWWi38V46bF2XZXPSHh_D7f2PmUHbIw-4DidCBnBnm-IbxtjXOFbdMMgpHUv4dYVH6TgnkucBgmlkgnY0gmlwhCJ6_HaJc2VjcDI1NmsxoQM06FsT6EJ57mzR_wiLu2Bz1dER2nUFSCpaXzCccQtnhYN0Y3CCdl-DdWRwgiMohXdha3UyDw"
}

Changes:

• status-im/infra-role-nim-waku@c59aaf4c - consul: add service with ENR URI in metadata
• https://github.com/status-im/infra-misc/commit/1057ad48 - status-fleets: add ENR addresses to JSON
• status-im/infra-waku-connect@3cb4bc10 - requirements: add ENR URI to consul services
• status-im/infra-status-legacy@d681c21c - requirements: add ENR URI to consul services
• 7018a019 - requirements: add ENR URI to consul services

But honestly, I'm not a fan of adding more stuff there.