fix(WAKU2-RPC): encode waku message payloads in base64 #572
Conversation
LNSD
added
the
track:nwaku-productionization
There was a problem hiding this comment.
Nice! base64 for byte arrays feel much more 'natural'.
Btw, TIL that private api was exclusive for relay. In go-waku, the methods to retrieve messages return them regardless of the protocol on which they were received (filter or relay), since filter does not have a way to decode messages, only get_waku_v2_filter_v1_messages; and neither relay did before this PR
There was a problem hiding this comment.
I certainly agree with the change to base64, but think we need to decide on two things:
- Using
WakuMessageas input forpostmethods now require applications to provide fields for which they may not have the info to fill it out. I.e. to use version1they must use the private api methods. This may be OK, but it's worth considering the impact. - I'm OK with introducing backwards incompatibilities and announcing it clearly in release notes, but we should consider using the built in versioning in method signatures to maintain the old
postmethod (at least for a release cycle or so, to allow people to have a time where they can change their API usage before it breaks).
| `WakuRelayMessage` is an `Object` containing the following fields: | ||
|
|
||
| | Field | Type | Inclusion | Description | | ||
| | ----: | :---: | :---: | ----------- | | ||
| | `payload` | `String` | mandatory | The payload being relayed as a hex encoded data string | | ||
| | `contentTopic` | `String` | optional | Message content topic for optional content-based filtering | | ||
| | `timestamp` | `Number` | optional | The time at which the message is generated by its sender. This field holds the Unix epoch time in nanoseconds as a 64-bits integer value. | | ||
| | `ephemeral` | `Boolean` | optional | This flag indicates the transient nature of the message. | |
There was a problem hiding this comment.
This API input Object was constructed to separate the fields that a user/app may want to provide to the API from what the API should provide itself. This required separating the WakuMessage wire format from the object provided to post calls. Currently the only field which is determined by the API is version, but wouldn't it make sense to maintain separation between what is published and the message constructed by the node/API?
| | `ephemeral` | `Boolean` | optional | This flag indicates the transient nature of the message. | | ||
|
|
||
| > **_NOTE:_** `WakuRelayMessage` maps directly to a [`WakuMessage`](#wakumessage), except that the latter contains an explicit message `version`. For `WakuRelay` purposes, the versioning is handled by the API. | ||
|
|
||
| ### `post_waku_v2_relay_v1_message` |
There was a problem hiding this comment.
This is a breaking change for this API method (also the Private API methods, but these are presumable very seldomly used). Since the Relay API is used by several applications/platforms, wouldn't it make sense to maintain backwards compatibility with this API version (at least in the implementation) and introduce post_waku_v2_relay_v2_message (note v2 indicates new API version). The versioning as part of the method signature was introduced to prevent backwards incompatibilities.
We could also deprecate this function, but it may be a good idea to maintain backwards compatibility for one release cycle (i.e. until nwaku v0.16.0).
I didn't know about that at all 🤯 But for me makes sense to have separate APIs for different protocols (separation of concerns and single responsibility principles). |
| ## Relay Private API | ||
|
|
||
| The Private API provides functionality to encrypt/decrypt `WakuMessage` payloads using either symmetric or asymmetric cryptography. This allows backwards compatibility with [Waku v1 nodes](https://github.com/vacp2p/specs/blob/master/specs/waku/v1/waku-1.md). | ||
| It is the API client's responsibility to keep track of the keys used for encrypted communication. Since keys must be cached by the client and provided to the node to encrypt/decrypt payloads, a Private API SHOULD NOT be exposed on non-local or untrusted nodes. |
There was a problem hiding this comment.
a Private API SHOULD NOT be exposed on [non-local] or untrusted nodes.
I find it to be an odd phrasing. How can one expose an API on an untrusted node? Would be good to rephrase.
I guess there are two warnings:
- Do not expose your private API to untrusted actors (but if the nwaku node does not hold the key then is the encryption/decryption really the reason not to do that?)
- Do not use the relay private API on a node you don't trust (they could be caching your keys)
| | ----: | :---: | :---: |----------- | | ||
| | `topic` | `String` | mandatory | The [PubSub `topic`](https://github.com/libp2p/specs/blob/master/pubsub/README.md#the-topic-descriptor) being published on | | ||
| | `message` | [`WakuMessage`](#wakumessage) | mandatory | The (unencrypted) `message` being relayed | | ||
| | `symkey` | `String` | mandatory | The hex encoded symmetric key to use for payload encryption. This field MUST be included if symmetric key cryptography is selected | |
There was a problem hiding this comment.
symkey
Maybe best to stick to camel case across the whole RFC?
|
@fryorcraken I agree with you. But it was not my intention to rewrite the RFC. I just collocated the Private API after the Relay API. The diff messes it up 😅 We'll also update the phrasing if further improvements to the JSON-RPC RFC are necessary. Thanks for your comments :) |
While the plan is to supersede the JSON-PRC API with a Rest API implementation, some conversations are pending about the Waku API design that should happen before resuming the Rest API initiative.
Meanwhile, some stakeholders are using the JSON-RPC API (e.g., the Logos Data Science Team), and as a consequence, some issues are being reported in the nwaku repository:
get_waku_v2_relay_v1_messagesreturn an array has payload instead of expected hex string waku-org/nwaku#1139This PR aims to replace the
WakuMessagepayloadhexencoding with the more efficient and ergonomic (e.g., for the browsers)base64encoding.WakuMessagepayload encoding fromhextobase64.WakuMessagetype.I aim to merge these JSON-RPC implementation changes as part of this week's
nwakurelease (v0.15.0).