Signed JSONRPC 2.0 request scheme for pool API

[shazow]

A vipnode-client who wants to participate in usage-metered pools will need to implement an additional protocol for reporting which nodes the client is connected to.

This is a rough draft of what this protocol would look like.

Requirements

• vipnode_client: Request for a set of vipnodes that are ready for nodeID to connect.
• vipnode_update: Periodic updates with details about connected peers to the vipnode pool.
• (Optional) vipnode_register: Automatic registration flow to update nodeID <- address mapping.

Question: Does it make sense to use a vipnode_ RPC method prefix for these, or should it be something more general?

Considerations

Can be implemented as a push to, or a pull from, the vipnode pool. It can be implemented as a streaming API (e.g. websocket) or separate request/response (e.g. HTTPS).

Question: Assuming that the target audience for the protocol is mobile clients, it's probably a better idea to do client-initiated polling request/response for the sake of battery life?

For the sake of consistency with the rest Ethereum's RPC, the calls will be JSON-RPC 2.0.

Payloads must be signed and verified with the node key to avoid pools from being able to fake paid activity.

Request/Response Examples:

Warning: These are out of date. See discussion.

vipnode_client:

TODO

vipnode_update:

(From the client)

Request POST https://pool.vipnode.org/api with body:

{
  "jsonrpc": "2.0",
  "method": "vipnode_peers",
  "params": {
    "nodeID": "19b5013d2424...0738ac974f6",
    "timestamp": "1527609234005",
    "signature": "<Signature of subset of payload by private node key>",
    "peers": [
      {
        "nodeID": "1234abcd...dcba4321"
      },
      {
        "nodeID": "1234abcd...dcba4321"
      }
    ]
  },
  "id": 1
}

Response 200 from server with body:

{
  "jsonrpc": "2.0",
  "result": {
    "balance": "xxx"
  },
  "id": 1
}

Questions

1. What other metadata do we need from the peers?
2. Which subset of the payload should the signature sign? Should it include everything except the signature field, using a key-sorted deterministic JSON encoding?
3. Should signature be a top-level field? This would allow us to only sign params without injecting a new key, but would it violate JSON-RPC 2.0 and make the implementation more difficult for some JSONRPC libraries?
4. Do we need anything else from the vipnode-pool in the response?

vipnode_register:

TODO, some related notes here: #7

[shazow]

@ligi Do you have any thoughts on the above? Is this something that would be easy enough for you to implement for the WallETH integration?

[ryanschneider]

What about standardizing signatures with something like:

{
...
"params": {
   "payload": { ... },
   "signature": "<signature of payload object>"
}

payload could be something better, but IMO still generic so it can be used for a signed messages.

[shazow]

@ryanschneider Yea, that could be workable. I think aesthetically, it might be best to make it a top-level thing like:

{
  "jsonrpc": "2.0",
  "method": "...",
  "params": { ... },
  "params_sig": "<signature of params>",
  "id": 1
}

But I'm investigating if this is Bad (https://twitter.com/shazow/status/1001505943484555264, if you feel like RT'ing :P).

[shazow]

Things like https://godoc.org/github.com/sourcegraph/jsonrpc2 include an optional top-level meta key.

[ryanschneider]

Maybe piggyback off of their meta key and put it in there? ``` { "meta": "params_sig": "...." } ``` I'd guess a fair number of projects use a higher-level JSONRPC lib, which might strip out or reject the extra key. Putting it in `meta` would at least allow using their lib for parsing. :)

…

On Tue, May 29, 2018 at 9:57 AM, Andrey Petrov ***@***.***> wrote: Things like https://godoc.org/github.com/sourcegraph/jsonrpc2 include an optional top-level meta key. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#10 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AADREIL5TuelKSx7Q8qD0nC5FLoLnwpnks5t3X4EgaJpZM4URw6_> .

[shazow]

Dumping some notes:

Layout

My current thinking is this layout:

{
  "jsonrpc": "2.0",
  "method": "foo",
  "params": {
    "signed": {
      "method": "foo",
      "nonce": "<epoch timestamp>",  // Or should this be "timestamp" and reject time deviation?
      "pubkey": "<base64 node id>",  // Or should this be "id"? nodeID?
      ... other params required for the method
    },
    "signature": "<base64 signature of 'signed' with 'pubkey'>",
  },
  "id": 1
}

Pros:

• Works with all existing JSONRPC clients (namely targeting https://godoc.org/github.com/ethereum/go-ethereum/rpc at the moment)
• Vaguely similar layout here https://api.random.org/json-rpc/1/signing (though different flow)

Cons:

• Method is redundant
• More nesting, cluttered

Questions:

• What kind of signature? (Discussed below)
• What about positional params? (Not supported in this configuration)

The alternative is to force positional-only params, like:

{
  "jsonrpc": "2.0",
  "method": "foo",
  "params": {
    "signed": [
      "foo",
      "<epoch timestamp>",
      "<base64 node id>",
      ... other positional params required for the method
    },
    "signature": "<base64 signature of 'signed' with node id>",
  },
  "id": 1
}

Makes the signing process a bit less ambiguous (no need to sort keys).

Signature

NodeIDs are base64 ecdsa public keys.

There are some handly helpers in https://godoc.org/github.com/ethereum/go-ethereum/crypto

go-ethereum generally uses crypto/ecdsa.Sign of a Keccak256 hash (which apparently does not need a MAC?).

This is all assuming we want to stick close to what Ethereum already does.

If we wanted to deviate entirely, we could just use nacl/sign or something instead. We could probably even get away from NodeIDs altogether and make our own keys to cross-sign the NodeIDs with, but that's probably a bad idea?

[ryanschneider]

At first I would've voted for the signed object, but after thinking a bit more, I actually like the symmetry of the signed array with the fact that ethereum's params are always an array for all methods.

In that case, I don't think you need to repeat the method as signed[0] though, just instead say that the signature is over .method + .params.signed[], unless there's a reason why the method would ever not be part of the signature.

Actually, now that I think about it, you might have some difficulty handling this RPC in-process later if params is an object and not an array, just because AFAICT go-ethereum assumes in params will be an array in the RPC layer (see ServerCodec.ParseRequestArguments in rpc/types.go. Something to consider if you ever plan on proposing this as an EIP, or want to handle these requests in a fork of geth or other ethereum clients vs. a standalone API.

[shazow]

@ryanschneider Hmm TIL about RPC in-progress not supporting objects. Are you sure about this? Their examples certainly work with objects: https://github.com/ethereum/go-ethereum/blob/master/rpc/client_test.go#L44

Or do you mean that the proper ethereum RPCs are registered as positional args?

I think that wouldn't restrict vipnode registering an RPC service with object args, but you make a good point regarding consistency.

The main reason I wanted to include the method in the params is to make it clearer what is "signed" by declaration, but .method + .params.signed[] is not too bad.

[ryanschneider]

Sorry I should've been more clear. Yes, they support objects, but the params field itself is always an array. There might be some magic I'm missing that converts {... "params": { "foo": "bar" } } into the equivalent of {... "params": [{"foo":"bar"}]} but I haven't seen it yet.

But really my main point is that if you look at all the existing RPCs, they all take params as an array, even the ones that take a single object as a param (e.g. eth_newFilter) expect params to look like [{...}].

Because of this, some ethereum client APIs that build RPC requests might assume the params is always an array as well. Actually geth's own client that you linked to the test for those this as well:

https://github.com/ethereum/go-ethereum/blob/1836366ac19e30f157570e61342fae53bc6c8a57/rpc/client.go#L449

While jsonrpcMessage defines Params as a json.RawMessage, the newMessage function I linked to above accepts paramsIn ...interface{}, which is always a slice, even for a single argument.

I would not be surprised if web3.js and other frameworks for interacting with ethereum clients also often make this assumption.

[shazow]

Regarding the signing stuff, this is how it's looking so far: https://github.com/vipnode/vipnode/tree/wip/request

Seems okay for now, will see how it's feeling after some usage.

[shazow]

Full working RPC example: https://github.com/vipnode/vipnode/blob/wip/pool/service_test.go#L60

I'm not super happy with it, primarily because the nonce ends up basically being a unique msg ID which is already part of the JSONRPC protocol so it's frustratingly redundant but can't get access to it from the client API.

I might spend a bit more time surveying the JSONRPC 2.0 libraries out there (see what's in JS/rust land) and see how plausible it is to put the signature back at the top-level and do a full-message signature including the message ID and method, or maybe see if other messages let you provide your own ID generator.

Approximately | | this close to starting to spec out JSONRPC 3.0. :P

[shazow]

The current signing scheme looks like this:

{
  "jsonrpc": "2.0",
  "method": "<method>",
  "params": [
    "<base 64 signature, described below>",
    "<nodeID, aka hex-encoded public key>",
    <nonce integer>,
    ...<remaining positional args>
    ]
  },
  "id": 1
}

The signature takes the form of the method concatenated with a JSON-encoded list of the public identifier (such as nodeID), nonce, and any other remaining positional args. For example,

vipnode_update["1234abcd...",1533328630,["somepeer","anotherpeer"]]

Would be what is signed by the public key "1234abcd..." for the RPC call to:

Vipnode.Update(peers []string) error

All the signing and verification stand-alone code is here: https://github.com/vipnode/vipnode/blob/master/request/request.go

Each RPC method has to call the signing and verifying accordingly: https://github.com/vipnode/vipnode/blob/master/pool/service.go#L75

It's not ideal but it should be compatible with existing jsonrpc2.0 libs.

[shazow]

Another idea: Investigate using JSON-LD Signatures as a possible format in the future.

At first glance, it could be compatible with what we need. Would be a weird mashup to use JSON-LD with JSON-RPC, but all of this is somewhat weird.

(Probably too late for the current milestone of vipnode, but maybe an option to evaluate if/when we have an opportunity to redesign the wire format.)