RPC API w/ Permissions

[amerameen]

ACs

• We are able to build a new program that imports the interface and is able to make calls to the RPC node
• RPC server & client make use of this interface

Notes

• Would be helpful to have name of method, the serializer for request, deserializer for response, request handler in one place
• Currently we build the server from a hardcoded builder command
• A crate for interacting with Filecoin (i.e. Forest & Lotus RPC servers)
• This would include a refactor of RPC methods in RPC router
• This would be the first step in testing our RPC programatically
• This would be like a web3.js in Rust for Filecoin

[cryptoquick]

After looking at Lotus, I think this feature will be essential for authorization and permissioning. It also shouldn't take me too long, and should unstuck me to better implement JWT authentication #997.

My approach I'm thinking is going to be to loosely replicate the data structure in this file:

https://github.com/filecoin-project/lotus/blob/b54b1303f9f6bcc3b57ae25cfc1174b59f44b943/api/apistruct/struct.go

It would be in two parts:

/// permissions.rs
pub enum Common {
    AuthNew {
        name: 'Filecoin.AuthNew',
        auth: Authorization::Admin,
        input: auth_api::auth_new::<DB, KS, B>,
        output: Result<String, jsonrpc_v2::Error>,
    }
}

And then I'll have a static const HashMap using https://github.com/matklad/once_cell where I can do iteration and lookup with the key as the method name string.

@creativcoder and I discussed options such as lazy_static and phf, but we wound up on once_cell so we can avoid the use of macros. This will create the static const HashMap between method name and its corresponding enum member at compile-time.

Thoughts?

[ec2]

Interesting. If we do method routing using the static HashMap, then we won't need to use jsonrpc_v2, if I'm understanding this correctly. Is that right?

[cryptoquick]

We will, I'll just build the router at runtime by iterating over the correct hashmap(s). Also, lotus can be configured to provide a few different APIs; for mining, storage, fullnode, etc.

[cryptoquick]

So, I've been working on this for about a week now (excluding my work on #1033), and I'm gonna give you guys an opportunity to call this a "research spike" and have me work on other things. However, if you do give me the opportunity to solve it, I think I have the knowledge needed to do so. I'll provide some of that knowledge here, to report on requirements and provide recommendations of priority.

So, first some background.

I looked into OpenRPC (such as via https://github.com/etclabscore/ethereum-json-rpc-specification), and I find it to be of limited utility. The API definition is in JSON. It'd be one thing if there were existing OpenRPC definitions, but that's unfortunately not the case.

Further, authorization (access levels) is not something OpenRPC has properties available for us to describe, which is the major "win" I wanted from this. Ideally, we'd have a builder pattern akin to the one here:

https://github.com/ChainSafe/forest/blob/d245f14cbcb0f7c0848fbddbdf802931b83b35e8/node/rpc/src/lib.rs#L50~L204

But it would also allow us to describe endpoints like this:

pub mod auth_new {
    pub const METHOD: &str = "Filecoin.AuthNew";
    pub type Params = (Vec<String>,);
    pub type Result = Vec<u8>;
}

These types and constants can then be referenced in handlers and router builders.

I also wanted to describe access in an enum like this:

/// Access levels to be checked against JWT claims
pub enum Access {
    Admin,
    Sign,
    Write,
    Read,
}

That way it could be easily matched against validated JWT claims.

I could do that by building an access table like this:

/// Access mapping between method names and access levels
/// Checked against JWT claims on every request
pub static ACCESS: Lazy<HashMap<&str, Access>> = Lazy::new(|| {
    let mut access = HashMap::new();

    access.insert(auth_new::METHOD, Access::Admin);
    access.insert(auth_verify::METHOD, Access::Read);

    access
});

And this would be done referencing the following Lotus code:

https://github.com/filecoin-project/lotus/blob/b54b1303f9f6bcc3b57ae25cfc1174b59f44b943/api/apistruct/struct.go#L50~L776

This approach would work fine for the jsonrpc-v2 backend, considering its implementation of the builder pattern for its router as linked above.

I also investigated Parity's work on jsonrpsee, and while promising, I find its practical utility limited by its use of procedural macros.

In fact, macros have been a major headache with regards to making a useful abstraction. The reason being, variables can't seem to be passed to jsonrpsee's client, and several other Rust JSON-RPC clients also employ this approach to describe the client API.

Ideally there would be a client builder pattern, similar to jsonrpc-v2's approach. I haven't been able to find anything existing that provides that, however.

I'd estimate it'd take me a month to drive this home, but I'd recommend sandbagging that estimate 2x, like the refactor of the HTTP server, so more like 2 months. I'm just being overly cautious, which I think is important to communicate while undertaking these more open-ended refactors.

I think it'd be very worthwhile to solve this, but only if we do it the right way. That said, I think we'd get more bang for the buck if we focused on the actual CLI client implementation using the existing 0.1.0 version of the jsonrpsee client. Work there will have limited reusability due to its use of macros, but I think the Forest client's lack of certain useful commands that the lotus client has reduces its utility much more, and thus, should be of higher priority to address.