lnrpc+lnd+rpc: add new Signer RPC service, and sub RPC server infrastructure #2081

In this commit, we introduce a new sub-package within the greater RPC package. This new sub-package will house a new set of sub-RPC servers to expose experimental features behind build flags for upstream consumers. In this commit, we add the first config for the service, which will simply expose the lnwallet.Signer interface over RPC. In the default file, we have what the config will be if the build tag (signerrpc) is off. In this case, the config parser won't detect any times, and if specified will error out. In the active file, we have the true config that the server will use. With this new set up, we'll exploit these build flags heavily in order to create a generalized framework for adding additional sub RPC servers.

In this commit, we add a new proto generation script to match the one in the main lnrpc package. This script differs, as we don't need to generate the REST proxy stuff (for now).

In this commit, we add the scafolding for the future sub-server RPC system. The idea is that each sub server will implement this particular interface. From there on, a "root" RPC server is able to query this registry, and dynamically create each sub-sever instance without knowing the details of each sub-server. In the init() method of the pacakge of a sub-server, the sub-server is to call: RegisterSubServer to claim its namespace. Afterwards, the root RPC server can use the RegisteredSubServers() method to obtain a slice of ALL regsitered sub-servers. Once this list is obtained, it can use the New() method of the SubServerDriver struct to create a new sub-server instance. Each sub-server needs to be able to locate it's primary config using the SubServerConfigDispatcher interface. This can be a map of maps, or a regular config structr. The main requirement is that the sub-server be able to find a config under the same name that it registered with. This string of abstractions will allow the main RPC server to find, create, and run each sub-server without knowing the details of its configuration or its role.

In this commit, we add a full implementation of the new SignerServer sub RPC service within the main root RPC service. This service is able to fully manage its macaroons, and service any connected clients. Atm, this service only has a single method: SignOutputRaw which mimics the existing lnwallet.Signer interface within lnd itself. As the API's are so similar, it will be possible for a client to directly use the lnwallet.Signer interface, and have a proxy that sends the request over RPC, and translates the proto layer on both sides. To the client, it doesn't know that it's using a remote, or local RPC.

In this commit, we create a lnrpc.SubServerDriver for signrpc. Note that this file will only have its init() method executed if the proper build flag is on. As a result, only if the build flag is set, will the RPC server be registered, and visible at the packge lnrpc level for the root server to manipulate.

…ework In this commit, we add the glue infrastructure to make the sub RPC server system work properly. Our high level goal is the following: using only the lnrpc package (with no visibility into the sub RPC servers), the RPC server is able to find, create, run, and manage the entire set of present and future sub RPC servers. In order to achieve this, we use the reflect package and build tags heavily to permit a loosely coupled configuration parsing system for the sub RPC servers. We start with a new `subRpcServerConfigs` struct which is _always_ present. This struct has its own group, and will house a series of sub-configs, one for each sub RPC server. Each sub-config is actually gated behind a build flag, and can be used to allow users on the command line or in the config to specify arguments related to the sub-server. If the config isn't present, then we don't attempt to parse it at all, if it is, then that means the RPC server has been registered, and we should parse the contents of its config. The `subRpcServerConfigs` struct has two main methods: `PopulateDependancies` and `FetchConfig`. The `PopulateDependancies` is used to dynamically locate and set the config fields for each new sub-server. As the config may not actually have any fields (if the build flag is off), we use the reflect pacakge to determine if things are compiled in or not, then if so, we dynamically set each of the config parameters. The `PopulateDependancies` method implements the `lnrpc.SubServerConfigDispatcher` interface. Our goal is to allow sub servers to look up their actual config in this main config struct. We achieve this by using reflect to look up the target field _as if it were a key in a map_. If the field is found, then we check if it has any actual attributes (it won't if the build flag is off), if it is, then we return it as we expect it to be populated already.

… sub-servers In this commit, we modify the existing rpcServer to fully manage the macaroons, gRPC server, and also seek out and create all sub-servers. With this change, the RPC server gains more responsibility, as it becomes the "root" server in the hierarchy of gRPC sub-servers. In addition to creating each sub-server, it will also merge the set of macaroon permissions for each sub-server, with the permissions of the rest of the RPC infra. As a result, each sub-server is able to independently specify what it needs w.r.t macaroon permissions and have that taken care of by the RPC server. In order to achieve this, we need to unify the creation of the RPC interceptors, and also fully manage the gRPC server ourselves. Some examples with various build configs: ``` ⛰i make build Building debug lnd and lncli. go build -v -tags="dev" -o lnd-debug -ldflags "-X github.com/lightningnetwork/lnd/build.Commit=v0.5-beta-143-gb2069914c4b76109b7c59320dc48f8a5f30deb75-dirty" github.com/lightningnetwork/lnd go build -v -tags="dev" -o lncli-debug -ldflags "-X github.com/lightningnetwork/lnd/build.Commit=v0.5-beta-143-gb2069914c4b76109b7c59320dc48f8a5f30deb75-dirty" github.com/lightningnetwork/lnd/cmd/lncli ⛰i ./lnd-debug --debuglevel=debug --signrpc.signermacaroonpath=~/sign.macaroon unknown flag `signrpc.signermacaroonpath' unknown flag `signrpc.signermacaroonpath' ⛰i make build tags=signerrpc Building debug lnd and lncli. go build -v -tags="dev signerrpc" -o lnd-debug -ldflags "-X github.com/lightningnetwork/lnd/build.Commit=v0.5-beta-143-gb2069914c4b76109b7c59320dc48f8a5f30deb75-dirty" github.com/lightningnetwork/lnd go build -v -tags="dev signerrpc" -o lncli-debug -ldflags "-X github.com/lightningnetwork/lnd/build.Commit=v0.5-beta-143-gb2069914c4b76109b7c59320dc48f8a5f30deb75-dirty" github.com/lightningnetwork/lnd/cmd/lncli ⛰i ./lnd-debug --debuglevel=debug --signrpc.signermacaroonpath=~/sign.macaroon 2018-10-22 17:31:01.132 [INF] LTND: Version: 0.5.0-beta commit=v0.5-beta-143-gb2069914c4b76109b7c59320dc48f8a5f30deb75-dirty, build=development, logging=default 2018-10-22 17:31:01.133 [INF] LTND: Active chain: Bitcoin (network=simnet) 2018-10-22 17:31:01.140 [INF] CHDB: Checking for schema update: latest_version=6, db_version=6 2018-10-22 17:31:01.236 [INF] LTND: Primary chain is set to: bitcoin 2018-10-22 17:31:02.391 [INF] LNWL: Opened wallet 2018-10-22 17:31:03.315 [INF] LNWL: The wallet has been unlocked without a time limit 2018-10-22 17:31:03.315 [INF] LTND: LightningWallet opened 2018-10-22 17:31:03.319 [INF] LNWL: Catching up block hashes to height 3060, this will take a while... 2018-10-22 17:31:03.320 [INF] HSWC: Restoring in-memory circuit state from disk 2018-10-22 17:31:03.320 [INF] LNWL: Done catching up block hashes 2018-10-22 17:31:03.320 [INF] HSWC: Payment circuits loaded: num_pending=0, num_open=0 2018-10-22 17:31:03.322 [DBG] LTND: Populating dependencies for sub RPC server: Signrpc ``` As for the config, an example is: ``` [signrpc] signrpc.signermacaroonpath=~/signer.macaroon ```

In this commit, we add a recursive proto generation script. This avoids having to add a new script for each upcoming sub-server.

In this commit, we add the ComputeInputScript which will allow callers to obtain witnesses for all outputs under control of the wallet. This allows external scripting of things like coin join, etc.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

lnrpc+lnd+rpc: add new Signer RPC service, and sub RPC server infrastructure #2081

lnrpc+lnd+rpc: add new Signer RPC service, and sub RPC server infrastructure #2081

Commits on Nov 29, 2018