rpc: Add GetRawBlocks and GetCFilters gRPC methods

[matheusd]

This adds the GetCFilters method to the WalletService gRPC service. This
allows clients to fetch the wallet-stored and header-validated version 2
committed filters so that non-wallet-related transactions can be
searched for.

It also adds a new service (NetworkService) with a GetRawBlock call. The
service is intended to be used for calls which fetch or push data that
might not be backed by the wallet into the P2P network.

This particular method is useful for external callers to fetch a full
block after they have determined it (probably) contains transactions
they are interested in by checking the respective cfilter.

A previous commit also fixes an issue in regenerating the api bindings from the api.proto file.

[jrick]

This may cause some problems in SPV mode if it is called concurrently with a request for the same block from the SPV syncer. These concurrent requests are disallowed by the p2p package (a restriction that allows for the more synchronous API).

[matheusd]

hmm. I could add a mutex to RemotePeer to protect concurrent invocations to Blocks()/Block(). Though it's going to be tricky to handle the cancelling of the ctx (the lock might be acquired only after the peer has already beed disconnected due to errors and the caller's ctx may have already been canceled so I'd need to handle all that)

[matheusd]

Added mutex in 57dd618

[matheusd]

I have another issue to handle as well (either on this or on a separate PR):

The wallet won't relay non-wallet-relevant txs due to not storing them. Any suggestions on how to handle this?

My initial ideas as alternatives:

• Add a sort of mempool to Wallet{} that would be checked whenever the spv syncer's receiveGetData is signalled to relay a new tx. Handling removing the txs on a given schedule would be tricky.

• Add callbacks on the wallet to create a client backed mempool. When querying for a tx, query via these callbacks. Might also be tricky, have possible recursion issues.

• Add a new method to Network implementations: ForcePublishTx. On RPC this maps to the regular PublishTransaction but on SPV this would map to something that publishes and waits for a corresponding GetData call to happen before returning.

[jrick]

Another possibility would be to send the tx out immediately instead of sending the inv first and waiting for a getdata. I don't like allowing this though...

If we add a temporary "mempool" for publishing recently inv'd transactions that aren't saved by the wallet, removing them after some duration, I think it's more appropriate to add that functionality to the SPV syncer instead of the wallet, since it really is the brains behind how the entire application interacts with the network and what messages are sent (vs p2p which is dumber and is mostly for directly sending and receiving those messages, and vs wallet which we don't want to taint with unrelated transactions).

[matheusd]

I think it's more appropriate to add that functionality to the SPV syncer

I considered that. The only tricky bit is determining when to store in the syncer's mempool or not. I'd have to either pass another argument to PublishTransactions or determine (during PublishTransactions) whether that tx is in the wallet or not to store it (but that would involve doing another db op which increases latency when sending out votes).

I'll poc using an additional argument and see if that's acceptable.

[matheusd]

Added the mempool to SPV syncer in 9527f8c

I guessed an eviction timeout of 60 minutes, can adjust as needed.

I see there's a conflict with current master, I'll rebase and fix that after you've had a chance to take a look at the current solution.

[jrick]

I think we should be able to avoid the signature change on PublishTransactions. The SPV syncer can call the wallet to determine if a transaction is relevant or not to determine whether to add it to the wallet, or manage it itself in memory.

Currently the wallet has an unexported isRelevantTx method for this, but we could provide an exported version.

[matheusd]

Changed last commit from changing the PublishTransactions signature to using DetermineRelevantTxs (so that only a single dbtx is done for the whole set of txs).

[jrick]

reads well and I don't anticipate any issues anymore after the p2p changes to allow concurrent requests of the same data.