Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

multi: add watchtower client RPC subserver #3184

Merged
merged 12 commits into from Jul 31, 2019

Conversation

@wpaulino
Copy link
Collaborator

commented Jun 11, 2019

In this PR, we introduce a new RPC subserver: WatchtowerClientRPC. So far, this RPC subserver allows us to interact with the daemon's backing watchtower client at runtime. Certain changes were needed for the wtclient.TowerClient and wtclient.DB interfaces in order to provide the functionality required by the RPC subserver.

@wpaulino wpaulino force-pushed the wpaulino:wtclient-subserver branch 2 times, most recently from 26eb7fc to 118cf23 Jun 11, 2019

@wpaulino wpaulino requested a review from cfromknecht Jun 14, 2019

@wpaulino wpaulino force-pushed the wpaulino:wtclient-subserver branch from 118cf23 to d2abc8c Jun 14, 2019

@Roasbeef Roasbeef requested a review from valentinewallace Jul 2, 2019

@wpaulino wpaulino added this to the 0.7.1 milestone Jul 2, 2019

@wpaulino wpaulino added the v0.7.1 label Jul 2, 2019

@wpaulino wpaulino force-pushed the wpaulino:wtclient-subserver branch 4 times, most recently from f0ddfb2 to 0ca7b98 Jul 4, 2019

@Roasbeef Roasbeef requested review from Roasbeef and removed request for valentinewallace Jul 9, 2019

@cfromknecht
Copy link
Collaborator

left a comment

did an initial pass, looks solid so far! tested on my node and was able to modify towers and get info about towers+sessions

watchtower/wtmock/client_db.go Outdated Show resolved Hide resolved
watchtower/wtmock/client_db.go Outdated Show resolved Hide resolved
watchtower/wtclient/client.go Outdated Show resolved Hide resolved
watchtower/wtclient/client.go Outdated Show resolved Hide resolved
watchtower/wtclient/client.go Outdated Show resolved Hide resolved
lnrpc/wtclientrpc/wtclient.proto Outdated Show resolved Hide resolved
watchtower/wtclient/client.go Outdated Show resolved Hide resolved
watchtower/wtclient/client.go Outdated Show resolved Hide resolved
watchtower/wtclient/client.go Outdated Show resolved Hide resolved

@wpaulino wpaulino force-pushed the wpaulino:wtclient-subserver branch from 0ca7b98 to e05b1dd Jul 11, 2019

@wpaulino wpaulino requested a review from cfromknecht Jul 11, 2019

@wpaulino wpaulino force-pushed the wpaulino:wtclient-subserver branch from e05b1dd to 408a96f Jul 11, 2019

lnrpc/wtclientrpc/watchtower_client.go Outdated Show resolved Hide resolved
lnrpc/wtclientrpc/wtclient.proto Outdated Show resolved Hide resolved
lnrpc/wtclientrpc/wtclient.proto Outdated Show resolved Hide resolved
lnrpc/wtclientrpc/wtclient.proto Show resolved Hide resolved
lnrpc/wtclientrpc/wtclient.proto Outdated Show resolved Hide resolved
lntest/node.go Show resolved Hide resolved
watchtower/wtclient/client.go Show resolved Hide resolved
watchtower/wtdb/client_session.go Show resolved Hide resolved

@wpaulino wpaulino force-pushed the wpaulino:wtclient-subserver branch 3 times, most recently from 8f419db to 7b01f4a Jul 11, 2019

lnrpc/wtclientrpc/watchtower_client.go Outdated Show resolved Hide resolved
cmd/lncli/wtclient_active.go Outdated Show resolved Hide resolved
cmd/lncli/wtclient_active.go Outdated Show resolved Hide resolved
cmd/lncli/wtclient_active.go Outdated Show resolved Hide resolved
cmd/lncli/wtclient_active.go Outdated Show resolved Hide resolved
lnrpc/wtclientrpc/driver.go Show resolved Hide resolved
lnrpc/wtclientrpc/watchtower_client.go Outdated Show resolved Hide resolved
watchtower/wtdb/client_db_test.go Show resolved Hide resolved

@wpaulino wpaulino force-pushed the wpaulino:wtclient-subserver branch from 7b01f4a to 9aa63c8 Jul 12, 2019

@wpaulino wpaulino requested a review from cfromknecht Jul 12, 2019

@cfromknecht
Copy link
Collaborator

left a comment

@wpaulino latest changes look solid! no major comments apart form maybe keeping the existing wtclient.private-tower-uris in the config to maintain backcompat for the point release, and removing fully in next major release

lncfg/wtclient.go Show resolved Hide resolved

@wpaulino wpaulino force-pushed the wpaulino:wtclient-subserver branch from 9aa63c8 to 917fda3 Jul 12, 2019

lncfg/wtclient.go Outdated Show resolved Hide resolved

@wpaulino wpaulino removed this from the 0.7.1 milestone Jul 23, 2019

@wpaulino wpaulino removed the v0.7.1 label Jul 23, 2019

@wpaulino wpaulino added this to the 0.8 milestone Jul 23, 2019

@wpaulino wpaulino force-pushed the wpaulino:wtclient-subserver branch from 917fda3 to 7a723e9 Jul 25, 2019

@cfromknecht
Copy link
Collaborator

left a comment

LGTM 💰

@Roasbeef Roasbeef added the v0.8.0 label Jul 30, 2019

watchtower/wtdb/client_db.go Show resolved Hide resolved
watchtower/wtdb/tower.go Show resolved Hide resolved
var towerPK towerPK
copy(towerPK[:], pubKey.SerializeCompressed())

towerID, ok := m.towerIndex[towerPK]

This comment has been minimized.

Copy link
@Roasbeef

Roasbeef Jul 30, 2019

Member

Out of the scope of this PR, so just reflecting, but it seems that we can remove much of the code in this mock tower, if we abstracted the storage backend from the actual tower DB a bit. So in this case, the business logic would be shared across the regular and the mock, with the only difference being storage in a map vs actually on disk.

Each time we modify the behavior of the actual tower DB, we must also take care to ensure that this is also updated, and implements the actual behavior perfectly (ideally). This adds an additional cost/diff to modifying this area of the codebase, as we essentially need two maintain the logic of two structs that do the same thing.

This comment has been minimized.

Copy link
@Roasbeef

Roasbeef Jul 30, 2019

Member

The mock DB had it place during initial development as we wanted to flesh out the biz logic without being burdened with actual DB details such as bucket structure, serialization, etc. However at this point, I don't see a good reason for it to exists in its current state.

This comment has been minimized.

Copy link
@wpaulino

wpaulino Jul 30, 2019

Author Collaborator

In other parts of the codebase we have mock DBs for unit tests to speed them up, but doesn't seem like there's something to gain here given that the tests still run with the actual bolt implementation.

This comment has been minimized.

Copy link
@cfromknecht

cfromknecht Jul 30, 2019

Collaborator

the db tests run both mock and boltdb implementations to assert they have identical behavior. all of the other packages use the mockdb to speed them up

var (
// macPermissions maps RPC calls to the permissions they require.
//
// TODO(wilmer): create tower macaroon?

This comment has been minimized.

Copy link
@Roasbeef

Roasbeef Jul 30, 2019

Member

This would be nice, as otherwise, the same macaroon (atm) that can add/remove towers is also able to do things like send coins on chain. On the other-hand, one could argue that being able to remove towers is possibly move dangerous than sending coins on chain...

I think we can leave this out for now, and leave the time from now until the first 0.8 release candidate to mull things over.

lnrpc/wtclientrpc/wtclient.go Show resolved Hide resolved
lnrpc/wtclientrpc/wtclient.go Show resolved Hide resolved
lnrpc/wtclientrpc/wtclient.go Show resolved Hide resolved

wpaulino added some commits Jun 8, 2019

watchtower/wtclient: remove stale sessionQueue TODO
The state updates that are pending to be acked by the tower are already
loaded within newSessionQueue.
watchtower: extend client db to filter sessions for a specific tower
This currently takes O(N) time as there does not exist an index of
active client sessions for each watchtower within the client's database.
This index is likely to be added in the future.
watchtower: extend client databse with CRUD operations for towers
These operations are currently unused, but will be integrated into the
TowerClient at a later point as future preparation for the
WatchtowerClient RPC subserver, which will allow users to add, remove,
and list the watchtowers currntly in use.
watchtower/wtclient: extend TowerClient with CRUD operations for towers
In this commit, we extend the wtclient.Client interface with the
following methods:

  * AddTower
  * RemoveTower
  * RegisteredTowers
  * LookupTower
  * Stats

Care has been taken to ensure that any in-memory state updates are
_only_ performed after a successful database update.

These methods are currently unused, but they serve as a dependency for
the upcoming WatchtowerClient RPC subserver.
watchtower: load persisted towers upon TowerClient creation
We do this as a convenience for WatchtowerClient users so that they do
not need to re-add towers upon restarts. We ensure not to re-add towers
that have been previously removed by determining whether it has any
lingering active sessions.
multi: only allow specifying towers to TowerClient through RPC
With the introduction of the WatchtowerClient RPC subserver, the lnd
configuration flag to specify private watchtowers for the client is no
longer needed and can lead to confusion upon users. Therefore, we remove
the flag completely, and only rely on the watchtower client being active
through a new --wtclient.active flag.
watchtower/wtclient: only reset iterator once we've exhausted all can…
…didates

Doing so allows us to load balance sessions better amongst all of the
tower candidates.

@wpaulino wpaulino force-pushed the wpaulino:wtclient-subserver branch from 7a723e9 to 0690c8f Jul 30, 2019

@wpaulino wpaulino requested a review from Roasbeef Jul 30, 2019

@Roasbeef
Copy link
Member

left a comment

LGTM 🌊

Gave this a spin on my testnet nodes, was able to modify the set of towers it was connected to w/o any issue!

@Roasbeef Roasbeef merged commit 8c9c4b5 into lightningnetwork:master Jul 31, 2019

1 of 2 checks passed

coverage/coveralls Coverage decreased (-0.02%) to 60.714%
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details

@wpaulino wpaulino deleted the wpaulino:wtclient-subserver branch Jul 31, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
3 participants
You can’t perform that action at this time.