[watchtower/lookout]: on-chain breach monitoring #2124

cfromknecht · 2018-10-30T09:38:58Z

This PR introduces the watchtower/lookout package, which handles the responsibility of monitoring the chain for possible breaches and responding by decrypting and broadcasting any justice transactions that its clients had previously uploaded. Together with the watchtower/server, which receives and stores encrypted blobs from clients, this represents that second primary service enabling the tower to act on behalf of the tower's clients.

At a high level, the lookout service receives input via two sources: block events and it's database. Incoming state updates are continually written to the database, and are made available to the lookout as soon as the tower successfully persists the encrypted blob. As new blocks come in, the tower searches for any breach hints matching the txid prefixes contained in the newly found block. If any matches are generated from the query, the lookout service will dispatch an attempt to decrypt and sweep the transaction on behalf of the user.

Some slight modifications have been made to the watchtower/blob package, most notably:

introducing support for padded sweep adresses
properly generating DER-encoded signatures for use in breach input witness stacks

Some open questions:

What is the ideal reward function for the tower? Currently only a proportional cut is taken, though perhaps a base + proportional is more suitable. As such, some edge cases around how to handle dust outputs are not accounted for in this PR. The intention is to revist these edge cases after some feedback and when finishing up full persistence in the watchtower/wtdb package.
Should we apply any sort of input/output sorting (a la BIP69) to the justice transactions? It is certainly possible, though would be interested in hearing thoughts on whether it's necessary considering the justice transactions are somewhat identifiable anyway.

Builds on #2122

This commit fixes an issue with the witness stack construction for to-local and to-remote inputs, that would cause the justice kit to return signatures as fixed-size, 64-byte signatures. The correct behavior is to return DER-encoded signatures so that they will properly verify on the network, since the consensus rules won't be able to understand the fixed-size variant.

Roasbeef

One small step for lnd, one giant leap for the Lightning Network! ⚡️🚀

I've completed an initial pass, will likely do another to cover the set of unit tests added. The only major comment concerns the nonce generation. I'm a bit weary of using a sequence based nonce, as it puts a lot of responsibility on the client to ensure it doesn't send with a duplicate nonce lest it leaks plaintext blobs.

Roasbeef · 2018-11-01T00:39:18Z

watchtower/wtdb/session_id.go

API design level comment: blob isn't the most descriptive name for a package. It's possible to ensure isolation at the unit test level, yet still intermingle components within a unified package.

not sure what you mean by the last comment, can you explain?

Roasbeef · 2018-11-01T00:41:57Z

watchtower/wtdb/session_id.go

What measures will we take on the client side in order to ensure this never happens? It seems certain fault cases on the client side could possibly cause it to re-encrypt with the same nonce sequence.

As an alternative, we can use randomized 192-bit (24 byte) nonce. I'm planning to go this route with the encrypted static backups. The chacha API in crypto/x also exposes an variant with larger nonces: https://godoc.org/golang.org/x/crypto/chacha20poly1305#NewX

Nice! Wasn't aware that the 192-bit variant was exposed, I agree the randomization is the safer approach.

with the existing nonce generation, the protections would be to use truly ephemeral keys (instead of HD keys) and to persist an encrypted update under for the session/seqnum so it can be resent on startup. arguably the first should still be done with a randomized nonce, though it does simplify the latter at the expense of tower storage

Roasbeef · 2018-11-01T00:54:16Z

watchtower/lookout/interface.go

Method name doesn't mach the godoc comment.

Roasbeef · 2018-11-01T00:57:11Z

watchtower/lookout/interface.go

Why not give it the entire block? As is, the block epochs just contain height+hash. If we give this method the entire block, then we don't require it to do any network I/O, and also it can be extended to batch search blocks.

On the second parameter: doesn't the database know the entire set of breach hints? So we can simplify to just take a set of blocks?

To be correct, a breach hint should be provided for each transaction included at the provided block epoch.
So we call this once we already know that the set of transactions has a breach within the block?

This just a database query, there's no network I/O here. The breach hints passed in are constructed from the block, though they could be constructed from multiple blocks.

The block epoch is passed just so that the database can record the last processed block height, so after startup it knows where to begin searching. i'm thinking about just making this a separate db call though.

Roasbeef · 2018-11-01T00:58:50Z

watchtower/lookout/interface.go

Why not just use something like:

Suggested change

// EpochSource delivers an in-order stream of blocks as they are seen on the

type EpochSource interface {

RegisterBlockEpochNtfn(*BlockEpoch) (*BlockEpochEvent, error)

Start() error

Stop() error

}

So a simplified interface that includes only the methods of the greater ChainNotifier interface that we actually care about.

not sure i follow, are you suggesting we'd pass in the notifier? at this point, it will have already been started and it's lifecycle would be tied to the watchtower

i removed the epoch stream entirely, so that block fetching happens w/in the lookout. i think the testabilty is roughly equivalent, but spares a lot of code duplication!

Roasbeef · 2018-11-01T01:12:05Z

watchtower/lookout/punisher.go

Spend tracking?

just a reminder to add persistent tracking of published justice txns and then monitor for spends to see if ours confirms or not. I've updated the comment to be more precise

Roasbeef · 2018-11-01T01:12:40Z

watchtower/lookout/punisher.go

Seems important? Otherwise, it'll keep attempting to re-broadcast on restart?

we won't rebroadcast at the moment, but these should be implemented together in a following PR

Roasbeef · 2018-11-01T01:14:45Z

watchtower/lookout/lookout.go

So it'll rescan all blocks since the last time a breached happend? Seems a bit wasteful, and instead we can rescan blocks based on the session height of a client to ensure that we don't miss any breaches that might've happend while they were down, just in case they missed a blob send.

it'll rescan since the last epoch that caused a db query

Roasbeef · 2018-11-01T01:15:05Z

watchtower/lookout/lookout.go

Similar Q as earlier: we shouldn't need to?

Roasbeef · 2018-11-01T01:17:30Z

watchtower/lookout/lookout.go

Why vars? (here and below)

This commit modifies the blob encryption scheme to use chacha20-poly1305 with a randomized 192-bit nonce. The previous approach used a deterministic nonce scheme, which is being replaced to simplify the requirements of a correct implementation. As a result, each payload gains an addtional 24-bytes prepended to the ciphertext.

The nonce is now passed in as the prefix to the ciphertext, and is generated randomly in calls to Encrypt.

Roasbeef

LGTM 🧬

cfromknecht force-pushed the wtlookout branch from dc02c8e to b701709 Compare October 31, 2018 03:31

Roasbeef added enhancement Improvements to existing features / behaviour safety General label for issues/PRs related to the safety of using the software security General label for issues/PRs related to the security of the software watchtower P2 should be fixed if one has time labels Oct 31, 2018

cfromknecht force-pushed the wtlookout branch from b701709 to d9dc2b9 Compare October 31, 2018 04:01

cfromknecht force-pushed the wtlookout branch from d9dc2b9 to 4e27940 Compare November 1, 2018 00:39

Roasbeef requested changes Nov 1, 2018

View reviewed changes

cfromknecht force-pushed the wtlookout branch from 4e27940 to bb8cff5 Compare November 1, 2018 03:37

cfromknecht added 12 commits October 31, 2018 20:42

watchtower/blob/justice_kit_test: remove external nonce

740e8fd

The nonce is now passed in as the prefix to the ciphertext, and is generated randomly in calls to Encrypt.

watchtower/wtdb/session_info: compute rewards outputs

4e6dc38

watchtower/wtdb/mock: adds lookout-related mock functions

b79bab0

watchtower/lookout/log: adds lookout subsystem logger

1d7e382

watchtower/lookout/interface: adds primary lookout ifaces

bd5ed47

watchtower/lookout/justice_descriptor: adds justice txn creation

0632520

watchtower/lookout/punisher: adds Punisher craft+bcast justice txn

3ab34f8

watchtower/lookout/lookout: adds Lookout

475ab01

watchtower/lookout/mock: adds mock backend

671be69

watchtower/lookout/lookout_test: adds simple lookout tests

fa63ba2

watchtower/lookout/justice_descriptor_test: add create txn test

8bc8964

cfromknecht force-pushed the wtlookout branch from bb8cff5 to 8bc8964 Compare November 1, 2018 03:43

Roasbeef approved these changes Nov 14, 2018

View reviewed changes

Roasbeef merged commit 2f0bc5c into lightningnetwork:master Nov 14, 2018

-// EpochSource delivers an in-order stream of blocks as they are seen on the
+type EpochSource interface {
+      RegisterBlockEpochNtfn(*BlockEpoch) (*BlockEpochEvent, error)
+      Start() error
+     Stop() error
+}

[watchtower/lookout]: on-chain breach monitoring #2124

[watchtower/lookout]: on-chain breach monitoring #2124

Uh oh!

Conversation

cfromknecht commented Oct 30, 2018

Uh oh!

Roasbeef left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Roasbeef left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!