Remove Channel's ChannelMonitor copy #667

valentinewallace · 2020-08-10T21:35:34Z

In service to the larger refactor of removing the Channel's reference
to its ChannelMonitor.

This also allows us to remove the Channel's channel_monitor() function. Next we can remove all of the Channel's logic for keeping its channel monitor up to date, as well as the update_monitor_ooo logic in ChannelMonitor.

~~Partially resolves~~ Closes #657

codecov · 2020-08-10T22:10:09Z

Codecov Report

Merging #667 into master will increase coverage by 0.10%.
The diff coverage is 92.47%.

@@            Coverage Diff             @@
##           master     #667      +/-   ##
==========================================
+ Coverage   91.36%   91.46%   +0.10%     
==========================================
  Files          35       35              
  Lines       21703    21888     +185     
==========================================
+ Hits        19828    20019     +191     
+ Misses       1875     1869       -6

Impacted Files	Coverage Δ
lightning/src/ln/channel.rs	`87.17% <ø> (-0.04%)`	⬇️
lightning/src/ln/reorg_tests.rs	`98.94% <ø> (ø)`
lightning/src/ln/functional_tests.rs	`97.15% <88.23%> (+0.14%)`	⬆️
lightning/src/ln/channelmonitor.rs	`95.56% <90.24%> (-0.15%)`	⬇️
lightning/src/ln/channelmanager.rs	`85.26% <96.96%> (+<0.01%)`	⬆️
lightning/src/util/test_utils.rs	`86.13% <100.00%> (ø)`
lightning/src/ln/onchaintx.rs	`93.75% <0.00%> (-0.20%)`	⬇️
lightning/src/routing/router.rs	`97.39% <0.00%> (+0.80%)`	⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update b3b4f43...28d9036. Read the comment docs.

valentinewallace · 2020-08-12T19:55:48Z

Oops, broke build -- will fix in a bit

valentinewallace · 2020-08-13T18:09:44Z

I looked into extracting some common HTLC selection functionality between monitor_would_broadcast and build_commitment_transaction but they're pretty different, they want different HTLCs, and monitor_would_broadcast needs local and remote HTLCs, and idk. Let me know if I'm missing something and it's definitely a good idea.

valentinewallace · 2020-08-13T19:22:41Z

Hmm, fuzzing seems slow still. Looking into it.

lightning/src/ln/channelmanager.rs

lightning/src/ln/channel.rs

arik-so · 2020-08-18T17:46:17Z

lightning/src/ln/channel.rs

@@ -908,7 +969,7 @@ impl<ChanSigner: ChannelKeys> Channel<ChanSigner> {
 				InboundHTLCState::AwaitingRemoteRevokeToAnnounce(_) => (!generated_by_local, "AwaitingRemoteRevokeToAnnounce"),


not so much related to this diff in particular, but is the string literal the most elegant way to get the state name?

Open to other options 😄 (for a future PR)

lightning/src/ln/channelmonitor.rs

lightning/src/ln/channel.rs

lightning/src/ln/channelmonitor.rs

ariard

I'm all in to remove Channel's ChannelMonitor copy but I think there is a easier way than dc9ef2c. If I understand new would_broadcast_at_height_functionality well, its purpose is only to verify that ChannelMonitor accomplish its job well and broadcast channel update accordingly.

I'm not sure if it's a great approach, in the future if we add new conditions in ChannelMonitor to force-close channel we would need to access them too. Like in case of mempools-congestion, preemptively close channel at risk.

Another way to solve this would be a) pass a reference to ChainWatchInterface to ChannelManager b) at funding_tx detection, return its output to watch c) monitor commitment transaction broadcast. Parsing positively commitment transaction is easy to do due to transaction pattern (one-input), LN commitment number watermark and overall funding outpoint spending. It's more robust also as we shouldn't assume that our ChannelManager and ChannelMonitor are running on the same block provider and thus may have block view latency.

I know that's already the current model to query ChannelMonitor to decide if we should shutdown and broadcast channel update, but I think it's fine to wait a bit for channel update broadcast. Shutdown/broadcast will happen either sooner after any attempt to unsuccessfully update monitors. Or latter after block processing.

If you still want to be quick in term of updating global network view of our channel maybe we can ChannelMonitor signal broadcast with a boolean and us periodically querying it like we're doing for get_and_clear_pending_htlcs_updated ?

Overall, I may miss some onchain/offchain coupling assumptions but it feels like we can short-cut some complexity ?

lightning/src/ln/channel.rs

TheBlueMatt · 2020-08-21T01:57:54Z

Ooooo, right, so I think that's true - would_broadcast_at_height is only used to close a channel and we could happily have the ChannelMonitor handle that (though currently Channel[Manager] handles that fully). My knee-jerk reaction was that there would be a race condition where we'd continue updating a channel state after its been closed (and the latest local commitment broadcast), but I think we could not even bother notifying the ChannelManager that we're closing the channel and wait for provide_latest_local_commitment_tx_info to return an Err because local_tx_signed is set to close the channel, so there's no race.

lightning/src/ln/channelmonitor.rs

TheBlueMatt

I had to admit I'm somewhat surprised that's the only test changes you need, but looks good.

lightning/src/ln/channelmanager.rs

lightning/src/ln/channelmonitor.rs

lightning/src/ln/channelmanager.rs

lightning/src/ln/channelmonitor.rs

TheBlueMatt

Aside from the one clarifying change I suggested and the two comment nits, looks good!

arik-so · 2020-08-25T06:25:51Z

lightning/src/ln/channelmonitor.rs

@@ -148,6 +148,16 @@ pub enum ChannelMonitorUpdateErr {
 #[derive(Debug)]
 pub struct MonitorUpdateError(pub &'static str);

+/// An event to be processed by the ChannelManager.
+#[derive(PartialEq)]
+pub enum MonitorEvent {


Out of curiosity, how do we decide which events are monitor events? For example, why is HTLCEvent for claiming one, but forwarding offered HTLCs isn't?

In this context, its basically "a thing which we detected on-chain which has an impact on our understanding of a channel's state" (ie, basically, "channel has been closed on chain" or "transaction on-chain resolved an HTLC which was outstanding when we went on chain").

lightning/src/ln/onchaintx.rs

lightning/src/ln/channelmonitor.rs

arik-so · 2020-08-25T06:39:08Z

lightning/src/ln/channelmonitor.rs

+				0 => MonitorEvent::HTLCEvent(Readable::read(reader)?),
+				1 => MonitorEvent::CommitmentTxBroadcasted(funding_info.0),
+				_ => return Err(DecodeError::InvalidValue)
+			};


One day we'll have versioning for reads lol :)

To do this, we replace get_and_clear_pending_htlcs_updated with get_and_clear_pending_monitor_events, and which still transmits HTLCUpdates as before, but now also transmits a new MonitorEvent::CommitmentTxBroadcasted event when a channel's commitment transaction is broadcasted.

ariard · 2020-08-26T20:29:59Z

lightning/src/ln/channelmonitor.rs

 			if let Some(commitment_tx) = self.onchain_tx_handler.get_fully_signed_local_tx(&self.funding_redeemscript) {
+				self.local_tx_signed = true;


@valentinewallace

I think this is breaking one of the property of our distributed watchtower infrastructure as documented here : https://github.com/rust-bitcoin/rust-lightning/blob/3defcc896266f3d67848ce28981a756e971a3f0c/lightning/src/ln/channelmonitor.rs#L1174

AFAICT, by adding logs flag is true beyond updating local view with latest local state.

I'll fix it and add coverage for this.

Logs flag? Ok I think I see what you mean though, thanks for catching this!

We're discussing on IRC still, but I don't think this change is correct - we should refuse to accept updates after we've signed a local tx irrespective of why we signed it.

Sorry, after discussion on IRC and testing I realized that the alleged property was only supported in my mind. In fact #667 makes it easy to actually implement it, so did so in #679 with test coverage.

valentinewallace force-pushed the remove-channels-chanmon branch from 26bf274 to 6b693e6 Compare August 12, 2020 19:46

valentinewallace force-pushed the remove-channels-chanmon branch from 613ff14 to a7f185f Compare August 13, 2020 18:05

valentinewallace changed the title ~~Add would_broadcast_at_height functionality to Channel~~ Remove Channel's ChannelMonitor copy Aug 13, 2020

valentinewallace force-pushed the remove-channels-chanmon branch from ad02c3d to a97ec8c Compare August 17, 2020 20:13

TheBlueMatt reviewed Aug 18, 2020

View reviewed changes

lightning/src/ln/channelmanager.rs Outdated Show resolved Hide resolved

lightning/src/ln/channel.rs Outdated Show resolved Hide resolved

lightning/src/ln/channel.rs Outdated Show resolved Hide resolved

lightning/src/ln/channel.rs Outdated Show resolved Hide resolved

valentinewallace mentioned this pull request Aug 19, 2020

Rust release support #471

Closed

valentinewallace force-pushed the remove-channels-chanmon branch from a97ec8c to d0056d2 Compare August 19, 2020 18:07

valentinewallace mentioned this pull request Aug 19, 2020

Bump MSRV to 1.30.0 #671

Merged

valentinewallace force-pushed the remove-channels-chanmon branch 2 times, most recently from 684dba8 to 70c58d9 Compare August 19, 2020 21:01

arik-so reviewed Aug 20, 2020

View reviewed changes

valentinewallace force-pushed the remove-channels-chanmon branch from 70c58d9 to 528cea5 Compare August 20, 2020 17:45

ariard reviewed Aug 21, 2020

View reviewed changes

lightning/src/ln/channel.rs Outdated Show resolved Hide resolved

valentinewallace force-pushed the remove-channels-chanmon branch 2 times, most recently from 905dbe2 to df55910 Compare August 23, 2020 02:35

TheBlueMatt reviewed Aug 23, 2020

View reviewed changes

lightning/src/ln/channelmonitor.rs Outdated Show resolved Hide resolved

valentinewallace force-pushed the remove-channels-chanmon branch from df55910 to 76fe424 Compare August 24, 2020 21:51

TheBlueMatt reviewed Aug 24, 2020

View reviewed changes

lightning/src/ln/channelmanager.rs Outdated Show resolved Hide resolved

lightning/src/ln/channelmonitor.rs Outdated Show resolved Hide resolved

valentinewallace force-pushed the remove-channels-chanmon branch 2 times, most recently from 2b86ba7 to 8b5502b Compare August 24, 2020 23:05

TheBlueMatt reviewed Aug 24, 2020

View reviewed changes

lightning/src/ln/channelmanager.rs Outdated Show resolved Hide resolved

TheBlueMatt reviewed Aug 24, 2020

View reviewed changes

lightning/src/ln/channelmanager.rs Outdated Show resolved Hide resolved

TheBlueMatt reviewed Aug 25, 2020

View reviewed changes

lightning/src/ln/channelmonitor.rs Outdated Show resolved Hide resolved

TheBlueMatt reviewed Aug 25, 2020

View reviewed changes

arik-so reviewed Aug 25, 2020

View reviewed changes

valentinewallace added 2 commits August 25, 2020 16:37

Remove Channel's ChannelMonitor copy

28d9036

valentinewallace force-pushed the remove-channels-chanmon branch from 84ea158 to 28d9036 Compare August 25, 2020 20:37

TheBlueMatt approved these changes Aug 25, 2020

View reviewed changes

TheBlueMatt merged commit 501974d into lightningdevkit:master Aug 25, 2020

ariard reviewed Aug 26, 2020

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Remove Channel's ChannelMonitor copy #667

Remove Channel's ChannelMonitor copy #667

valentinewallace commented Aug 10, 2020 •

edited

Loading

codecov bot commented Aug 10, 2020 •

edited

Loading

valentinewallace commented Aug 12, 2020

valentinewallace commented Aug 13, 2020

valentinewallace commented Aug 13, 2020

arik-so Aug 18, 2020

valentinewallace Aug 25, 2020 •

edited

Loading

ariard left a comment

TheBlueMatt commented Aug 21, 2020

TheBlueMatt left a comment

TheBlueMatt left a comment

arik-so Aug 25, 2020

TheBlueMatt Aug 25, 2020

arik-so Aug 25, 2020

valentinewallace Aug 25, 2020

ariard Aug 26, 2020

valentinewallace Aug 26, 2020

TheBlueMatt Aug 26, 2020

ariard Aug 28, 2020 •

edited

Loading

		@@ -908,7 +969,7 @@ impl<ChanSigner: ChannelKeys> Channel<ChanSigner> {
		InboundHTLCState::AwaitingRemoteRevokeToAnnounce(_) => (!generated_by_local, "AwaitingRemoteRevokeToAnnounce"),

		if let Some(commitment_tx) = self.onchain_tx_handler.get_fully_signed_local_tx(&self.funding_redeemscript) {
		self.local_tx_signed = true;

Remove Channel's ChannelMonitor copy #667

Remove Channel's ChannelMonitor copy #667

Conversation

valentinewallace commented Aug 10, 2020 • edited Loading

codecov bot commented Aug 10, 2020 • edited Loading

Codecov Report

valentinewallace commented Aug 12, 2020

valentinewallace commented Aug 13, 2020

valentinewallace commented Aug 13, 2020

Choose a reason for hiding this comment

valentinewallace Aug 25, 2020 • edited Loading

Choose a reason for hiding this comment

ariard left a comment

Choose a reason for hiding this comment

TheBlueMatt commented Aug 21, 2020

TheBlueMatt left a comment

Choose a reason for hiding this comment

TheBlueMatt left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ariard Aug 28, 2020 • edited Loading

Choose a reason for hiding this comment

valentinewallace commented Aug 10, 2020 •

edited

Loading

codecov bot commented Aug 10, 2020 •

edited

Loading

valentinewallace Aug 25, 2020 •

edited

Loading

ariard Aug 28, 2020 •

edited

Loading