chainntnfs: handle spend notifications within TxConfNotifier #2004
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
wpaulino
force-pushed
the
spend-tx-notifier
branch
4 times, most recently
from
4012931
to
38e0203
Compare
wpaulino
added
notifications
backend
wpaulino
force-pushed
the
spend-tx-notifier
branch
3 times, most recently
from
40c57fb
to
f4c2e0c
Compare
halseth
reviewed
Oct 6, 2018
chainntnfs/tx_notifier.go
Outdated
| // transactions. Each height hint represents the earliest height at | ||
| // which the transactions could have been confirmed within the chain. | ||
| hintCache ConfirmHintCache | ||
| confirmHintCache ConfirmHintCache |
There was a problem hiding this comment.
Good job on the step-by-step commits here, makes it easy to review! 🎉
wpaulino
force-pushed
the
spend-tx-notifier
branch
3 times, most recently
from
a879b38
to
7e0b74b
Compare
|
Addressed remaining comments and extended the tests a bit more to correctly test the behavior of the |
wpaulino
force-pushed
the
spend-tx-notifier
branch
3 times, most recently
from
366d7d9
to
368e623
Compare
halseth
reviewed
Oct 16, 2018
wpaulino
force-pushed
the
spend-tx-notifier
branch
3 times, most recently
from
cd826ae
to
02381fe
Compare
halseth
reviewed
Oct 23, 2018
wpaulino
force-pushed
the
spend-tx-notifier
branch
from
02381fe
to
82dd815
Compare
cfromknecht
reviewed
Oct 25, 2018
There was a problem hiding this comment.
first sweep done, looking really solid! great to see so much duplicated code removed. have some small comments, mostly regarding differences between how confs are handled in #1787 and how spends are handled here
| func NewTxNotifier(startHeight uint32, reorgSafetyLimit uint32, | ||
| confirmHintCache ConfirmHintCache) *TxNotifier { | ||
| confirmHintCache ConfirmHintCache, |
There was a problem hiding this comment.
still don't see the use in having these separate when the can just share a HintCache, but not a blocker
There was a problem hiding this comment.
Agreed - we already have the unified HeightHintCache struct that handles both of these cases. Perhaps this should be done within #2044 or even another PR?
There was a problem hiding this comment.
Best to just do in this PR imo, to avoid ever introducing the spendHintCache.
There was a problem hiding this comment.
just remembering that it is already done in #1767, though it was never merged
wpaulino
force-pushed
the
spend-tx-notifier
branch
3 times, most recently
from
b238e5c
to
19c1aa6
Compare
In this commit, we modify the set of tests that start the different backend notifiers with UnsafeStart to stop them within the tests themselves. This prevents us from running into a panic when attempting to run the package-level tests with a filter (using test.run).
In this commit, we modify our height hint cache to no longer start a database transaction if no outpoints/txids are provided to update the height hints for.
In this commit, we add a new channel within the SpendEvent struct that will be sent upon whenever the spending transaction of the registered outpoint gets reorged out of the chain. This will pave the road for successfully handling a funding transaction getting reorged out of the chain among other things.
In this commit, we introduce the required fields for the TxNotifier to properly carry its duties in notifying its registered clients about the spend of an outpoint. These are not yet used, but will be throughout the some of the following commits.
In this commit, we introduce the registration logic for spend notifications to the TxNotifier. Most of this logic was taken from the different existing ChainNotifier implementations, however, it features some useful additions in order to make the ChainNotifier a bit more robust. Some of these additions include the following: 1. RegisterSpend will now return a HistoricalSpendDispatch struct, which includes the details for successfully determining if an outpoint was spent in the past. A HistoricalSpendDispatch will only be returned upon the first registration of an outpoint. This is done as, previously, if multiple clients registered for the same outpoint, then multiple historical rescans would also be dispatched, incurring a toll on the backend itself. 2. UpdateSpendDetails will now be used to determine when a historical rescan has completed, no matter if a spending transaction was found or not. This is needed in order to responsibly update the spend hints for outpoints at tip, otherwise we'd attempt to update them even though we haven't yet determined if they have been spent or not. This will dispatch notifications to all currently registered clients for the same outpoint. In the event that another client registers later on, then the spending details are cached in memory in order to prevent further historical rescans.
In this commit, we add support to allow the TxNotifier to properly determine whether a new block extending the chain contains a transaction that spends a registered outpoint. In the event that it does, spend notifications will be dispatched to all active registered clients for such outpoint.
…istered outpoints In this commit, we introduce support to the TxNotifier to detect spending transactions of registered outpoints being reorged out of the chain. In the event that a reorg does occur, we'll consume the Spend notification if it hasn't been consumed yet, and dispatch a Reorg notification instead.
…updates In this commit, we address an issue w.r.t. updating the confirm hints for transactions and spend hints for outpoints on chain updates. Previously, upon a block being disconnected, we'd attempt to commit a new height hint for all outstanding confirmation notifications. This is not correct because we'll end up modifying the height hint for things that have confirmed at a previous height than the one being disconnected. This would cause issues on restart when attempting a historical dispatch, as we would start scanning at a height above which the transaction actually confirmed in. This has been addressed by only updating the hints for outstanding notifications that are still unconfirmed/unspent and for notifications that were confirmed/spent within the block being connected/disconnected.
…TxNotifier In this commit, we modify the logic within RegisterSpendNtfn for the BitcoindNotifier to account for the recent changes made to the TxNotifier. Since it is now able to handle spend notification registration and dispatch, we can bypass all the current logic within the BitcoindNotifier and interact directly with the TxNotifier instead. The most notable changes include the following: 1. We'll only attempt a historical rescan if the TxNotifier tells us so. 2. We'll dispatch the historical rescan within the main goroutine to prevent WaitGroup panics, due to the asynchronous nature of the notifier.
In this commit, we remove the old spend notification logic within the BitcoindNotifier as it's been phased out by the TxNotifier.
…tifier In this commit, we modify the logic within RegisterSpendNtfn for the BtcdNotifier to account for the recent changes made to the TxNotifier. Since it is now able to handle spend notification registration and dispatch, we can bypass all the current logic within the BtcdNotifier and interact directly with the TxNotifier instead. The most notable change is that now we'll only attempt a historical rescan if the TxNotifier tells us so.
In this commit, we remove the old spend notification logic within the BtcdNotifier as it's been phased out by the TxNotifier.
…TxNotifier In this commit, we modify the logic within RegisterSpendNtfn for the NeutrinoNotifier to account for the recent changes made to the TxNotifier. Since it is now able to handle spend notification registration and dispatch, we can bypass all the current logic within the NeutrinoNotifier and interact directly with the TxNotifier instead. The most notable change is that now we'll only attempt a historical rescan if the TxNotifier tells us so.
In this commit, we remove the old spend notification logic within the NeutrinoNotifier as it's been phased out by the TxNotifier.
In this commit, we'll now commit the current height of the TxNotifier as the height hint for an outpoint/transaction after a rescan has been completed and has determined that the outpoint is unspent/transaction is unconfirmed. We do this to prevent another potentially long rescan if the daemon is restarted before a new block comes in (which is when the hints will be updated again).
In this commit, we modify the notifier to handle filter updates synchronously. We do this to prevent race conditions between new block notifications and filter updates. Otherwise, it's possible for a new block to come in that should match our filter, but doesn't due to the filter being updated after. We also modify their order so that the filter is updated first. We do this so we can immediately start watching for the event at tip while the rescan is ongoing.
wpaulino
force-pushed
the
spend-tx-notifier
branch
from
74147a0
to
e6b1a27
Compare
cfromknecht
approved these changes
Oct 31, 2018
Roasbeef
approved these changes
Oct 31, 2018
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
In this PR, we overhaul the
TxConfNotifierstruct to also track spend notifications. This introduces a number of benefits, such as: removing duplicate spend notification logic within the differentChainNotifierimplementations, spend reorg detection, and many other minor optimizations. This will serve as a stepping stone for re-enabling the height hint cache, as it previously had issues under reorgs. Details for the most important commits can be found inline, explaining what each one does and why.Depends on #1787.