Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

channeldb+routing+discovery: reject zombie announcements #2777

Merged
merged 8 commits into from Mar 28, 2019

Conversation

@wpaulino
Copy link
Collaborator

@wpaulino wpaulino commented Mar 13, 2019

In this PR, we introduce a new persistent index for zombie edges to quickly reject announcements for edges we've previously deemed as zombies. Care has been taken to ensure we don't reject fresh updates for edges we've considered zombies.

channeldb/graph.go Show resolved Hide resolved
channeldb/graph.go Outdated Show resolved Hide resolved
discovery/gossiper.go Outdated Show resolved Hide resolved
discovery/gossiper.go Outdated Show resolved Hide resolved
discovery/gossiper.go Outdated Show resolved Hide resolved
routing/router.go Outdated Show resolved Hide resolved
@wpaulino wpaulino force-pushed the wpaulino:reject-zombie-anns branch 2 times, most recently from d0233f8 to b389ade Mar 18, 2019
@wpaulino wpaulino force-pushed the wpaulino:reject-zombie-anns branch from b389ade to a062952 Mar 20, 2019
Copy link
Member

@Roasbeef Roasbeef left a comment

Excited to finally exterminate the Zombies in a permanent manner....LOK TAR OGAR!!!! ✊🏾

Solid set of changes, completed first pass and will start testing on various nodes. No major comments other than the sig validation and node pruning comments you'll find in-line.

channeldb/graph.go Show resolved Hide resolved
channeldb/graph.go Outdated Show resolved Hide resolved
MarkEdgeLive(chanID lnwire.ShortChannelID) error

// IsZombieEdge returns whether the edge is considered zombie.
IsZombieEdge(chanID lnwire.ShortChannelID) bool

This comment has been minimized.

@Roasbeef

Roasbeef Mar 20, 2019
Member

This interface is getting a bit thicc...in the future I think we should slim it unbundle it a bit by breaking it into distinct interfaces that have very narrow purposes. As is now, many of the callers that accept this interface don't use most of the methods they're forced to know about.

This comment has been minimized.

@wpaulino

wpaulino Mar 26, 2019
Author Collaborator

Slimmed it down a bit since the gossiper now only requires knowing of MarkEdgeLive.

routing/router.go Show resolved Hide resolved
discovery/gossiper.go Outdated Show resolved Hide resolved
case <-time.After(2 * trickleDelay):
}

time.Sleep(time.Second)

This comment has been minimized.

@Roasbeef

Roasbeef Mar 20, 2019
Member

Why's this sleep needed? Perhaps we should instead return to the caller immediately even if we get an update for an ann we don't know of.

This comment has been minimized.

@wpaulino

wpaulino Mar 26, 2019
Author Collaborator

Since we have to re-process it, if we return to the caller immediately then they won't be informed of any errors that occurred while re-processing it. Turns out the sleep wasn't needed though.

"as zombie: %v", chanToPrune.ChannelPoint, err)
}

err = r.cfg.Graph.DeleteChannelEdge(&chanToPrune.ChannelPoint)

This comment has been minimized.

@Roasbeef

Roasbeef Mar 20, 2019
Member

Perhaps we should combine both operations into a single call? If we do, then this means that someone can't give us a super old channel and make us re-scan the graph with GetUTXO or the like.

This comment has been minimized.

@cfromknecht

cfromknecht Mar 23, 2019
Collaborator

yeah the more i think about this, i think its a good idea to have the transition from live -> zombie be atomic

This comment has been minimized.

@wpaulino

wpaulino Mar 26, 2019
Author Collaborator

Good point! Fixed.

// channels from the graph based on their spentness, but whether they
// are considered zombies or not.
if r.cfg.AssumeChannelValid {
if err := r.pruneZombieChans(); err != nil {

This comment has been minimized.

@Roasbeef

Roasbeef Mar 20, 2019
Member

This skips calling PruneGraphNodes.

This comment has been minimized.

@wpaulino

wpaulino Mar 26, 2019
Author Collaborator

Will be including these changes in another PR.

@wpaulino wpaulino force-pushed the wpaulino:reject-zombie-anns branch 3 times, most recently from 89eb9bd to baae58d Mar 26, 2019
@wpaulino
Copy link
Collaborator Author

@wpaulino wpaulino commented Mar 26, 2019

Comments addressed, PTAL @cfromknecht @Roasbeef.

@wpaulino wpaulino force-pushed the wpaulino:reject-zombie-anns branch from baae58d to d9f0cfc Mar 26, 2019
if err != nil {
return errors.Errorf("unable to reconstruct message: %v", err)

This comment has been minimized.

@cfromknecht

cfromknecht Mar 27, 2019
Collaborator

👍 one day, we will kill go-errors

Copy link
Collaborator

@cfromknecht cfromknecht left a comment

LGTM, will begin testing this on my node 🔥

@wpaulino wpaulino force-pushed the wpaulino:reject-zombie-anns branch 3 times, most recently from 4bf9074 to 53ddc24 Mar 27, 2019
@wpaulino wpaulino force-pushed the wpaulino:reject-zombie-anns branch 2 times, most recently from f496d0a to a9b85d1 Mar 27, 2019
Copy link
Collaborator

@halseth halseth left a comment

💀🔫

channeldb/graph.go Outdated Show resolved Hide resolved
channeldb/graph.go Show resolved Hide resolved
routing/router.go Outdated Show resolved Hide resolved
routing/router.go Outdated Show resolved Hide resolved
discovery/gossiper.go Outdated Show resolved Hide resolved
discovery/gossiper.go Show resolved Hide resolved
discovery/gossiper.go Outdated Show resolved Hide resolved
discovery/gossiper_test.go Outdated Show resolved Hide resolved
discovery/gossiper_test.go Outdated Show resolved Hide resolved
discovery/gossiper_test.go Outdated Show resolved Hide resolved
wpaulino added 4 commits Mar 27, 2019
In this commit, we add a zombie edge index to the database. This allows
us to quickly determine across restarts whether we're attempting to
process an edge we've previously deemed as zombie.
We mark the edges as zombies when pruning them to ensure we don't
attempt to reprocess them later on. This also applies to channels that
have been removed from the graph due to being stale.
In this commit, we extend the graph's FetchChannelEdgesByID and
HasChannelEdge methods to also check the zombie index whenever the edge
to be looked up doesn't exist within the edge index. We do this to
signal to callers that the edge is known, but only as a zombie, and the
only information that we have about the edge are the node public keys of
the two parties involved in the edge.

In the event that an edge does exist within the zombie index, we make
an additional check on edge policies to ensure they are not within the
router's pruning window, indicating that it is a fresh update.
wpaulino added 4 commits Mar 27, 2019
In this commit, we leverage the recently introduced zombie edge index to
quickly reject announcements for edges we've previously deemed as
zombies. Care has been taken to ensure we don't reject fresh updates for
edges we've considered zombies.
@wpaulino wpaulino force-pushed the wpaulino:reject-zombie-anns branch from a9b85d1 to 5cec451 Mar 27, 2019
// signed by the correct party. The least-significant
// bit in the flag on the channel update tells us which
// edge is being updated.
var pubKey *btcec.PublicKey

This comment has been minimized.

@cfromknecht

cfromknecht Mar 27, 2019
Collaborator

not a blocker, just noting that this same block of code exists in many places w/in the code base. would be great to have a method like

func (c *ChannelEdgeInfo) DirectionKey(flags lnwire.ChannelFlags) *btcec.PublicKey
Copy link
Member

@Roasbeef Roasbeef left a comment

LGTM 🎎

return nil
}

exists = true
isZombie = false

This comment has been minimized.

@Roasbeef

Roasbeef Mar 28, 2019
Member

Non blocking nit: not actually needed since the default value of a bool is false.

@Roasbeef Roasbeef merged commit 43ba4a5 into lightningnetwork:master Mar 28, 2019
2 checks passed
2 checks passed
continuous-integration/travis-ci/pr The Travis CI build passed
Details
coverage/coveralls First build on reject-zombie-anns at 59.795%
Details
@wpaulino wpaulino deleted the wpaulino:reject-zombie-anns branch Mar 28, 2019
"graph(shortChanID=%v), saving for "+
"reprocessing later", shortChanID)

// NOTE: We don't return anything on the error channel

This comment has been minimized.

@halseth

halseth Mar 28, 2019
Collaborator

Can this potentially cause the gossiper to deadlock if the announcement never comes through? 🤔

This comment has been minimized.

@wpaulino

wpaulino Mar 28, 2019
Author Collaborator

No because we return. The errChan in which we're sending the error back is given to the caller.

This comment has been minimized.

@halseth

halseth Apr 1, 2019
Collaborator

Was just wondering who waits for the value on the channel, but looks like we never actually read it for remote annonucements...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

None yet

4 participants