Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

channeldb+routing+discovery: reject zombie announcements #2777

Merged
merged 8 commits into from Mar 28, 2019

Conversation

Projects
None yet
4 participants
@wpaulino
Copy link
Collaborator

wpaulino commented Mar 13, 2019

In this PR, we introduce a new persistent index for zombie edges to quickly reject announcements for edges we've previously deemed as zombies. Care has been taken to ensure we don't reject fresh updates for edges we've considered zombies.

@wpaulino wpaulino added this to the 0.6 milestone Mar 14, 2019

Show resolved Hide resolved channeldb/graph.go
Show resolved Hide resolved channeldb/graph.go Outdated
Show resolved Hide resolved discovery/gossiper.go Outdated
Show resolved Hide resolved discovery/gossiper.go Outdated
Show resolved Hide resolved discovery/gossiper.go Outdated
Show resolved Hide resolved routing/router.go Outdated

@wpaulino wpaulino force-pushed the wpaulino:reject-zombie-anns branch 2 times, most recently from d0233f8 to b389ade Mar 18, 2019

@wpaulino wpaulino force-pushed the wpaulino:reject-zombie-anns branch from b389ade to a062952 Mar 20, 2019

@Roasbeef
Copy link
Member

Roasbeef left a comment

Excited to finally exterminate the Zombies in a permanent manner....LOK TAR OGAR!!!! ✊🏾

Solid set of changes, completed first pass and will start testing on various nodes. No major comments other than the sig validation and node pruning comments you'll find in-line.

Show resolved Hide resolved channeldb/graph.go
Show resolved Hide resolved channeldb/graph.go Outdated
MarkEdgeLive(chanID lnwire.ShortChannelID) error

// IsZombieEdge returns whether the edge is considered zombie.
IsZombieEdge(chanID lnwire.ShortChannelID) bool

This comment has been minimized.

Copy link
@Roasbeef

Roasbeef Mar 20, 2019

Member

This interface is getting a bit thicc...in the future I think we should slim it unbundle it a bit by breaking it into distinct interfaces that have very narrow purposes. As is now, many of the callers that accept this interface don't use most of the methods they're forced to know about.

This comment has been minimized.

Copy link
@wpaulino

wpaulino Mar 26, 2019

Author Collaborator

Slimmed it down a bit since the gossiper now only requires knowing of MarkEdgeLive.

Show resolved Hide resolved routing/router.go
Show resolved Hide resolved discovery/gossiper.go Outdated
case <-time.After(2 * trickleDelay):
}

time.Sleep(time.Second)

This comment has been minimized.

Copy link
@Roasbeef

Roasbeef Mar 20, 2019

Member

Why's this sleep needed? Perhaps we should instead return to the caller immediately even if we get an update for an ann we don't know of.

This comment has been minimized.

Copy link
@wpaulino

wpaulino Mar 26, 2019

Author Collaborator

Since we have to re-process it, if we return to the caller immediately then they won't be informed of any errors that occurred while re-processing it. Turns out the sleep wasn't needed though.

"as zombie: %v", chanToPrune.ChannelPoint, err)
}

err = r.cfg.Graph.DeleteChannelEdge(&chanToPrune.ChannelPoint)

This comment has been minimized.

Copy link
@Roasbeef

Roasbeef Mar 20, 2019

Member

Perhaps we should combine both operations into a single call? If we do, then this means that someone can't give us a super old channel and make us re-scan the graph with GetUTXO or the like.

This comment has been minimized.

Copy link
@cfromknecht

cfromknecht Mar 23, 2019

Collaborator

yeah the more i think about this, i think its a good idea to have the transition from live -> zombie be atomic

This comment has been minimized.

Copy link
@wpaulino

wpaulino Mar 26, 2019

Author Collaborator

Good point! Fixed.

// channels from the graph based on their spentness, but whether they
// are considered zombies or not.
if r.cfg.AssumeChannelValid {
if err := r.pruneZombieChans(); err != nil {

This comment has been minimized.

Copy link
@Roasbeef

Roasbeef Mar 20, 2019

Member

This skips calling PruneGraphNodes.

This comment has been minimized.

Copy link
@wpaulino

wpaulino Mar 26, 2019

Author Collaborator

Will be including these changes in another PR.

@wpaulino wpaulino force-pushed the wpaulino:reject-zombie-anns branch 3 times, most recently from 89eb9bd to baae58d Mar 26, 2019

@wpaulino

This comment has been minimized.

Copy link
Collaborator Author

wpaulino commented Mar 26, 2019

Comments addressed, PTAL @cfromknecht @Roasbeef.

@wpaulino wpaulino force-pushed the wpaulino:reject-zombie-anns branch from baae58d to d9f0cfc Mar 26, 2019

if err != nil {
return errors.Errorf("unable to reconstruct message: %v", err)

This comment has been minimized.

Copy link
@cfromknecht

cfromknecht Mar 27, 2019

Collaborator

👍 one day, we will kill go-errors

@cfromknecht
Copy link
Collaborator

cfromknecht left a comment

LGTM, will begin testing this on my node 🔥

@wpaulino wpaulino force-pushed the wpaulino:reject-zombie-anns branch 3 times, most recently from 4bf9074 to 53ddc24 Mar 27, 2019

@wpaulino wpaulino force-pushed the wpaulino:reject-zombie-anns branch 2 times, most recently from f496d0a to a9b85d1 Mar 27, 2019

@halseth
Copy link
Collaborator

halseth left a comment

💀🔫

Show resolved Hide resolved channeldb/graph.go Outdated
Show resolved Hide resolved channeldb/graph.go
Show resolved Hide resolved routing/router.go Outdated
Show resolved Hide resolved routing/router.go Outdated
Show resolved Hide resolved discovery/gossiper.go Outdated
Show resolved Hide resolved discovery/gossiper.go
Show resolved Hide resolved discovery/gossiper.go Outdated
Show resolved Hide resolved discovery/gossiper_test.go Outdated
Show resolved Hide resolved discovery/gossiper_test.go Outdated
Show resolved Hide resolved discovery/gossiper_test.go Outdated

wpaulino added some commits Mar 27, 2019

channeldb: add zombie edge index
In this commit, we add a zombie edge index to the database. This allows
us to quickly determine across restarts whether we're attempting to
process an edge we've previously deemed as zombie.
channeldb: extend DeleteChannelEdge to mark edge as zombie
We mark the edges as zombies when pruning them to ensure we don't
attempt to reprocess them later on. This also applies to channels that
have been removed from the graph due to being stale.
channeldb+routing: extend edge lookup methods with zombie index check
In this commit, we extend the graph's FetchChannelEdgesByID and
HasChannelEdge methods to also check the zombie index whenever the edge
to be looked up doesn't exist within the edge index. We do this to
signal to callers that the edge is known, but only as a zombie, and the
only information that we have about the edge are the node public keys of
the two parties involved in the edge.

In the event that an edge does exist within the zombie index, we make
an additional check on edge policies to ensure they are not within the
router's pruning window, indicating that it is a fresh update.

wpaulino added some commits Mar 27, 2019

discovery: reject announcements for known zombie edges
In this commit, we leverage the recently introduced zombie edge index to
quickly reject announcements for edges we've previously deemed as
zombies. Care has been taken to ensure we don't reject fresh updates for
edges we've considered zombies.

@wpaulino wpaulino force-pushed the wpaulino:reject-zombie-anns branch from a9b85d1 to 5cec451 Mar 27, 2019

// signed by the correct party. The least-significant
// bit in the flag on the channel update tells us which
// edge is being updated.
var pubKey *btcec.PublicKey

This comment has been minimized.

Copy link
@cfromknecht

cfromknecht Mar 27, 2019

Collaborator

not a blocker, just noting that this same block of code exists in many places w/in the code base. would be great to have a method like

func (c *ChannelEdgeInfo) DirectionKey(flags lnwire.ChannelFlags) *btcec.PublicKey
@Roasbeef
Copy link
Member

Roasbeef left a comment

LGTM 🎎

return nil
}

exists = true
isZombie = false

This comment has been minimized.

Copy link
@Roasbeef

Roasbeef Mar 28, 2019

Member

Non blocking nit: not actually needed since the default value of a bool is false.

@Roasbeef Roasbeef merged commit 43ba4a5 into lightningnetwork:master Mar 28, 2019

2 checks passed

continuous-integration/travis-ci/pr The Travis CI build passed
Details
coverage/coveralls First build on reject-zombie-anns at 59.795%
Details

@wpaulino wpaulino deleted the wpaulino:reject-zombie-anns branch Mar 28, 2019

"graph(shortChanID=%v), saving for "+
"reprocessing later", shortChanID)

// NOTE: We don't return anything on the error channel

This comment has been minimized.

Copy link
@halseth

halseth Mar 28, 2019

Collaborator

Can this potentially cause the gossiper to deadlock if the announcement never comes through? 🤔

This comment has been minimized.

Copy link
@wpaulino

wpaulino Mar 28, 2019

Author Collaborator

No because we return. The errChan in which we're sending the error back is given to the caller.

This comment has been minimized.

Copy link
@halseth

halseth Apr 1, 2019

Collaborator

Was just wondering who waits for the value on the channel, but looks like we never actually read it for remote annonucements...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.