channeldb+rpcserver: expose legacy payments as multi-path payments #3499

cfromknecht · 2019-09-12T23:51:08Z

Builds on #3442

This PR extends the ListPayments, RouterRPC.SendPayment, and RouterRPC.TrackPayment to include an experimental field for displaying HTLC attempts for a given payment. This will us to display multiple HTLCs when MPP is fully integrated. Legacy payments will have at most one attempt, and we simply convert them into a more generic MPPayment struct after loading from disk.

In the future we will perform a migration to convert the legacy on-disk format into real MPP payments on disk, but this allows us to fleshing out the API before the internal work is completed.

channeldb/payments.go

rpcserver.go

channeldb/mp_payment.go

lnrpc/rpc.proto

channeldb/mp_payment.go

channeldb/payments.go

lnrpc/rpc.proto

halseth

Nice, I really like the limited scope of this PR! Makes it easy to review, and it is a great first step :)

Gave it a high level look, looks more or less good to me, but will need to see it in tandem with upcoming MPP PRs to make sure things fit 👍

channeldb/payments.go

lnrpc/rpc.proto

channeldb/mp_payment.go

Roasbeef

Solid base groundwork towards multi-pathing all the things 👍

lnrpc/rpc.proto

cmd/lncli/types.go

Roasbeef

What about the PaymentStatus call in the router sub-server?

cfromknecht · 2019-11-02T03:13:54Z

What about the PaymentStatus call in the router sub-server?

Last few commits now add the HTLC attempts to PaymentStatus

I rebased everything over #3442, which also allowed me test that the MPP record is being set properly in the new sendtoroute itests. This actually pointed out a bug in the test where we would send before receiving the node ann, causing us to downgrade to legacy payments for some of the htlcs. That bug has been fixed in #3442, and here we extend the itest with additional checks on the listpayments rpc.

channeldb/payments.go

channeldb/mp_payment.go

lnrpc/routerrpc/router_backend.go

halseth

I think this is very close, no major comments :)

channeldb/mp_payment.go

channeldb/payments.go

lnrpc/rpc.proto

rpcserver.go

routing/control_tower.go

halseth

LGTM 😱

channeldb/mp_payment.go

channeldb/payments.go

halseth · 2019-11-11T12:24:24Z

channeldb/payment_control.go

@@ -192,16 +191,17 @@ func (p *PaymentControl) RegisterAttempt(paymentHash lntypes.Hash,
 // duplicate payments to the same payment hash. The provided preimage is
 // atomically saved to the DB for record keeping.
 func (p *PaymentControl) Success(paymentHash lntypes.Hash,
-	preimage lntypes.Preimage) (*route.Route, error) {
+	preimage lntypes.Preimage) (*MPPayment, error) {


nit: would be nice if these channeldb changes could get their own commit

i attempted to do so originally, but it ended up causing more code shuffling than it was worth. the extraction code ends up being duplicated and move around, the tests end up double testing various aspects, etc over 4 commits. so i squashed it all into one since it's i found it easier to just look at the whole change

lnrpc/routerrpc/router_server.go

channeldb/mp_payment.go

channeldb/payments.go

lnrpc/rpc.proto

lnrpc/routerrpc/router_backend.go

routing/control_tower_test.go

Preliminary step to exposing PaymentPreimage() as a method that can be shared between legacy payments and mutli-path payments.

Prior name is too long XD

This commit modifies the FetchPayment method to return MPPayment structs converted from the legacy on-disk format. This allows us to attach the HTLCs to the events given to clients subscribing to the outcome of an HTLC. This commit also bubbles up to the routerrpc/router_server, by populating HTLCAttempts in the response and extracting the legacy route field from the HTLCAttempts.

joostjager

LGTM. Only wondering how relevant this change is now that we aim for single shot mpp first?

joostjager · 2019-11-20T09:47:40Z

lnrpc/routerrpc/router_server.go

+			case htlc.Failure != nil && result.FailureReason ==
+				channeldb.FailureReasonPaymentDetails:
+
+				legacyRoute = &htlc.Route


Is this better than just an if statement? Or at least use fallthrough?

joostjager · 2019-11-20T10:05:19Z

channeldb/mp_payment.go

+	Route route.Route
+
+	// AttemptTime is the time at which this HTLC was attempted.
+	AttemptTime time.Time


This field doesn't seem to be set at all in this pr. Remove and introduce in the follow up?

joostjager

Discussed offline, proceeding in parallel with single shot mpp

cfromknecht requested review from halseth, joostjager and Roasbeef as code owners September 12, 2019 23:51

joostjager reviewed Sep 13, 2019

View reviewed changes

cfromknecht force-pushed the mpp-payments-rpc branch 3 times, most recently from 24852d6 to 0369eb7 Compare September 16, 2019 18:54

halseth reviewed Sep 17, 2019

View reviewed changes

channeldb/payments.go Outdated Show resolved Hide resolved

lnrpc/rpc.proto Outdated Show resolved Hide resolved

Roasbeef added routing database Related to the database/storage of LND P2 should be fixed if one has time rpc Related to the RPC interface amp and removed routing labels Oct 3, 2019

alexbosworth reviewed Oct 8, 2019

View reviewed changes

channeldb/mp_payment.go Outdated Show resolved Hide resolved

alexbosworth reviewed Oct 8, 2019

View reviewed changes

channeldb/mp_payment.go Outdated Show resolved Hide resolved

Roasbeef reviewed Oct 30, 2019

View reviewed changes

lnrpc/rpc.proto Outdated Show resolved Hide resolved

lnrpc/rpc.proto Outdated Show resolved Hide resolved

cmd/lncli/types.go Outdated Show resolved Hide resolved

Roasbeef requested changes Oct 30, 2019

View reviewed changes

cfromknecht mentioned this pull request Oct 31, 2019

single-shot, sender-side mpp via sendtoroute #3442

Merged

cfromknecht added this to the 0.9.0 milestone Nov 2, 2019

cfromknecht force-pushed the mpp-payments-rpc branch 2 times, most recently from 5ce2ebc to b1db05a Compare November 2, 2019 03:09

cfromknecht requested review from Roasbeef, joostjager and halseth November 2, 2019 03:10

cfromknecht force-pushed the mpp-payments-rpc branch 2 times, most recently from 6b01397 to 36798d3 Compare November 2, 2019 21:17

joostjager reviewed Nov 5, 2019

View reviewed changes

channeldb/payments.go Show resolved Hide resolved

channeldb/mp_payment.go Outdated Show resolved Hide resolved

lnrpc/routerrpc/router_backend.go Show resolved Hide resolved

cfromknecht force-pushed the mpp-payments-rpc branch from f5ee562 to 17b0ef8 Compare November 5, 2019 21:36

joostjager changed the title ~~channeldb+rcpserver: expose legacy payments as mutli-path payments~~ channeldb+rpcserver: expose legacy payments as mutli-path payments Nov 6, 2019

joostjager changed the title ~~channeldb+rpcserver: expose legacy payments as mutli-path payments~~ channeldb+rpcserver: expose legacy payments as multi-path payments Nov 6, 2019

halseth suggested changes Nov 6, 2019

View reviewed changes

cfromknecht mentioned this pull request Nov 6, 2019

bolt 11 payment addr + single shot mpp for SendPayment #3679

Merged

5 tasks

cfromknecht force-pushed the mpp-payments-rpc branch from 17b0ef8 to 9a3efd6 Compare November 6, 2019 21:31

cfromknecht requested review from halseth and joostjager November 6, 2019 21:31

cfromknecht force-pushed the mpp-payments-rpc branch 2 times, most recently from b77b77a to 4a62ad9 Compare November 8, 2019 11:40

halseth approved these changes Nov 11, 2019

View reviewed changes

joostjager reviewed Nov 11, 2019

View reviewed changes

channeldb/mp_payment.go Outdated Show resolved Hide resolved

channeldb/payments.go Outdated Show resolved Hide resolved

lnrpc/rpc.proto Show resolved Hide resolved

lnrpc/routerrpc/router_backend.go Show resolved Hide resolved

routing/control_tower_test.go Outdated Show resolved Hide resolved

cfromknecht added 9 commits November 19, 2019 20:40

multi: rename Payment.PaymentPreimage to Payment.Preimage

fbd599a

Preliminary step to exposing PaymentPreimage() as a method that can be shared between legacy payments and mutli-path payments.

routing/control_tower: fix PaymentResult godocs

9e01940

channeldb: add MPPayment from FetchPayments

7760245

lnrpc: expose new MPP fields on list payments

d1f245e

lnrpc+rpcserver: populate experimental MPP fields in ListPayment rpc

997500b

lntest: assert mpp fields in listpayments

e9eabed

multi: rename to FailureReasonPaymentDetails

01cecb1

Prior name is too long XD

routerrpc: add HTLCAttempts to PaymentStatus

68916eb

cfromknecht force-pushed the mpp-payments-rpc branch from 4a62ad9 to 063f24f Compare November 20, 2019 04:45

joostjager reviewed Nov 20, 2019

View reviewed changes

joostjager approved these changes Nov 20, 2019

View reviewed changes

Roasbeef merged commit 6f9fcfa into lightningnetwork:master Nov 22, 2019

cfromknecht deleted the mpp-payments-rpc branch November 22, 2019 09:07

cfromknecht mentioned this pull request Nov 22, 2019

lnrpc: fix nanonsecond typo #3757

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

channeldb+rpcserver: expose legacy payments as multi-path payments #3499

channeldb+rpcserver: expose legacy payments as multi-path payments #3499

cfromknecht commented Sep 12, 2019 •

edited

Loading

halseth left a comment

Roasbeef left a comment

Roasbeef left a comment

cfromknecht commented Nov 2, 2019 •

edited

Loading

halseth left a comment

halseth left a comment

halseth Nov 11, 2019

cfromknecht Nov 20, 2019

joostjager left a comment

joostjager Nov 20, 2019

joostjager Nov 20, 2019

joostjager left a comment

channeldb+rpcserver: expose legacy payments as multi-path payments #3499

channeldb+rpcserver: expose legacy payments as multi-path payments #3499

Conversation

cfromknecht commented Sep 12, 2019 • edited Loading

halseth left a comment

Choose a reason for hiding this comment

Roasbeef left a comment

Choose a reason for hiding this comment

Roasbeef left a comment

Choose a reason for hiding this comment

cfromknecht commented Nov 2, 2019 • edited Loading

halseth left a comment

Choose a reason for hiding this comment

halseth left a comment

Choose a reason for hiding this comment

halseth Nov 11, 2019

Choose a reason for hiding this comment

cfromknecht Nov 20, 2019

Choose a reason for hiding this comment

joostjager left a comment

Choose a reason for hiding this comment

joostjager Nov 20, 2019

Choose a reason for hiding this comment

joostjager Nov 20, 2019

Choose a reason for hiding this comment

joostjager left a comment

Choose a reason for hiding this comment

cfromknecht commented Sep 12, 2019 •

edited

Loading

cfromknecht commented Nov 2, 2019 •

edited

Loading