Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Intercept forward htlc #4018

Open
wants to merge 3 commits into
base: master
from

Conversation

@roeierez
Copy link
Contributor

roeierez commented Feb 19, 2020

This PR introduces the ability to intercept htlc forward events that are handled at the htlcswitch package and to override the switch default behavior. Basically, similar to HODL invoices, this PR enables routing nodes to HODL HTLCs and to run custom validations or processes before forwarding (or cancelling) the HTLC.
More specifically for every HTLC that is intended to be forwarded, the implementor can choose to hold the forward and signal the switch to wait for a later resolution. Such resolution can be one of the following:

  1. Resume - Resume execution to the default behavior (probably just forward).
  2. Fail - Fail the htlc backward, the implementor is not interested in forwarding it.
  3. Settle - For a given preimage that was revealed in a way not known to the switch the implementor can settle the htlc.
    Having more control over the forward behavior enables new important use cases to be implemented.
    We are specifically looking to support the following use cases:

Pay to Open
This is about being able to pay someone with no channel or with a low capacity channel by opening a channel during the payment.
Let's say Alice (that has a channel) wants to pay Bob (that doesn't have a channel).

  1. Bob is connected to a node and issues an invoice with a routing hint that points to a fake channel between Bob and the node.
  2. When Alice pays Bob's invoice, the node intercepts the HTLC and holds it.
  3. At this stage one of the two options can follow:
    • The node opens a channel to Bob and upon confirmation (or not) resumes the forward.
    • The node opens a channel to Bob, pays Bob the original Alices amount minus service fee (in a different invoice), gets the preimage from Bob (in exchange to the payment) and settles the hold forward.

Notify to Pay
Let's say Alice wants to pay Bob, a mobile user, which is connected to a node via private channel.

  1. Alice pays, the node intercept the forward, checks Bob's availability and notices Bob is offline (inactive).
  2. The node notifies Bob using an agreed-upon notification mechanism.
  3. The node waits for Bob to be online and once the channel is active, it resumes the forward.

I am still missing things such as RPC layer, startup behavior, proper testing etc... but I wanted to share and get an initial feedback.
We would be interested to get your thoughts on the overall approach.

@roeierez roeierez requested review from cfromknecht and Roasbeef as code owners Feb 19, 2020
@halseth

This comment has been minimized.

Copy link
Collaborator

halseth commented Feb 20, 2020

Can be integrated with the HTLC notifier? #3781
cc @carlaKC

@carlaKC

This comment has been minimized.

Copy link
Collaborator

carlaKC commented Feb 20, 2020

Can be integrated with the HTLC notifier? #3781

I think they're different sides of the same coin? HTLCNotifier notifies forwarding decisions post-fact, once we have made the forwarding/failing decision.

My understanding of this is that it's preemptive: receive forward, callout to external forwarding predicate and then return, as specified in #3953.

@roeierez

This comment has been minimized.

Copy link
Contributor Author

roeierez commented Feb 20, 2020

Can be integrated with the HTLC notifier? #3781

@halseth thanks for bringing this to the discussion.
Some things to note when thinking about integration:

  1. The notifications are dispatched after the forward has been processed while the interception should be hooked before that, this means the current htlcNotifier interface needs to be extended.
  2. Interception is bidirectional and need to handle responses as well, this affects both the middleware and the RPC layer.
  3. Controlling the forwards will probably need different RPC permissions than getting htlc notifications.
  4. One example I see in the code that seems similar to this case is the channel events vs the channel acceptor. Two different mechanisms that have similar path in the code.

I think it will be an attempt to mix different things together.
Hope that helps.

@Roasbeef

This comment has been minimized.

Copy link
Member

Roasbeef commented Feb 21, 2020

Interception should be done at the switch, not the link. Also extreme care must be taken to not block the entire forwarding fabric, as the switch manages its state synchronously atm.

@Roasbeef Roasbeef added this to the 0.11.0 milestone Feb 21, 2020
@champo

This comment has been minimized.

Copy link
Contributor

champo commented Feb 21, 2020

@roeierez I really like the look of the API here. I've also been working on something similar. It allows you to register virtual channels and control HTLCs there through gRPC. It adds a new Link implementation and the intercept happens there. As far as the switch is concerned, nothing changes from a regular channel, which I think goes along the lines of what @Roasbeef suggests.

The highly-WIP code is at b3b2f7f and an usage example at https://gist.github.com/champo/8e90ce1ef38f2dd546814ff7898988c3 A knonw issue is that it doesn't close circuits so it kinda stops working after a success settle. It's easy to fix but I haven't gotten around to it yet.

@roeierez roeierez force-pushed the breez:intercept-forward-htlc branch 8 times, most recently from df7f3de to a199479 Feb 23, 2020
@roeierez

This comment has been minimized.

Copy link
Contributor Author

roeierez commented Feb 25, 2020

Interception should be done at the switch, not the link. Also extreme care must be taken to not block the entire forwarding fabric, as the switch manages its state synchronously atm.

I have moved the interception to the switch, added unit tests and the Router RPC implementation.

uint64 htlc_id = 2;
}

message ForwardHtlcInterceptRequest {

This comment has been minimized.

Copy link
@champo

champo Mar 1, 2020

Contributor

It'd be great to include the sphinx packet in this message. It would allow a receiver to check amount and expiry as it would if the HTLC had been forwarded through a link. This is useful for reverse swaps.

This comment has been minimized.

Copy link
@roeierez

roeierez Mar 2, 2020

Author Contributor

@champo thanks for the feedback.
Currently the fields amount_sat expiry are what you are looking for as they are the actual values that would have been used in the forwarded packet in the default switch behavior.

This comment has been minimized.

Copy link
@champo

champo Mar 2, 2020

Contributor

A third-party receiver (say a mobile app) shouldn't trust the fields. They would want to check the information against the sphinx. It would also enable usage of new fields like payment secret which is only known to the receiver.

This comment has been minimized.

Copy link
@roeierez

roeierez Mar 5, 2020

Author Contributor

@champo I agree. At first I exposed only the fields that are required for the use cases I mentioned in the PR description. I do think it makes sense to expose all the forwarding info (such as outgoing channel, tlv fields and raw sphinx packet) as other use cases may need this info in order to take a decision at interception time.

@roeierez roeierez force-pushed the breez:intercept-forward-htlc branch from a199479 to b1cdbc7 Mar 4, 2020
@roeierez roeierez changed the title Intercept forward htlc [WIP] Intercept forward htlc Mar 4, 2020
}

// RemoveForwardHtlcHandler removes a ForwardHtlcHandler.
func (c *Middleware) RemoveForwardHtlcHandler(id uint64) {

This comment has been minimized.

Copy link
@champo

champo Mar 11, 2020

Contributor

nit: maybe have a type alias for the id so it doesn't leak implementation detail of using uint16

func (c *Middleware) AddForwardHtlcHandler(
handler ForwardHtlcHandler) uint64 {

id := atomic.AddUint64(&c.fwdHandlerID, 1)

This comment has been minimized.

Copy link
@champo

champo Mar 11, 2020

Contributor

do you need an atomic here when fwdHandlers is always accessed with a lock?

This comment has been minimized.

Copy link
@roeierez

roeierez Mar 29, 2020

Author Contributor

No, moved to within the lock scope.

return err
}

delete(c.holdForwards, circuitKey)

This comment has been minimized.

Copy link
@champo

champo Mar 11, 2020

Contributor

shoudn't this be protected by the lock too? concurrent writes to maps are not supported

This comment has been minimized.

Copy link
@roeierez

roeierez Mar 29, 2020

Author Contributor

We should! Done.

@@ -63,6 +64,50 @@ var (
ErrUnreadableFailureMessage = errors.New("unreadable failure message")
)

// forwardResolver implements the htlcinterceptor.Interceptor interface.

This comment has been minimized.

Copy link
@champo

champo Mar 11, 2020

Contributor
Suggested change
// forwardResolver implements the htlcinterceptor.Interceptor interface.
// forwardResolver implements the htlcinterceptor.ForwardResolver interface.

// Fail forward a failed packet to the switch.
func (h *forwardResolver) Fail() error {
return h.resolve(&lnwire.UpdateFailHTLC{})

This comment has been minimized.

Copy link
@champo

champo Mar 11, 2020

Contributor

is it possible to set a Reason? what will it default to?

circuitInKey: &circuitInKey,
htlc: message,
}
return h.htlcSwitch.handlePacketForward(pkt)

This comment has been minimized.

Copy link
@champo

champo Mar 11, 2020

Contributor

I think this should either be calling ForwardPackets, routeAsync or route. As far as I can tell, calls to handlePacketForward always happen in the context of hltcForwarder go routine to keep the switch "sync". Calling it here directly might be messing up some assumptions made down the line. The 3 methods suggested before make sure the message ends up in hltcForwarder.

This comment has been minimized.

Copy link
@roeierez

roeierez Mar 29, 2020

Author Contributor

Good one! Used route.

// In case we got the circuitInKey populated it means that we should
// use it to find the circuit as there is no open circuit probably due
// to interception of this packet.
circuit = s.circuits.LookupCircuit(*pkt.circuitInKey)

This comment has been minimized.

Copy link
@champo

champo Mar 11, 2020

Contributor

is this circuit ever deleted?

This comment has been minimized.

Copy link
@roeierez
@roeierez roeierez force-pushed the breez:intercept-forward-htlc branch from b1cdbc7 to 5d72a87 Mar 29, 2020
roeierez added 3 commits Feb 18, 2020
This commit adds a new package that exposes a way to intercept an htlc
forward and override the regular switch behavior.
It also allows the caller to hold a forward and in a later time, resume,
cancel or settle the htlc.
The Interceptor interface is intentded to be propagated to the switch that
in his turn will pass every forward through it.
The Middleware is intended to be passed to other systems (such as RPC server)
so they can implement their own interception logic.
This commit connects the htlc Interceptor and Middleware into both rpc
server and the switch.
The rpc server should later expose RPC endpoints to communicate with the
Middleware to enable higher layers add their own interception logic.
@roeierez roeierez force-pushed the breez:intercept-forward-htlc branch from 5d72a87 to c416106 Mar 29, 2020
@roeierez roeierez requested a review from joostjager as a code owner Mar 29, 2020
@roeierez roeierez force-pushed the breez:intercept-forward-htlc branch from c416106 to ef5113a Mar 29, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

None yet

5 participants
You can’t perform that action at this time.