New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
renepay: accomodate fees in the payment flow #6893
renepay: accomodate fees in the payment flow #6893
Conversation
221a7b6
to
c03168d
Compare
65fcc77
to
5de111b
Compare
Notes:
|
de94391
to
025d047
Compare
025d047
to
3fc82bf
Compare
3fc82bf
to
0822bb6
Compare
Added an error check in uncertainty_network_update_from_listpeerchannels and rebased and squashed commits. |
910f2f9
to
a49806d
Compare
I tried to rebase this, but it was non-trivial and the tests failed. So I'd ask you to do it please? |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Trivial feedback only... this approach looks much nicer than my random hack!!!
Rebase needed though
plugins/renepay/flow.c
Outdated
|
||
/* Try to increase the value we send until we fail to fulfill the | ||
* fee inequality. It takes only one iteration though. */ | ||
for(int i=0;i<10;++i) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nitpick: I prefer size_t i
if it's a simple loop like this
plugins/renepay/flow.c
Outdated
= amount_msat_min( | ||
channel_liquidity(gossmap,chan_extra_map,flow->path[0],flow->dirs[0]), | ||
channel_max_htlc(flow->path[0],flow->dirs[0])); | ||
for(int i=1;i<tal_count(flow->path);++i) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
here too...
plugins/renepay/flow.c
Outdated
* probability. If the deficit is very small with respect to the amount | ||
* each flow carries then optimization here will not make much | ||
* difference. */ | ||
for(int i=0;i<tal_count(flows) && !amount_msat_zero(deficit);++i) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
here too...
Min. Cost Flow does not take into account fees when computing a flow with liquidity constraints. This is a work-around solution that reduces the amount on every route to respect the liquidity bound. The deficity in the delivered amount is solved by running MCF once again. Changes: 1. the function `flow_complete` allocates amounts to send over the set of routes computed by the MCF algorithm, but it does not allocate more than liquidity bound of the route. For this reason `minflow` returns a set of routes that satisfy the liquidity bounds but it is not guaranteed that the total payment reaches the destination therefore there could a deficit in the delivery: `deficit = amount_to_deliver - delivering`. 2. in the function `add_payflows` after `minflow` returns a set of routes we call `flows_fit_amount` that tries to a allocate the `deficit` in the routes that the MCF have computed. 3. if the resulting flows pass all payment constraints then we update `amount_to_deliver = amount_to_deliver - delivering`, and the loop repeats as long as `amount_to_deliver` is not zero. In other words, the excess amount, beyond the liquidity bound, in the routes is removed and then we try to allocate it into known routes, otherwise we do a whole MCF again just for the remaining amount. Fixes issue ElementsProject#6599
a49806d
to
8d87c19
Compare
I have rebased and fixed the conflicts. If the CI test pass, this will be ready for merging. |
Flake is unrelated, opened PR #7021 for that. |
Min. Cost Flow (MCF) does not resolve the correct amount in channel forwarding
because the algorithm dismisses the amounts paid as fees to the routing
nodes. The root of this problem is that flow conservation is assumed for
simplicity such that the sender is the only source and the recepient is the
only sink. This in reality is not true, because also forwarding nodes are
sinks since they consume fees.
When real amounts including fees are then added to the payment routes by
the function
flow_complete
, some channels might be requested to forward anamount which could be greater than
known_max - htlc_total
.For remote channels this issue is likely inadverted because we are anyways
caping the probability of success down to 5% in our linearization scheme,
see this line,
ie. 5% of the channel probable liquidity is reserved which could be in most
cases sufficient to let fees go through.
However, if the conditional capacity (ie.
know_max - known_min
) is smallthen its 5% may not be sufficient to let fees go through, specially if the MCF
has saturated the available liquidity of that particular channel.
The easiest way this might happen is with local channels, because
known_max-knon_min = 0
therefore there is 0 breath space for fees, furthermorethe MCF is always favorable to send as much flow as possible through local
channels because their fee and uncertainty cost is zero.
This was reported in issue #6599.
A temporary fix #6601 was merged that puts a fee reserve of 1% in local
channels.
This commit is a more robust solution that works-around the problem by reducing
a portion of the amount in those routes that exceed the liquidity bound.
Then the deficit in the deliver amount is either sent along the other routes
or sent through a new set of flows.
Liquidity bound in a channel
The liquidity bound in a channel is defined as
known_max - htlc_total
, itrepresents the maximum amount we can forward through this channel given our
current knowledge. This is implemented in this function:
Maximum forward
Bolt 7 specifies that a channel should forward an amount
out
if thefee
is greater equal toIf there is a limit
in
to the amount we can pay to a node to forward a payment onour behalf (through a specific channel) there will be a limit to the amount that
the channel can forward for us. Since we would like to send as much as possible we
need to solve the following problem:
given the value
in
, find the maximum valueout
such that:Which is equivalent to
Let's denote the bound in the right hand side as
B(in,out)
,which depends on
out
and that makes the fee equation difficult to invert.But consider another simpler expression
B_simple(in)
One can quickly verify that
B_simple(in) <= B(in,out)
so that one could useB_simple(in)
as an upper bound forout
.In fact we can even show that
for every value of
out
.Therefore if we set
we will automatically satisfy Bolt7 inequality.
We can also try if
out+1
also satisfies the inequality, if it does then thatnumber is the maximum value we can forward.
The function
computes the maximum value of
out
such thatusing the technique we explained here.
Liquidity bound of a route
Another component of this PR is the liquidity bound of a given route.
It is defined as the maximum amount the last node can receive considering the
liquidity bound of each channel and the fees that are paid.
It can be computed greedly in a single pass from the source to the destination.
The implementation is in the function:
For completeness we also constrain the liquidity of the route by taking into account the
htlc_max
.Changes
the function
flow_complete
allocates amounts to send over the set of routescomputed by the MCF algorithm, but it does not allocate more than liquidity
bound of the route. For this reason
minflow
returns a set of routes thatsatisfy the liquidity bounds but it is not guaranteed that the total payment
reaches the destination therefore there could a deficit in the delivery:
deficit = amount_to_deliver - delivering
.in the function
add_payflows
afterminflow
returns a set of routes wecall
flows_fit_amount
that tries to a allocate thedeficit
in the routesthat the MCF have computed.
if the resulting flows pass all payment constraints then we update
amount_to_deliver = amount_to_deliver - delivering
, and the looprepeats as long as
amount_to_deliver
is not zero.In other words, the excess amount, beyond the liquidity bound,
in the routes is removed and then we try to allocate it
into known routes, otherwise we do a whole MCF again just for the
remaining amount.
Task list