Dynamic fees for lightning gateways

[justinmoon]

Outgoing payments:

• Fees currently hard-coded at 1%

Incoming payments:

• The client doesn't add any fee. They would do so by creating an invoice for slightly more than they want to receive. Or maybe we could add a large fee to the final (imaginary) path between the gateway and federation (???).
• The gateway just checks that they don't lose any money.

In principle I think the gateways should advertise the fees they will charge and clients would use this information to choose which gateway to use.

[bradleystachurski]

This is interesting! I'm happy to look into this.

Do you think it's in scope to include the interaction of gateways advertising fees and clients optimizing selection or should we leave that as a future improvement?

[justinmoon]

Do you think it's in scope to include the interaction of gateways advertising fees and clients optimizing selection or should we leave that as a future improvement?

I would do this in 3 PRs:

• Add configurable fees but default them to current values so everything continues to work
• Gateways advertise their fees and user clients use these fees when making a payment
• Have user take fees into consideration when selecting a gateway. But that should also take into account other metrics like these.

[justinmoon]

Also, any client API which actually pays lightning invoice (e.g. ln-pay cli command) might need to add a confirmation step where it presents the fees associated with paying this invoice and asks if that's OK. Maybe we don't need to do that for our CLI now, but GUI clients would definitely want to present this information before blindly paying the invoice + fees.

[okjodom]

On the implementation,

Step 1: Add configurable fees but default them to current values so everything continues to work

I think the LightningGateway should declare routing fees as proportional_millionths plus a base_msat fee. See https://github.com/lightningdevkit/rust-lightning/blob/main/lightning/src/routing/gossip.rs#L1057-L1065 . But we can start off simple, by making a configurable proportional_millionths only, then migrate to the full RoutingFees definition

See #2246
Also includes

Step 2: "Gateways advertise their fees and user clients use these fees when making a payment"

[okjodom]

Originally posted as Issue #524

[okjodom]

After today's discussion in deep dive, we started sketching full requirements and spec for dynamic gateway routing fees

The two halves of fees are

1. Outgoing Payments
   

Here, it seems possible to represent gateway fees as:

• fixed base fee
• percent of amount to be paid

Similar to RoutingFees

2. Incoming Payments
   

For incoming payments, we may want to support a fee spec where either the sender / receiver can pay the gateway fir routing payments into the federation. It's unclear what that representation entails, and if we can use the same spec as outgoing payments

[nassersaazi]

@okjodom @m1sterc001guy any PRs that have been opened following this discussion that i can look at?

[justinmoon]

#2246 is only progress so far

[otech47]

Unlike outgoing payments, I think incoming gateway fees should be expressed as a per-federation fee, since the gateway needs to take on the risk of holding the ecash notes for that federation.

This make sense

In the future, Bob could be using a fedimint client that is smart enough to contact the federation directly and buy the preimage himself. In that case, there should be no gateway fee.

Yes, this sounds like a situation where Alice and Bob should just determine whether Bob has enough ecash for the payment before even bothering to go through the gateway for a swap?

So I don't think it makes sense for the receiver to pay the gateway fee, since the receiver does not necessarily know who is going to pay for the invoice. Bob is essentially paying a tad more in fees because his client is "dumb" and only knows how to speak lightning, not lightning and fedimint. I believe this is also consistent with how lightning works today.

This could be fine as the default case, but I can see a situation where the receiver may want some visibility on fees when they generate the payment request / LN invoice and perhaps even some ability to cover fees.

In a P2P IRL interaction, Alice might want to be paid in ecash for selling a laptop to Bob but Bob only has sats in a lightning wallet.

Maybe Bob should have to pay fees because he doesn't have enough of the currency that Alice accepts (ecash)... or maybe Alice should have to pay fees because she can't take custody of any sats because her lightning channel is not balanced. The acceptable fee burden might be relative to the social interaction and scope of commerce but if we allow them to at least meet in the middle it might be something like:

• Alice can check what fees any current gateways are charging for incoming lightning sats to be converted to her federation's ecash, say it is 1% of a price of 1M sats = 10k sats
• Bob sees the invoice and needs to take notice that there is a 1% fee on top of the price that Alice set on the laptop
• When Bob scans the invoice, his lightning wallet estimates there will be 0.5% in routing fees, so in total 1.5% in fees will be paid by the sender of the payment
• If for whatever reason either of these 2 fee rates are unacceptable to Bob, at least Alice has a chance to lower the price of the laptop to share the fee burden

So as far as fedimint is concerned the calculation of the invoice amount just needs to be dynamic in the case of multiple gateways setting competitive fees and at most, clients should get the best fee rate using a 1D matrix of gateways that have enough ecash given an amount.

There might be another layer of complexity to explore if there is any negotiation that can happen between Alice & the gateway(s) when updating this fee rate. Can federations do anything to protect their users from unexpectedly high fees if a gateway starts to raise the fee rate too high or too fast? Would gateways choose to hold more ecash based on the federation's willingness to tolerate higher fees or greater fee flexibility?

[otech47]

Another idea that I'm not sure would work is putting all gateways into the RouteHints array when creating the invoice, but in order of preference?

Alice still has to choose among gateways X, Y, and Z to determine the amount after comparing fee rates, but could that invoice also contain route hints for the other gateway options?

Would lightning wallets evaluate each route hint in order such that if no route is found for gateway X, it would try to find a path to gateway Y, next all when paying a single invoice? or are we stuck juggling 3 distinct BOLT11 invoices when deciding which to display for Bob to pay?

Could be I don't understand how exactly we're using the route hints...

What I am getting at is if there is some way to use the routing hints to influence the recommended fee limit a lightning wallet will display when making a payment... since that might increase chance of payment success on the sender side

[m1sterc001guy]

If for whatever reason either of these 2 fee rates are unacceptable to Bob, at least Alice has a chance to lower the price of the laptop to share the fee burden

This scenario is actually simpler than that. In this case, Bob won't even know that the "gateway fee" is a gateway fee, he will see it as a lightning routing fee. So, in your example, the total fee rate will still be 1.5%, but that will communicated to Bob as lightning routing fees. He won't have any idea that he's paying a lightning gateway.

In the future, Bob could be using a fedimint client that is smart enough to contact the federation directly and buy the preimage himself. In that case, there should be no gateway fee.

We actually implemented this already, which is great! #2602

Can federations do anything to protect their users from unexpectedly high fees if a gateway starts to raise the fee rate too high or too fast? Would gateways choose to hold more ecash based on the federation's willingness to tolerate higher fees or greater fee flexibility?

I don't think the federations need to do anything here, it should be on the clients to recognize that gateways are charging higher fees and the client can switch gateways. We still need to build that part.

I think gateway's choosing federations based on federation fees is out of scope for this issue right now (IMO we should first focus on clients selecting gateways based on their fees), but that might be an area of exploration in the future.

Another idea that I'm not sure would work is putting all gateways into the RouteHints array when creating the invoice, but in order of preference?

See this issue: #2634. We need to balance the size of the QR code vs the amount of route hints in the QR code. The route hints are put in the invoice after an "active" gateway is selected. So multiple gateways are not included in a single invoice. I believe gateway selection here should occur by the receiver (i.e the client generating the invoice) and the receiver should select the gateway that will minimize fees for the sender before the invoice is generated. So we will probably need to add some logic before invoice generation that selects an "optimal" gateway.

[otech47]

is it possible right now for the federation to change the price of a preimage? Assuming the gateway is ready to purchase the preimage with ecash to settle an HTLC they intercepted, seems like the gateway could actually get the chance to ask Alice for a lower price in the scenario where the gateway would be losing money and should just fail the HTLC. Alice may want to just cover the difference in ecash to avoid a payment failure and the hassle of choosing a different gateway?

[m1sterc001guy]

No support for this today. The workaround would just be to fail the incoming payment so the contract is refunded, then re-create the contract.

[elsirion]

Since LN is sender-routed there is no need to ever re-negotiate incoming payments. Wither the sender accepts the fees communicated by the intermediate hops or they don't. The Gateway->User step looks like a normal LN hop over a private channel from the outside if that channel is expensive that's the sender's problem.

[elsirion]

Only the outgoing case is interesting imo, the incoming one should be transparent to the sender and recipient (looks and behaves like normal LN fees). When it come to outgoing payments we are faced with an optimization problem for which is probably unsolvable:

Which gateway will be the cheapest in total, once gateway fee and routing cost are combined.

The problem here is that we cannot tell ahead of time which gateway will require how much fees to route a payment. To approximate a solution we need a "quote" API endpoint where the gateway estimates, through probing, how much fees will have to be paid to route the payment. For example Bitcoin Beach Wallet implements a similar functionality. The interesting point about the quote API is that we don't need to announce gateway fees at all because only the total fee paid is relevant.

For us, with multiple independent gateways, this becomes a lot trickier. Each time a gateway fails we need to cancel one contract and fund another one with potentially a different gateway (the next cheapest one), costing at least one epoch in time. Imo setting an upper bound for fees paid and letting the state machines in the background figure out the details would likely be the best UX.

The same behavior could be achieved without a quote API, but there are some bad incentives to set GW fees to zero while pretending LN fees are costlier to maximize earnings since clients would need to guess a certain upper limit in fees they are willing to pay before trying another GW.