[Feature request] option to read estimated fees from a file

[xmrk-btc]

I would like an option to read the estimated fees from a text file, instead of using the current fees. The file format could be: each line contains one channel id and the fee in ppm, separated by whitespace. For channels which are not found in the file, the current fee would be used (same behaviour as now), so in other words, the file would override the current fees.

Additionally, if the file contains word default where the channel id is expected, that fee is applied to all channels not found in the file.
So for example, if the file with estimates is:

11111111 10
2222222 20

then channel 11111111 is treated as if its outgoing fee was 10 ppm, 2222222 - as if outgoing fee was 20 ppm, and for 3333333 (which is not in the file) the estimated fee is the current fee of 3333333.

Example with added default:

11111111 10
2222222 20
default 50

Now, the estimated earnings for 3333333 are calculated using 50 ppm - the default value specified.

Background

I am running rebalance-lnd and charge-lnd, and they do not interact well. charge-lnd sets very high fees when the channel's local balance is depleted, which attracts rebalance-lnd to do costly rebalancing, because it appears profitable. But after that rebalancing, charge-lnd lowers the fee, so the rebalancing we have just done is not profitable, if we take into account the new fee - which is what will be charged when forwarding.

For example, we start with an almost depleted channel with fee 400 ppm. rebalance-lnd sends btc to this channel (local balance increases), happily paying 300 ppm, which is less than 400 ppm, so looks profitable. Then, charge-lnd sees increased local balance, so the forwarding fee is lowered to say 100 ppm. So when someone does the forwarding, I earn 100 ppm, while I paid 300 ppm, so I am losing money. (Maybe my charge-lnd config is not ideal, maybe the fee raises too quickly. But I still think it makes sense to set high fee to discourage routing, and I only want to discourage routing, not encourage rebalancing.)

To solve that, I would like to use the average fee ratio calculated from the fees collected in, say, last 7 days. So in my example lnd would no longer estimate the fee as 400 ppm, unless there really is a significant volume of forwardings while the channel's fee is over 400 ppm. I guess it is not a perfect method, but good enough.

I prefer the generic method to prevent feature creep: instead of implementing 10 methods of estimating fees in rebalance-lnd, users can implement their own method.

[C-Otto]

I'm not a fan of the fee settings you described. Why do you lower the fee rate to 100ppm if you think 400ppm are suitable? Why don't you leave the fee rate at 400ppm all the time? I don't think that rebalance-lnd should be involved with fee rate changes. You as the node operator need to configure fees that are realistic, i.e. fees that meet both your demands and the demands of your users. Go as high as you can. If you do that, you can use rebalance-lnd without any adjustments (aside from, possibly, some --fee-factor below 1.0).

[xmrk-btc]

Well, I hoped that charge-lnd would let me determine the right fee automatically, that I just need to set the fee range and then the fee would raise until an equilibrium between inflow and outflow is reached. But I saw this is not sufficient, I saw some my channels drained, this is why I started rebalancing.

But ok, I haven't though much about the big picture. Guess I need to see your recent video on the topic. :)

[TrezorHannes]

FWIW I can understand both perspectives.
@xmrk-btc would like to combine both the fee-incentive charge-lnd provides for rebalancing, with

• high fees when channels are on the remote
• low fees to stipulate outgoing payments when channels are high on the local

In either of the extremes, the computation rebalance-lnd takes into account are the current fees, which neglects both the past (average fees of the last 7days or 30days) and the future (post rebalance). The file would allow some computation outside of lnd to make demand and supply of past and future available to rebalancing costs.

In theory, this could be made applicable by using the external file as proposed, and use --fee-ppm-limit & reckless. All the smart logic would be outsourced, and rebalance-lnd acts as executioner only.