Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bump dust exposure threshold #2354

Merged

Conversation

alecchendev
Copy link
Contributor

@alecchendev alecchendev commented Jun 13, 2023

Addresses #2261. Adds a config knob allowing a user to set a multiplier on the high priority fee estimate to control their maximum dust exposure.

With fee rates rising dramatically in mid-April 2023, thresholds for what is considered dust have risen, often exceeding our previous default dust exposure threshold of 5k sats. This causes all payments and HTLC forwards between 5k sats and new dust thresholds to fail. Having a multiplier allows the maximum dust exposure to scale automatically with changing fee rates.

(This PR originally just bumped the default to 25k sats but has been changed to this new approach.)

Copy link

@ariard ariard left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The 25k threshold sounds good to me - The ideal balance between dust exposure risks from miner-as-counterparties and HTLC throughput on a channel would be a dynamic limit scaled up from mempool levels of congestion probed at periodic interval…However as the concern is going away with anchor channels, and we should ship them soon(tm) I don’t think we should think too much on the ideal value.

lightning/src/util/config.rs Outdated Show resolved Hide resolved
@codecov-commenter
Copy link

codecov-commenter commented Jun 13, 2023

Codecov Report

Patch coverage: 91.50% and project coverage change: +0.14 🎉

Comparison is base (89aa7ac) 90.32% compared to head (b040335) 90.46%.

❗ Your organization is not using the GitHub App Integration. As a result you may experience degraded service beginning May 15th. Please install the Github App Integration for your organization. Read more.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #2354      +/-   ##
==========================================
+ Coverage   90.32%   90.46%   +0.14%     
==========================================
  Files         106      106              
  Lines       54948    58190    +3242     
  Branches    54948    58190    +3242     
==========================================
+ Hits        49630    52643    +3013     
- Misses       5318     5547     +229     
Impacted Files Coverage Δ
lightning/src/util/config.rs 56.56% <56.25%> (-1.58%) ⬇️
lightning/src/ln/channelmanager.rs 89.98% <94.73%> (+3.71%) ⬆️
lightning/src/ln/functional_tests.rs 98.26% <96.77%> (+<0.01%) ⬆️
lightning/src/ln/channel.rs 89.47% <100.00%> (+0.05%) ⬆️
lightning/src/ln/functional_test_utils.rs 88.10% <100.00%> (+0.27%) ⬆️
lightning/src/ln/onion_route_tests.rs 98.27% <100.00%> (+0.01%) ⬆️
lightning/src/ln/priv_short_conf_tests.rs 97.64% <100.00%> (+<0.01%) ⬆️
lightning/src/util/ser_macros.rs 66.18% <100.00%> (-3.30%) ⬇️

... and 8 files with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Do you have feedback about the report comment? Let us know in this issue.

@TheBlueMatt
Copy link
Collaborator

With the current hard-limit design I don't think $1.50 is nearly high enough. Sadly, the "right answer" really depends on the fee market - if transaction fees are $0.50 (they are today), $1.50 seems low - you have to pay a third of that to go on chain, why are you worried about losing that to dust? If someone is doing some attack they're gonna do it for more than that. If fees go back down to where transactions cost $0.05, then $1.50 is probably not a crazy level.

The discussion on the issue veered into some other ideas like an HTLC count limit, rather than an HTLC value limit, which may make sense, though that really relies on our counterparty's fee estimator for inbound channels, rather than ours. Another option would be to set the dust limit as some (configurable) multiplier on the HighPriority fee estimate - we have to pay that to get funds back, you should be willing to lose at least a few of those at a time.

@alecchendev
Copy link
Contributor Author

if transaction fees are $0.50 (they are today), $1.50 seems low - you have to pay a third of that to go on chain, why are you worried about losing that to dust?

Hmm that's a good point

The discussion on the issue veered into some other ideas like an HTLC count limit, rather than an HTLC value limit, which may make sense, though that really relies on our counterparty's fee estimator for inbound channels, rather than ours. Another option would be to set the dust limit as some (configurable) multiplier on the HighPriority fee estimate - we have to pay that to get funds back, you should be willing to lose at least a few of those at a time.

Okay, I'll think about these some more and try to get something up soon, if not I may drop this in favor of someone else picking it up since the milestone is coming up and I initially thought this change was going to be more trivial

@alecchendev
Copy link
Contributor Author

alecchendev commented Jun 19, 2023

Okay, I chose to go with having a configurable multiplier of the high priority feerate to set the dust exposure threshold since it seems the most straight forward without introducing any possible new issues IIUC (open to another approach if people think it's better however). I've mostly implemented it, just need to adjusts/write tests (done now).

I also saw in the issue some discussion about making it a global dust exposure limit rather than per-channel at the cost of some parallelism - is that worth implementing here as well?

@alecchendev alecchendev force-pushed the 2023-06-bump-default-dust-exp branch 2 times, most recently from c24d052 to 47fbe19 Compare June 19, 2023 21:54
Copy link
Collaborator

@TheBlueMatt TheBlueMatt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Didnt review the test changes but in general I think this looks great!

@TheBlueMatt TheBlueMatt added this to the 0.0.116 milestone Jun 19, 2023
@alecchendev alecchendev force-pushed the 2023-06-bump-default-dust-exp branch from 47fbe19 to ad4b09a Compare June 20, 2023 22:40
lightning/src/util/config.rs Outdated Show resolved Hide resolved
lightning/src/util/config.rs Outdated Show resolved Hide resolved
lightning/src/util/config.rs Outdated Show resolved Hide resolved
lightning/src/ln/channel.rs Outdated Show resolved Hide resolved
lightning/src/ln/channel.rs Outdated Show resolved Hide resolved
lightning/src/util/config.rs Outdated Show resolved Hide resolved
@alecchendev alecchendev force-pushed the 2023-06-bump-default-dust-exp branch from ad4b09a to 3e0280e Compare June 22, 2023 22:08
@alecchendev
Copy link
Contributor Author

Rebased, fixed fuzzing, switched to using an enum instead of an extra field which I think is much nicer.

@TheBlueMatt
Copy link
Collaborator

Ugh, needs rebase again sorry about that.

@alecchendev alecchendev force-pushed the 2023-06-bump-default-dust-exp branch 2 times, most recently from b43cdbe to 1106b2f Compare June 27, 2023 21:06
@alecchendev
Copy link
Contributor Author

Rebased

Copy link

@ariard ariard left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Moving to a real-time feerate-adjusted max_dust_htlc_exposure_msat sounds good, note the caveat about LDK clients relying on third-party fee-estimators.

pub enum MaxDustHTLCExposure {
/// This sets a fixed limit on the total dust exposure in millisatoshis. Setting this too low
/// may prevent the sending or receipt of low-value HTLCs on high-traffic nodes, and this limit
/// is very important to prevent stealing of dust HTLCs by miners.
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we would like to be informative towards LDK-based routing hops operators, I think we can indicate the risk of fee griefing attacks, where counterparties in collusion hold a HTLC payment path through a target and force-close the incoming commitment transaction to burn the non-settled dust HTLCs.

See the original description of dust exposure here: https://lists.linuxfoundation.org/pipermail/lightning-dev/2020-May/002714.html

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I slightly reworded and added the link - let me know if you think that's enough or not

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for doing it, well new comment missed the point of dissociating the two types of dust exposure risks:

  • one by counterparties with mining capabilities both payment senders/routing hops
  • one by counterparties without mining capabilities for routing hops only.

Enhanced the comment here ariard@ed9eabb (feel free to take it, squash it)

There is an added link to the disclosure post of October 2021, that is adding more context for node operators.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh I see, thanks.

There is an added link to the disclosure post of October 2021, that is adding more context for node operators.

In ariard@ed9eabb it looks like the two links are the same?

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In ariard@ed9eabb it looks like the two links are the same?

Fuck my bad, here In ariard@ed9eabb

The correct link is this one https://lists.linuxfoundation.org/pipermail/lightning-dev/2021-October/003257.html, which is the most informative source on the mailing list on dust exposure, to the best of my knowledge.

lightning/src/util/config.rs Show resolved Hide resolved
lightning/src/util/config.rs Show resolved Hide resolved
Copy link
Collaborator

@TheBlueMatt TheBlueMatt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this basically LGTM, feel free to squash the fields.

lightning/src/util/config.rs Show resolved Hide resolved
@alecchendev alecchendev force-pushed the 2023-06-bump-default-dust-exp branch from 1106b2f to 6fb3fa0 Compare June 30, 2023 00:57
Copy link

@ariard ariard left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review ACK 6fb3fa0 modulo the better comment on “dust exposure”.

Verified the high_priority_feerate_per_kw * value computation for FeeRateMultiplier in get_max_dust_htlc_exposure_msat and how it’s used through the codebase, though I didn’t check the tests changes.

TheBlueMatt
TheBlueMatt previously approved these changes Jul 2, 2023
Copy link
Collaborator

@TheBlueMatt TheBlueMatt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some trivial comments, mostly docs, feel free to squash when you address them, will find another reviewer.

lightning/src/util/config.rs Show resolved Hide resolved
lightning/src/util/config.rs Outdated Show resolved Hide resolved
lightning/src/util/config.rs Outdated Show resolved Hide resolved
lightning/src/util/config.rs Outdated Show resolved Hide resolved
lightning/src/util/config.rs Outdated Show resolved Hide resolved
lightning/src/util/config.rs Outdated Show resolved Hide resolved
lightning/src/ln/channelmanager.rs Outdated Show resolved Hide resolved
@alecchendev alecchendev force-pushed the 2023-06-bump-default-dust-exp branch from 4184ea2 to ac9617d Compare July 6, 2023 23:14
@alecchendev alecchendev force-pushed the 2023-06-bump-default-dust-exp branch from ac9617d to eae4be1 Compare July 6, 2023 23:22
wpaulino
wpaulino previously approved these changes Jul 7, 2023
@wpaulino
Copy link
Contributor

wpaulino commented Jul 7, 2023

Hmm, on second thought, maybe we shouldn't have gone the enum route to keep both around and have the feerate multiplier only apply to pre-anchor channels.

@TheBlueMatt
Copy link
Collaborator

Yea, that's something I was wondering about, but even if we're using anchors it's not unreasonable to want our dust exposure limit to float upwards. In the future we should probably think about the protocol letting us tweak our (but not our counterparty's) dust limit for anchor channels and then we'll want this.

@TheBlueMatt
Copy link
Collaborator

And in the mean time users are free to set a different limit on different types of channels.

@wpaulino
Copy link
Contributor

wpaulino commented Jul 7, 2023

And in the mean time users are free to set a different limit on different types of channels.

Yeah that's what I figured, but it doesn't seem very clear that users can do so. Maybe we add a sentence or two to let them know setting the feerate multiplier variant is mostly intended for pre-anchor channels in it's current form?

With fee rates rising dramatically in mid-April 2023, thresholds for
what is considered dust have risen, often exceeding our previous dust
exposure threshold of 5k sats. This causes all payments and HTLC
forwards between 5k sats and new dust thresholds to fail.

This commit changes our max dust exposure config knob from a fixed
upper limit to a `MaxDustHTLCExposure` enum with an additional variant
to allow setting our max dust exposure to a multiplier on the current
high priority feerate.

To remain backwards compatible we'll always write the fixed limit if
it's set, or its default value in its currently reserved TLV.

We also now write an odd TLV for the new enum, so that previous
versions can safely ignore it upon downgrading, while allowing us to
make use of the new type when it's written.
This commit makes use of the added enum to calculate the dust
exposure threshold based on the current fee rate. This also updates
tests to ensure it works as intended.
@alecchendev
Copy link
Contributor Author

Maybe we add a sentence or two to let them know setting the feerate multiplier variant is mostly intended for pre-anchor channels in it's current form?

Added a comment on the config knob and the variant's docs.

Will anchor channels be the default in 0.0.116? If they are, does it still make sense for the default value here to be the multiplier variant?

@wpaulino
Copy link
Contributor

wpaulino commented Jul 7, 2023

Will anchor channels be the default in 0.0.116? If they are, does it still make sense for the default value here to be the multiplier variant?

Probably won't happen for another 2 releases or so. We can consider changing the default max exposure then.

Copy link

@ariard ariard left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review ACK b040335712d, looked again into how we’re checking dust exposure in send_htlc(), send_update_fee() and update_fee(). Didn’t check at all backward compatibility issues.

Nice if we can add the good link on the most informative explanation of “dust htlc exposure”, though can be done as follow-up to avoid invalidating reviews so far.

@TheBlueMatt TheBlueMatt merged commit 6ebb6d1 into lightningdevkit:main Jul 8, 2023
14 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Bump dust exposure threshold so we can send HTLCs
5 participants