Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add an option to make the success probability estimation nonlinear #2547

Merged

Commits on Sep 19, 2023

  1. Split out success probability calculation to allow for changes

    Our "what is the success probability of paying over a channel with
    the given liquidity bounds" calculation is reused in a few places,
    and is a key assumption across our main score calculation and the
    historical bucket score calculations.
    
    Here we break it out into a function to make it easier to
    experiment with different success probability calculations.
    
    Note that this drops the numerator +1 in the liquidity scorer,
    which was added to compensate for the divisor + 1 (which exists to
    avoid divide-by-zero), making the new math slightly less correct
    but not by any material amount.
    TheBlueMatt committed Sep 19, 2023
    Configuration menu
    Copy the full SHA
    7543890 View commit details
    Browse the repository at this point in the history
  2. Scale the success probability of channels without info down by 75%

    If we are examining a channel for which we have no information at
    all, we traditionally assume the HTLC success probability is
    proportional to the channel's capacity. While this may be the case,
    it is not the case that a tiny payment over a huge channel is
    guaranteed to succeed, as we assume. Rather, the probability of
    such success is likely closer to 50% than 100%.
    
    Here we try to capture this by simply scaling the success
    probability for channels where we have no information down
    linearly. We pick 75% as the upper bound rather arbitrarily - while
    50% may be more accurate, its possible it would lead to an
    over-reliance on channels which we have paid through in the past,
    which aren't necessarily always the best candidates.
    
    Note that we only do this scaling for the historical bucket
    tracker, as there we can be confident we've never seen a successful
    HTLC completion on the given channel. If we were to apply the same
    scaling to the simple liquidity bounds based scoring we'd penalize
    channels we've never tried over those we've only ever fails to pay
    over, which is obviously not a good outcome.
    TheBlueMatt committed Sep 19, 2023
    Configuration menu
    Copy the full SHA
    5f98c39 View commit details
    Browse the repository at this point in the history

Commits on Sep 20, 2023

  1. Score in-flight amounts as amounts, not a capacity reduction

    When we started considering the in-flight amounts when scoring, we
    took the approach of considering the in-flight amount as an
    effective reduction in the channel's total capacity. When we were
    scoring using a flat success probability PDF, that was fine,
    however in the next commit we'll move to a highly nonlinear one,
    which makes this a pretty confusing heuristic.
    
    Here, instead, we move to considering the in-flight amount as
    simply an extension of the amount we're trying to send over the
    channel, which is equivalent for the flat success probability PDF,
    but makes much more sense in a nonlinear world.
    TheBlueMatt committed Sep 20, 2023
    Configuration menu
    Copy the full SHA
    df52da7 View commit details
    Browse the repository at this point in the history
  2. Add an option to make the success probability estimation nonlinear

    Our "what is the success probability of paying over a channel with
    the given liquidity bounds" calculation currently assumes the
    probability of where the liquidity lies in a channel is constant
    across the entire capacity of a channel. This is obviously a
    somewhat dubious assumption given most nodes don't materially
    rebalance and flows within the network often push liquidity
    "towards the edges".
    
    Here we add an option to consider this when scoring channels during
    routefinding. Specifically, if a new `linear_success_probability`
    flag is unset on `ProbabilisticScoringFeeParameters`, rather than
    assuming a PDF of `1` (across the channel's capacity scaled from 0
    to 1), we use `(x - 0.5)^2`.
    
    This assumes liquidity is likely to be near the edges, which
    matches experimental results. Further, calculating the CDF (i.e.
    integral) between arbitrary points on the PDF is trivial, which we
    do as our main scoring function.
    
    While this (finally) introduces floats in our scoring, its not
    practical to exponentiate using fixed-precision, and benchmarks
    show this is a performance regression, but not a huge one, more
    than made up for by the increase in payment success rates.
    TheBlueMatt committed Sep 20, 2023
    Configuration menu
    Copy the full SHA
    259ceb0 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    f2b2920 View commit details
    Browse the repository at this point in the history