Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

adds `fundall` flag to `openchannel` #4029

Open
wants to merge 3 commits into
base: master
from

Conversation

@bjarnemagnussen
Copy link
Contributor

bjarnemagnussen commented Feb 25, 2020

Related to PR #4024, but reopened as a new PR as this feature has undergone major changes.

Description

This PR introduces a new flag fundall for openchannel, which when opening a new channel uses the whole available wallet balance for the channel capacity up to the allowed maximum funding limit (currently ~0.16 btc).

Motivation

See also #619

The command sendcoins has a flag sweepall that will spend all the coins from the wallet. However no such flag exists when opening new channels using openchannel.

Wanting to use the total available wallet balance in the local amount when opening a channel requires calculating the needed fee and subtracting it from the balance manually.

Implementation

A new coin select function CoinSelectUpToAmount has been introduced, which selects coins up to the requested amount, or below if there are not sufficient funds available. Fees and dust amounts are added or subtracted from this amount or the change output depending on the available funds, see also the new test cases.

Considerations

This implementation should be concurrent safe, becuase it locks the coins prior to selecting them, as currently done by the wallet assembler for the coin select subroutine.

Pull Request Checklist

  • If this is your first time contributing, we recommend you read the Code
    Contribution Guidelines
  • All changes are Go version 1.12 compliant
  • The code being submitted is commented according to Code Documentation and Commenting
  • For new code: Code is accompanied by tests which exercise both
    the positive and negative (error paths) conditions (if applicable)
  • Code has been formatted with go fmt
  • For code and documentation: lines are wrapped at 80 characters
    (the tab character should be counted as 8 characters, not 4, as some IDEs do
    per default)
  • Running make check does not fail any tests
  • Running go vet does not report any issues
  • Running make lint does not report any new issues that did not
    already exist
  • All commits build properly and pass tests. Only in exceptional
    cases it can be justifiable to violate this condition. In that case, the
    reason should be stated in the commit message.
  • Commits have a logical structure according to Ideal Git Commit Structure
When enabling this flag all the available wallet balance up to the allowed maximum funding limit will be used with a channel commitment.
// the amount to be pushed to the remote or
// the allowed minimum funding size.
if remoteInitialBalance >= minChanFundingSize {
localFundingAmt = remoteInitialBalance + 1

This comment has been minimized.

Copy link
@bjarnemagnussen

bjarnemagnussen Feb 25, 2020

Author Contributor

One requirement for local amount is to be greater than the remote initial balance. Is it therefore enough for the local amount to be 1 Satoshi greater?

This comment has been minimized.

Copy link
@halseth

halseth Mar 4, 2020

Collaborator

Where does this requirement come from?

This comment has been minimized.

Copy link
@bjarnemagnussen

bjarnemagnussen Mar 4, 2020

Author Contributor

This requirement comes from the RPC and is because it defines the push amount that must be available with the local funding.

lnd/rpcserver.go

Lines 1570 to 1578 in 1467cd4

// Ensure that the initial balance of the remote party (if pushing
// satoshis) does not exceed the amount the local party has requested
// for funding.
//
// TODO(roasbeef): incorporate base fee?
if remoteInitialBalance >= localFundingAmt {
return fmt.Errorf("amount pushed to remote peer for initial " +
"state must be below the local funding amount")
}

@@ -214,3 +214,136 @@ func CoinSelectSubtractFees(feeRate chainfee.SatPerKWeight, amt,

return selectedUtxos, outputAmt, changeAmt, nil
}

// CoinSelectUpToAmount attempts to select coins such that we'll spend up

This comment has been minimized.

Copy link
@Roasbeef

Roasbeef Feb 25, 2020

Member

Why can't CraftSweepAllTx be used for this purpose?

This comment has been minimized.

Copy link
@Roasbeef

Roasbeef Feb 25, 2020

Member

This also doesn't lock outpoints at all, which can lead to race conditions.

This comment has been minimized.

Copy link
@bjarnemagnussen

bjarnemagnussen Feb 25, 2020

Author Contributor

As far as I can see, the CraftSweepAllTx will spend all inputs. But for opening channels there is a maximum constant lnd.MaxFundingAmount that a channel funding is not allowed to exceed.
It would probably be possible to change CraftSweepAllTx to allow to specify a maximum amount, but would logically not make sense in a sweep package?

In addition, opening channels requires sending many messages. E.g. before attempting to open a channel LND will check if the remote node is responding, using dust limits depending on the chain used, etc. From the workflow of opening a channel it wasn't obvious to me how to get the funding output to use as "delivery" address, without specifying the funding amount in the first place. And I also didn't want to change this workflow when fundall is used.

Regarding locking of outpoints, as far as I see this should already happen in the wallet_assembler.go like for the other coin select subroutines:

// Lock the selected coins. These coins are now "reserved",
// this prevents concurrent funding requests from referring to
// and this double-spending the same set of coins.
for _, coin := range selectedCoins {
outpoint := coin.OutPoint
w.cfg.CoinLocker.LockOutpoint(outpoint)
}

@halseth

This comment has been minimized.

Copy link
Collaborator

halseth commented Feb 25, 2020

The wallet/fundingmanager already has a subtractFees flag that can be used for this purpose:

SubtractFees bool

It should significantly reduce the complexity of this PR I think.

@bjarnemagnussen

This comment has been minimized.

Copy link
Contributor Author

bjarnemagnussen commented Feb 25, 2020

The wallet/fundingmanager already has a subtractFees flag that can be used for this purpose:

SubtractFees bool

It should significantly reduce the complexity of this PR I think.

I was trying to use this flag, but since the maximum funding amount is limited to the constant lnd.MaxFundingAmount, we don't always want to use it. It is only if the balance is less than lnd.MaxFundingAmount, and in special cases where the balance is just slightly greater than lnd.MaxFundingAmount but not enough to cover the fees. Then we actually will subtract the fee from the total balance which will give a funding amount a little less than this constant.

But I did use the CoinSelectSubtractFee function for inspiration a lot, and it may be that I should try to combine the new CoinSelectUpToAmount with it?

I saw there may also be an open TODO in that regard for the excess fee. Furthermore, CoinSelect does not make correct fee calculation if change amount is dust, which is solved by CoinSelectSubtractFee and CoinSelectUpToAmount.

@halseth

This comment has been minimized.

Copy link
Collaborator

halseth commented Feb 28, 2020

@bjarnemagnussen I'm not sure I understand the limitation of the current API. If you do a regular channel opening with the MaxFundingAmt value, it will open that channel if the balance is available for that channel size + the fee. If the balance is not available, one can use the subtractFees flag, wit a value set to the remaining wallet balance. Wouldn't this work?

Refactors the function `CoinSelectUpToAmount` to make use of the existing API.
@bjarnemagnussen

This comment has been minimized.

Copy link
Contributor Author

bjarnemagnussen commented Feb 29, 2020

@halseth This works well, and I have added a commit to do as you suggest! :) Hopefully this is something along the line of what you were thinking?

Mismatch between coin select functions

There is this little mismatch between CoinSelect and CoinSelectSubtractFees that probably should be considered.

The CoinSelect function always uses a fixed fee calculation for one input and one change output. Let's assume the balance is one satoshi greater than the sum of MaxFundingAmt and a fee for one input and no change output. Then the CoinSelect function will return an error for insufficient funds, and the CoinSelectSubtractFees function takes over.

Inside CoinSelectSubtractFees we use the whole balance but the smaller fee calculation for one input and no change output. The final output amount returned from CoinSelectSubtractFees will therefore be the smaller fee subtracted from the total balance, which will result in an amount one satoshi greater than the MaxFundingAmt.

An easy fix is to use a sanity check that simply makes sure that we do not return an amount greater than what we asked for. Any difference should be small (fee difference between one and no change output) and can directly go towards the fee.

Test cases

I have for now added a test case "WIP: mismatch of fee calculation between functions" that will pass due to the added sanity check explained above, but otherwise would give rise to an output amount greater than the specified maximum.

There is also an oddity regarding testing for dust change, as it depends on which branch inside CoinSelectUpToAmount is used. Since CoinSelectUpToAmount now makes use of CoinSelect and CoinSelectSubtractFees it depends on their different ways of handling dust change. I have removed testing for dust change as those should be handled by the other coin select functions. I have therefore also removed the test case "no amount to pay fee for change".

Copy link
Collaborator

halseth left a comment

This looks pretty good! Could an alternative way to communicate our intention to use all our funds to the wallet be a fundUpTo bool? That way we would reuse the localAmt field for our maximum channels size, but communicate with the boolean that we allow creating a smaller channel if the funds are not available.

We could also take this a step further and create a chanSizeType enum that would have values SubtractFees, FundUpTo and ExactChanSize.

cli.BoolFlag{
Name: "fundall",
Usage: "(optional) if set, then the local_amt field will " +
"be ignored, and the wallet will attempt to commit the " +

This comment has been minimized.

Copy link
@halseth

halseth Mar 4, 2020

Collaborator

should state that should not be set together with localAmt

This comment has been minimized.

Copy link
@bjarnemagnussen

bjarnemagnussen Mar 4, 2020

Author Contributor

Oh, right!

// the amount to be pushed to the remote or
// the allowed minimum funding size.
if remoteInitialBalance >= minChanFundingSize {
localFundingAmt = remoteInitialBalance + 1

This comment has been minimized.

Copy link
@halseth

halseth Mar 4, 2020

Collaborator

Where does this requirement come from?

// current limit of the maximum funding amount.
var fundUpToMaxAmt btcutil.Amount
if msg.fundAll {
fundUpToMaxAmt = MaxFundingAmount

This comment has been minimized.

Copy link
@halseth

halseth Mar 4, 2020

Collaborator

should we return an error if the user attempts fundAll but have more funds than MaxFundingAmount?

This comment has been minimized.

Copy link
@bjarnemagnussen

bjarnemagnussen Mar 4, 2020

Author Contributor

I am not sure how the code here is related to your comment?

Regarding your comment: Do you prefer fundAll to fail if there would be any leftovers? Because the funds could be slightly more than MaxFundingAmount but due to fees the local amount would be less and without any leftovers.

Personally, I prefer to have a flag that I could set to try to open a channel with the maximum possible funding amount, regardless if it spends all funds or only the allowed maximum amount with leftovers.

Copy link
Contributor Author

bjarnemagnussen left a comment

Thanks @halseth for your review and the helpful inputs!

Could an alternative way to communicate our intention to use all our funds to the wallet be a fundUpTo bool? That way we would reuse the localAmt field for our maximum channels size, but communicate with the boolean that we allow creating a smaller channel if the funds are not available.

I have been considering this. But there were a few things I fell over:

  • There is the minimum channel funding limit minChanFundingSize defined in lnd that we always want to exceed. And we may therefore want to be able to define a minimum value at the channel funding when fundAll is set.
  • When called from the RPC it must also be assured that the funding amount is at least the amount we will afterwards try to push to the remote (see comment with the code).
  • With a minimum channel funding limit set inside localAmt we keep the behaviour that the channel opening will fail with funds less than what is set with the localAmt.

I am therefore not sure if we can work around this in another way?

We could also take this a step further and create a chanSizeType enum that would have values SubtractFees, FundUpTo and ExactChanSize.

I like your idea of defining a chanSizeType enum for the different cases! But how it will look probably depends on if the minimum amount is needed or not from the discussion above.

// the amount to be pushed to the remote or
// the allowed minimum funding size.
if remoteInitialBalance >= minChanFundingSize {
localFundingAmt = remoteInitialBalance + 1

This comment has been minimized.

Copy link
@bjarnemagnussen

bjarnemagnussen Mar 4, 2020

Author Contributor

This requirement comes from the RPC and is because it defines the push amount that must be available with the local funding.

lnd/rpcserver.go

Lines 1570 to 1578 in 1467cd4

// Ensure that the initial balance of the remote party (if pushing
// satoshis) does not exceed the amount the local party has requested
// for funding.
//
// TODO(roasbeef): incorporate base fee?
if remoteInitialBalance >= localFundingAmt {
return fmt.Errorf("amount pushed to remote peer for initial " +
"state must be below the local funding amount")
}

cli.BoolFlag{
Name: "fundall",
Usage: "(optional) if set, then the local_amt field will " +
"be ignored, and the wallet will attempt to commit the " +

This comment has been minimized.

Copy link
@bjarnemagnussen

bjarnemagnussen Mar 4, 2020

Author Contributor

Oh, right!

// current limit of the maximum funding amount.
var fundUpToMaxAmt btcutil.Amount
if msg.fundAll {
fundUpToMaxAmt = MaxFundingAmount

This comment has been minimized.

Copy link
@bjarnemagnussen

bjarnemagnussen Mar 4, 2020

Author Contributor

I am not sure how the code here is related to your comment?

Regarding your comment: Do you prefer fundAll to fail if there would be any leftovers? Because the funds could be slightly more than MaxFundingAmount but due to fees the local amount would be less and without any leftovers.

Personally, I prefer to have a flag that I could set to try to open a channel with the maximum possible funding amount, regardless if it spends all funds or only the allowed maximum amount with leftovers.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

None yet

3 participants
You can’t perform that action at this time.