Simplify coin selection for sendcoins #8516

Chinwendu20 · 2024-03-04T05:51:04Z

Change Description

This PR fixes this issue: #6949 (comment)
Depends on: btcsuite/btcwallet#912 and #8729

In this well detailed issue comment, the process of sending funds using utxos selected by the sender currently has over five steps in which one has to call various psbt APIs.

The goal of this PR is to improve user experience by including utxos to the sendcoins command and corresponding rpc structs (SendCoinsRequest and SendCoinsResponse ) which would enable the sender described to achieve the same aim with one command.

This is done by adding a new field to rpc structs SendCoinsRequest and SendCoinsResponse to enable users request for that functionality and enable the driving function (in which we modify its functionality as well) use these utxos when crafting the transaction for this functionality.

Additionally, the sweepall field in the above mentioned request and response rpc structs when true now not only sweeps ALL funds in the wallet but also ALL funds in the selected utxos when used in conjunction with the new select utxos field.

The lncli sendcoins command was updated to include the flag, utxo to enable this functionality on that end.

There were also slight internal logic change, where a slice of utxos are now accepted by relevant functions as variadic argument, functional options to enable this functionality.

Steps to Test

Steps for reviewers to follow to test the change.

Pull Request Checklist

Testing

Your PR passes all CI checks.
Tests covering the positive and negative (error paths) are included.
Bug fixes contain tests triggering the bug to prevent regressions.

Code Style and Documentation

The change obeys the Code Documentation and Commenting guidelines, and lines wrap at 80.
Commits follow the Ideal Git Commit Structure.
Any new logging statements use an appropriate subsystem and logging level.
Any new lncli commands have appropriate tags in the comments for the rpc in the proto file.
There is a change description in the release notes, or [skip ci] in the commit message for small changes.

📝 Please see our Contribution Guidelines for further guidance.

coderabbitai · 2024-03-04T05:51:08Z

Important

Review skipped

Auto reviews are limited to specific labels.

Labels to auto review (1)

llm-review

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

Share

Tips

Chat

There are 3 ways to chat with CodeRabbit:

Review comments: Directly reply to a review comment made by CodeRabbit. Example:
- I pushed a fix in commit <commit_id>.
- Generate unit testing code for this file.
- Open a follow-up GitHub issue for this discussion.
Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
- @coderabbitai generate unit testing code for this file.
- @coderabbitai modularize this function.
PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
- @coderabbitai generate interesting stats about this repository and render them as a table.
- @coderabbitai show all the console.log statements in this repository.
- @coderabbitai read src/utils.ts and generate unit testing code.
- @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
- @coderabbitai help me debug CodeRabbit configuration file.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (invoked as PR comments)

@coderabbitai pause to pause the reviews on a PR.
@coderabbitai resume to resume the paused reviews.
@coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
@coderabbitai full review to do a full review from scratch and review all the files again.
@coderabbitai summary to regenerate the summary of the PR.
@coderabbitai resolve resolve all the CodeRabbit review comments.
@coderabbitai configuration to show the current CodeRabbit configuration for the repository.
@coderabbitai help to get help.

Additionally, you can add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.

CodeRabbit Configration File (`.coderabbit.yaml`)

You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
Please see the configuration documentation for more information.
If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

Visit our Documentation for detailed information on how to use CodeRabbit.
Join our Discord Community to get help, request features, and share feedback.
Follow us on X/Twitter for updates and announcements.

Chinwendu20 · 2024-03-04T06:40:35Z

Hello @hieblmi here is the PR

sweep/walletsweep.go

Chinwendu20 · 2024-03-04T11:07:14Z

I would be using the psbt API instead as this approach would not craft a transaction with all inputs selected by the user as pointed out here:
btcsuite/btcwallet#912 (comment)

Modified it, no need for this

itest/lnd_channel_funding_fund_max_test.go

Chinwendu20 · 2024-03-05T06:25:44Z

Hello @sputn1ck, do you think I should create a different function instead of adding an arg to an exported one as per your comment here: btcsuite/btcwallet#912 (comment)

bennyhodl · 2024-03-12T18:59:31Z

tACK.

Tested locally and worked as expected with one utxo, sweep, and multiple utxos. Would like to see this merged as I am signing PSBTs manually because my application requires 1 input --> 1 output transactions.

bennyhodl · 2024-03-26T14:42:51Z

Is there an update on this @Chinwendu20 ?

Chinwendu20 · 2024-03-26T14:44:14Z

Is there an update on this @Chinwendu20 ?

Thanks I should push an update by the end of this week..

hieblmi

tACK, apart from the failing itest cases which seem to be unrelated to this PR I have nits left.

Great work again on this @Chinwendu20.

hieblmi · 2024-06-07T10:21:15Z

lnwallet/btcwallet/btcwallet.go

@@ -974,12 +974,13 @@ func (b *BtcWallet) ImportTaprootScript(scope waddrmgr.KeyScope,
 // the specified outputs. In the case the wallet has insufficient funds, or the
 // outputs are non-standard, a non-nil error will be returned.
 //
-// NOTE: This method requires the global coin selection lock to be held.
+// NOTE: This method requires the global coin selection lock to be held.n


nit: remove n

this is not addressed

hieblmi · 2024-06-07T10:23:52Z

cmd/lncli/commands.go

+		cli.StringSliceFlag{
+			Name: "utxo",
+			Usage: "a utxo specified as outpoint(tx:idx) which " +
+				"will be used as input for the transaction; " +


nit: period instead of ;

hieblmi · 2024-06-07T10:26:11Z

docs/release-notes/release-notes-0.18.0.md

@@ -384,6 +384,9 @@
  required](https://github.com/lightningnetwork/lnd/pull/8681) to be consistent
  with the flow when a payment request isn't used. 

+* [Added `outpoints` to `SendCoinsRequest`](


move to release notes for v0.18.1-beta.

Signed-off-by: Ononiwu Maureen <59079323+Chinwendu20@users.noreply.github.com>

Chinwendu20 · 2024-06-13T06:42:08Z

@guggero we have two approvals now, I think we can merge?

hieblmi · 2024-06-13T08:46:37Z

@guggero we have two approvals now, I think we can merge?

Wanted to make sure that @yyforyongyu's CRs were addressed, would be good to get his go too.

yyforyongyu

There's a missing check that we need to ensure provided utxos are known to the wallet.

yyforyongyu · 2024-06-14T08:14:01Z

lnwallet/btcwallet/btcwallet.go

@@ -974,12 +974,13 @@ func (b *BtcWallet) ImportTaprootScript(scope waddrmgr.KeyScope,
 // the specified outputs. In the case the wallet has insufficient funds, or the
 // outputs are non-standard, a non-nil error will be returned.
 //
-// NOTE: This method requires the global coin selection lock to be held.
+// NOTE: This method requires the global coin selection lock to be held.n


this is not addressed

yyforyongyu · 2024-06-14T08:18:30Z

lnwallet/btcwallet/btcwallet.go

@@ -974,7 +974,7 @@ func (b *BtcWallet) ImportTaprootScript(scope waddrmgr.KeyScope,
 // the specified outputs. In the case the wallet has insufficient funds, or the
 // outputs are non-standard, a non-nil error will be returned.
 //
-// NOTE: This method requires the global coin selection lock to be held.n
+// NOTE: This method requires the global coin selection lock to be held.


ok i see it now, this should be cherry-picked to where it's introduced.

yyforyongyu · 2024-06-14T08:39:46Z

sweep/walletsweep.go

 func CraftSweepAllTx(feeRate, maxFeeRate chainfee.SatPerKWeight,
 	blockHeight uint32, deliveryAddrs []DeliveryAddr,
 	changeAddr btcutil.Address, coinSelectLocker CoinSelectionLocker,
 	utxoSource UtxoSource, outputLeaser OutputLeaser,
-	signer input.Signer, minConfs int32) (*WalletSweepPackage, error) {
+	signer input.Signer, minConfs int32,
+	selectUtxos ...wire.OutPoint) (*WalletSweepPackage, error) {


Why do we want to use a variadic param rather than a slice here? I think it's easier if it's selectUtxos []wire.OutPoint) and we can pass the selectOutpoints instead of selectOutpoints... in SendCoins?

Not really easier in my opinion as that would also mean we would have to make changes other places this function was called

I think the diff is fairly simple,

diff --git a/rpcserver.go b/rpcserver.go index 80cff0763..619c635a8 100644 --- a/rpcserver.go +++ b/rpcserver.go @@ -1389,7 +1389,7 @@ func (r *rpcServer) SendCoins(ctx context.Context, sweepTxPkg, err := sweep.CraftSweepAllTx( feePerKw, maxFeeRate, uint32(bestHeight), nil, targetAddr, wallet, wallet, wallet.WalletController, - r.server.cc.Signer, minConfs, selectOutpoints..., + r.server.cc.Signer, minConfs, selectOutpoints, ) if err != nil { return nil, err @@ -1444,7 +1444,7 @@ func (r *rpcServer) SendCoins(ctx context.Context, outputs, targetAddr, wallet, wallet, wallet.WalletController, r.server.cc.Signer, minConfs, - selectOutpoints..., + selectOutpoints, ) if err != nil { return nil, err diff --git a/sweep/walletsweep.go b/sweep/walletsweep.go index f0ec70e7c..1fa2068ea 100644 --- a/sweep/walletsweep.go +++ b/sweep/walletsweep.go @@ -220,7 +220,7 @@ func CraftSweepAllTx(feeRate, maxFeeRate chainfee.SatPerKWeight, changeAddr btcutil.Address, coinSelectLocker CoinSelectionLocker, utxoSource UtxoSource, outputLeaser OutputLeaser, signer input.Signer, minConfs int32, - selectUtxos ...wire.OutPoint) (*WalletSweepPackage, error) { + selectUtxos []wire.OutPoint) (*WalletSweepPackage, error) { // TODO(roasbeef): turn off ATPL as well when available? diff --git a/sweep/walletsweep_test.go b/sweep/walletsweep_test.go index 8cc93c941..724b9d85d 100644 --- a/sweep/walletsweep_test.go +++ b/sweep/walletsweep_test.go @@ -346,7 +346,7 @@ func TestCraftSweepAllTxCoinSelectFail(t *testing.T) { _, err := CraftSweepAllTx( 0, 0, 10, nil, nil, coinSelectLocker, utxoSource, utxoLeaser, - nil, 0, + nil, 0, nil, ) // Since we instructed the coin select locker to fail above, we should @@ -372,7 +372,7 @@ func TestCraftSweepAllTxUnknownWitnessType(t *testing.T) { _, err := CraftSweepAllTx( 0, 0, 10, nil, nil, coinSelectLocker, utxoSource, utxoLeaser, - nil, 0, + nil, 0, nil, ) // Since passed in a p2wsh output, which is unknown, we should fail to @@ -461,7 +461,7 @@ func TestCraftSweepAllTx(t *testing.T) { sweepPkg, err := CraftSweepAllTx( 0, 0, 10, nil, deliveryAddr, coinSelectLocker, utxoSource, utxoLeaser, signer, 0, - tc.selectUtxos..., + tc.selectUtxos, ) if tc.errString != "" { require.ErrorContains(t, err, tc.errString)

yyforyongyu · 2024-06-14T08:40:00Z

sweep/walletsweep.go

@@ -260,6 +263,19 @@ func CraftSweepAllTx(feeRate, maxFeeRate chainfee.SatPerKWeight,

 		log.Trace("[WithCoinSelectLock] finished fetching UTXOs")

+		// Use select utxos, if provided.
+		if len(selectUtxos) > 0 {
+			if fn.HasDuplicates(selectUtxos) {


This check has been repeated multiple times - if it's already done in SendCoins so there's no need to check it here

CraftSweepAllTx is an independent function it is not bound to be used in sendcoins only, so I think it makes sense that it carries out this check as well

In that case, all other places should remove this check, there's no need to duplicate this logic and we should implement it once - easier to test and maintain.

A better option is to use set (map) - that way we don't even need to worry about duplicates.

Or maybe the Set structure in the fn package? What do you think?

yyforyongyu · 2024-06-14T08:42:12Z

sweep/walletsweep_test.go

+			selectUtxos: nil,
+		},
+		{
+			name: "sweep select utxos in wallet, no error",


nit: select -> selected, same below

select can actually be used as an adjective

yyforyongyu · 2024-06-14T08:47:43Z

cmd/lncli/commands.go

+	if ctx.IsSet("utxo") {
+		utxos := ctx.StringSlice("utxo")
+
+		if fn.HasDuplicates(utxos) {


This is checked again - we should enforce the single responsibility principle and have only one place to implement this logic.

prelim checks. I think I have seen this done a couple of places for lncli

In that case, we should also fix them in future PRs.

yyforyongyu · 2024-06-14T08:51:40Z

lntest/rpc/lnd.go

@@ -376,14 +376,15 @@ func (h *HarnessRPC) SendCoins(
 	return resp
 }

-// SendCoinsAssertErr sends a given amount of money to the specified address
-// from the passed node and asserts an error has returned.
-func (h *HarnessRPC) SendCoinsAssertErr(req *lnrpc.SendCoinsRequest) {


Can we stick to the XXXAssertErr pattern? That's the pattern used in all other assertions, OpenChannelAssertErr, ConnectPeerAssertErr, etc.

There is acctually a logic change in this function. Hence AssertErr might not be really descriptive

yyforyongyu · 2024-06-14T08:51:55Z

lntest/rpc/lnd.go

 	ctxt, cancel := context.WithTimeout(h.runCtx, DefaultTimeout)
 	defer cancel()

 	_, err := h.LN.SendCoins(ctxt, req)
-	require.Error(h, err, "node %s didn't not return an error", h.Name)


I think we can keep this assertion.

#8516 (comment)

yyforyongyu · 2024-06-14T08:53:33Z

go.mod

@@ -13,6 +13,7 @@ require (
 	github.com/btcsuite/btcwallet v0.16.10-0.20240404104514-b2f31f9045fb
 	github.com/btcsuite/btcwallet/wallet/txauthor v1.3.4
 	github.com/btcsuite/btcwallet/wallet/txrules v1.2.1
+	github.com/btcsuite/btcwallet/wallet/txsizes v1.2.4


hmmm how did you create this commit? Like what was the command used?

make tidy-module

yyforyongyu · 2024-06-14T09:12:37Z

itest/lnd_misc_test.go

-			Txid:      sweepHash[:],
-			Label:     "label that will not work",
-			Overwrite: false,
+	tcs := []struct {


I've laid out my view on table-driven test here and I think we should avoid it in itest.

In general, are you saying that if a function has more than two args we should not use TDD for it?

I like using table-driven tests when the outputs of a function are only determined by one or two inputs in a test setup,

It depends - but we should discourage using it in itest because it's difficult to maintain.

I am sorry I am having difficulty refactoring this to not use TDD. I am not sure what the non TDD version should look like. Would it be something like this?

similarfunction := func(t *testing.T, req *sendcoinsRequest, someconditionalsthatwecheck){ ---- } // With select utxos ht.Run(){ similarfunction() } // With no select utxos ht.Run(){ similarfunction() }

Would I need to refactor testsendcoinsselectutxos as well?

Chinwendu20 · 2024-06-14T10:09:43Z

@yyforyongyu I think we do that already? I responded to that in the specific code piece you wrote the review in. Let me know if that works.

Signed-off-by: Ononiwu Maureen <59079323+Chinwendu20@users.noreply.github.com>

This commit changes `SendCoinsAssertErr` to `SendCoinsReturnErr`. The function now returns the resulting error from calling `sendCoins`. Signed-off-by: Ononiwu Maureen <59079323+Chinwendu20@users.noreply.github.com>

Signed-off-by: Ononiwu Maureen <59079323+Chinwendu20@users.noreply.github.com>

lightninglabs-deploy · 2024-08-05T12:48:34Z

@yyforyongyu: review reminder
@Chinwendu20, remember to re-request review from reviewers when ready

guggero · 2024-08-05T12:52:01Z

Replaced by #8955.

Chinwendu20 requested review from yyforyongyu, ziggie1984, Roasbeef, Crypt-iQ and ellemouton as code owners March 4, 2024 05:51

Chinwendu20 mentioned this pull request Mar 4, 2024

Use selected Utxos while crafting txn btcsuite/btcwallet#912

Merged

Chinwendu20 force-pushed the sendcoins branch from 5b80b46 to 7f7b064 Compare March 4, 2024 06:15

hieblmi assigned Chinwendu20 Mar 4, 2024

guggero requested review from guggero and removed request for Roasbeef, yyforyongyu and ziggie1984 March 4, 2024 08:44

Chinwendu20 commented Mar 4, 2024

View reviewed changes

sweep/walletsweep.go Outdated Show resolved Hide resolved

Chinwendu20 marked this pull request as draft March 4, 2024 11:05

Chinwendu20 commented Mar 5, 2024

View reviewed changes

itest/lnd_channel_funding_fund_max_test.go Outdated Show resolved Hide resolved

ellemouton requested review from sputn1ck and removed request for Crypt-iQ and ellemouton March 11, 2024 13:42

guggero removed their request for review March 13, 2024 16:30

bennyhodl mentioned this pull request Mar 27, 2024

Specify OutPoint in SendCoinsRequest Synota-io/tonic_lnd#12

Draft

Chinwendu20 force-pushed the sendcoins branch 3 times, most recently from c800df5 to 3e6ab6a Compare March 29, 2024 11:27

Chinwendu20 force-pushed the sendcoins branch 2 times, most recently from b6bede7 to 3677d4b Compare June 7, 2024 10:15

hieblmi approved these changes Jun 7, 2024

View reviewed changes

Chinwendu20 force-pushed the sendcoins branch from 3677d4b to 6266698 Compare June 7, 2024 12:28

saubyk added this to the v0.18.1 milestone Jun 7, 2024

lnrpc: Add Outpoint field in SendCoinsRequest

8aa947b

Signed-off-by: Ononiwu Maureen <59079323+Chinwendu20@users.noreply.github.com>

Chinwendu20 force-pushed the sendcoins branch 2 times, most recently from 23df67f to 7065b64 Compare June 12, 2024 14:00

yyforyongyu requested changes Jun 14, 2024

View reviewed changes

saubyk removed the request for review from sputn1ck June 17, 2024 03:25

Chinwendu20 added 9 commits June 17, 2024 07:33

lntest+lnwallet: Adapt walletctrl, UTXO sendcoins

ff63676

Signed-off-by: Ononiwu Maureen <59079323+Chinwendu20@users.noreply.github.com>

sweep: Add selectUtxos to CraftSweepAllTx args

a329592

Signed-off-by: Ononiwu Maureen <59079323+Chinwendu20@users.noreply.github.com>

cmd: Export and move utxosToOutpoint.

577e472

Signed-off-by: Ononiwu Maureen <59079323+Chinwendu20@users.noreply.github.com>

lnd: modify sendcoin rpc impl for select utxos

2b72b48

Signed-off-by: Ononiwu Maureen <59079323+Chinwendu20@users.noreply.github.com>

lncli: Add outpoints flag to sendcoins command

1e91cb8

Signed-off-by: Ononiwu Maureen <59079323+Chinwendu20@users.noreply.github.com>

itest+lntest: Change SendCoinsAssertErr

5e9521e

This commit changes `SendCoinsAssertErr` to `SendCoinsReturnErr`. The function now returns the resulting error from calling `sendCoins`. Signed-off-by: Ononiwu Maureen <59079323+Chinwendu20@users.noreply.github.com>

itest: Update itest with sendcoins with utxo itest

0093067

Signed-off-by: Ononiwu Maureen <59079323+Chinwendu20@users.noreply.github.com>

gomod: fix lint

f4ded8d

Signed-off-by: Ononiwu Maureen <59079323+Chinwendu20@users.noreply.github.com>

docs: Added release note

16a45f6

Signed-off-by: Ononiwu Maureen <59079323+Chinwendu20@users.noreply.github.com>

Chinwendu20 force-pushed the sendcoins branch from 7065b64 to 16a45f6 Compare June 18, 2024 07:25

saubyk added the P3 might get fixed, nice to have label Jun 25, 2024

Chinwendu20 requested a review from yyforyongyu July 15, 2024 12:13

yyforyongyu mentioned this pull request Jul 30, 2024

Allow selecting coins in sendcoins #8955

Merged

saubyk removed this from the v0.18.3 milestone Aug 1, 2024

guggero closed this Aug 5, 2024

Simplify coin selection for sendcoins #8516

Simplify coin selection for sendcoins #8516

Conversation

Chinwendu20 commented Mar 4, 2024 • edited Loading

Change Description

Steps to Test

Pull Request Checklist

Testing

Code Style and Documentation

coderabbitai bot commented Mar 4, 2024 • edited Loading

Review skipped

Chat

CodeRabbit Commands (invoked as PR comments)

CodeRabbit Configration File (.coderabbit.yaml)

Documentation and Community

Chinwendu20 commented Mar 4, 2024

Chinwendu20 commented Mar 4, 2024 • edited Loading

Chinwendu20 commented Mar 5, 2024

bennyhodl commented Mar 12, 2024

bennyhodl commented Mar 26, 2024

Chinwendu20 commented Mar 26, 2024

hieblmi left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Chinwendu20 commented Jun 13, 2024

hieblmi commented Jun 13, 2024

yyforyongyu left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Chinwendu20 commented Jun 14, 2024

lightninglabs-deploy commented Aug 5, 2024

guggero commented Aug 5, 2024

Chinwendu20 commented Mar 4, 2024 •

edited

Loading

coderabbitai bot commented Mar 4, 2024 •

edited

Loading

CodeRabbit Configration File (`.coderabbit.yaml`)

Chinwendu20 commented Mar 4, 2024 •

edited

Loading