Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Funds not returned to internal wallet upon channel closure #793

Closed
newbold opened this issue Jan 26, 2018 · 7 comments
Closed

Funds not returned to internal wallet upon channel closure #793

newbold opened this issue Jan 26, 2018 · 7 comments

Comments

@newbold
Copy link

newbold commented Jan 26, 2018

Closed channel 505809:1198:1, waited 144 blocks, but the channel funds do not appear when running listfunds. I do see the transaction that should have returned them in the blockchain, but my node doesn't appear to be aware of it.

Working with @cdecker in Slack, I've tried bumping up the bip32_max_index value (UPDATE vars SET val=val+1 WHERE name='bip32_max_index';) and restarting the node, but no luck so far.

(To be clear, not worried about the potentially lost funds, just want to help with testing/debugging!)
@rustyrussell
Copy link
Contributor

BTW, smartbit.com.au gives a nicer result for segwit txs.

In this case, that tx is a p2wsh output: it's the responsibility of onchaind not the wallet code. If you run with --log-level=debug and grep for onchaind, it should spell out exactly what it's doing...

@newbold
Copy link
Author

newbold commented Jan 27, 2018 •
edited by cdecker
Loading

Thanks for the smartbit.com.au tip -- that's super nice. Switching as of now. 😄

OK, grepping for onchaind in the log, I see this:

lightning_onchaind(17291): TRACE: Tracking output 0 of e3766a049556fda654477c8a02cff875d2de8175f6d59db9d3a6c082031086f4: OUR_UNILATERAL/OUR_HTLC

That happens to be the txid of the returned funds that are missing! So this is starting to make more sense now. Below that, there's this:

lightning_onchaind(17291): TRACE: backtrace: common/subdaemon.c:33 (crashdump) 0x40e322
lightning_onchaind(17291): TRACE: backtrace: (null):0 ((null)) 0x7f18ddeb04af
lightning_onchaind(17291): TRACE: backtrace: (null):0 ((null)) 0x7f18ddeb0428
lightning_onchaind(17291): TRACE: backtrace: (null):0 ((null)) 0x7f18ddeb2029
lightning_onchaind(17291): TRACE: backtrace: src/secp256k1.c:36 (default_illegal_callback_fn) 0x4377e8
lightning_onchaind(17291): TRACE: backtrace: src/util.h:24 (secp256k1_callback_call) 0x4424d3
lightning_onchaind(17291): TRACE: backtrace: src/secp256k1.c:321 (secp256k1_ecdsa_verify) 0x4424d3
lightning_onchaind(17291): TRACE: backtrace: bitcoin/signature.c:134 (check_signed_hash) 0x4176e6
lightning_onchaind(17291): TRACE: backtrace: bitcoin/signature.c:153 (check_tx_sig) 0x41779a
lightning_onchaind(17291): TRACE: backtrace: onchaind/onchain.c:176 (grind_feerate) 0x402745
lightning_onchaind(17291): TRACE: backtrace: onchaind/onchain.c:1050 (resolve_our_htlc_ourcommit) 0x404a95
lightning_onchaind(17291): TRACE: backtrace: onchaind/onchain.c:1375 (handle_our_unilateral) 0x40574c
lightning_onchaind(17291): TRACE: backtrace: onchaind/onchain.c:2076 (main) 0x407334
lightning_onchaind(17291): TRACE: backtrace: (null):0 ((null)) 0x7f18dde9b82f
lightning_onchaind(17291): TRACE: backtrace: (null):0 ((null)) 0x402338
lightning_onchaind(17291): TRACE: backtrace: (null):0 ((null)) 0xffffffffffffffff
lightning_onchaind(17291): STATUS_FAIL_INTERNAL_ERROR: FATAL SIGNAL 6
lightningd(16557): peer 035f1498c929d4cefba4701ae36a554691f526ff60b1766badd5a49b3c8b68e1d8: Peer transient failure in ONCHAIND_OUR_UNILATERAL: Owning subdaemon lightning_onchaind died (33536)

I guess the process that should catch the output and put it in my wallet is dying, but I'm not sure why. Glad to test anything else if you have any suggestions.

@cdecker
Copy link
Member

cdecker commented Jan 27, 2018

It appears we are assuming the struct tracked_output *out has a valid remote_htlc_sig, when it doesn't. Checking the length of the remote_htlc_sigs array it's empty. In addition I think we should be selecting a specific sig instead of passing in the entire array here:

lightning/onchaind/onchain.c

Lines 1368 to 1375 in 1da27c5

out = new_tracked_output(&outs, txid,
tx_blockheight,
OUR_UNILATERAL, i,
tx->output[i].amount,
OUR_HTLC,
&htlcs[j], htlc_scripts[j],
remote_htlc_sigs);
resolve_our_htlc_ourcommit(out);

Not really all that familiar with the onchaind code, @rustyrussell can probably add more context here.

@cdecker
Copy link
Member

cdecker commented Jan 27, 2018

Just as an additional datapoint, the full remaining balance in the channel is in an HTLC that has no preimage yet:

sqlite> select id, channel_htlc_id, direction, msatoshi, HEX(payment_hash), HEX(payment_key) from channel_htlcs;
1|0|1|141105110|BD0671BAE4CAA023AE70E4C30AEF3FA32D8DB63D2845822FBE2D74A37C759568|

@noledge
Copy link

noledge commented Feb 9, 2018

Have you been able to revive those funds so far?
I think I have the exact same problem here.
One Channel died about a week ago, I can see the transaction in the block explorer but it is not returned to my funds.

Instead, this happens:

2018-02-07T09:01:07.860Z lightningd(30945): peer 0387e3780a4325eb38421fb83000a6f6c0ffa4a69ea0c81db3f00e8e5015c9e8a1: Peer permanent failure in ONCHAIND_OUR_UNILATERAL: Funding transaction spent
2018-02-07T09:01:07.860Z lightningd(30945):      (tx 689ec3ffa49f19a52ff701d3a7bfcfa0b0bc425fe1a10a308089193fd747f4a2)
2018-02-07T09:01:07.862Z lightningd(30945): peer 0387e3780a4325eb38421fb83000a6f6c0ffa4a69ea0c81db3f00e8e5015c9e8a1: State changed from ONCHAIND_OUR_UNILATERAL to FUNDING_SPEND_SEEN
2018-02-07T09:01:08.059Z lightningd(30945): peer 0387e3780a4325eb38421fb83000a6f6c0ffa4a69ea0c81db3f00e8e5015c9e8a1: State changed from FUNDING_SPEND_SEEN to ONCHAIND_OUR_UNILATERAL
[libsecp256k1] illegal argument: sig != NULL
lightning_onchaind: Fatal signal 6
0x40e695 crashdump
        common/subdaemon.c:30
0x7f56c94994af ???
        ???:0
0x7f56c9499428 ???
        ???:0
0x7f56c949b029 ???
        ???:0
0x43db48 default_illegal_callback_fn
        src/secp256k1.c:41
0x449c3d secp256k1_callback_call
        src/util.h:24
0x449c3d secp256k1_ecdsa_verify
        src/secp256k1.c:310
0x417af4 check_signed_hash
        bitcoin/signature.c:134
0x417ba8 check_tx_sig
        bitcoin/signature.c:153
0x4027ad grind_feerate
        onchaind/onchain.c:176
0x404b3a resolve_our_htlc_ourcommit
        onchaind/onchain.c:1050
0x405811 handle_our_unilateral
        onchaind/onchain.c:1375
0x407451 main
        onchaind/onchain.c:2076
0x7f56c948482f ???
        ???:0
0x402388 ???
        ???:0
0xffffffffffffffff ???
        ???:0
2018-02-07T09:01:08.071Z lightningd(30945): lightning_onchaind(0387e3780a4325eb38421fb83000a6f6c0ffa4a69ea0c81db3f00e8e5015c9e8a1): STATUS_FAIL_INTERNAL_ERROR: FATAL SIGNAL 6
2018-02-07T09:01:08.071Z lightningd(30945): peer 0387e3780a4325eb38421fb83000a6f6c0ffa4a69ea0c81db3f00e8e5015c9e8a1: Peer transient failure in ONCHAIND_OUR_UNILATERAL: Owning subdaemon lightning_onchaind died (33536)

Also happens with latest commits
"version": "v0.5.2-2016-11-21-1904-g2a979a2",

@CryptoChronicus
Copy link

i have this exact same problem.
"lightning_onchaind(17291): STATUS_FAIL_INTERNAL_ERROR: FATAL SIGNAL 6"
channels been showing ONCHAIND_OUR_UNILATERAL for a long time.
Depth around 3329.

"Owning subdaemon lightning_onchaind died (33536)"
"funding tx announce ready, but peer state ONCHAIN_OUR_UNILATERAL owned by none"

etcetera.

I woulda just posted a log but I am away from the node right now, figured I ask if there ever was an obvious solution to this?

@cdecker cdecker added the state::stale This issue/PR has not seen any activity for a while, and has no actionable step. Will be closed soon label Jun 30, 2018
@noledge
Copy link

noledge commented Jul 31, 2018 •
edited
Loading

saw this being marked as "stale"

this still happens with the newest snapshot, if that is the question.

2018-07-31T10:53:43.824Z lightningd(9394): 0387e3780a4325eb38421fb83000a6f6c0ffa4a69ea0c81db3f00e8e5015c9e8a1 chan #2: Peer permanent failure in ONCHAIN: Funding transaction spent
2018-07-31T10:53:43.824Z lightningd(9394):       (tx 689ec3ffa49f19a52ff701d3a7bfcfa0b0bc425fe1a10a308089193fd747f4a2)
2018-07-31T10:53:43.827Z lightningd(9394): 0387e3780a4325eb38421fb83000a6f6c0ffa4a69ea0c81db3f00e8e5015c9e8a1 chan #2: State changed from ONCHAIN to FUNDING_SPEND_SEEN
2018-07-31T10:53:44.061Z lightningd(9394): 0387e3780a4325eb38421fb83000a6f6c0ffa4a69ea0c81db3f00e8e5015c9e8a1 chan #2: State changed from FUNDING_SPEND_SEEN to ONCHAIN
[libsecp256k1] illegal argument: sig != NULL
lightning_onchaind: Fatal signal 6 (version v0.5.2-2016-11-21-3145-g1fca7ab)
0x40a963 crashdump
        common/daemon.c:37
0x7f79e11da4af ???
        ???:0
0x7f79e11da428 ???
        ???:0
0x7f79e11dc029 ???
        ???:0
0x4466d8 default_illegal_callback_fn
        src/secp256k1.c:41
0x4527cd secp256k1_callback_call
        src/util.h:24
0x4527cd secp256k1_ecdsa_verify
        src/secp256k1.c:310
0x4180e1 check_signed_hash
        bitcoin/signature.c:134
0x418195 check_tx_sig
        bitcoin/signature.c:153
0x4026ff grind_htlc_tx_fee
        onchaind/onchain.c:141
0x4027fa set_htlc_timeout_fee
        onchaind/onchain.c:172
0x405735 resolve_our_htlc_ourcommit
        onchaind/onchain.c:1328
0x406212 handle_our_unilateral
        onchaind/onchain.c:1606
0x407b56 main
        onchaind/onchain.c:2236
0x7f79e11c582f ???
        ???:0
0x402438 ???
        ???:0
0xffffffffffffffff ???
        ???:0
2018-07-31T10:53:44.090Z lightningd(9394): lightning_onchaind-0387e3780a4325eb38421fb83000a6f6c0ffa4a69ea0c81db3f00e8e5015c9e8a1 chan #2: FATAL SIGNAL 6 (version v0.5.2-2016-11-21-3145-g1fca7ab)
2018-07-31T10:53:44.091Z lightningd(9394): lightning_onchaind-0387e3780a4325eb38421fb83000a6f6c0ffa4a69ea0c81db3f00e8e5015c9e8a1 chan #2: backtrace: common/daemon.c:42 (crashdump) 0x40a9b3
2018-07-31T10:53:44.091Z lightningd(9394): lightning_onchaind-0387e3780a4325eb38421fb83000a6f6c0ffa4a69ea0c81db3f00e8e5015c9e8a1 chan #2: backtrace: (null):0 ((null)) 0x7f79e11da4af
2018-07-31T10:53:44.091Z lightningd(9394): lightning_onchaind-0387e3780a4325eb38421fb83000a6f6c0ffa4a69ea0c81db3f00e8e5015c9e8a1 chan #2: backtrace: (null):0 ((null)) 0x7f79e11da428
2018-07-31T10:53:44.091Z lightningd(9394): lightning_onchaind-0387e3780a4325eb38421fb83000a6f6c0ffa4a69ea0c81db3f00e8e5015c9e8a1 chan #2: backtrace: (null):0 ((null)) 0x7f79e11dc029
2018-07-31T10:53:44.091Z lightningd(9394): lightning_onchaind-0387e3780a4325eb38421fb83000a6f6c0ffa4a69ea0c81db3f00e8e5015c9e8a1 chan #2: backtrace: src/secp256k1.c:41 (default_illegal_callback_fn) 0x4466d8
2018-07-31T10:53:44.091Z lightningd(9394): lightning_onchaind-0387e3780a4325eb38421fb83000a6f6c0ffa4a69ea0c81db3f00e8e5015c9e8a1 chan #2: backtrace: src/util.h:24 (secp256k1_callback_call) 0x4527cd
2018-07-31T10:53:44.091Z lightningd(9394): lightning_onchaind-0387e3780a4325eb38421fb83000a6f6c0ffa4a69ea0c81db3f00e8e5015c9e8a1 chan #2: backtrace: src/secp256k1.c:310 (secp256k1_ecdsa_verify) 0x4527cd
2018-07-31T10:53:44.091Z lightningd(9394): lightning_onchaind-0387e3780a4325eb38421fb83000a6f6c0ffa4a69ea0c81db3f00e8e5015c9e8a1 chan #2: backtrace: bitcoin/signature.c:134 (check_signed_hash) 0x4180e1
2018-07-31T10:53:44.091Z lightningd(9394): lightning_onchaind-0387e3780a4325eb38421fb83000a6f6c0ffa4a69ea0c81db3f00e8e5015c9e8a1 chan #2: backtrace: bitcoin/signature.c:153 (check_tx_sig) 0x418195
2018-07-31T10:53:44.091Z lightningd(9394): lightning_onchaind-0387e3780a4325eb38421fb83000a6f6c0ffa4a69ea0c81db3f00e8e5015c9e8a1 chan #2: backtrace: onchaind/onchain.c:141 (grind_htlc_tx_fee) 0x4026ff
2018-07-31T10:53:44.091Z lightningd(9394): lightning_onchaind-0387e3780a4325eb38421fb83000a6f6c0ffa4a69ea0c81db3f00e8e5015c9e8a1 chan #2: backtrace: onchaind/onchain.c:172 (set_htlc_timeout_fee) 0x4027fa
2018-07-31T10:53:44.091Z lightningd(9394): lightning_onchaind-0387e3780a4325eb38421fb83000a6f6c0ffa4a69ea0c81db3f00e8e5015c9e8a1 chan #2: backtrace: onchaind/onchain.c:1328 (resolve_our_htlc_ourcommit) 0x405735
2018-07-31T10:53:44.091Z lightningd(9394): lightning_onchaind-0387e3780a4325eb38421fb83000a6f6c0ffa4a69ea0c81db3f00e8e5015c9e8a1 chan #2: backtrace: onchaind/onchain.c:1606 (handle_our_unilateral) 0x406212
2018-07-31T10:53:44.091Z lightningd(9394): lightning_onchaind-0387e3780a4325eb38421fb83000a6f6c0ffa4a69ea0c81db3f00e8e5015c9e8a1 chan #2: backtrace: onchaind/onchain.c:2236 (main) 0x407b56
2018-07-31T10:53:44.091Z lightningd(9394): lightning_onchaind-0387e3780a4325eb38421fb83000a6f6c0ffa4a69ea0c81db3f00e8e5015c9e8a1 chan #2: backtrace: (null):0 ((null)) 0x7f79e11c582f
2018-07-31T10:53:44.091Z lightningd(9394): lightning_onchaind-0387e3780a4325eb38421fb83000a6f6c0ffa4a69ea0c81db3f00e8e5015c9e8a1 chan #2: backtrace: (null):0 ((null)) 0x402438
2018-07-31T10:53:44.091Z lightningd(9394): lightning_onchaind-0387e3780a4325eb38421fb83000a6f6c0ffa4a69ea0c81db3f00e8e5015c9e8a1 chan #2: backtrace: (null):0 ((null)) 0xffffffffffffffff
2018-07-31T10:53:44.091Z lightningd(9394): lightning_onchaind-0387e3780a4325eb38421fb83000a6f6c0ffa4a69ea0c81db3f00e8e5015c9e8a1 chan #2: STATUS_FAIL_INTERNAL_ERROR: FATAL SIGNAL
2018-07-31T10:53:44.091Z lightningd(9394): 0387e3780a4325eb38421fb83000a6f6c0ffa4a69ea0c81db3f00e8e5015c9e8a1 chan #2: Owning subdaemon lightning_onchaind died (61952)





$ lightning/cli/lightning-cli listfunds | jq
{
  "outputs": [],
  "channels": [
    {
      "peer_id": "0387e3780a4325eb38421fb83000a6f6c0ffa4a69ea0c81db3f00e8e5015c9e8a1",
      "short_channel_id": "507106:1671:1",
      "channel_sat": 350000,
      "channel_total_sat": 350000,
      "funding_txid": "521de83883cac03c6d97063c70345ed0254aeb44cdc5ec1ab42ade092003b14a"
    }
  ]
}





$ lightning/cli/lightning-cli listpeers | jq
{
  "peers": [
    {
      "id": "0387e3780a4325eb38421fb83000a6f6c0ffa4a69ea0c81db3f00e8e5015c9e8a1",
      "connected": false,
      "channels": [
        {
          "state": "ONCHAIN",
          "short_channel_id": "507106:1671:1",
          "channel_id": "4ab1032009de2ab41aecc5cd44eb4a25d05e34703c06976d3cc0ca8338e81d53",
          "funding_txid": "521de83883cac03c6d97063c70345ed0254aeb44cdc5ec1ab42ade092003b14a",
          "msatoshi_to_us": 350000000,
          "msatoshi_to_us_min": 350000000,
          "msatoshi_to_us_max": 350000000,
          "msatoshi_total": 350000000,
          "dust_limit_satoshis": 546,
          "max_htlc_value_in_flight_msat": 18446744073709552000,
          "their_channel_reserve_satoshis": 0,
          "our_channel_reserve_satoshis": 3500,
          "spendable_msatoshi": 346500000,
          "htlc_minimum_msat": 0,
          "their_to_self_delay": 144,
          "our_to_self_delay": 144,
          "max_accepted_htlcs": 483,
          "status": [
            "ONCHAIN:Owning subdaemon lightning_onchaind died (61952)"
          ],
          "in_payments_offered": 0,
          "in_msatoshi_offered": 0,
          "in_payments_fulfilled": 0,
          "in_msatoshi_fulfilled": 0,
          "out_payments_offered": 0,
          "out_msatoshi_offered": 0,
          "out_payments_fulfilled": 0,
          "out_msatoshi_fulfilled": 0
        }
      ]
    }
  ]
}

@cdecker cdecker removed the state::stale This issue/PR has not seen any activity for a while, and has no actionable step. Will be closed soon label Aug 1, 2018
@cdecker cdecker closed this as completed Jan 10, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@cdecker @rustyrussell @newbold @noledge @CryptoChronicus