Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jasan/guilt configure rebased ammended #3

Closed
wants to merge 47 commits into from
Closed

Jasan/guilt configure rebased ammended #3

wants to merge 47 commits into from

Conversation

jsarenik
Copy link

@jsarenik jsarenik commented Jun 6, 2018

This PR is already rebased to current master.

rustyrussell and others added 30 commits June 6, 2018 03:25
We generate new ones anyway; removing this code changes fixes coming
up which now only need to change one place.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
…haviour.

The gossip-query spec enhancements say not to forward an channel_announcement until
you have receive a channel_update.  This test fails for now.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
…pdate.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
…roadcast.

handle_pending_cannouncement might not actually add the announcment,
as it could be waiting for a channel_update.  We need to wait for
the actual announcement before considering announcing our node.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
In general, we need to only publish node announcements after
publishing channel announcements, though we can accept node
announcements as soon as we see channel announcements.  So we keep a
flag for those node_announcement which haven't been broadcast yet.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
From BOLT #9 proposed update.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This includes the gossip query messages.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We use the same system as for gossip: we trickle out replies when we're
otherwise idle.

This is minimal infrastructure: we don't actually process the
query_short_channel_ids message yet, nor do we append node
announcements.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We use the same system as for gossip: we trickle out replies when we're
otherwise idle.

As we trickle out replies to query_short_channel_ids, we remember the
pubkeys of nodes we mention.  At the end, we sort and uniquify, and
then send any node_announcements we have for those.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This doesn't handle zlib yet.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
And write the test for it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This lets us filter by timestamp.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This is kind of orthogonal to the other changes, but makes sense: if we
would instantly or never prune the message, don't accept it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
And initialize filter (to "never") when we negotiated LOCAL_GOSSIP_QUERIES,
and send initial filter message.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We have a function called 'wake_pkt_out' which is really 'start
gossiping', so rename it to 'wake_gossip_out'.

In addition, it's fired both on a timer, and in response to our first
gossip_timestamp_filter, which leads to very confusing (though,
technically, not incorrect) behavior.

Keep a single timer at all times, which now doubles as the flag to
indicating we're syncing right now.  Set it once we're done syncing
gossip.

Technically this means we got from once-every-60-seconds to
quiet-for-60-seconds-between-gossip, but that's OK.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
…filtering.

Since we currently only (ab)use it to send everything, we need a way to
generate boutique queries for testing.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We send them all the short_channel_ids we have in a given range.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We keep a crappy bitmap, and finish when their replies cover
everything we asked.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
…eplies.

We cap each reply at a single one, which forces the code into our
recursion logic.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We still use uncompressed if zlib turns out to be larger.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Previous replies weren't large enough; add another channel and then
use IO tracing to make sure the reply is zlib encoded.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We *accept* a node_announce if we have a channel_announce, but we
can't queue it until we queue the channel_announce, which we only do
once we have recieved a channel_update.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Suggested-by: @cdecker
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
The whole test is under DEVELOPER anyway, but even if it weren't it
would work fine.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Suggested-by: @cdecker
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Signed-off-by: Christian Decker <decker.christian@gmail.com>
rustyrussell added a commit that referenced this pull request Jun 12, 2019
Direct leak of 16 byte(s) in 1 object(s) allocated from:
    #0 0x7ff02889063e in calloc (/lib/x86_64-linux-gnu/libasan.so.5+0x10c63e)
    #1 0x555ce2ad8d2e in htable_default_alloc ccan/ccan/htable/htable.c:19
    #2 0x555ce2ad9698 in double_table ccan/ccan/htable/htable.c:226
    #3 0x555ce2ad9b62 in htable_add_ ccan/ccan/htable/htable.c:331
    #4 0x555ce2a638e4 in htlc_in_map_add lightningd/htlc_end.h:113
    #5 0x555ce2a63beb in connect_htlc_in lightningd/htlc_end.c:39
    #6 0x555ce2a85cbc in channel_added_their_htlc lightningd/peer_htlcs.c:1382
    #7 0x555ce2a860e1 in peer_got_commitsig lightningd/peer_htlcs.c:1466
    #8 0x555ce2a5db04 in channel_msg lightningd/channel_control.c:228
    #9 0x555ce2a8d393 in sd_msg_read lightningd/subd.c:474
    #10 0x555ce2ada157 in next_plan ccan/ccan/io/io.c:59
    #11 0x555ce2adacd4 in do_plan ccan/ccan/io/io.c:407
    #12 0x555ce2adad12 in io_ready ccan/ccan/io/io.c:417
    #13 0x555ce2adcd67 in io_loop ccan/ccan/io/poll.c:445
    ElementsProject#14 0x555ce2a67c66 in io_loop_with_timers lightningd/io_loop_with_timers.c:24
    ElementsProject#15 0x555ce2a6e56b in main lightningd/lightningd.c:822
    ElementsProject#16 0x7ff028242b6a in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x26b6a)

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
rustyrussell added a commit that referenced this pull request Jun 12, 2019
Direct leak of 1024 byte(s) in 2 object(s) allocated from:
    #0 0x7f4c84ce4448 in malloc (/lib/x86_64-linux-gnu/libasan.so.5+0x10c448)
    #1 0x55d11b782c96 in timer_default_alloc ccan/ccan/timer/timer.c:16
    #2 0x55d11b7832b7 in add_level ccan/ccan/timer/timer.c:166
    #3 0x55d11b783864 in timer_fast_forward ccan/ccan/timer/timer.c:334
    #4 0x55d11b78396a in timers_expire ccan/ccan/timer/timer.c:359
    #5 0x55d11b774993 in io_loop ccan/ccan/io/poll.c:395
    #6 0x55d11b72322f in plugins_init lightningd/plugin.c:1013
    #7 0x55d11b7060ea in main lightningd/lightningd.c:664
    #8 0x7f4c84696b6a in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x26b6a)

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
rustyrussell added a commit that referenced this pull request Jun 12, 2019
Indirect leak of 48 byte(s) in 1 object(s) allocated from:
    #0 0x7f4c84ce4448 in malloc (/lib/x86_64-linux-gnu/libasan.so.5+0x10c448)
    #1 0x55d11b77d270 in strmap_add_ ccan/ccan/strmap/strmap.c:90
    #2 0x55d11b704603 in command_set_usage lightningd/jsonrpc.c:891
    #3 0x55d11b733cb5 in param common/param.c:295
    #4 0x55d11b6f7b37 in json_connect lightningd/connect_control.c:96
    #5 0x55d11b7042ef in setup_command_usage lightningd/jsonrpc.c:841
    #6 0x55d11b70443b in jsonrpc_command_add_perm lightningd/jsonrpc.c:863
    #7 0x55d11b704533 in jsonrpc_setup lightningd/jsonrpc.c:876
    #8 0x55d11b705695 in new_lightningd lightningd/lightningd.c:210
    #9 0x55d11b706062 in main lightningd/lightningd.c:644
    #10 0x7f4c84696b6a in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x26b6a)

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
rustyrussell added a commit that referenced this pull request Jun 12, 2019
Direct leak of 64 byte(s) in 1 object(s) allocated from:
    #0 0x7f4dc279163e in calloc (/lib/x86_64-linux-gnu/libasan.so.5+0x10c63e)
    #1 0x564ee8a24bb1 in htable_default_alloc ccan/ccan/htable/htable.c:19
    #2 0x564ee8a2551b in double_table ccan/ccan/htable/htable.c:226
    #3 0x564ee8a259e5 in htable_add_ ccan/ccan/htable/htable.c:331
    #4 0x564ee89a5300 in block_map_add lightningd/chaintopology.h:83
    #5 0x564ee89a6ece in add_tip lightningd/chaintopology.c:626
    #6 0x564ee89a72c3 in have_new_block lightningd/chaintopology.c:694
    #7 0x564ee89a3ab0 in process_rawblock lightningd/bitcoind.c:466
    #8 0x564ee89a2fb4 in bcli_finished lightningd/bitcoind.c:214
    #9 0x564ee8a284d6 in destroy_conn ccan/ccan/io/poll.c:244
    #10 0x564ee8a284f6 in destroy_conn_close_fd ccan/ccan/io/poll.c:250
    #11 0x564ee8a34a0d in notify ccan/ccan/tal/tal.c:235
    #12 0x564ee8a34efc in del_tree ccan/ccan/tal/tal.c:397
    #13 0x564ee8a35288 in tal_free ccan/ccan/tal/tal.c:481
    ElementsProject#14 0x564ee8a26cf5 in io_close ccan/ccan/io/io.c:450
    ElementsProject#15 0x564ee8a28c11 in io_loop ccan/ccan/io/poll.c:449
    ElementsProject#16 0x564ee89b3c3b in io_loop_with_timers lightningd/io_loop_with_timers.c:24
    ElementsProject#17 0x564ee89ba540 in main lightningd/lightningd.c:822
    ElementsProject#18 0x7f4dc2143b6a in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x26b6a)

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
rustyrussell added a commit that referenced this pull request Jun 12, 2019
Direct leak of 32 byte(s) in 1 object(s) allocated from:
    #0 0x7f7678ee863e in calloc (/lib/x86_64-linux-gnu/libasan.so.5+0x10c63e)
    #1 0x55f8c7b0fce5 in htable_default_alloc ccan/ccan/htable/htable.c:19
    #2 0x55f8c7b1064f in double_table ccan/ccan/htable/htable.c:226
    #3 0x55f8c7b10b19 in htable_add_ ccan/ccan/htable/htable.c:331
    #4 0x55f8c7afac63 in scriptpubkeyset_add wallet/txfilter.c:30
    #5 0x55f8c7afafce in txfilter_add_scriptpubkey wallet/txfilter.c:77
    #6 0x55f8c7afb05f in txfilter_add_derkey wallet/txfilter.c:91
    #7 0x55f8c7aa4d67 in init_txfilter lightningd/lightningd.c:482
    #8 0x55f8c7aa52d8 in main lightningd/lightningd.c:721
    #9 0x7f767889ab6a in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x26b6a)

Direct leak of 16 byte(s) in 1 object(s) allocated from:
    #0 0x7f05f389563e in calloc (/lib/x86_64-linux-gnu/libasan.so.5+0x10c63e)
    #1 0x55cac1e6bc99 in htable_default_alloc ccan/ccan/htable/htable.c:19
    #2 0x55cac1e6c603 in double_table ccan/ccan/htable/htable.c:226
    #3 0x55cac1e6cacd in htable_add_ ccan/ccan/htable/htable.c:331
    #4 0x55cac1e56e48 in outpointset_add wallet/txfilter.c:61
    #5 0x55cac1e57162 in outpointfilter_add wallet/txfilter.c:116
    #6 0x55cac1e5ea3a in wallet_utxoset_add wallet/wallet.c:2365
    #7 0x55cac1deddc2 in topo_add_utxos lightningd/chaintopology.c:603
    #8 0x55cac1dedeac in add_tip lightningd/chaintopology.c:620
    #9 0x55cac1dee2de in have_new_block lightningd/chaintopology.c:694
    #10 0x55cac1deaab0 in process_rawblock lightningd/bitcoind.c:466
    #11 0x55cac1de9fb4 in bcli_finished lightningd/bitcoind.c:214
    #12 0x55cac1e6f5be in destroy_conn ccan/ccan/io/poll.c:244
    #13 0x55cac1e6f5de in destroy_conn_close_fd ccan/ccan/io/poll.c:250
    ElementsProject#14 0x55cac1e7baf5 in notify ccan/ccan/tal/tal.c:235
    ElementsProject#15 0x55cac1e7bfe4 in del_tree ccan/ccan/tal/tal.c:397
    ElementsProject#16 0x55cac1e7c370 in tal_free ccan/ccan/tal/tal.c:481
    ElementsProject#17 0x55cac1e6dddd in io_close ccan/ccan/io/io.c:450
    ElementsProject#18 0x55cac1e6fcf9 in io_loop ccan/ccan/io/poll.c:449
    ElementsProject#19 0x55cac1dfac66 in io_loop_with_timers lightningd/io_loop_with_timers.c:24
    ElementsProject#20 0x55cac1e0156b in main lightningd/lightningd.c:822
    ElementsProject#21 0x7f05f3247b6a in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x26b6a)

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
rustyrussell added a commit that referenced this pull request Jun 12, 2019
Direct leak of 16 byte(s) in 1 object(s) allocated from:
    #0 0x7ff02889063e in calloc (/lib/x86_64-linux-gnu/libasan.so.5+0x10c63e)
    #1 0x555ce2ad8d2e in htable_default_alloc ccan/ccan/htable/htable.c:19
    #2 0x555ce2ad9698 in double_table ccan/ccan/htable/htable.c:226
    #3 0x555ce2ad9b62 in htable_add_ ccan/ccan/htable/htable.c:331
    #4 0x555ce2a638e4 in htlc_in_map_add lightningd/htlc_end.h:113
    #5 0x555ce2a63beb in connect_htlc_in lightningd/htlc_end.c:39
    #6 0x555ce2a85cbc in channel_added_their_htlc lightningd/peer_htlcs.c:1382
    #7 0x555ce2a860e1 in peer_got_commitsig lightningd/peer_htlcs.c:1466
    #8 0x555ce2a5db04 in channel_msg lightningd/channel_control.c:228
    #9 0x555ce2a8d393 in sd_msg_read lightningd/subd.c:474
    #10 0x555ce2ada157 in next_plan ccan/ccan/io/io.c:59
    #11 0x555ce2adacd4 in do_plan ccan/ccan/io/io.c:407
    #12 0x555ce2adad12 in io_ready ccan/ccan/io/io.c:417
    #13 0x555ce2adcd67 in io_loop ccan/ccan/io/poll.c:445
    ElementsProject#14 0x555ce2a67c66 in io_loop_with_timers lightningd/io_loop_with_timers.c:24
    ElementsProject#15 0x555ce2a6e56b in main lightningd/lightningd.c:822
    ElementsProject#16 0x7ff028242b6a in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x26b6a)

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
rustyrussell added a commit that referenced this pull request Jun 14, 2019
Direct leak of 1024 byte(s) in 2 object(s) allocated from:
    #0 0x7f4c84ce4448 in malloc (/lib/x86_64-linux-gnu/libasan.so.5+0x10c448)
    #1 0x55d11b782c96 in timer_default_alloc ccan/ccan/timer/timer.c:16
    #2 0x55d11b7832b7 in add_level ccan/ccan/timer/timer.c:166
    #3 0x55d11b783864 in timer_fast_forward ccan/ccan/timer/timer.c:334
    #4 0x55d11b78396a in timers_expire ccan/ccan/timer/timer.c:359
    #5 0x55d11b774993 in io_loop ccan/ccan/io/poll.c:395
    #6 0x55d11b72322f in plugins_init lightningd/plugin.c:1013
    #7 0x55d11b7060ea in main lightningd/lightningd.c:664
    #8 0x7f4c84696b6a in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x26b6a)

To fix this, we actually make 'ld->timers' a pointer, so we can clean
it up last of all.  We can't free it before ld, because that causes
timers to be destroyed.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
rustyrussell added a commit that referenced this pull request Jun 14, 2019
Indirect leak of 48 byte(s) in 1 object(s) allocated from:
    #0 0x7f4c84ce4448 in malloc (/lib/x86_64-linux-gnu/libasan.so.5+0x10c448)
    #1 0x55d11b77d270 in strmap_add_ ccan/ccan/strmap/strmap.c:90
    #2 0x55d11b704603 in command_set_usage lightningd/jsonrpc.c:891
    #3 0x55d11b733cb5 in param common/param.c:295
    #4 0x55d11b6f7b37 in json_connect lightningd/connect_control.c:96
    #5 0x55d11b7042ef in setup_command_usage lightningd/jsonrpc.c:841
    #6 0x55d11b70443b in jsonrpc_command_add_perm lightningd/jsonrpc.c:863
    #7 0x55d11b704533 in jsonrpc_setup lightningd/jsonrpc.c:876
    #8 0x55d11b705695 in new_lightningd lightningd/lightningd.c:210
    #9 0x55d11b706062 in main lightningd/lightningd.c:644
    #10 0x7f4c84696b6a in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x26b6a)

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
rustyrussell added a commit that referenced this pull request Jun 14, 2019
Direct leak of 64 byte(s) in 1 object(s) allocated from:
    #0 0x7f4dc279163e in calloc (/lib/x86_64-linux-gnu/libasan.so.5+0x10c63e)
    #1 0x564ee8a24bb1 in htable_default_alloc ccan/ccan/htable/htable.c:19
    #2 0x564ee8a2551b in double_table ccan/ccan/htable/htable.c:226
    #3 0x564ee8a259e5 in htable_add_ ccan/ccan/htable/htable.c:331
    #4 0x564ee89a5300 in block_map_add lightningd/chaintopology.h:83
    #5 0x564ee89a6ece in add_tip lightningd/chaintopology.c:626
    #6 0x564ee89a72c3 in have_new_block lightningd/chaintopology.c:694
    #7 0x564ee89a3ab0 in process_rawblock lightningd/bitcoind.c:466
    #8 0x564ee89a2fb4 in bcli_finished lightningd/bitcoind.c:214
    #9 0x564ee8a284d6 in destroy_conn ccan/ccan/io/poll.c:244
    #10 0x564ee8a284f6 in destroy_conn_close_fd ccan/ccan/io/poll.c:250
    #11 0x564ee8a34a0d in notify ccan/ccan/tal/tal.c:235
    #12 0x564ee8a34efc in del_tree ccan/ccan/tal/tal.c:397
    #13 0x564ee8a35288 in tal_free ccan/ccan/tal/tal.c:481
    ElementsProject#14 0x564ee8a26cf5 in io_close ccan/ccan/io/io.c:450
    ElementsProject#15 0x564ee8a28c11 in io_loop ccan/ccan/io/poll.c:449
    ElementsProject#16 0x564ee89b3c3b in io_loop_with_timers lightningd/io_loop_with_timers.c:24
    ElementsProject#17 0x564ee89ba540 in main lightningd/lightningd.c:822
    ElementsProject#18 0x7f4dc2143b6a in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x26b6a)

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
rustyrussell added a commit that referenced this pull request Jun 14, 2019
Direct leak of 32 byte(s) in 1 object(s) allocated from:
    #0 0x7f7678ee863e in calloc (/lib/x86_64-linux-gnu/libasan.so.5+0x10c63e)
    #1 0x55f8c7b0fce5 in htable_default_alloc ccan/ccan/htable/htable.c:19
    #2 0x55f8c7b1064f in double_table ccan/ccan/htable/htable.c:226
    #3 0x55f8c7b10b19 in htable_add_ ccan/ccan/htable/htable.c:331
    #4 0x55f8c7afac63 in scriptpubkeyset_add wallet/txfilter.c:30
    #5 0x55f8c7afafce in txfilter_add_scriptpubkey wallet/txfilter.c:77
    #6 0x55f8c7afb05f in txfilter_add_derkey wallet/txfilter.c:91
    #7 0x55f8c7aa4d67 in init_txfilter lightningd/lightningd.c:482
    #8 0x55f8c7aa52d8 in main lightningd/lightningd.c:721
    #9 0x7f767889ab6a in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x26b6a)

Direct leak of 16 byte(s) in 1 object(s) allocated from:
    #0 0x7f05f389563e in calloc (/lib/x86_64-linux-gnu/libasan.so.5+0x10c63e)
    #1 0x55cac1e6bc99 in htable_default_alloc ccan/ccan/htable/htable.c:19
    #2 0x55cac1e6c603 in double_table ccan/ccan/htable/htable.c:226
    #3 0x55cac1e6cacd in htable_add_ ccan/ccan/htable/htable.c:331
    #4 0x55cac1e56e48 in outpointset_add wallet/txfilter.c:61
    #5 0x55cac1e57162 in outpointfilter_add wallet/txfilter.c:116
    #6 0x55cac1e5ea3a in wallet_utxoset_add wallet/wallet.c:2365
    #7 0x55cac1deddc2 in topo_add_utxos lightningd/chaintopology.c:603
    #8 0x55cac1dedeac in add_tip lightningd/chaintopology.c:620
    #9 0x55cac1dee2de in have_new_block lightningd/chaintopology.c:694
    #10 0x55cac1deaab0 in process_rawblock lightningd/bitcoind.c:466
    #11 0x55cac1de9fb4 in bcli_finished lightningd/bitcoind.c:214
    #12 0x55cac1e6f5be in destroy_conn ccan/ccan/io/poll.c:244
    #13 0x55cac1e6f5de in destroy_conn_close_fd ccan/ccan/io/poll.c:250
    ElementsProject#14 0x55cac1e7baf5 in notify ccan/ccan/tal/tal.c:235
    ElementsProject#15 0x55cac1e7bfe4 in del_tree ccan/ccan/tal/tal.c:397
    ElementsProject#16 0x55cac1e7c370 in tal_free ccan/ccan/tal/tal.c:481
    ElementsProject#17 0x55cac1e6dddd in io_close ccan/ccan/io/io.c:450
    ElementsProject#18 0x55cac1e6fcf9 in io_loop ccan/ccan/io/poll.c:449
    ElementsProject#19 0x55cac1dfac66 in io_loop_with_timers lightningd/io_loop_with_timers.c:24
    ElementsProject#20 0x55cac1e0156b in main lightningd/lightningd.c:822
    ElementsProject#21 0x7f05f3247b6a in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x26b6a)

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
rustyrussell added a commit that referenced this pull request Jun 14, 2019
Direct leak of 16 byte(s) in 1 object(s) allocated from:
    #0 0x7ff02889063e in calloc (/lib/x86_64-linux-gnu/libasan.so.5+0x10c63e)
    #1 0x555ce2ad8d2e in htable_default_alloc ccan/ccan/htable/htable.c:19
    #2 0x555ce2ad9698 in double_table ccan/ccan/htable/htable.c:226
    #3 0x555ce2ad9b62 in htable_add_ ccan/ccan/htable/htable.c:331
    #4 0x555ce2a638e4 in htlc_in_map_add lightningd/htlc_end.h:113
    #5 0x555ce2a63beb in connect_htlc_in lightningd/htlc_end.c:39
    #6 0x555ce2a85cbc in channel_added_their_htlc lightningd/peer_htlcs.c:1382
    #7 0x555ce2a860e1 in peer_got_commitsig lightningd/peer_htlcs.c:1466
    #8 0x555ce2a5db04 in channel_msg lightningd/channel_control.c:228
    #9 0x555ce2a8d393 in sd_msg_read lightningd/subd.c:474
    #10 0x555ce2ada157 in next_plan ccan/ccan/io/io.c:59
    #11 0x555ce2adacd4 in do_plan ccan/ccan/io/io.c:407
    #12 0x555ce2adad12 in io_ready ccan/ccan/io/io.c:417
    #13 0x555ce2adcd67 in io_loop ccan/ccan/io/poll.c:445
    ElementsProject#14 0x555ce2a67c66 in io_loop_with_timers lightningd/io_loop_with_timers.c:24
    ElementsProject#15 0x555ce2a6e56b in main lightningd/lightningd.c:822
    ElementsProject#16 0x7ff028242b6a in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x26b6a)

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
rustyrussell added a commit that referenced this pull request Jun 14, 2019
Direct leak of 64 byte(s) in 1 object(s) allocated from:
    #0 0x7f4dc279163e in calloc (/lib/x86_64-linux-gnu/libasan.so.5+0x10c63e)
    #1 0x564ee8a24bb1 in htable_default_alloc ccan/ccan/htable/htable.c:19
    #2 0x564ee8a2551b in double_table ccan/ccan/htable/htable.c:226
    #3 0x564ee8a259e5 in htable_add_ ccan/ccan/htable/htable.c:331
    #4 0x564ee89a5300 in block_map_add lightningd/chaintopology.h:83
    #5 0x564ee89a6ece in add_tip lightningd/chaintopology.c:626
    #6 0x564ee89a72c3 in have_new_block lightningd/chaintopology.c:694
    #7 0x564ee89a3ab0 in process_rawblock lightningd/bitcoind.c:466
    #8 0x564ee89a2fb4 in bcli_finished lightningd/bitcoind.c:214
    #9 0x564ee8a284d6 in destroy_conn ccan/ccan/io/poll.c:244
    #10 0x564ee8a284f6 in destroy_conn_close_fd ccan/ccan/io/poll.c:250
    #11 0x564ee8a34a0d in notify ccan/ccan/tal/tal.c:235
    #12 0x564ee8a34efc in del_tree ccan/ccan/tal/tal.c:397
    #13 0x564ee8a35288 in tal_free ccan/ccan/tal/tal.c:481
    ElementsProject#14 0x564ee8a26cf5 in io_close ccan/ccan/io/io.c:450
    ElementsProject#15 0x564ee8a28c11 in io_loop ccan/ccan/io/poll.c:449
    ElementsProject#16 0x564ee89b3c3b in io_loop_with_timers lightningd/io_loop_with_timers.c:24
    ElementsProject#17 0x564ee89ba540 in main lightningd/lightningd.c:822
    ElementsProject#18 0x7f4dc2143b6a in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x26b6a)

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
rustyrussell added a commit that referenced this pull request Jun 14, 2019
Direct leak of 32 byte(s) in 1 object(s) allocated from:
    #0 0x7f7678ee863e in calloc (/lib/x86_64-linux-gnu/libasan.so.5+0x10c63e)
    #1 0x55f8c7b0fce5 in htable_default_alloc ccan/ccan/htable/htable.c:19
    #2 0x55f8c7b1064f in double_table ccan/ccan/htable/htable.c:226
    #3 0x55f8c7b10b19 in htable_add_ ccan/ccan/htable/htable.c:331
    #4 0x55f8c7afac63 in scriptpubkeyset_add wallet/txfilter.c:30
    #5 0x55f8c7afafce in txfilter_add_scriptpubkey wallet/txfilter.c:77
    #6 0x55f8c7afb05f in txfilter_add_derkey wallet/txfilter.c:91
    #7 0x55f8c7aa4d67 in init_txfilter lightningd/lightningd.c:482
    #8 0x55f8c7aa52d8 in main lightningd/lightningd.c:721
    #9 0x7f767889ab6a in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x26b6a)

Direct leak of 16 byte(s) in 1 object(s) allocated from:
    #0 0x7f05f389563e in calloc (/lib/x86_64-linux-gnu/libasan.so.5+0x10c63e)
    #1 0x55cac1e6bc99 in htable_default_alloc ccan/ccan/htable/htable.c:19
    #2 0x55cac1e6c603 in double_table ccan/ccan/htable/htable.c:226
    #3 0x55cac1e6cacd in htable_add_ ccan/ccan/htable/htable.c:331
    #4 0x55cac1e56e48 in outpointset_add wallet/txfilter.c:61
    #5 0x55cac1e57162 in outpointfilter_add wallet/txfilter.c:116
    #6 0x55cac1e5ea3a in wallet_utxoset_add wallet/wallet.c:2365
    #7 0x55cac1deddc2 in topo_add_utxos lightningd/chaintopology.c:603
    #8 0x55cac1dedeac in add_tip lightningd/chaintopology.c:620
    #9 0x55cac1dee2de in have_new_block lightningd/chaintopology.c:694
    #10 0x55cac1deaab0 in process_rawblock lightningd/bitcoind.c:466
    #11 0x55cac1de9fb4 in bcli_finished lightningd/bitcoind.c:214
    #12 0x55cac1e6f5be in destroy_conn ccan/ccan/io/poll.c:244
    #13 0x55cac1e6f5de in destroy_conn_close_fd ccan/ccan/io/poll.c:250
    ElementsProject#14 0x55cac1e7baf5 in notify ccan/ccan/tal/tal.c:235
    ElementsProject#15 0x55cac1e7bfe4 in del_tree ccan/ccan/tal/tal.c:397
    ElementsProject#16 0x55cac1e7c370 in tal_free ccan/ccan/tal/tal.c:481
    ElementsProject#17 0x55cac1e6dddd in io_close ccan/ccan/io/io.c:450
    ElementsProject#18 0x55cac1e6fcf9 in io_loop ccan/ccan/io/poll.c:449
    ElementsProject#19 0x55cac1dfac66 in io_loop_with_timers lightningd/io_loop_with_timers.c:24
    ElementsProject#20 0x55cac1e0156b in main lightningd/lightningd.c:822
    ElementsProject#21 0x7f05f3247b6a in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x26b6a)

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
rustyrussell added a commit that referenced this pull request Jun 14, 2019
Direct leak of 16 byte(s) in 1 object(s) allocated from:
    #0 0x7ff02889063e in calloc (/lib/x86_64-linux-gnu/libasan.so.5+0x10c63e)
    #1 0x555ce2ad8d2e in htable_default_alloc ccan/ccan/htable/htable.c:19
    #2 0x555ce2ad9698 in double_table ccan/ccan/htable/htable.c:226
    #3 0x555ce2ad9b62 in htable_add_ ccan/ccan/htable/htable.c:331
    #4 0x555ce2a638e4 in htlc_in_map_add lightningd/htlc_end.h:113
    #5 0x555ce2a63beb in connect_htlc_in lightningd/htlc_end.c:39
    #6 0x555ce2a85cbc in channel_added_their_htlc lightningd/peer_htlcs.c:1382
    #7 0x555ce2a860e1 in peer_got_commitsig lightningd/peer_htlcs.c:1466
    #8 0x555ce2a5db04 in channel_msg lightningd/channel_control.c:228
    #9 0x555ce2a8d393 in sd_msg_read lightningd/subd.c:474
    #10 0x555ce2ada157 in next_plan ccan/ccan/io/io.c:59
    #11 0x555ce2adacd4 in do_plan ccan/ccan/io/io.c:407
    #12 0x555ce2adad12 in io_ready ccan/ccan/io/io.c:417
    #13 0x555ce2adcd67 in io_loop ccan/ccan/io/poll.c:445
    ElementsProject#14 0x555ce2a67c66 in io_loop_with_timers lightningd/io_loop_with_timers.c:24
    ElementsProject#15 0x555ce2a6e56b in main lightningd/lightningd.c:822
    ElementsProject#16 0x7ff028242b6a in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x26b6a)

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
rustyrussell added a commit that referenced this pull request Jun 30, 2019
Direct leak of 1024 byte(s) in 2 object(s) allocated from:
    #0 0x7f4c84ce4448 in malloc (/lib/x86_64-linux-gnu/libasan.so.5+0x10c448)
    #1 0x55d11b782c96 in timer_default_alloc ccan/ccan/timer/timer.c:16
    #2 0x55d11b7832b7 in add_level ccan/ccan/timer/timer.c:166
    #3 0x55d11b783864 in timer_fast_forward ccan/ccan/timer/timer.c:334
    #4 0x55d11b78396a in timers_expire ccan/ccan/timer/timer.c:359
    #5 0x55d11b774993 in io_loop ccan/ccan/io/poll.c:395
    #6 0x55d11b72322f in plugins_init lightningd/plugin.c:1013
    #7 0x55d11b7060ea in main lightningd/lightningd.c:664
    #8 0x7f4c84696b6a in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x26b6a)

To fix this, we actually make 'ld->timers' a pointer, so we can clean
it up last of all.  We can't free it before ld, because that causes
timers to be destroyed.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
rustyrussell added a commit that referenced this pull request Jun 30, 2019
Indirect leak of 48 byte(s) in 1 object(s) allocated from:
    #0 0x7f4c84ce4448 in malloc (/lib/x86_64-linux-gnu/libasan.so.5+0x10c448)
    #1 0x55d11b77d270 in strmap_add_ ccan/ccan/strmap/strmap.c:90
    #2 0x55d11b704603 in command_set_usage lightningd/jsonrpc.c:891
    #3 0x55d11b733cb5 in param common/param.c:295
    #4 0x55d11b6f7b37 in json_connect lightningd/connect_control.c:96
    #5 0x55d11b7042ef in setup_command_usage lightningd/jsonrpc.c:841
    #6 0x55d11b70443b in jsonrpc_command_add_perm lightningd/jsonrpc.c:863
    #7 0x55d11b704533 in jsonrpc_setup lightningd/jsonrpc.c:876
    #8 0x55d11b705695 in new_lightningd lightningd/lightningd.c:210
    #9 0x55d11b706062 in main lightningd/lightningd.c:644
    #10 0x7f4c84696b6a in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x26b6a)

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
rustyrussell added a commit that referenced this pull request Jun 30, 2019
Direct leak of 64 byte(s) in 1 object(s) allocated from:
    #0 0x7f4dc279163e in calloc (/lib/x86_64-linux-gnu/libasan.so.5+0x10c63e)
    #1 0x564ee8a24bb1 in htable_default_alloc ccan/ccan/htable/htable.c:19
    #2 0x564ee8a2551b in double_table ccan/ccan/htable/htable.c:226
    #3 0x564ee8a259e5 in htable_add_ ccan/ccan/htable/htable.c:331
    #4 0x564ee89a5300 in block_map_add lightningd/chaintopology.h:83
    #5 0x564ee89a6ece in add_tip lightningd/chaintopology.c:626
    #6 0x564ee89a72c3 in have_new_block lightningd/chaintopology.c:694
    #7 0x564ee89a3ab0 in process_rawblock lightningd/bitcoind.c:466
    #8 0x564ee89a2fb4 in bcli_finished lightningd/bitcoind.c:214
    #9 0x564ee8a284d6 in destroy_conn ccan/ccan/io/poll.c:244
    #10 0x564ee8a284f6 in destroy_conn_close_fd ccan/ccan/io/poll.c:250
    #11 0x564ee8a34a0d in notify ccan/ccan/tal/tal.c:235
    #12 0x564ee8a34efc in del_tree ccan/ccan/tal/tal.c:397
    #13 0x564ee8a35288 in tal_free ccan/ccan/tal/tal.c:481
    ElementsProject#14 0x564ee8a26cf5 in io_close ccan/ccan/io/io.c:450
    ElementsProject#15 0x564ee8a28c11 in io_loop ccan/ccan/io/poll.c:449
    ElementsProject#16 0x564ee89b3c3b in io_loop_with_timers lightningd/io_loop_with_timers.c:24
    ElementsProject#17 0x564ee89ba540 in main lightningd/lightningd.c:822
    ElementsProject#18 0x7f4dc2143b6a in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x26b6a)

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
rustyrussell added a commit that referenced this pull request Jun 30, 2019
Direct leak of 32 byte(s) in 1 object(s) allocated from:
    #0 0x7f7678ee863e in calloc (/lib/x86_64-linux-gnu/libasan.so.5+0x10c63e)
    #1 0x55f8c7b0fce5 in htable_default_alloc ccan/ccan/htable/htable.c:19
    #2 0x55f8c7b1064f in double_table ccan/ccan/htable/htable.c:226
    #3 0x55f8c7b10b19 in htable_add_ ccan/ccan/htable/htable.c:331
    #4 0x55f8c7afac63 in scriptpubkeyset_add wallet/txfilter.c:30
    #5 0x55f8c7afafce in txfilter_add_scriptpubkey wallet/txfilter.c:77
    #6 0x55f8c7afb05f in txfilter_add_derkey wallet/txfilter.c:91
    #7 0x55f8c7aa4d67 in init_txfilter lightningd/lightningd.c:482
    #8 0x55f8c7aa52d8 in main lightningd/lightningd.c:721
    #9 0x7f767889ab6a in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x26b6a)

Direct leak of 16 byte(s) in 1 object(s) allocated from:
    #0 0x7f05f389563e in calloc (/lib/x86_64-linux-gnu/libasan.so.5+0x10c63e)
    #1 0x55cac1e6bc99 in htable_default_alloc ccan/ccan/htable/htable.c:19
    #2 0x55cac1e6c603 in double_table ccan/ccan/htable/htable.c:226
    #3 0x55cac1e6cacd in htable_add_ ccan/ccan/htable/htable.c:331
    #4 0x55cac1e56e48 in outpointset_add wallet/txfilter.c:61
    #5 0x55cac1e57162 in outpointfilter_add wallet/txfilter.c:116
    #6 0x55cac1e5ea3a in wallet_utxoset_add wallet/wallet.c:2365
    #7 0x55cac1deddc2 in topo_add_utxos lightningd/chaintopology.c:603
    #8 0x55cac1dedeac in add_tip lightningd/chaintopology.c:620
    #9 0x55cac1dee2de in have_new_block lightningd/chaintopology.c:694
    #10 0x55cac1deaab0 in process_rawblock lightningd/bitcoind.c:466
    #11 0x55cac1de9fb4 in bcli_finished lightningd/bitcoind.c:214
    #12 0x55cac1e6f5be in destroy_conn ccan/ccan/io/poll.c:244
    #13 0x55cac1e6f5de in destroy_conn_close_fd ccan/ccan/io/poll.c:250
    ElementsProject#14 0x55cac1e7baf5 in notify ccan/ccan/tal/tal.c:235
    ElementsProject#15 0x55cac1e7bfe4 in del_tree ccan/ccan/tal/tal.c:397
    ElementsProject#16 0x55cac1e7c370 in tal_free ccan/ccan/tal/tal.c:481
    ElementsProject#17 0x55cac1e6dddd in io_close ccan/ccan/io/io.c:450
    ElementsProject#18 0x55cac1e6fcf9 in io_loop ccan/ccan/io/poll.c:449
    ElementsProject#19 0x55cac1dfac66 in io_loop_with_timers lightningd/io_loop_with_timers.c:24
    ElementsProject#20 0x55cac1e0156b in main lightningd/lightningd.c:822
    ElementsProject#21 0x7f05f3247b6a in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x26b6a)

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
rustyrussell added a commit that referenced this pull request Jun 30, 2019
Direct leak of 16 byte(s) in 1 object(s) allocated from:
    #0 0x7ff02889063e in calloc (/lib/x86_64-linux-gnu/libasan.so.5+0x10c63e)
    #1 0x555ce2ad8d2e in htable_default_alloc ccan/ccan/htable/htable.c:19
    #2 0x555ce2ad9698 in double_table ccan/ccan/htable/htable.c:226
    #3 0x555ce2ad9b62 in htable_add_ ccan/ccan/htable/htable.c:331
    #4 0x555ce2a638e4 in htlc_in_map_add lightningd/htlc_end.h:113
    #5 0x555ce2a63beb in connect_htlc_in lightningd/htlc_end.c:39
    #6 0x555ce2a85cbc in channel_added_their_htlc lightningd/peer_htlcs.c:1382
    #7 0x555ce2a860e1 in peer_got_commitsig lightningd/peer_htlcs.c:1466
    #8 0x555ce2a5db04 in channel_msg lightningd/channel_control.c:228
    #9 0x555ce2a8d393 in sd_msg_read lightningd/subd.c:474
    #10 0x555ce2ada157 in next_plan ccan/ccan/io/io.c:59
    #11 0x555ce2adacd4 in do_plan ccan/ccan/io/io.c:407
    #12 0x555ce2adad12 in io_ready ccan/ccan/io/io.c:417
    #13 0x555ce2adcd67 in io_loop ccan/ccan/io/poll.c:445
    ElementsProject#14 0x555ce2a67c66 in io_loop_with_timers lightningd/io_loop_with_timers.c:24
    ElementsProject#15 0x555ce2a6e56b in main lightningd/lightningd.c:822
    ElementsProject#16 0x7ff028242b6a in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x26b6a)

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
rustyrussell added a commit that referenced this pull request Jul 25, 2019
I'm deeply reluctant to do this, as I'd thought this was fixed with
recent lnd versions.  Logs below show that it continues, with channel
loss on almost every restart.

At this rate, we risk bifurcating the network.  In fact, only four
errors my node have ever been NOT "sync error".

2018-09-12T01:21:40.671Z lightningd(1263): 03e50492eab4107a773141bb419e107bda3de3d55652e6e1a41225f06a0bbf2d56 chan #3: Peer permanent failure in CHANNELD_NORMAL: lightning_channeld: received ERROR channel b7008735ad2425ab92bcffa2f255ba93f63e0b5c685368f308e76ca0d2a30a41: sync error

2018-12-07T06:41:26.209Z lightningd(1215): 03da1c27ca77872ac5b3e568af30673e599a47a5e4497f85c7b5da42048807b3ed chan ElementsProject#1038: Peer permanent failure in CHANNELD_NORMAL: lightning_channeld: received ERROR channel 48858b0d55ae982596932ceb72584d4bb31363b9ecbaa56721b158ca4d18f5f8: sync error
2018-12-07T06:41:43.707Z lightningd(1215): 0219c2f8818bd2124dcc41827b726fd486c13cdfb6edf4e1458194663fb07891c7 chan ElementsProject#2508: Peer permanent failure in CHANNELD_AWAITING_LOCKIN: lightning_channeld: received ERROR channel 388b653e433773d20d74a151c552df647b74e240ef983d21a6d6c5816523b858: sync error
2018-12-07T06:41:45.553Z lightningd(1215): 03e50492eab4107a773141bb419e107bda3de3d55652e6e1a41225f06a0bbf2d56 chan ElementsProject#1044: Peer permanent failure in CHANNELD_NORMAL: lightning_channeld: received ERROR channel b58e9391383bfbe848da881ab9ddd9a8987c76318d421dac6f552b0d451ff957: sync error
2018-12-07T06:41:46.501Z lightningd(1215): 0390b5d4492dc2f5318e5233ab2cebf6d48914881a33ef6a9c6bcdbb433ad986d0 chan ElementsProject#871: Peer permanent failure in CHANNELD_NORMAL: lightning_channeld: received ERROR channel 91f43cb6a8c37d0be237a7c491f11d9dfad48534699fb4f076b2c0cbde964424: sync error
2018-12-07T06:41:46.985Z lightningd(1215): 03e5ea100e6b1ef3959f79627cb575606b19071235c48b3e7f9808ebcd6d12e87d chan ElementsProject#1026: Peer permanent failure in CHANNELD_NORMAL: lightning_channeld: received ERROR channel 6cc360db0627b19df146ccd971570c14597b22662bbc0907a233042480e50be7: sync error
2018-12-07T06:41:47.340Z lightningd(1215): 03c2abfa93eacec04721c019644584424aab2ba4dff3ac9bdab4e9c97007491dda chan ElementsProject#1420: Peer permanent failure in CHANNELD_NORMAL: lightning_channeld: received ERROR channel f363d174390bf819b47e568cb5890c8e432d61c03ba0d38d7c53996679080a74: sync error
2018-12-07T06:41:47.641Z lightningd(1215): 032679fec1213e5b0a23e066c019d7b991b95c6e4d28806b9ebd1362f9e32775cf chan ElementsProject#1058: Peer permanent failure in CHANNELD_NORMAL: lightning_channeld: received ERROR channel 602dc88c7f333ed88f24c6f2c760cb53fa359a4299dfab677f6a81ca33613231: sync error

2019-01-06T10:56:47.332Z lightningd(1202): 02cdf83ef8e45908b1092125d25c68dcec7751ca8d39f557775cd842e5bc127469 chan ElementsProject#2608: Peer permanent failure in CHANNELD_NORMAL: lightning_channeld: received ERROR channel 17b7c895c3feb6ae5b8209ef05044b0aa125629ef1ebc2ce6b2efb27e231533b: sync error
2019-01-06T10:57:08.896Z lightningd(1202): 0219c2f8818bd2124dcc41827b726fd486c13cdfb6edf4e1458194663fb07891c7 chan ElementsProject#2610: Peer permanent failure in CHANNELD_NORMAL: lightning_channeld: received ERROR channel 52d5e3717c7b4f6b06f2b7d55aa8d904a0558706e18be981c82d2c11d4bdf82c: sync error
2019-01-06T10:57:08.950Z lightningd(1202): 02ad6fb8d693dc1e4569bcedefadf5f72a931ae027dc0f0c544b34c1c6f3b9a02b chan ElementsProject#7185: Peer permanent failure in CHANNELD_NORMAL: lightning_channeld: received ERROR channel 245438c15a986b53da7694114c646b77ab663d236d7928732764f5b9251cd2d1: sync error

2019-01-15T09:15:26.882Z lightningd(1191): 03a76b80027d7c067e0da77da95880faaf89e9bf87b73a7d57bd4a3f2a124b764f chan ElementsProject#7430: Peer permanent failure in CHANNELD_AWAITING_LOCKIN: lightning_channeld: received ERROR channel 97c1e01612faf5653af2980abdf382c0f3b24d8a5961b6a3a1eb12444cf9db2e: sync error

2019-05-02T11:32:06.511Z lightningd(14815): 036e8a8efeb26f3cffce99f462839ef6ea3b1691d569d59c402be0d3d6cef9b79c chan ElementsProject#7573: Peer permanent failure in CHANNELD_NORMAL: lightning_channeld: received ERROR channel 6766b0b14013de753f9b354ce7a4b6e4756165ef970aae2650aeda990cfe5687: sync error

2019-06-12T10:38:57.503Z lightningd(1264): 024d2387409269f3b79e2708bb39b895c9f4b6a8322153af54eba487d4993bf60f chan #9607: Peer permanent failure in CHANNELD_NORMAL: lightning_channeld: received ERROR channel 1f3111399c670dab87b4e3d7bac22865c29d4c9992df71fdce9e8893666a08bc: sync error
2019-06-12T10:41:00.435Z lightningd(1264): 02809e936f0e82dfce13bcc47c77112db068f569e1db29e7bf98bcdd68b838ee84 chan #9332: Peer permanent failure in CHANNELD_NORMAL: lightning_channeld: received ERROR channel a31b5252be9b001f573e00310ea9098532c81322389aa8721946185b1b70ca4c: sync error
2019-06-12T10:46:23.097Z lightningd(1264): 02fcdb04f51d61dddc0481c10751173d523e3408ebe3a848a1d6cb34b1f5df6668 chan ElementsProject#7586: Peer permanent failure in CHANNELD_NORMAL: lightning_channeld: received ERROR channel bd18e98f5bd56ac73e7b2eb7fd70f6dbe3a4dda1e5bebe7bf6484c3a0f6b55e7: sync error
2019-06-12T10:46:24.627Z lightningd(1264): 03bb88ccc444534da7b5b64b4f7b15e1eccb18e102db0e400d4b9cfe93763aa26d chan #9626: Peer permanent failure in CHANNELD_NORMAL: lightning_channeld: received ERROR channel 345e89c2f0100257940aff7413c1e29786d08b0a1ea1e259d577650d18791872: sync error
2019-06-12T10:46:26.381Z lightningd(1264): 0331f80652fb840239df8dc99205792bba2e559a05469915804c08420230e23c7c chan #9677: Peer permanent failure in CHANNELD_NORMAL: lightning_channeld: received ERROR channel d38752727ed5dab33abb06c5671e9d7d467feb469f0d249aa488f45e304221c1: sync error
2019-06-12T12:12:51.261Z lightningd(1264): 02d3366059edde4179fc0d071828b4bd726effba7225c3851f3d86a6a827f934a2 chan #9804: Peer permanent failure in CHANNELD_NORMAL: lightning_channeld: received ERROR channel d00c9eb31bb0c1f5794804114117be3cc75a756a1e4c08099b7188a5fd9f7215: sync error

2019-06-13T03:19:28.212Z lightningd(1218): 03e5ea100e6b1ef3959f79627cb575606b19071235c48b3e7f9808ebcd6d12e87d chan #10792: Peer permanent failure in CHANNELD_NORMAL: lightning_channeld: received ERROR channel 873a526043bbc680ea4398c7a45b9742762d782dea285c661bb90ab8f165976d: sync error
2019-06-13T06:19:52.486Z lightningd(1230): 030995c0c0217d763c2274aa6ed69a0bb85fa2f7d118f93631550f3b6219a577f5 chan #10743: Peer permanent failure in CHANNELD_NORMAL: lightning_channeld: received ERROR channel 29157b32dd0c13bcf4f785c5527d067159e102d62516e3a00fbf2c0f33bf59ec: sync error

2019-06-14T01:25:37.598Z lightningd(1235): 02cf60741c586aa54ff24381beab1aebf45eda61a8c49b043cf1f6e203e611e581 chan #12786: Peer permanent failure in CHANNELD_NORMAL: lightning_channeld: received ERROR channel 827472a7167ab1fecd680e4f28e1ee74bcd25d04dcdea5d1295ba381b6543661: sync error

2019-07-17T03:37:12.703Z UNUSUAL lightningd(1262): 03021c5f5f57322740e4ee6936452add19dc7ea7ccf90635f95119ab82a62ae268 chan #14764: Peer permanent failure in CHANNELD_NORMAL: lightning_channeld: received ERROR channel 5ff0890d9f1fbb63439a7d793c28cb74c3baef8c9b610c51c64b8a6497237540: sync error
2019-07-17T03:37:14.964Z UNUSUAL lightningd(1262): 030c3f19d742ca294a55c00376b3b355c3c90d61c6b6b39554dbc7ac19b141c14f chan #14839: Peer permanent failure in CHANNELD_NORMAL: lightning_channeld: received ERROR channel 79525ec2c4eaffb5fd6893957f330db81b7383c50d57113d5bf8ffee3c121bdc: sync error
2019-07-17T03:37:16.048Z UNUSUAL lightningd(1262): 028c1da32603fce64118e469ffe2cfeec04d1c4bd88205efb4e8b4208f77a8064e chan #14996: Peer permanent failure in CHANNELD_NORMAL: lightning_channeld: received ERROR channel 6913067c9c89404d9451df25fed1a6cc98b9d9ef801b623d5e8e90aa43ca3077: sync error

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
rustyrussell added a commit that referenced this pull request Jul 25, 2019
I'm deeply reluctant to do this, as I'd thought this was fixed with
recent lnd versions.  Logs below show that it continues, with channel
loss on almost every restart.

At this rate, we risk bifurcating the network.  In fact, only four
errors my node have ever been NOT "sync error".

2018-09-12T01:21:40.671Z lightningd(1263): 03e50492eab4107a773141bb419e107bda3de3d55652e6e1a41225f06a0bbf2d56 chan #3: Peer permanent failure in CHANNELD_NORMAL: lightning_channeld: received ERROR channel b7008735ad2425ab92bcffa2f255ba93f63e0b5c685368f308e76ca0d2a30a41: sync error

2018-12-07T06:41:26.209Z lightningd(1215): 03da1c27ca77872ac5b3e568af30673e599a47a5e4497f85c7b5da42048807b3ed chan ElementsProject#1038: Peer permanent failure in CHANNELD_NORMAL: lightning_channeld: received ERROR channel 48858b0d55ae982596932ceb72584d4bb31363b9ecbaa56721b158ca4d18f5f8: sync error
2018-12-07T06:41:43.707Z lightningd(1215): 0219c2f8818bd2124dcc41827b726fd486c13cdfb6edf4e1458194663fb07891c7 chan ElementsProject#2508: Peer permanent failure in CHANNELD_AWAITING_LOCKIN: lightning_channeld: received ERROR channel 388b653e433773d20d74a151c552df647b74e240ef983d21a6d6c5816523b858: sync error
2018-12-07T06:41:45.553Z lightningd(1215): 03e50492eab4107a773141bb419e107bda3de3d55652e6e1a41225f06a0bbf2d56 chan ElementsProject#1044: Peer permanent failure in CHANNELD_NORMAL: lightning_channeld: received ERROR channel b58e9391383bfbe848da881ab9ddd9a8987c76318d421dac6f552b0d451ff957: sync error
2018-12-07T06:41:46.501Z lightningd(1215): 0390b5d4492dc2f5318e5233ab2cebf6d48914881a33ef6a9c6bcdbb433ad986d0 chan ElementsProject#871: Peer permanent failure in CHANNELD_NORMAL: lightning_channeld: received ERROR channel 91f43cb6a8c37d0be237a7c491f11d9dfad48534699fb4f076b2c0cbde964424: sync error
2018-12-07T06:41:46.985Z lightningd(1215): 03e5ea100e6b1ef3959f79627cb575606b19071235c48b3e7f9808ebcd6d12e87d chan ElementsProject#1026: Peer permanent failure in CHANNELD_NORMAL: lightning_channeld: received ERROR channel 6cc360db0627b19df146ccd971570c14597b22662bbc0907a233042480e50be7: sync error
2018-12-07T06:41:47.340Z lightningd(1215): 03c2abfa93eacec04721c019644584424aab2ba4dff3ac9bdab4e9c97007491dda chan ElementsProject#1420: Peer permanent failure in CHANNELD_NORMAL: lightning_channeld: received ERROR channel f363d174390bf819b47e568cb5890c8e432d61c03ba0d38d7c53996679080a74: sync error
2018-12-07T06:41:47.641Z lightningd(1215): 032679fec1213e5b0a23e066c019d7b991b95c6e4d28806b9ebd1362f9e32775cf chan ElementsProject#1058: Peer permanent failure in CHANNELD_NORMAL: lightning_channeld: received ERROR channel 602dc88c7f333ed88f24c6f2c760cb53fa359a4299dfab677f6a81ca33613231: sync error

2019-01-06T10:56:47.332Z lightningd(1202): 02cdf83ef8e45908b1092125d25c68dcec7751ca8d39f557775cd842e5bc127469 chan ElementsProject#2608: Peer permanent failure in CHANNELD_NORMAL: lightning_channeld: received ERROR channel 17b7c895c3feb6ae5b8209ef05044b0aa125629ef1ebc2ce6b2efb27e231533b: sync error
2019-01-06T10:57:08.896Z lightningd(1202): 0219c2f8818bd2124dcc41827b726fd486c13cdfb6edf4e1458194663fb07891c7 chan ElementsProject#2610: Peer permanent failure in CHANNELD_NORMAL: lightning_channeld: received ERROR channel 52d5e3717c7b4f6b06f2b7d55aa8d904a0558706e18be981c82d2c11d4bdf82c: sync error
2019-01-06T10:57:08.950Z lightningd(1202): 02ad6fb8d693dc1e4569bcedefadf5f72a931ae027dc0f0c544b34c1c6f3b9a02b chan ElementsProject#7185: Peer permanent failure in CHANNELD_NORMAL: lightning_channeld: received ERROR channel 245438c15a986b53da7694114c646b77ab663d236d7928732764f5b9251cd2d1: sync error

2019-01-15T09:15:26.882Z lightningd(1191): 03a76b80027d7c067e0da77da95880faaf89e9bf87b73a7d57bd4a3f2a124b764f chan ElementsProject#7430: Peer permanent failure in CHANNELD_AWAITING_LOCKIN: lightning_channeld: received ERROR channel 97c1e01612faf5653af2980abdf382c0f3b24d8a5961b6a3a1eb12444cf9db2e: sync error

2019-05-02T11:32:06.511Z lightningd(14815): 036e8a8efeb26f3cffce99f462839ef6ea3b1691d569d59c402be0d3d6cef9b79c chan ElementsProject#7573: Peer permanent failure in CHANNELD_NORMAL: lightning_channeld: received ERROR channel 6766b0b14013de753f9b354ce7a4b6e4756165ef970aae2650aeda990cfe5687: sync error

2019-06-12T10:38:57.503Z lightningd(1264): 024d2387409269f3b79e2708bb39b895c9f4b6a8322153af54eba487d4993bf60f chan #9607: Peer permanent failure in CHANNELD_NORMAL: lightning_channeld: received ERROR channel 1f3111399c670dab87b4e3d7bac22865c29d4c9992df71fdce9e8893666a08bc: sync error
2019-06-12T10:41:00.435Z lightningd(1264): 02809e936f0e82dfce13bcc47c77112db068f569e1db29e7bf98bcdd68b838ee84 chan #9332: Peer permanent failure in CHANNELD_NORMAL: lightning_channeld: received ERROR channel a31b5252be9b001f573e00310ea9098532c81322389aa8721946185b1b70ca4c: sync error
2019-06-12T10:46:23.097Z lightningd(1264): 02fcdb04f51d61dddc0481c10751173d523e3408ebe3a848a1d6cb34b1f5df6668 chan ElementsProject#7586: Peer permanent failure in CHANNELD_NORMAL: lightning_channeld: received ERROR channel bd18e98f5bd56ac73e7b2eb7fd70f6dbe3a4dda1e5bebe7bf6484c3a0f6b55e7: sync error
2019-06-12T10:46:24.627Z lightningd(1264): 03bb88ccc444534da7b5b64b4f7b15e1eccb18e102db0e400d4b9cfe93763aa26d chan #9626: Peer permanent failure in CHANNELD_NORMAL: lightning_channeld: received ERROR channel 345e89c2f0100257940aff7413c1e29786d08b0a1ea1e259d577650d18791872: sync error
2019-06-12T10:46:26.381Z lightningd(1264): 0331f80652fb840239df8dc99205792bba2e559a05469915804c08420230e23c7c chan #9677: Peer permanent failure in CHANNELD_NORMAL: lightning_channeld: received ERROR channel d38752727ed5dab33abb06c5671e9d7d467feb469f0d249aa488f45e304221c1: sync error
2019-06-12T12:12:51.261Z lightningd(1264): 02d3366059edde4179fc0d071828b4bd726effba7225c3851f3d86a6a827f934a2 chan #9804: Peer permanent failure in CHANNELD_NORMAL: lightning_channeld: received ERROR channel d00c9eb31bb0c1f5794804114117be3cc75a756a1e4c08099b7188a5fd9f7215: sync error

2019-06-13T03:19:28.212Z lightningd(1218): 03e5ea100e6b1ef3959f79627cb575606b19071235c48b3e7f9808ebcd6d12e87d chan #10792: Peer permanent failure in CHANNELD_NORMAL: lightning_channeld: received ERROR channel 873a526043bbc680ea4398c7a45b9742762d782dea285c661bb90ab8f165976d: sync error
2019-06-13T06:19:52.486Z lightningd(1230): 030995c0c0217d763c2274aa6ed69a0bb85fa2f7d118f93631550f3b6219a577f5 chan #10743: Peer permanent failure in CHANNELD_NORMAL: lightning_channeld: received ERROR channel 29157b32dd0c13bcf4f785c5527d067159e102d62516e3a00fbf2c0f33bf59ec: sync error

2019-06-14T01:25:37.598Z lightningd(1235): 02cf60741c586aa54ff24381beab1aebf45eda61a8c49b043cf1f6e203e611e581 chan #12786: Peer permanent failure in CHANNELD_NORMAL: lightning_channeld: received ERROR channel 827472a7167ab1fecd680e4f28e1ee74bcd25d04dcdea5d1295ba381b6543661: sync error

2019-07-17T03:37:12.703Z UNUSUAL lightningd(1262): 03021c5f5f57322740e4ee6936452add19dc7ea7ccf90635f95119ab82a62ae268 chan #14764: Peer permanent failure in CHANNELD_NORMAL: lightning_channeld: received ERROR channel 5ff0890d9f1fbb63439a7d793c28cb74c3baef8c9b610c51c64b8a6497237540: sync error
2019-07-17T03:37:14.964Z UNUSUAL lightningd(1262): 030c3f19d742ca294a55c00376b3b355c3c90d61c6b6b39554dbc7ac19b141c14f chan #14839: Peer permanent failure in CHANNELD_NORMAL: lightning_channeld: received ERROR channel 79525ec2c4eaffb5fd6893957f330db81b7383c50d57113d5bf8ffee3c121bdc: sync error
2019-07-17T03:37:16.048Z UNUSUAL lightningd(1262): 028c1da32603fce64118e469ffe2cfeec04d1c4bd88205efb4e8b4208f77a8064e chan #14996: Peer permanent failure in CHANNELD_NORMAL: lightning_channeld: received ERROR channel 6913067c9c89404d9451df25fed1a6cc98b9d9ef801b623d5e8e90aa43ca3077: sync error

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
rustyrussell added a commit that referenced this pull request Jul 26, 2019
I'm deeply reluctant to do this, as I'd thought this was fixed with
recent lnd versions.  Logs below show that it continues, with channel
loss on almost every restart.

At this rate, we risk bifurcating the network.  In fact, only four
errors my node have ever been NOT "sync error".

2018-09-12T01:21:40.671Z lightningd(1263): 03e50492eab4107a773141bb419e107bda3de3d55652e6e1a41225f06a0bbf2d56 chan #3: Peer permanent failure in CHANNELD_NORMAL: lightning_channeld: received ERROR channel b7008735ad2425ab92bcffa2f255ba93f63e0b5c685368f308e76ca0d2a30a41: sync error

2018-12-07T06:41:26.209Z lightningd(1215): 03da1c27ca77872ac5b3e568af30673e599a47a5e4497f85c7b5da42048807b3ed chan ElementsProject#1038: Peer permanent failure in CHANNELD_NORMAL: lightning_channeld: received ERROR channel 48858b0d55ae982596932ceb72584d4bb31363b9ecbaa56721b158ca4d18f5f8: sync error
2018-12-07T06:41:43.707Z lightningd(1215): 0219c2f8818bd2124dcc41827b726fd486c13cdfb6edf4e1458194663fb07891c7 chan ElementsProject#2508: Peer permanent failure in CHANNELD_AWAITING_LOCKIN: lightning_channeld: received ERROR channel 388b653e433773d20d74a151c552df647b74e240ef983d21a6d6c5816523b858: sync error
2018-12-07T06:41:45.553Z lightningd(1215): 03e50492eab4107a773141bb419e107bda3de3d55652e6e1a41225f06a0bbf2d56 chan ElementsProject#1044: Peer permanent failure in CHANNELD_NORMAL: lightning_channeld: received ERROR channel b58e9391383bfbe848da881ab9ddd9a8987c76318d421dac6f552b0d451ff957: sync error
2018-12-07T06:41:46.501Z lightningd(1215): 0390b5d4492dc2f5318e5233ab2cebf6d48914881a33ef6a9c6bcdbb433ad986d0 chan ElementsProject#871: Peer permanent failure in CHANNELD_NORMAL: lightning_channeld: received ERROR channel 91f43cb6a8c37d0be237a7c491f11d9dfad48534699fb4f076b2c0cbde964424: sync error
2018-12-07T06:41:46.985Z lightningd(1215): 03e5ea100e6b1ef3959f79627cb575606b19071235c48b3e7f9808ebcd6d12e87d chan ElementsProject#1026: Peer permanent failure in CHANNELD_NORMAL: lightning_channeld: received ERROR channel 6cc360db0627b19df146ccd971570c14597b22662bbc0907a233042480e50be7: sync error
2018-12-07T06:41:47.340Z lightningd(1215): 03c2abfa93eacec04721c019644584424aab2ba4dff3ac9bdab4e9c97007491dda chan ElementsProject#1420: Peer permanent failure in CHANNELD_NORMAL: lightning_channeld: received ERROR channel f363d174390bf819b47e568cb5890c8e432d61c03ba0d38d7c53996679080a74: sync error
2018-12-07T06:41:47.641Z lightningd(1215): 032679fec1213e5b0a23e066c019d7b991b95c6e4d28806b9ebd1362f9e32775cf chan ElementsProject#1058: Peer permanent failure in CHANNELD_NORMAL: lightning_channeld: received ERROR channel 602dc88c7f333ed88f24c6f2c760cb53fa359a4299dfab677f6a81ca33613231: sync error

2019-01-06T10:56:47.332Z lightningd(1202): 02cdf83ef8e45908b1092125d25c68dcec7751ca8d39f557775cd842e5bc127469 chan ElementsProject#2608: Peer permanent failure in CHANNELD_NORMAL: lightning_channeld: received ERROR channel 17b7c895c3feb6ae5b8209ef05044b0aa125629ef1ebc2ce6b2efb27e231533b: sync error
2019-01-06T10:57:08.896Z lightningd(1202): 0219c2f8818bd2124dcc41827b726fd486c13cdfb6edf4e1458194663fb07891c7 chan ElementsProject#2610: Peer permanent failure in CHANNELD_NORMAL: lightning_channeld: received ERROR channel 52d5e3717c7b4f6b06f2b7d55aa8d904a0558706e18be981c82d2c11d4bdf82c: sync error
2019-01-06T10:57:08.950Z lightningd(1202): 02ad6fb8d693dc1e4569bcedefadf5f72a931ae027dc0f0c544b34c1c6f3b9a02b chan ElementsProject#7185: Peer permanent failure in CHANNELD_NORMAL: lightning_channeld: received ERROR channel 245438c15a986b53da7694114c646b77ab663d236d7928732764f5b9251cd2d1: sync error

2019-01-15T09:15:26.882Z lightningd(1191): 03a76b80027d7c067e0da77da95880faaf89e9bf87b73a7d57bd4a3f2a124b764f chan ElementsProject#7430: Peer permanent failure in CHANNELD_AWAITING_LOCKIN: lightning_channeld: received ERROR channel 97c1e01612faf5653af2980abdf382c0f3b24d8a5961b6a3a1eb12444cf9db2e: sync error

2019-05-02T11:32:06.511Z lightningd(14815): 036e8a8efeb26f3cffce99f462839ef6ea3b1691d569d59c402be0d3d6cef9b79c chan ElementsProject#7573: Peer permanent failure in CHANNELD_NORMAL: lightning_channeld: received ERROR channel 6766b0b14013de753f9b354ce7a4b6e4756165ef970aae2650aeda990cfe5687: sync error

2019-06-12T10:38:57.503Z lightningd(1264): 024d2387409269f3b79e2708bb39b895c9f4b6a8322153af54eba487d4993bf60f chan #9607: Peer permanent failure in CHANNELD_NORMAL: lightning_channeld: received ERROR channel 1f3111399c670dab87b4e3d7bac22865c29d4c9992df71fdce9e8893666a08bc: sync error
2019-06-12T10:41:00.435Z lightningd(1264): 02809e936f0e82dfce13bcc47c77112db068f569e1db29e7bf98bcdd68b838ee84 chan #9332: Peer permanent failure in CHANNELD_NORMAL: lightning_channeld: received ERROR channel a31b5252be9b001f573e00310ea9098532c81322389aa8721946185b1b70ca4c: sync error
2019-06-12T10:46:23.097Z lightningd(1264): 02fcdb04f51d61dddc0481c10751173d523e3408ebe3a848a1d6cb34b1f5df6668 chan ElementsProject#7586: Peer permanent failure in CHANNELD_NORMAL: lightning_channeld: received ERROR channel bd18e98f5bd56ac73e7b2eb7fd70f6dbe3a4dda1e5bebe7bf6484c3a0f6b55e7: sync error
2019-06-12T10:46:24.627Z lightningd(1264): 03bb88ccc444534da7b5b64b4f7b15e1eccb18e102db0e400d4b9cfe93763aa26d chan #9626: Peer permanent failure in CHANNELD_NORMAL: lightning_channeld: received ERROR channel 345e89c2f0100257940aff7413c1e29786d08b0a1ea1e259d577650d18791872: sync error
2019-06-12T10:46:26.381Z lightningd(1264): 0331f80652fb840239df8dc99205792bba2e559a05469915804c08420230e23c7c chan #9677: Peer permanent failure in CHANNELD_NORMAL: lightning_channeld: received ERROR channel d38752727ed5dab33abb06c5671e9d7d467feb469f0d249aa488f45e304221c1: sync error
2019-06-12T12:12:51.261Z lightningd(1264): 02d3366059edde4179fc0d071828b4bd726effba7225c3851f3d86a6a827f934a2 chan #9804: Peer permanent failure in CHANNELD_NORMAL: lightning_channeld: received ERROR channel d00c9eb31bb0c1f5794804114117be3cc75a756a1e4c08099b7188a5fd9f7215: sync error

2019-06-13T03:19:28.212Z lightningd(1218): 03e5ea100e6b1ef3959f79627cb575606b19071235c48b3e7f9808ebcd6d12e87d chan #10792: Peer permanent failure in CHANNELD_NORMAL: lightning_channeld: received ERROR channel 873a526043bbc680ea4398c7a45b9742762d782dea285c661bb90ab8f165976d: sync error
2019-06-13T06:19:52.486Z lightningd(1230): 030995c0c0217d763c2274aa6ed69a0bb85fa2f7d118f93631550f3b6219a577f5 chan #10743: Peer permanent failure in CHANNELD_NORMAL: lightning_channeld: received ERROR channel 29157b32dd0c13bcf4f785c5527d067159e102d62516e3a00fbf2c0f33bf59ec: sync error

2019-06-14T01:25:37.598Z lightningd(1235): 02cf60741c586aa54ff24381beab1aebf45eda61a8c49b043cf1f6e203e611e581 chan #12786: Peer permanent failure in CHANNELD_NORMAL: lightning_channeld: received ERROR channel 827472a7167ab1fecd680e4f28e1ee74bcd25d04dcdea5d1295ba381b6543661: sync error

2019-07-17T03:37:12.703Z UNUSUAL lightningd(1262): 03021c5f5f57322740e4ee6936452add19dc7ea7ccf90635f95119ab82a62ae268 chan #14764: Peer permanent failure in CHANNELD_NORMAL: lightning_channeld: received ERROR channel 5ff0890d9f1fbb63439a7d793c28cb74c3baef8c9b610c51c64b8a6497237540: sync error
2019-07-17T03:37:14.964Z UNUSUAL lightningd(1262): 030c3f19d742ca294a55c00376b3b355c3c90d61c6b6b39554dbc7ac19b141c14f chan #14839: Peer permanent failure in CHANNELD_NORMAL: lightning_channeld: received ERROR channel 79525ec2c4eaffb5fd6893957f330db81b7383c50d57113d5bf8ffee3c121bdc: sync error
2019-07-17T03:37:16.048Z UNUSUAL lightningd(1262): 028c1da32603fce64118e469ffe2cfeec04d1c4bd88205efb4e8b4208f77a8064e chan #14996: Peer permanent failure in CHANNELD_NORMAL: lightning_channeld: received ERROR channel 6913067c9c89404d9451df25fed1a6cc98b9d9ef801b623d5e8e90aa43ca3077: sync error

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
rustyrussell added a commit that referenced this pull request Jul 28, 2019
I'm deeply reluctant to do this, as I'd thought this was fixed with
recent lnd versions.  Logs below show that it continues, with channel
loss on almost every restart.

At this rate, we risk bifurcating the network.  In fact, only four
errors my node have ever been NOT "sync error".

2018-09-12T01:21:40.671Z lightningd(1263): 03e50492eab4107a773141bb419e107bda3de3d55652e6e1a41225f06a0bbf2d56 chan #3: Peer permanent failure in CHANNELD_NORMAL: lightning_channeld: received ERROR channel b7008735ad2425ab92bcffa2f255ba93f63e0b5c685368f308e76ca0d2a30a41: sync error

2018-12-07T06:41:26.209Z lightningd(1215): 03da1c27ca77872ac5b3e568af30673e599a47a5e4497f85c7b5da42048807b3ed chan ElementsProject#1038: Peer permanent failure in CHANNELD_NORMAL: lightning_channeld: received ERROR channel 48858b0d55ae982596932ceb72584d4bb31363b9ecbaa56721b158ca4d18f5f8: sync error
2018-12-07T06:41:43.707Z lightningd(1215): 0219c2f8818bd2124dcc41827b726fd486c13cdfb6edf4e1458194663fb07891c7 chan ElementsProject#2508: Peer permanent failure in CHANNELD_AWAITING_LOCKIN: lightning_channeld: received ERROR channel 388b653e433773d20d74a151c552df647b74e240ef983d21a6d6c5816523b858: sync error
2018-12-07T06:41:45.553Z lightningd(1215): 03e50492eab4107a773141bb419e107bda3de3d55652e6e1a41225f06a0bbf2d56 chan ElementsProject#1044: Peer permanent failure in CHANNELD_NORMAL: lightning_channeld: received ERROR channel b58e9391383bfbe848da881ab9ddd9a8987c76318d421dac6f552b0d451ff957: sync error
2018-12-07T06:41:46.501Z lightningd(1215): 0390b5d4492dc2f5318e5233ab2cebf6d48914881a33ef6a9c6bcdbb433ad986d0 chan ElementsProject#871: Peer permanent failure in CHANNELD_NORMAL: lightning_channeld: received ERROR channel 91f43cb6a8c37d0be237a7c491f11d9dfad48534699fb4f076b2c0cbde964424: sync error
2018-12-07T06:41:46.985Z lightningd(1215): 03e5ea100e6b1ef3959f79627cb575606b19071235c48b3e7f9808ebcd6d12e87d chan ElementsProject#1026: Peer permanent failure in CHANNELD_NORMAL: lightning_channeld: received ERROR channel 6cc360db0627b19df146ccd971570c14597b22662bbc0907a233042480e50be7: sync error
2018-12-07T06:41:47.340Z lightningd(1215): 03c2abfa93eacec04721c019644584424aab2ba4dff3ac9bdab4e9c97007491dda chan ElementsProject#1420: Peer permanent failure in CHANNELD_NORMAL: lightning_channeld: received ERROR channel f363d174390bf819b47e568cb5890c8e432d61c03ba0d38d7c53996679080a74: sync error
2018-12-07T06:41:47.641Z lightningd(1215): 032679fec1213e5b0a23e066c019d7b991b95c6e4d28806b9ebd1362f9e32775cf chan ElementsProject#1058: Peer permanent failure in CHANNELD_NORMAL: lightning_channeld: received ERROR channel 602dc88c7f333ed88f24c6f2c760cb53fa359a4299dfab677f6a81ca33613231: sync error

2019-01-06T10:56:47.332Z lightningd(1202): 02cdf83ef8e45908b1092125d25c68dcec7751ca8d39f557775cd842e5bc127469 chan ElementsProject#2608: Peer permanent failure in CHANNELD_NORMAL: lightning_channeld: received ERROR channel 17b7c895c3feb6ae5b8209ef05044b0aa125629ef1ebc2ce6b2efb27e231533b: sync error
2019-01-06T10:57:08.896Z lightningd(1202): 0219c2f8818bd2124dcc41827b726fd486c13cdfb6edf4e1458194663fb07891c7 chan ElementsProject#2610: Peer permanent failure in CHANNELD_NORMAL: lightning_channeld: received ERROR channel 52d5e3717c7b4f6b06f2b7d55aa8d904a0558706e18be981c82d2c11d4bdf82c: sync error
2019-01-06T10:57:08.950Z lightningd(1202): 02ad6fb8d693dc1e4569bcedefadf5f72a931ae027dc0f0c544b34c1c6f3b9a02b chan ElementsProject#7185: Peer permanent failure in CHANNELD_NORMAL: lightning_channeld: received ERROR channel 245438c15a986b53da7694114c646b77ab663d236d7928732764f5b9251cd2d1: sync error

2019-01-15T09:15:26.882Z lightningd(1191): 03a76b80027d7c067e0da77da95880faaf89e9bf87b73a7d57bd4a3f2a124b764f chan ElementsProject#7430: Peer permanent failure in CHANNELD_AWAITING_LOCKIN: lightning_channeld: received ERROR channel 97c1e01612faf5653af2980abdf382c0f3b24d8a5961b6a3a1eb12444cf9db2e: sync error

2019-05-02T11:32:06.511Z lightningd(14815): 036e8a8efeb26f3cffce99f462839ef6ea3b1691d569d59c402be0d3d6cef9b79c chan ElementsProject#7573: Peer permanent failure in CHANNELD_NORMAL: lightning_channeld: received ERROR channel 6766b0b14013de753f9b354ce7a4b6e4756165ef970aae2650aeda990cfe5687: sync error

2019-06-12T10:38:57.503Z lightningd(1264): 024d2387409269f3b79e2708bb39b895c9f4b6a8322153af54eba487d4993bf60f chan #9607: Peer permanent failure in CHANNELD_NORMAL: lightning_channeld: received ERROR channel 1f3111399c670dab87b4e3d7bac22865c29d4c9992df71fdce9e8893666a08bc: sync error
2019-06-12T10:41:00.435Z lightningd(1264): 02809e936f0e82dfce13bcc47c77112db068f569e1db29e7bf98bcdd68b838ee84 chan #9332: Peer permanent failure in CHANNELD_NORMAL: lightning_channeld: received ERROR channel a31b5252be9b001f573e00310ea9098532c81322389aa8721946185b1b70ca4c: sync error
2019-06-12T10:46:23.097Z lightningd(1264): 02fcdb04f51d61dddc0481c10751173d523e3408ebe3a848a1d6cb34b1f5df6668 chan ElementsProject#7586: Peer permanent failure in CHANNELD_NORMAL: lightning_channeld: received ERROR channel bd18e98f5bd56ac73e7b2eb7fd70f6dbe3a4dda1e5bebe7bf6484c3a0f6b55e7: sync error
2019-06-12T10:46:24.627Z lightningd(1264): 03bb88ccc444534da7b5b64b4f7b15e1eccb18e102db0e400d4b9cfe93763aa26d chan #9626: Peer permanent failure in CHANNELD_NORMAL: lightning_channeld: received ERROR channel 345e89c2f0100257940aff7413c1e29786d08b0a1ea1e259d577650d18791872: sync error
2019-06-12T10:46:26.381Z lightningd(1264): 0331f80652fb840239df8dc99205792bba2e559a05469915804c08420230e23c7c chan #9677: Peer permanent failure in CHANNELD_NORMAL: lightning_channeld: received ERROR channel d38752727ed5dab33abb06c5671e9d7d467feb469f0d249aa488f45e304221c1: sync error
2019-06-12T12:12:51.261Z lightningd(1264): 02d3366059edde4179fc0d071828b4bd726effba7225c3851f3d86a6a827f934a2 chan #9804: Peer permanent failure in CHANNELD_NORMAL: lightning_channeld: received ERROR channel d00c9eb31bb0c1f5794804114117be3cc75a756a1e4c08099b7188a5fd9f7215: sync error

2019-06-13T03:19:28.212Z lightningd(1218): 03e5ea100e6b1ef3959f79627cb575606b19071235c48b3e7f9808ebcd6d12e87d chan #10792: Peer permanent failure in CHANNELD_NORMAL: lightning_channeld: received ERROR channel 873a526043bbc680ea4398c7a45b9742762d782dea285c661bb90ab8f165976d: sync error
2019-06-13T06:19:52.486Z lightningd(1230): 030995c0c0217d763c2274aa6ed69a0bb85fa2f7d118f93631550f3b6219a577f5 chan #10743: Peer permanent failure in CHANNELD_NORMAL: lightning_channeld: received ERROR channel 29157b32dd0c13bcf4f785c5527d067159e102d62516e3a00fbf2c0f33bf59ec: sync error

2019-06-14T01:25:37.598Z lightningd(1235): 02cf60741c586aa54ff24381beab1aebf45eda61a8c49b043cf1f6e203e611e581 chan #12786: Peer permanent failure in CHANNELD_NORMAL: lightning_channeld: received ERROR channel 827472a7167ab1fecd680e4f28e1ee74bcd25d04dcdea5d1295ba381b6543661: sync error

2019-07-17T03:37:12.703Z UNUSUAL lightningd(1262): 03021c5f5f57322740e4ee6936452add19dc7ea7ccf90635f95119ab82a62ae268 chan #14764: Peer permanent failure in CHANNELD_NORMAL: lightning_channeld: received ERROR channel 5ff0890d9f1fbb63439a7d793c28cb74c3baef8c9b610c51c64b8a6497237540: sync error
2019-07-17T03:37:14.964Z UNUSUAL lightningd(1262): 030c3f19d742ca294a55c00376b3b355c3c90d61c6b6b39554dbc7ac19b141c14f chan #14839: Peer permanent failure in CHANNELD_NORMAL: lightning_channeld: received ERROR channel 79525ec2c4eaffb5fd6893957f330db81b7383c50d57113d5bf8ffee3c121bdc: sync error
2019-07-17T03:37:16.048Z UNUSUAL lightningd(1262): 028c1da32603fce64118e469ffe2cfeec04d1c4bd88205efb4e8b4208f77a8064e chan #14996: Peer permanent failure in CHANNELD_NORMAL: lightning_channeld: received ERROR channel 6913067c9c89404d9451df25fed1a6cc98b9d9ef801b623d5e8e90aa43ca3077: sync error

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
rustyrussell added a commit that referenced this pull request Dec 6, 2020
As reported by Wladimir J. van der Laan.  Valgrind will complain
about padding and unset fields, so memset the structs.

```
==42653== Syscall param socketcall.connect(serv_addr..sa_len) points to uninitialised byte(s)
==42653==    at 0x4C7D19A: _connect (in /lib/libc.so.7)
==42653==    by 0x4EE1F35: ??? (in /lib/libthr.so.3)
==42653==    by 0x249D57: get_local_sockname (netaddress.c:212)
==42653==    by 0x249CDB: guess_address (netaddress.c:242)
==42653==    by 0x2473D0: public_address (connectd.c:1003)
==42653==    by 0x246CE4: setup_listeners (connectd.c:0)
==42653==    by 0x246566: connect_init (connectd.c:1311)
==42653==    by 0x270CEB: next_plan (io.c:59)
==42653==    by 0x2713EE: io_ready (io.c:417)
==42653==    by 0x2726B1: io_loop (poll.c:445)
==42653==    by 0x24618A: main (connectd.c:1703)
==42653==  Address 0x7fc000690 is on thread 1's stack
==42653==  in frame #3, created by guess_address (netaddress.c:231)
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
rustyrussell added a commit that referenced this pull request Dec 9, 2020
As reported by Wladimir J. van der Laan.  Valgrind will complain
about padding and unset fields, so memset the structs.

```
==42653== Syscall param socketcall.connect(serv_addr..sa_len) points to uninitialised byte(s)
==42653==    at 0x4C7D19A: _connect (in /lib/libc.so.7)
==42653==    by 0x4EE1F35: ??? (in /lib/libthr.so.3)
==42653==    by 0x249D57: get_local_sockname (netaddress.c:212)
==42653==    by 0x249CDB: guess_address (netaddress.c:242)
==42653==    by 0x2473D0: public_address (connectd.c:1003)
==42653==    by 0x246CE4: setup_listeners (connectd.c:0)
==42653==    by 0x246566: connect_init (connectd.c:1311)
==42653==    by 0x270CEB: next_plan (io.c:59)
==42653==    by 0x2713EE: io_ready (io.c:417)
==42653==    by 0x2726B1: io_loop (poll.c:445)
==42653==    by 0x24618A: main (connectd.c:1703)
==42653==  Address 0x7fc000690 is on thread 1's stack
==42653==  in frame #3, created by guess_address (netaddress.c:231)
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
rustyrussell pushed a commit that referenced this pull request Aug 23, 2021
The variable `block` (instace of `struct block`) is
allocated on the stack without being initialized, i.e. its
member `prev` points to nowhere. This causes a segmentation
fault on my machine on the binding of "prev_hash" on running
`wallet_block_add`, as the following core-dump analysis
shows:

    $ egdb ./wallet/test/run-wallet ./run-wallet.core
    [...]
    Core was generated by `run-wallet'.
    Program terminated with signal SIGSEGV, Segmentation fault.
    ---Type <return> to continue, or q <return> to quit---
    #0  0x000008f67a04b660 in memcpy (dst0=<optimized out>, src0=0x100007f8c, length=32) at /usr/src/lib/libc/string/memcpy.c:97
    97                      TLOOP1(*dst++ = *src++);
    (gdb) bt
    #0  0x000008f67a04b660 in memcpy (dst0=<optimized out>, src0=0x100007f8c, length=32) at /usr/src/lib/libc/string/memcpy.c:97
    #1  0x000008f73e838f60 in sqlite3VdbeMemSetStr () from /usr/local/lib/libsqlite3.so.37.12
    #2  0x000008f73e83cb11 in bindText () from /usr/local/lib/libsqlite3.so.37.12
    #3  0x000008f44bc91345 in db_sqlite3_query (stmt=0x8f6845bf028) at wallet/db_sqlite3.c:77
    #4  0x000008f44bc91122 in db_sqlite3_exec (stmt=0x8f6845bf028) at wallet/db_sqlite3.c:110
    #5  0x000008f44bcbb3b2 in db_exec_prepared_v2 (stmt=0x8f6845bf028) at ./wallet/db.c:2055
    #6  0x000008f44bcc6890 in wallet_block_add (w=0x8f688b5bba8, b=0x7f7ffffca788) at ./wallet/wallet.c:3556
    #7  0x000008f44bce2607 in test_wallet_outputs (ld=0x8f6a35a7828, ctx=0x8f6a35c0268) at wallet/test/run-wallet.c:1104
    #8  0x000008f44bcddec0 in main (argc=1, argv=0x7f7ffffcaaf8) at wallet/test/run-wallet.c:1930

Fix by explicitely setting the whole structure to zero.

[ Rebuilt generated files, too --RR ]
rustyrussell pushed a commit that referenced this pull request Feb 14, 2023
This will fix a crash that I caused on armv7
and by looking inside the coredump with gdb
(by adding an assert on n that must be
different from null) I get the following stacktrace

```
(gdb) bt
\#0  0x00000000 in ?? ()
\#1  0x0043a038 in send_backtrace (why=0xbe9e3600 "FATAL SIGNAL 11") at common/daemon.c:36
\#2  0x0043a0ec in crashdump (sig=11) at common/daemon.c:46
\#3  <signal handler called>
\#4  0x00406d04 in node_announcement (map=0x938ecc, nann_off=495146) at common/gossmap.c:586
\#5  0x00406fec in map_catchup (map=0x938ecc, num_rejected=0xbe9e3a40) at common/gossmap.c:643
\#6  0x004073a4 in load_gossip_store (map=0x938ecc, num_rejected=0xbe9e3a40) at common/gossmap.c:697
\#7  0x00408244 in gossmap_load (ctx=0x0, filename=0x4e16b8 "gossip_store", num_channel_updates_rejected=0xbe9e3a40) at common/gossmap.c:976
\#8  0x0041a548 in init (p=0x93831c, buf=0x9399d4 "\n\n{\"jsonrpc\":\"2.0\",\"id\":\"cln:init#25\",\"method\":\"init\",\"params\":{\"options\":{},\"configuration\":{\"lightning-dir\":\"/home/vincent/.lightning/testnet\",\"rpc-file\":\"lightning-rpc\",\"startup\":true,\"network\":\"te"..., config=0x939cdc) at plugins/topology.c:622
\#9  0x0041e5d0 in handle_init (cmd=0x938934, buf=0x9399d4 "\n\n{\"jsonrpc\":\"2.0\",\"id\":\"cln:init#25\",\"method\":\"init\",\"params\":{\"options\":{},\"configuration\":{\"lightning-dir\":\"/home/vincent/.lightning/testnet\",\"rpc-file\":\"lightning-rpc\",\"startup\":true,\"network\":\"te"..., params=0x939c8c)
    at plugins/libplugin.c:1208
\#10 0x0041fc04 in ld_command_handle (plugin=0x93831c, toks=0x939bec) at plugins/libplugin.c:1572
\#11 0x00420050 in ld_read_json_one (plugin=0x93831c) at plugins/libplugin.c:1667
\#12 0x004201bc in ld_read_json (conn=0x9391c4, plugin=0x93831c) at plugins/libplugin.c:1687
\#13 0x004cb82c in next_plan (conn=0x9391c4, plan=0x9391d8) at ccan/ccan/io/io.c:59
\ElementsProject#14 0x004cc67c in do_plan (conn=0x9391c4, plan=0x9391d8, idle_on_epipe=false) at ccan/ccan/io/io.c:407
\ElementsProject#15 0x004cc6dc in io_ready (conn=0x9391c4, pollflags=1) at ccan/ccan/io/io.c:417
\ElementsProject#16 0x004cf8cc in io_loop (timers=0x9383c4, expired=0xbe9e3ce4) at ccan/ccan/io/poll.c:453
\ElementsProject#17 0x00420af4 in plugin_main (argv=0xbe9e3eb4, init=0x41a46c <init>, restartability=PLUGIN_STATIC, init_rpc=true, features=0x0, commands=0x6167e8 <commands>, num_commands=4, notif_subs=0x0, num_notif_subs=0, hook_subs=0x0, num_hook_subs=0, notif_topics=0x0, num_notif_topics=0) at plugins/libplugin.c:1891
\ElementsProject#18 0x0041a6f8 in main (argc=1, argv=0xbe9e3eb4) at plugins/topology.c:679
```

I do not know if this is a solution because I do not know
when I can parse a node announcement for a node that
it is not longer in the gossip map.

So, I hope this is just usefult for @rustyrussell

Changelog-Fixed: fixes `FATAL SIGNAL 11` on gossmap node announcement parsing.

Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
rustyrussell pushed a commit that referenced this pull request Mar 23, 2023
The issue is that common_setup() wasn't called by the fuzz target,
leaving secp256k1_ctx as NULL.

UBSan error:

$ UBSAN_OPTIONS="print_stacktrace=1:halt_on_error=1" \
    ./fuzz-channel_id crash-1575b41ef09e62e4c09c165e6dc037a110b113f2

INFO: Running with entropic power schedule (0xFF, 100).
INFO: Seed: 1153355603
INFO: Loaded 1 modules   (25915 inline 8-bit counters): 25915 [0x563bae7ac3a8, 0x563bae7b28e3),
INFO: Loaded 1 PC tables (25915 PCs): 25915 [0x563bae7b28e8,0x563bae817c98),
./fuzz-channel_id: Running 1 inputs 1 time(s) each.
Running: crash-1575b41ef09e62e4c09c165e6dc037a110b113f2
bitcoin/pubkey.c:22:33: runtime error: null pointer passed as argument 1, which is declared to never be null
external/libwally-core/src/secp256k1/include/secp256k1.h:373:3: note: nonnull attribute specified here
    #0 0x563bae41e3db in pubkey_from_der bitcoin/pubkey.c:19:7
    #1 0x563bae4205e0 in fromwire_pubkey bitcoin/pubkey.c:111:7
    #2 0x563bae46437c in run tests/fuzz/fuzz-channel_id.c:42:3
    #3 0x563bae2f6016 in LLVMFuzzerTestOneInput tests/fuzz/libfuzz.c:23:2
    #4 0x563bae20a450 in fuzzer::Fuzzer::ExecuteCallback(unsigned char const*, unsigned long)
    #5 0x563bae1f4c3f in fuzzer::RunOneTest(fuzzer::Fuzzer*, char const*, unsigned long)
    #6 0x563bae1fa6e6 in fuzzer::FuzzerDriver(int*, char***, int (*)(unsigned char const*, unsigned long))
    #7 0x563bae223052 in main (tests/fuzz/fuzz-channel_id+0x181052) (BuildId: f7f56e14ffc06df54ab732d79ea922e773de1f25)
    #8 0x7fa7fa113082 in __libc_start_main
    #9 0x563bae1efbdd in _start

SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior bitcoin/pubkey.c:22:33 in
rustyrussell pushed a commit that referenced this pull request Jun 6, 2023
It is possible for db_column_bytes() to return 0 and for
db_column_blob() to return NULL even when db_column_is_null() returns
false. We need to short circuit in this case.

Detected by UBSan:

  db/bindings.c:479:12: runtime error: null pointer passed as argument 2, which is declared to never be null
  /usr/include/string.h:44:28: note: nonnull attribute specified here

  #0 0x95f117 in db_col_arr_ db/bindings.c:479:2
  #1 0x95ef85 in db_col_channel_type db/bindings.c:459:32
  #2 0x852c03 in wallet_stmt2channel wallet/wallet.c:1483:9
  #3 0x81f396 in wallet_channels_load_active wallet/wallet.c:1749:23
  #4 0x81f03d in wallet_init_channels wallet/wallet.c:1765:9
  #5 0x72f1f9 in load_channels_from_wallet lightningd/peer_control.c:2257:7
  #6 0x672856 in main lightningd/lightningd.c:1121:25
rustyrussell pushed a commit that referenced this pull request Jun 6, 2023
Fixes nullability errors detected by UBSan:

wire/fromwire.c:173:46: runtime error: null pointer passed as argument 1, which is declared to never be null
external/libwally-core/src/secp256k1/include/secp256k1.h:432:3: note: nonnull attribute specified here
    #0 0x65214a in fromwire_secp256k1_ecdsa_signature wire/fromwire.c:173:6
    #1 0x659500 in printwire_secp256k1_ecdsa_signature devtools/print_wire.c:331:1
    #2 0x646ba2 in printwire_channel_update wire/peer_printgen.c:1900:7
    #3 0x637182 in printpeer_wire_message wire/peer_printgen.c:128:11
    #4 0x65a097 in main devtools/decodemsg.c:85:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants