Don't use u32 locktime internally. #6

rustyrussell · 2015-10-01T00:37:47Z

We use it for both relative timeouts, where it's actually used in nSequence and OP_CSV, and absolute timeouts in locktime. Over the wire it's represented explicitly, but converting to locktime then BIP68 nSequence is just weird.

The latest version of the BIP doesn't use inversion, but does use bitshifts. It also uncovered a bug in the test scripts: the block timestamps creep forward when we generate large numbers of blocks (UpdateTime insists it be > GetMedianTimePast() so it's valid). We need to take this into account when waiting for the median to move (reduced it from 60 to 30 seconds, since that adds about 14 seconds). Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

rustyrussell · 2016-09-27T05:50:28Z

Fixed in 4b02c6f

I actually hit this very hard to reproduce race: if we haven't process the channeld message when block #6 comes in, we won't send the gossip message. We wait for logs, but don't generate new blocks, and timeout on l1.daemon.wait_for_log('peer_out WIRE_ANNOUNCEMENT_SIGNATURES'). The solution, which also tests that we don't send announcement signatures immediately, is to generate a single block, wait for CHANNELD_NORMAL, then (in gossip tests), generate 5 more. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

We hit: assert(!peer->handle_master_reply); #4 0x000055bba3b030a0 in master_sync_reply (peer=0x55bba41c0030, msg=0x55bba41c6a80 "", replytype=WIRE_CHANNEL_GOT_COMMITSIG_REPLY, handle=0x55bba3b041cf <handle_reply_wake_peer>) at channeld/channel.c:518 #5 0x000055bba3b049bc in handle_peer_commit_sig (conn=0x55bba41c10d0, peer=0x55bba41c0030, msg=0x55bba41c6a80 "") at channeld/channel.c:959 #6 0x000055bba3b05c69 in peer_in (conn=0x55bba41c10d0, peer=0x55bba41c0030, msg=0x55bba41c67c0 "") at channeld/channel.c:1339 #7 0x000055bba3b123eb in peer_decrypt_body (conn=0x55bba41c10d0, pcs=0x55bba41c0030) at common/cryptomsg.c:155 #8 0x000055bba3b2c63b in next_plan (conn=0x55bba41c10d0, plan=0x55bba41c1100) at ccan/ccan/io/io.c:59 We got a commit_sig from the peer while waiting for the master to reply to acknowledge the commitsig we want to send (handle_sending_commitsig_reply). The fix is to go always talk to the master synchronous, and not try to process anything but messages from the master daemon. This avoids the whole class of problems. There's a fairly simple way to do this, as ccan/io lets you override its poll call: we process any outstanding master requests there, or add the master fd to the pollfds array. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

We hit: assert(!peer->handle_master_reply); #4 0x000055bba3b030a0 in master_sync_reply (peer=0x55bba41c0030, msg=0x55bba41c6a80 "", replytype=WIRE_CHANNEL_GOT_COMMITSIG_REPLY, handle=0x55bba3b041cf <handle_reply_wake_peer>) at channeld/channel.c:518 #5 0x000055bba3b049bc in handle_peer_commit_sig (conn=0x55bba41c10d0, peer=0x55bba41c0030, msg=0x55bba41c6a80 "") at channeld/channel.c:959 #6 0x000055bba3b05c69 in peer_in (conn=0x55bba41c10d0, peer=0x55bba41c0030, msg=0x55bba41c67c0 "") at channeld/channel.c:1339 #7 0x000055bba3b123eb in peer_decrypt_body (conn=0x55bba41c10d0, pcs=0x55bba41c0030) at common/cryptomsg.c:155 #8 0x000055bba3b2c63b in next_plan (conn=0x55bba41c10d0, plan=0x55bba41c1100) at ccan/ccan/io/io.c:59 We got a commit_sig from the peer while waiting for the master to reply to acknowledge the commitsig we want to send (handle_sending_commitsig_reply). The fix is to go always talk to the master synchronous, and not try to process anything but messages from the master daemon. This avoids the whole class of problems. There's a fairly simple way to do this, as ccan/io lets you override its poll call: we process any outstanding master requests there, or add the master fd to the pollfds array. Fixes: ElementsProject#266 Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

We hit: assert(!peer->handle_master_reply); #4 0x000055bba3b030a0 in master_sync_reply (peer=0x55bba41c0030, msg=0x55bba41c6a80 "", replytype=WIRE_CHANNEL_GOT_COMMITSIG_REPLY, handle=0x55bba3b041cf <handle_reply_wake_peer>) at channeld/channel.c:518 #5 0x000055bba3b049bc in handle_peer_commit_sig (conn=0x55bba41c10d0, peer=0x55bba41c0030, msg=0x55bba41c6a80 "") at channeld/channel.c:959 #6 0x000055bba3b05c69 in peer_in (conn=0x55bba41c10d0, peer=0x55bba41c0030, msg=0x55bba41c67c0 "") at channeld/channel.c:1339 #7 0x000055bba3b123eb in peer_decrypt_body (conn=0x55bba41c10d0, pcs=0x55bba41c0030) at common/cryptomsg.c:155 #8 0x000055bba3b2c63b in next_plan (conn=0x55bba41c10d0, plan=0x55bba41c1100) at ccan/ccan/io/io.c:59 We got a commit_sig from the peer while waiting for the master to reply to acknowledge the commitsig we want to send (handle_sending_commitsig_reply). The fix is to go always talk to the master synchronous, and not try to process anything but messages from the master daemon. This avoids the whole class of problems. There's a fairly simple way to do this, as ccan/io lets you override its poll call: we process any outstanding master requests there, or add the master fd to the pollfds array. Fixes: #266 Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

distributing the keys securely

Don't do this: (gdb) bt #0 0x00007f37ae667c40 in ?? () from /lib/x86_64-linux-gnu/libz.so.1 #1 0x00007f37ae668b38 in ?? () from /lib/x86_64-linux-gnu/libz.so.1 #2 0x00007f37ae669907 in deflate () from /lib/x86_64-linux-gnu/libz.so.1 #3 0x00007f37ae674c65 in compress2 () from /lib/x86_64-linux-gnu/libz.so.1 #4 0x000000000040cfe3 in zencode_scids (ctx=0xc1f118, scids=0x2599bc49 "\a\325{", len=176320) at gossipd/gossipd.c:218 #5 0x000000000040d0b3 in encode_short_channel_ids_end (encoded=0x7fff8f98d9f0, max_bytes=65490) at gossipd/gossipd.c:236 #6 0x000000000040dd28 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17290511, number_of_blocks=8) at gossipd/gossipd.c:576 #7 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17290511, number_of_blocks=16) at gossipd/gossipd.c:595 #8 0x000000000040ddee in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17290495, number_of_blocks=32) at gossipd/gossipd.c:596 #9 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17290495, number_of_blocks=64) at gossipd/gossipd.c:595 #10 0x000000000040ddee in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17290431, number_of_blocks=128) at gossipd/gossipd.c:596 #11 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17290431, number_of_blocks=256) at gossipd/gossipd.c:595 #12 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17290431, number_of_blocks=512) at gossipd/gossipd.c:595 #13 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17290431, number_of_blocks=1024) at gossipd/gossipd.c:595 ElementsProject#14 0x000000000040ddee in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=2047) at gossipd/gossipd.c:596 ElementsProject#15 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=4095) at gossipd/gossipd.c:595 ElementsProject#16 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=8191) at gossipd/gossipd.c:595 ElementsProject#17 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=16382) at gossipd/gossipd.c:595 ElementsProject#18 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=32764) at gossipd/gossipd.c:595 ElementsProject#19 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=65528) at gossipd/gossipd.c:595 ElementsProject#20 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=131056) at gossipd/gossipd.c:595 ElementsProject#21 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=262112) at gossipd/gossipd.c:595 ElementsProject#22 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=524225) at gossipd/gossipd.c:595 ElementsProject#23 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=1048450) at gossipd/gossipd.c:595 ElementsProject#24 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=2096900) at gossipd/gossipd.c:595 ElementsProject#25 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=4193801) at gossipd/gossipd.c:595 ElementsProject#26 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=8387603) at gossipd/gossipd.c:595 ElementsProject#27 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=16775207) at gossipd/gossipd.c:595 ElementsProject#28 0x000000000040ddee in queue_channel_ranges (peer=0x3868fc8, first_blocknum=514201, number_of_blocks=33550414) at gossipd/gossipd.c:596 ElementsProject#29 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=514201, number_of_blocks=67100829) at gossipd/gossipd.c:595 ElementsProject#30 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=514201, number_of_blocks=134201659) at gossipd/gossipd.c:595 ElementsProject#31 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=514201, number_of_blocks=268403318) at gossipd/gossipd.c:595 ElementsProject#32 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=514201, number_of_blocks=536806636) at gossipd/gossipd.c:595 ElementsProject#33 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=514201, number_of_blocks=1073613273) at gossipd/gossipd.c:595 ElementsProject#34 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=514201, number_of_blocks=2147226547) at gossipd/gossipd.c:595 ElementsProject#35 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=514201, number_of_blocks=4294453094) at gossipd/gossipd.c:595 ElementsProject#36 0x000000000040df26 in handle_query_channel_range (peer=0x3868fc8, msg=0x37e0678 "\001\ao\342\214\n\266\361\263r\301\246\242F\256c\367O\223\036\203e\341Z\b\234h\326\031") at gossipd/gossipd.c:625 The cause was that converting a block number to an scid truncates it at 24 bits. When we look through the index from (truncated number) to (real end number) we get every channel, which is too large to encode, so we iterate again. This fixes both that problem, and also the issue that we'd end up dividing into many empty sections until we get to the highest block number. Instead, we just tack the empty blocks on to then end of the final query. Reported-by: George Vaccaro Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

Don't do this: (gdb) bt #0 0x00007f37ae667c40 in ?? () from /lib/x86_64-linux-gnu/libz.so.1 #1 0x00007f37ae668b38 in ?? () from /lib/x86_64-linux-gnu/libz.so.1 #2 0x00007f37ae669907 in deflate () from /lib/x86_64-linux-gnu/libz.so.1 #3 0x00007f37ae674c65 in compress2 () from /lib/x86_64-linux-gnu/libz.so.1 #4 0x000000000040cfe3 in zencode_scids (ctx=0xc1f118, scids=0x2599bc49 "\a\325{", len=176320) at gossipd/gossipd.c:218 #5 0x000000000040d0b3 in encode_short_channel_ids_end (encoded=0x7fff8f98d9f0, max_bytes=65490) at gossipd/gossipd.c:236 #6 0x000000000040dd28 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17290511, number_of_blocks=8) at gossipd/gossipd.c:576 #7 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17290511, number_of_blocks=16) at gossipd/gossipd.c:595 #8 0x000000000040ddee in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17290495, number_of_blocks=32) at gossipd/gossipd.c:596 #9 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17290495, number_of_blocks=64) at gossipd/gossipd.c:595 #10 0x000000000040ddee in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17290431, number_of_blocks=128) at gossipd/gossipd.c:596 #11 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17290431, number_of_blocks=256) at gossipd/gossipd.c:595 #12 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17290431, number_of_blocks=512) at gossipd/gossipd.c:595 #13 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17290431, number_of_blocks=1024) at gossipd/gossipd.c:595 ElementsProject#14 0x000000000040ddee in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=2047) at gossipd/gossipd.c:596 ElementsProject#15 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=4095) at gossipd/gossipd.c:595 ElementsProject#16 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=8191) at gossipd/gossipd.c:595 ElementsProject#17 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=16382) at gossipd/gossipd.c:595 ElementsProject#18 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=32764) at gossipd/gossipd.c:595 ElementsProject#19 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=65528) at gossipd/gossipd.c:595 ElementsProject#20 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=131056) at gossipd/gossipd.c:595 ElementsProject#21 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=262112) at gossipd/gossipd.c:595 ElementsProject#22 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=524225) at gossipd/gossipd.c:595 ElementsProject#23 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=1048450) at gossipd/gossipd.c:595 ElementsProject#24 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=2096900) at gossipd/gossipd.c:595 ElementsProject#25 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=4193801) at gossipd/gossipd.c:595 ElementsProject#26 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=8387603) at gossipd/gossipd.c:595 ElementsProject#27 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=16775207) at gossipd/gossipd.c:595 ElementsProject#28 0x000000000040ddee in queue_channel_ranges (peer=0x3868fc8, first_blocknum=514201, number_of_blocks=33550414) at gossipd/gossipd.c:596 ElementsProject#29 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=514201, number_of_blocks=67100829) at gossipd/gossipd.c:595 ElementsProject#30 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=514201, number_of_blocks=134201659) at gossipd/gossipd.c:595 ElementsProject#31 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=514201, number_of_blocks=268403318) at gossipd/gossipd.c:595 ElementsProject#32 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=514201, number_of_blocks=536806636) at gossipd/gossipd.c:595 ElementsProject#33 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=514201, number_of_blocks=1073613273) at gossipd/gossipd.c:595 ElementsProject#34 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=514201, number_of_blocks=2147226547) at gossipd/gossipd.c:595 ElementsProject#35 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=514201, number_of_blocks=4294453094) at gossipd/gossipd.c:595 ElementsProject#36 0x000000000040df26 in handle_query_channel_range (peer=0x3868fc8, msg=0x37e0678 "\001\ao\342\214\n\266\361\263r\301\246\242F\256c\367O\223\036\203e\341Z\b\234h\326\031") at gossipd/gossipd.c:625 The cause was that converting a block number to an scid truncates it at 24 bits. When we look through the index from (truncated number) to (real end number) we get every channel, which is too large to encode, so we iterate again. This fixes both that problem, and also the issue that we'd end up dividing into many empty sections until we get to the highest block number. Instead, we just tack the empty blocks on to then end of the final query. (My initial version requested 0xFFFFFFFE blocks, but the dev code which records what blocks were returned can't make a bitmap that big on 32 bit). Reported-by: George Vaccaro Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

Don't do this: (gdb) bt #0 0x00007f37ae667c40 in ?? () from /lib/x86_64-linux-gnu/libz.so.1 #1 0x00007f37ae668b38 in ?? () from /lib/x86_64-linux-gnu/libz.so.1 #2 0x00007f37ae669907 in deflate () from /lib/x86_64-linux-gnu/libz.so.1 #3 0x00007f37ae674c65 in compress2 () from /lib/x86_64-linux-gnu/libz.so.1 #4 0x000000000040cfe3 in zencode_scids (ctx=0xc1f118, scids=0x2599bc49 "\a\325{", len=176320) at gossipd/gossipd.c:218 #5 0x000000000040d0b3 in encode_short_channel_ids_end (encoded=0x7fff8f98d9f0, max_bytes=65490) at gossipd/gossipd.c:236 #6 0x000000000040dd28 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17290511, number_of_blocks=8) at gossipd/gossipd.c:576 #7 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17290511, number_of_blocks=16) at gossipd/gossipd.c:595 #8 0x000000000040ddee in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17290495, number_of_blocks=32) at gossipd/gossipd.c:596 #9 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17290495, number_of_blocks=64) at gossipd/gossipd.c:595 #10 0x000000000040ddee in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17290431, number_of_blocks=128) at gossipd/gossipd.c:596 #11 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17290431, number_of_blocks=256) at gossipd/gossipd.c:595 #12 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17290431, number_of_blocks=512) at gossipd/gossipd.c:595 #13 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17290431, number_of_blocks=1024) at gossipd/gossipd.c:595 #14 0x000000000040ddee in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=2047) at gossipd/gossipd.c:596 #15 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=4095) at gossipd/gossipd.c:595 #16 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=8191) at gossipd/gossipd.c:595 #17 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=16382) at gossipd/gossipd.c:595 #18 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=32764) at gossipd/gossipd.c:595 #19 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=65528) at gossipd/gossipd.c:595 #20 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=131056) at gossipd/gossipd.c:595 #21 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=262112) at gossipd/gossipd.c:595 #22 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=524225) at gossipd/gossipd.c:595 #23 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=1048450) at gossipd/gossipd.c:595 #24 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=2096900) at gossipd/gossipd.c:595 #25 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=4193801) at gossipd/gossipd.c:595 #26 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=8387603) at gossipd/gossipd.c:595 #27 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=16775207) at gossipd/gossipd.c:595 #28 0x000000000040ddee in queue_channel_ranges (peer=0x3868fc8, first_blocknum=514201, number_of_blocks=33550414) at gossipd/gossipd.c:596 #29 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=514201, number_of_blocks=67100829) at gossipd/gossipd.c:595 #30 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=514201, number_of_blocks=134201659) at gossipd/gossipd.c:595 #31 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=514201, number_of_blocks=268403318) at gossipd/gossipd.c:595 #32 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=514201, number_of_blocks=536806636) at gossipd/gossipd.c:595 #33 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=514201, number_of_blocks=1073613273) at gossipd/gossipd.c:595 #34 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=514201, number_of_blocks=2147226547) at gossipd/gossipd.c:595 #35 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=514201, number_of_blocks=4294453094) at gossipd/gossipd.c:595 #36 0x000000000040df26 in handle_query_channel_range (peer=0x3868fc8, msg=0x37e0678 "\001\ao\342\214\n\266\361\263r\301\246\242F\256c\367O\223\036\203e\341Z\b\234h\326\031") at gossipd/gossipd.c:625 The cause was that converting a block number to an scid truncates it at 24 bits. When we look through the index from (truncated number) to (real end number) we get every channel, which is too large to encode, so we iterate again. This fixes both that problem, and also the issue that we'd end up dividing into many empty sections until we get to the highest block number. Instead, we just tack the empty blocks on to then end of the final query. (My initial version requested 0xFFFFFFFE blocks, but the dev code which records what blocks were returned can't make a bitmap that big on 32 bit). Reported-by: George Vaccaro Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

Don't do this: (gdb) bt #0 0x00007f37ae667c40 in ?? () from /lib/x86_64-linux-gnu/libz.so.1 #1 0x00007f37ae668b38 in ?? () from /lib/x86_64-linux-gnu/libz.so.1 #2 0x00007f37ae669907 in deflate () from /lib/x86_64-linux-gnu/libz.so.1 #3 0x00007f37ae674c65 in compress2 () from /lib/x86_64-linux-gnu/libz.so.1 #4 0x000000000040cfe3 in zencode_scids (ctx=0xc1f118, scids=0x2599bc49 "\a\325{", len=176320) at gossipd/gossipd.c:218 ElementsProject#5 0x000000000040d0b3 in encode_short_channel_ids_end (encoded=0x7fff8f98d9f0, max_bytes=65490) at gossipd/gossipd.c:236 ElementsProject#6 0x000000000040dd28 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17290511, number_of_blocks=8) at gossipd/gossipd.c:576 ElementsProject#7 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17290511, number_of_blocks=16) at gossipd/gossipd.c:595 ElementsProject#8 0x000000000040ddee in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17290495, number_of_blocks=32) at gossipd/gossipd.c:596 ElementsProject#9 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17290495, number_of_blocks=64) at gossipd/gossipd.c:595 ElementsProject#10 0x000000000040ddee in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17290431, number_of_blocks=128) at gossipd/gossipd.c:596 ElementsProject#11 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17290431, number_of_blocks=256) at gossipd/gossipd.c:595 ElementsProject#12 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17290431, number_of_blocks=512) at gossipd/gossipd.c:595 ElementsProject#13 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17290431, number_of_blocks=1024) at gossipd/gossipd.c:595 ElementsProject#14 0x000000000040ddee in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=2047) at gossipd/gossipd.c:596 ElementsProject#15 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=4095) at gossipd/gossipd.c:595 ElementsProject#16 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=8191) at gossipd/gossipd.c:595 ElementsProject#17 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=16382) at gossipd/gossipd.c:595 ElementsProject#18 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=32764) at gossipd/gossipd.c:595 ElementsProject#19 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=65528) at gossipd/gossipd.c:595 ElementsProject#20 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=131056) at gossipd/gossipd.c:595 ElementsProject#21 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=262112) at gossipd/gossipd.c:595 ElementsProject#22 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=524225) at gossipd/gossipd.c:595 ElementsProject#23 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=1048450) at gossipd/gossipd.c:595 ElementsProject#24 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=2096900) at gossipd/gossipd.c:595 ElementsProject#25 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=4193801) at gossipd/gossipd.c:595 ElementsProject#26 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=8387603) at gossipd/gossipd.c:595 ElementsProject#27 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=16775207) at gossipd/gossipd.c:595 ElementsProject#28 0x000000000040ddee in queue_channel_ranges (peer=0x3868fc8, first_blocknum=514201, number_of_blocks=33550414) at gossipd/gossipd.c:596 ElementsProject#29 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=514201, number_of_blocks=67100829) at gossipd/gossipd.c:595 ElementsProject#30 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=514201, number_of_blocks=134201659) at gossipd/gossipd.c:595 ElementsProject#31 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=514201, number_of_blocks=268403318) at gossipd/gossipd.c:595 ElementsProject#32 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=514201, number_of_blocks=536806636) at gossipd/gossipd.c:595 ElementsProject#33 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=514201, number_of_blocks=1073613273) at gossipd/gossipd.c:595 ElementsProject#34 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=514201, number_of_blocks=2147226547) at gossipd/gossipd.c:595 ElementsProject#35 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=514201, number_of_blocks=4294453094) at gossipd/gossipd.c:595 ElementsProject#36 0x000000000040df26 in handle_query_channel_range (peer=0x3868fc8, msg=0x37e0678 "\001\ao\342\214\n\266\361\263r\301\246\242F\256c\367O\223\036\203e\341Z\b\234h\326\031") at gossipd/gossipd.c:625 The cause was that converting a block number to an scid truncates it at 24 bits. When we look through the index from (truncated number) to (real end number) we get every channel, which is too large to encode, so we iterate again. This fixes both that problem, and also the issue that we'd end up dividing into many empty sections until we get to the highest block number. Instead, we just tack the empty blocks on to then end of the final query. (My initial version requested 0xFFFFFFFE blocks, but the dev code which records what blocks were returned can't make a bitmap that big on 32 bit). Reported-by: George Vaccaro Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

Don't do this: (gdb) bt #0 0x00007f37ae667c40 in ?? () from /lib/x86_64-linux-gnu/libz.so.1 ElementsProject#1 0x00007f37ae668b38 in ?? () from /lib/x86_64-linux-gnu/libz.so.1 ElementsProject#2 0x00007f37ae669907 in deflate () from /lib/x86_64-linux-gnu/libz.so.1 ElementsProject#3 0x00007f37ae674c65 in compress2 () from /lib/x86_64-linux-gnu/libz.so.1 ElementsProject#4 0x000000000040cfe3 in zencode_scids (ctx=0xc1f118, scids=0x2599bc49 "\a\325{", len=176320) at gossipd/gossipd.c:218 ElementsProject#5 0x000000000040d0b3 in encode_short_channel_ids_end (encoded=0x7fff8f98d9f0, max_bytes=65490) at gossipd/gossipd.c:236 ElementsProject#6 0x000000000040dd28 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17290511, number_of_blocks=8) at gossipd/gossipd.c:576 ElementsProject#7 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17290511, number_of_blocks=16) at gossipd/gossipd.c:595 ElementsProject#8 0x000000000040ddee in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17290495, number_of_blocks=32) at gossipd/gossipd.c:596 ElementsProject#9 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17290495, number_of_blocks=64) at gossipd/gossipd.c:595 ElementsProject#10 0x000000000040ddee in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17290431, number_of_blocks=128) at gossipd/gossipd.c:596 ElementsProject#11 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17290431, number_of_blocks=256) at gossipd/gossipd.c:595 ElementsProject#12 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17290431, number_of_blocks=512) at gossipd/gossipd.c:595 ElementsProject#13 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17290431, number_of_blocks=1024) at gossipd/gossipd.c:595 ElementsProject#14 0x000000000040ddee in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=2047) at gossipd/gossipd.c:596 ElementsProject#15 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=4095) at gossipd/gossipd.c:595 ElementsProject#16 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=8191) at gossipd/gossipd.c:595 ElementsProject#17 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=16382) at gossipd/gossipd.c:595 ElementsProject#18 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=32764) at gossipd/gossipd.c:595 ElementsProject#19 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=65528) at gossipd/gossipd.c:595 ElementsProject#20 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=131056) at gossipd/gossipd.c:595 ElementsProject#21 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=262112) at gossipd/gossipd.c:595 ElementsProject#22 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=524225) at gossipd/gossipd.c:595 ElementsProject#23 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=1048450) at gossipd/gossipd.c:595 ElementsProject#24 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=2096900) at gossipd/gossipd.c:595 ElementsProject#25 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=4193801) at gossipd/gossipd.c:595 ElementsProject#26 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=8387603) at gossipd/gossipd.c:595 ElementsProject#27 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=16775207) at gossipd/gossipd.c:595 ElementsProject#28 0x000000000040ddee in queue_channel_ranges (peer=0x3868fc8, first_blocknum=514201, number_of_blocks=33550414) at gossipd/gossipd.c:596 ElementsProject#29 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=514201, number_of_blocks=67100829) at gossipd/gossipd.c:595 ElementsProject#30 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=514201, number_of_blocks=134201659) at gossipd/gossipd.c:595 ElementsProject#31 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=514201, number_of_blocks=268403318) at gossipd/gossipd.c:595 ElementsProject#32 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=514201, number_of_blocks=536806636) at gossipd/gossipd.c:595 ElementsProject#33 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=514201, number_of_blocks=1073613273) at gossipd/gossipd.c:595 ElementsProject#34 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=514201, number_of_blocks=2147226547) at gossipd/gossipd.c:595 ElementsProject#35 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=514201, number_of_blocks=4294453094) at gossipd/gossipd.c:595 ElementsProject#36 0x000000000040df26 in handle_query_channel_range (peer=0x3868fc8, msg=0x37e0678 "\001\ao\342\214\n\266\361\263r\301\246\242F\256c\367O\223\036\203e\341Z\b\234h\326\031") at gossipd/gossipd.c:625 The cause was that converting a block number to an scid truncates it at 24 bits. When we look through the index from (truncated number) to (real end number) we get every channel, which is too large to encode, so we iterate again. This fixes both that problem, and also the issue that we'd end up dividing into many empty sections until we get to the highest block number. Instead, we just tack the empty blocks on to then end of the final query. (My initial version requested 0xFFFFFFFE blocks, but the dev code which records what blocks were returned can't make a bitmap that big on 32 bit). Reported-by: George Vaccaro Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

Direct leak of 1024 byte(s) in 2 object(s) allocated from: #0 0x7f4c84ce4448 in malloc (/lib/x86_64-linux-gnu/libasan.so.5+0x10c448) #1 0x55d11b782c96 in timer_default_alloc ccan/ccan/timer/timer.c:16 #2 0x55d11b7832b7 in add_level ccan/ccan/timer/timer.c:166 #3 0x55d11b783864 in timer_fast_forward ccan/ccan/timer/timer.c:334 #4 0x55d11b78396a in timers_expire ccan/ccan/timer/timer.c:359 #5 0x55d11b774993 in io_loop ccan/ccan/io/poll.c:395 #6 0x55d11b72322f in plugins_init lightningd/plugin.c:1013 #7 0x55d11b7060ea in main lightningd/lightningd.c:664 #8 0x7f4c84696b6a in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x26b6a) Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

Indirect leak of 48 byte(s) in 1 object(s) allocated from: #0 0x7f4c84ce4448 in malloc (/lib/x86_64-linux-gnu/libasan.so.5+0x10c448) #1 0x55d11b77d270 in strmap_add_ ccan/ccan/strmap/strmap.c:90 #2 0x55d11b704603 in command_set_usage lightningd/jsonrpc.c:891 #3 0x55d11b733cb5 in param common/param.c:295 #4 0x55d11b6f7b37 in json_connect lightningd/connect_control.c:96 #5 0x55d11b7042ef in setup_command_usage lightningd/jsonrpc.c:841 #6 0x55d11b70443b in jsonrpc_command_add_perm lightningd/jsonrpc.c:863 #7 0x55d11b704533 in jsonrpc_setup lightningd/jsonrpc.c:876 #8 0x55d11b705695 in new_lightningd lightningd/lightningd.c:210 #9 0x55d11b706062 in main lightningd/lightningd.c:644 #10 0x7f4c84696b6a in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x26b6a) Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

Direct leak of 64 byte(s) in 1 object(s) allocated from: #0 0x7f4dc279163e in calloc (/lib/x86_64-linux-gnu/libasan.so.5+0x10c63e) #1 0x564ee8a24bb1 in htable_default_alloc ccan/ccan/htable/htable.c:19 #2 0x564ee8a2551b in double_table ccan/ccan/htable/htable.c:226 #3 0x564ee8a259e5 in htable_add_ ccan/ccan/htable/htable.c:331 #4 0x564ee89a5300 in block_map_add lightningd/chaintopology.h:83 #5 0x564ee89a6ece in add_tip lightningd/chaintopology.c:626 #6 0x564ee89a72c3 in have_new_block lightningd/chaintopology.c:694 #7 0x564ee89a3ab0 in process_rawblock lightningd/bitcoind.c:466 #8 0x564ee89a2fb4 in bcli_finished lightningd/bitcoind.c:214 #9 0x564ee8a284d6 in destroy_conn ccan/ccan/io/poll.c:244 #10 0x564ee8a284f6 in destroy_conn_close_fd ccan/ccan/io/poll.c:250 #11 0x564ee8a34a0d in notify ccan/ccan/tal/tal.c:235 #12 0x564ee8a34efc in del_tree ccan/ccan/tal/tal.c:397 #13 0x564ee8a35288 in tal_free ccan/ccan/tal/tal.c:481 ElementsProject#14 0x564ee8a26cf5 in io_close ccan/ccan/io/io.c:450 ElementsProject#15 0x564ee8a28c11 in io_loop ccan/ccan/io/poll.c:449 ElementsProject#16 0x564ee89b3c3b in io_loop_with_timers lightningd/io_loop_with_timers.c:24 ElementsProject#17 0x564ee89ba540 in main lightningd/lightningd.c:822 ElementsProject#18 0x7f4dc2143b6a in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x26b6a) Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

Direct leak of 32 byte(s) in 1 object(s) allocated from: #0 0x7f7678ee863e in calloc (/lib/x86_64-linux-gnu/libasan.so.5+0x10c63e) #1 0x55f8c7b0fce5 in htable_default_alloc ccan/ccan/htable/htable.c:19 #2 0x55f8c7b1064f in double_table ccan/ccan/htable/htable.c:226 #3 0x55f8c7b10b19 in htable_add_ ccan/ccan/htable/htable.c:331 #4 0x55f8c7afac63 in scriptpubkeyset_add wallet/txfilter.c:30 #5 0x55f8c7afafce in txfilter_add_scriptpubkey wallet/txfilter.c:77 #6 0x55f8c7afb05f in txfilter_add_derkey wallet/txfilter.c:91 #7 0x55f8c7aa4d67 in init_txfilter lightningd/lightningd.c:482 #8 0x55f8c7aa52d8 in main lightningd/lightningd.c:721 #9 0x7f767889ab6a in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x26b6a) Direct leak of 16 byte(s) in 1 object(s) allocated from: #0 0x7f05f389563e in calloc (/lib/x86_64-linux-gnu/libasan.so.5+0x10c63e) #1 0x55cac1e6bc99 in htable_default_alloc ccan/ccan/htable/htable.c:19 #2 0x55cac1e6c603 in double_table ccan/ccan/htable/htable.c:226 #3 0x55cac1e6cacd in htable_add_ ccan/ccan/htable/htable.c:331 #4 0x55cac1e56e48 in outpointset_add wallet/txfilter.c:61 #5 0x55cac1e57162 in outpointfilter_add wallet/txfilter.c:116 #6 0x55cac1e5ea3a in wallet_utxoset_add wallet/wallet.c:2365 #7 0x55cac1deddc2 in topo_add_utxos lightningd/chaintopology.c:603 #8 0x55cac1dedeac in add_tip lightningd/chaintopology.c:620 #9 0x55cac1dee2de in have_new_block lightningd/chaintopology.c:694 #10 0x55cac1deaab0 in process_rawblock lightningd/bitcoind.c:466 #11 0x55cac1de9fb4 in bcli_finished lightningd/bitcoind.c:214 #12 0x55cac1e6f5be in destroy_conn ccan/ccan/io/poll.c:244 #13 0x55cac1e6f5de in destroy_conn_close_fd ccan/ccan/io/poll.c:250 ElementsProject#14 0x55cac1e7baf5 in notify ccan/ccan/tal/tal.c:235 ElementsProject#15 0x55cac1e7bfe4 in del_tree ccan/ccan/tal/tal.c:397 ElementsProject#16 0x55cac1e7c370 in tal_free ccan/ccan/tal/tal.c:481 ElementsProject#17 0x55cac1e6dddd in io_close ccan/ccan/io/io.c:450 ElementsProject#18 0x55cac1e6fcf9 in io_loop ccan/ccan/io/poll.c:449 ElementsProject#19 0x55cac1dfac66 in io_loop_with_timers lightningd/io_loop_with_timers.c:24 ElementsProject#20 0x55cac1e0156b in main lightningd/lightningd.c:822 ElementsProject#21 0x7f05f3247b6a in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x26b6a) Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

Direct leak of 16 byte(s) in 1 object(s) allocated from: #0 0x7ff02889063e in calloc (/lib/x86_64-linux-gnu/libasan.so.5+0x10c63e) #1 0x555ce2ad8d2e in htable_default_alloc ccan/ccan/htable/htable.c:19 #2 0x555ce2ad9698 in double_table ccan/ccan/htable/htable.c:226 #3 0x555ce2ad9b62 in htable_add_ ccan/ccan/htable/htable.c:331 #4 0x555ce2a638e4 in htlc_in_map_add lightningd/htlc_end.h:113 #5 0x555ce2a63beb in connect_htlc_in lightningd/htlc_end.c:39 #6 0x555ce2a85cbc in channel_added_their_htlc lightningd/peer_htlcs.c:1382 #7 0x555ce2a860e1 in peer_got_commitsig lightningd/peer_htlcs.c:1466 #8 0x555ce2a5db04 in channel_msg lightningd/channel_control.c:228 #9 0x555ce2a8d393 in sd_msg_read lightningd/subd.c:474 #10 0x555ce2ada157 in next_plan ccan/ccan/io/io.c:59 #11 0x555ce2adacd4 in do_plan ccan/ccan/io/io.c:407 #12 0x555ce2adad12 in io_ready ccan/ccan/io/io.c:417 #13 0x555ce2adcd67 in io_loop ccan/ccan/io/poll.c:445 ElementsProject#14 0x555ce2a67c66 in io_loop_with_timers lightningd/io_loop_with_timers.c:24 ElementsProject#15 0x555ce2a6e56b in main lightningd/lightningd.c:822 ElementsProject#16 0x7ff028242b6a in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x26b6a) Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

Direct leak of 1024 byte(s) in 2 object(s) allocated from: #0 0x7f4c84ce4448 in malloc (/lib/x86_64-linux-gnu/libasan.so.5+0x10c448) #1 0x55d11b782c96 in timer_default_alloc ccan/ccan/timer/timer.c:16 #2 0x55d11b7832b7 in add_level ccan/ccan/timer/timer.c:166 #3 0x55d11b783864 in timer_fast_forward ccan/ccan/timer/timer.c:334 #4 0x55d11b78396a in timers_expire ccan/ccan/timer/timer.c:359 #5 0x55d11b774993 in io_loop ccan/ccan/io/poll.c:395 #6 0x55d11b72322f in plugins_init lightningd/plugin.c:1013 #7 0x55d11b7060ea in main lightningd/lightningd.c:664 #8 0x7f4c84696b6a in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x26b6a) To fix this, we actually make 'ld->timers' a pointer, so we can clean it up last of all. We can't free it before ld, because that causes timers to be destroyed. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

Indirect leak of 48 byte(s) in 1 object(s) allocated from: #0 0x7f4c84ce4448 in malloc (/lib/x86_64-linux-gnu/libasan.so.5+0x10c448) #1 0x55d11b77d270 in strmap_add_ ccan/ccan/strmap/strmap.c:90 #2 0x55d11b704603 in command_set_usage lightningd/jsonrpc.c:891 #3 0x55d11b733cb5 in param common/param.c:295 #4 0x55d11b6f7b37 in json_connect lightningd/connect_control.c:96 #5 0x55d11b7042ef in setup_command_usage lightningd/jsonrpc.c:841 #6 0x55d11b70443b in jsonrpc_command_add_perm lightningd/jsonrpc.c:863 #7 0x55d11b704533 in jsonrpc_setup lightningd/jsonrpc.c:876 #8 0x55d11b705695 in new_lightningd lightningd/lightningd.c:210 #9 0x55d11b706062 in main lightningd/lightningd.c:644 #10 0x7f4c84696b6a in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x26b6a) Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

Direct leak of 64 byte(s) in 1 object(s) allocated from: #0 0x7f4dc279163e in calloc (/lib/x86_64-linux-gnu/libasan.so.5+0x10c63e) #1 0x564ee8a24bb1 in htable_default_alloc ccan/ccan/htable/htable.c:19 #2 0x564ee8a2551b in double_table ccan/ccan/htable/htable.c:226 #3 0x564ee8a259e5 in htable_add_ ccan/ccan/htable/htable.c:331 #4 0x564ee89a5300 in block_map_add lightningd/chaintopology.h:83 #5 0x564ee89a6ece in add_tip lightningd/chaintopology.c:626 #6 0x564ee89a72c3 in have_new_block lightningd/chaintopology.c:694 #7 0x564ee89a3ab0 in process_rawblock lightningd/bitcoind.c:466 #8 0x564ee89a2fb4 in bcli_finished lightningd/bitcoind.c:214 #9 0x564ee8a284d6 in destroy_conn ccan/ccan/io/poll.c:244 #10 0x564ee8a284f6 in destroy_conn_close_fd ccan/ccan/io/poll.c:250 #11 0x564ee8a34a0d in notify ccan/ccan/tal/tal.c:235 #12 0x564ee8a34efc in del_tree ccan/ccan/tal/tal.c:397 #13 0x564ee8a35288 in tal_free ccan/ccan/tal/tal.c:481 #14 0x564ee8a26cf5 in io_close ccan/ccan/io/io.c:450 #15 0x564ee8a28c11 in io_loop ccan/ccan/io/poll.c:449 #16 0x564ee89b3c3b in io_loop_with_timers lightningd/io_loop_with_timers.c:24 #17 0x564ee89ba540 in main lightningd/lightningd.c:822 #18 0x7f4dc2143b6a in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x26b6a) Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

Direct leak of 32 byte(s) in 1 object(s) allocated from: #0 0x7f7678ee863e in calloc (/lib/x86_64-linux-gnu/libasan.so.5+0x10c63e) #1 0x55f8c7b0fce5 in htable_default_alloc ccan/ccan/htable/htable.c:19 #2 0x55f8c7b1064f in double_table ccan/ccan/htable/htable.c:226 #3 0x55f8c7b10b19 in htable_add_ ccan/ccan/htable/htable.c:331 #4 0x55f8c7afac63 in scriptpubkeyset_add wallet/txfilter.c:30 #5 0x55f8c7afafce in txfilter_add_scriptpubkey wallet/txfilter.c:77 #6 0x55f8c7afb05f in txfilter_add_derkey wallet/txfilter.c:91 #7 0x55f8c7aa4d67 in init_txfilter lightningd/lightningd.c:482 #8 0x55f8c7aa52d8 in main lightningd/lightningd.c:721 #9 0x7f767889ab6a in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x26b6a) Direct leak of 16 byte(s) in 1 object(s) allocated from: #0 0x7f05f389563e in calloc (/lib/x86_64-linux-gnu/libasan.so.5+0x10c63e) #1 0x55cac1e6bc99 in htable_default_alloc ccan/ccan/htable/htable.c:19 #2 0x55cac1e6c603 in double_table ccan/ccan/htable/htable.c:226 #3 0x55cac1e6cacd in htable_add_ ccan/ccan/htable/htable.c:331 #4 0x55cac1e56e48 in outpointset_add wallet/txfilter.c:61 #5 0x55cac1e57162 in outpointfilter_add wallet/txfilter.c:116 #6 0x55cac1e5ea3a in wallet_utxoset_add wallet/wallet.c:2365 #7 0x55cac1deddc2 in topo_add_utxos lightningd/chaintopology.c:603 #8 0x55cac1dedeac in add_tip lightningd/chaintopology.c:620 #9 0x55cac1dee2de in have_new_block lightningd/chaintopology.c:694 #10 0x55cac1deaab0 in process_rawblock lightningd/bitcoind.c:466 #11 0x55cac1de9fb4 in bcli_finished lightningd/bitcoind.c:214 #12 0x55cac1e6f5be in destroy_conn ccan/ccan/io/poll.c:244 #13 0x55cac1e6f5de in destroy_conn_close_fd ccan/ccan/io/poll.c:250 #14 0x55cac1e7baf5 in notify ccan/ccan/tal/tal.c:235 #15 0x55cac1e7bfe4 in del_tree ccan/ccan/tal/tal.c:397 #16 0x55cac1e7c370 in tal_free ccan/ccan/tal/tal.c:481 #17 0x55cac1e6dddd in io_close ccan/ccan/io/io.c:450 #18 0x55cac1e6fcf9 in io_loop ccan/ccan/io/poll.c:449 #19 0x55cac1dfac66 in io_loop_with_timers lightningd/io_loop_with_timers.c:24 #20 0x55cac1e0156b in main lightningd/lightningd.c:822 #21 0x7f05f3247b6a in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x26b6a) Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

Direct leak of 16 byte(s) in 1 object(s) allocated from: #0 0x7ff02889063e in calloc (/lib/x86_64-linux-gnu/libasan.so.5+0x10c63e) #1 0x555ce2ad8d2e in htable_default_alloc ccan/ccan/htable/htable.c:19 #2 0x555ce2ad9698 in double_table ccan/ccan/htable/htable.c:226 #3 0x555ce2ad9b62 in htable_add_ ccan/ccan/htable/htable.c:331 #4 0x555ce2a638e4 in htlc_in_map_add lightningd/htlc_end.h:113 #5 0x555ce2a63beb in connect_htlc_in lightningd/htlc_end.c:39 #6 0x555ce2a85cbc in channel_added_their_htlc lightningd/peer_htlcs.c:1382 #7 0x555ce2a860e1 in peer_got_commitsig lightningd/peer_htlcs.c:1466 #8 0x555ce2a5db04 in channel_msg lightningd/channel_control.c:228 #9 0x555ce2a8d393 in sd_msg_read lightningd/subd.c:474 #10 0x555ce2ada157 in next_plan ccan/ccan/io/io.c:59 #11 0x555ce2adacd4 in do_plan ccan/ccan/io/io.c:407 #12 0x555ce2adad12 in io_ready ccan/ccan/io/io.c:417 #13 0x555ce2adcd67 in io_loop ccan/ccan/io/poll.c:445 #14 0x555ce2a67c66 in io_loop_with_timers lightningd/io_loop_with_timers.c:24 #15 0x555ce2a6e56b in main lightningd/lightningd.c:822 #16 0x7ff028242b6a in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x26b6a) Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

Direct leak of 1024 byte(s) in 2 object(s) allocated from: #0 0x7f4c84ce4448 in malloc (/lib/x86_64-linux-gnu/libasan.so.5+0x10c448) #1 0x55d11b782c96 in timer_default_alloc ccan/ccan/timer/timer.c:16 #2 0x55d11b7832b7 in add_level ccan/ccan/timer/timer.c:166 #3 0x55d11b783864 in timer_fast_forward ccan/ccan/timer/timer.c:334 #4 0x55d11b78396a in timers_expire ccan/ccan/timer/timer.c:359 #5 0x55d11b774993 in io_loop ccan/ccan/io/poll.c:395 #6 0x55d11b72322f in plugins_init lightningd/plugin.c:1013 #7 0x55d11b7060ea in main lightningd/lightningd.c:664 #8 0x7f4c84696b6a in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x26b6a) To fix this, we actually make 'ld->timers' a pointer, so we can clean it up last of all. We can't free it before ld, because that causes timers to be destroyed. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

Indirect leak of 48 byte(s) in 1 object(s) allocated from: #0 0x7f4c84ce4448 in malloc (/lib/x86_64-linux-gnu/libasan.so.5+0x10c448) #1 0x55d11b77d270 in strmap_add_ ccan/ccan/strmap/strmap.c:90 #2 0x55d11b704603 in command_set_usage lightningd/jsonrpc.c:891 #3 0x55d11b733cb5 in param common/param.c:295 #4 0x55d11b6f7b37 in json_connect lightningd/connect_control.c:96 #5 0x55d11b7042ef in setup_command_usage lightningd/jsonrpc.c:841 #6 0x55d11b70443b in jsonrpc_command_add_perm lightningd/jsonrpc.c:863 #7 0x55d11b704533 in jsonrpc_setup lightningd/jsonrpc.c:876 #8 0x55d11b705695 in new_lightningd lightningd/lightningd.c:210 #9 0x55d11b706062 in main lightningd/lightningd.c:644 #10 0x7f4c84696b6a in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x26b6a) Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

Direct leak of 64 byte(s) in 1 object(s) allocated from: #0 0x7f4dc279163e in calloc (/lib/x86_64-linux-gnu/libasan.so.5+0x10c63e) #1 0x564ee8a24bb1 in htable_default_alloc ccan/ccan/htable/htable.c:19 #2 0x564ee8a2551b in double_table ccan/ccan/htable/htable.c:226 #3 0x564ee8a259e5 in htable_add_ ccan/ccan/htable/htable.c:331 #4 0x564ee89a5300 in block_map_add lightningd/chaintopology.h:83 #5 0x564ee89a6ece in add_tip lightningd/chaintopology.c:626 #6 0x564ee89a72c3 in have_new_block lightningd/chaintopology.c:694 #7 0x564ee89a3ab0 in process_rawblock lightningd/bitcoind.c:466 #8 0x564ee89a2fb4 in bcli_finished lightningd/bitcoind.c:214 #9 0x564ee8a284d6 in destroy_conn ccan/ccan/io/poll.c:244 #10 0x564ee8a284f6 in destroy_conn_close_fd ccan/ccan/io/poll.c:250 #11 0x564ee8a34a0d in notify ccan/ccan/tal/tal.c:235 #12 0x564ee8a34efc in del_tree ccan/ccan/tal/tal.c:397 #13 0x564ee8a35288 in tal_free ccan/ccan/tal/tal.c:481 #14 0x564ee8a26cf5 in io_close ccan/ccan/io/io.c:450 #15 0x564ee8a28c11 in io_loop ccan/ccan/io/poll.c:449 #16 0x564ee89b3c3b in io_loop_with_timers lightningd/io_loop_with_timers.c:24 #17 0x564ee89ba540 in main lightningd/lightningd.c:822 #18 0x7f4dc2143b6a in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x26b6a) Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

Direct leak of 32 byte(s) in 1 object(s) allocated from: #0 0x7f7678ee863e in calloc (/lib/x86_64-linux-gnu/libasan.so.5+0x10c63e) #1 0x55f8c7b0fce5 in htable_default_alloc ccan/ccan/htable/htable.c:19 #2 0x55f8c7b1064f in double_table ccan/ccan/htable/htable.c:226 #3 0x55f8c7b10b19 in htable_add_ ccan/ccan/htable/htable.c:331 #4 0x55f8c7afac63 in scriptpubkeyset_add wallet/txfilter.c:30 #5 0x55f8c7afafce in txfilter_add_scriptpubkey wallet/txfilter.c:77 #6 0x55f8c7afb05f in txfilter_add_derkey wallet/txfilter.c:91 #7 0x55f8c7aa4d67 in init_txfilter lightningd/lightningd.c:482 #8 0x55f8c7aa52d8 in main lightningd/lightningd.c:721 #9 0x7f767889ab6a in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x26b6a) Direct leak of 16 byte(s) in 1 object(s) allocated from: #0 0x7f05f389563e in calloc (/lib/x86_64-linux-gnu/libasan.so.5+0x10c63e) #1 0x55cac1e6bc99 in htable_default_alloc ccan/ccan/htable/htable.c:19 #2 0x55cac1e6c603 in double_table ccan/ccan/htable/htable.c:226 #3 0x55cac1e6cacd in htable_add_ ccan/ccan/htable/htable.c:331 #4 0x55cac1e56e48 in outpointset_add wallet/txfilter.c:61 #5 0x55cac1e57162 in outpointfilter_add wallet/txfilter.c:116 #6 0x55cac1e5ea3a in wallet_utxoset_add wallet/wallet.c:2365 #7 0x55cac1deddc2 in topo_add_utxos lightningd/chaintopology.c:603 #8 0x55cac1dedeac in add_tip lightningd/chaintopology.c:620 #9 0x55cac1dee2de in have_new_block lightningd/chaintopology.c:694 #10 0x55cac1deaab0 in process_rawblock lightningd/bitcoind.c:466 #11 0x55cac1de9fb4 in bcli_finished lightningd/bitcoind.c:214 #12 0x55cac1e6f5be in destroy_conn ccan/ccan/io/poll.c:244 #13 0x55cac1e6f5de in destroy_conn_close_fd ccan/ccan/io/poll.c:250 #14 0x55cac1e7baf5 in notify ccan/ccan/tal/tal.c:235 #15 0x55cac1e7bfe4 in del_tree ccan/ccan/tal/tal.c:397 #16 0x55cac1e7c370 in tal_free ccan/ccan/tal/tal.c:481 #17 0x55cac1e6dddd in io_close ccan/ccan/io/io.c:450 #18 0x55cac1e6fcf9 in io_loop ccan/ccan/io/poll.c:449 #19 0x55cac1dfac66 in io_loop_with_timers lightningd/io_loop_with_timers.c:24 #20 0x55cac1e0156b in main lightningd/lightningd.c:822 #21 0x7f05f3247b6a in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x26b6a) Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

Direct leak of 16 byte(s) in 1 object(s) allocated from: #0 0x7ff02889063e in calloc (/lib/x86_64-linux-gnu/libasan.so.5+0x10c63e) #1 0x555ce2ad8d2e in htable_default_alloc ccan/ccan/htable/htable.c:19 #2 0x555ce2ad9698 in double_table ccan/ccan/htable/htable.c:226 #3 0x555ce2ad9b62 in htable_add_ ccan/ccan/htable/htable.c:331 #4 0x555ce2a638e4 in htlc_in_map_add lightningd/htlc_end.h:113 #5 0x555ce2a63beb in connect_htlc_in lightningd/htlc_end.c:39 #6 0x555ce2a85cbc in channel_added_their_htlc lightningd/peer_htlcs.c:1382 #7 0x555ce2a860e1 in peer_got_commitsig lightningd/peer_htlcs.c:1466 #8 0x555ce2a5db04 in channel_msg lightningd/channel_control.c:228 #9 0x555ce2a8d393 in sd_msg_read lightningd/subd.c:474 #10 0x555ce2ada157 in next_plan ccan/ccan/io/io.c:59 #11 0x555ce2adacd4 in do_plan ccan/ccan/io/io.c:407 #12 0x555ce2adad12 in io_ready ccan/ccan/io/io.c:417 #13 0x555ce2adcd67 in io_loop ccan/ccan/io/poll.c:445 #14 0x555ce2a67c66 in io_loop_with_timers lightningd/io_loop_with_timers.c:24 #15 0x555ce2a6e56b in main lightningd/lightningd.c:822 #16 0x7ff028242b6a in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x26b6a) Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

The variable `block` (instace of `struct block`) is allocated on the stack without being initialized, i.e. its member `prev` points to nowhere. This causes a segmentation fault on my machine on the binding of "prev_hash" on running `wallet_block_add`, as the following core-dump analysis shows: $ egdb ./wallet/test/run-wallet ./run-wallet.core [...] Core was generated by `run-wallet'. Program terminated with signal SIGSEGV, Segmentation fault. ---Type <return> to continue, or q <return> to quit--- #0 0x000008f67a04b660 in memcpy (dst0=<optimized out>, src0=0x100007f8c, length=32) at /usr/src/lib/libc/string/memcpy.c:97 97 TLOOP1(*dst++ = *src++); (gdb) bt #0 0x000008f67a04b660 in memcpy (dst0=<optimized out>, src0=0x100007f8c, length=32) at /usr/src/lib/libc/string/memcpy.c:97 ElementsProject#1 0x000008f73e838f60 in sqlite3VdbeMemSetStr () from /usr/local/lib/libsqlite3.so.37.12 ElementsProject#2 0x000008f73e83cb11 in bindText () from /usr/local/lib/libsqlite3.so.37.12 ElementsProject#3 0x000008f44bc91345 in db_sqlite3_query (stmt=0x8f6845bf028) at wallet/db_sqlite3.c:77 ElementsProject#4 0x000008f44bc91122 in db_sqlite3_exec (stmt=0x8f6845bf028) at wallet/db_sqlite3.c:110 ElementsProject#5 0x000008f44bcbb3b2 in db_exec_prepared_v2 (stmt=0x8f6845bf028) at ./wallet/db.c:2055 ElementsProject#6 0x000008f44bcc6890 in wallet_block_add (w=0x8f688b5bba8, b=0x7f7ffffca788) at ./wallet/wallet.c:3556 ElementsProject#7 0x000008f44bce2607 in test_wallet_outputs (ld=0x8f6a35a7828, ctx=0x8f6a35c0268) at wallet/test/run-wallet.c:1104 ElementsProject#8 0x000008f44bcddec0 in main (argc=1, argv=0x7f7ffffcaaf8) at wallet/test/run-wallet.c:1930 Fix by explicitely setting the whole structure to zero. [ Rebuilt generated files, too --RR ]

The variable `block` (instace of `struct block`) is allocated on the stack without being initialized, i.e. its member `prev` points to nowhere. This causes a segmentation fault on my machine on the binding of "prev_hash" on running `wallet_block_add`, as the following core-dump analysis shows: $ egdb ./wallet/test/run-wallet ./run-wallet.core [...] Core was generated by `run-wallet'. Program terminated with signal SIGSEGV, Segmentation fault. ---Type <return> to continue, or q <return> to quit--- #0 0x000008f67a04b660 in memcpy (dst0=<optimized out>, src0=0x100007f8c, length=32) at /usr/src/lib/libc/string/memcpy.c:97 97 TLOOP1(*dst++ = *src++); (gdb) bt #0 0x000008f67a04b660 in memcpy (dst0=<optimized out>, src0=0x100007f8c, length=32) at /usr/src/lib/libc/string/memcpy.c:97 #1 0x000008f73e838f60 in sqlite3VdbeMemSetStr () from /usr/local/lib/libsqlite3.so.37.12 #2 0x000008f73e83cb11 in bindText () from /usr/local/lib/libsqlite3.so.37.12 #3 0x000008f44bc91345 in db_sqlite3_query (stmt=0x8f6845bf028) at wallet/db_sqlite3.c:77 #4 0x000008f44bc91122 in db_sqlite3_exec (stmt=0x8f6845bf028) at wallet/db_sqlite3.c:110 #5 0x000008f44bcbb3b2 in db_exec_prepared_v2 (stmt=0x8f6845bf028) at ./wallet/db.c:2055 #6 0x000008f44bcc6890 in wallet_block_add (w=0x8f688b5bba8, b=0x7f7ffffca788) at ./wallet/wallet.c:3556 #7 0x000008f44bce2607 in test_wallet_outputs (ld=0x8f6a35a7828, ctx=0x8f6a35c0268) at wallet/test/run-wallet.c:1104 #8 0x000008f44bcddec0 in main (argc=1, argv=0x7f7ffffcaaf8) at wallet/test/run-wallet.c:1930 Fix by explicitely setting the whole structure to zero. [ Rebuilt generated files, too --RR ]

The variable `block` (instace of `struct block`) is allocated on the stack without being initialized, i.e. its member `prev` points to nowhere. This causes a segmentation fault on my machine on the binding of "prev_hash" on running `wallet_block_add`, as the following core-dump analysis shows: $ egdb ./wallet/test/run-wallet ./run-wallet.core [...] Core was generated by `run-wallet'. Program terminated with signal SIGSEGV, Segmentation fault. ---Type <return> to continue, or q <return> to quit--- #0 0x000008f67a04b660 in memcpy (dst0=<optimized out>, src0=0x100007f8c, length=32) at /usr/src/lib/libc/string/memcpy.c:97 97 TLOOP1(*dst++ = *src++); (gdb) bt #0 0x000008f67a04b660 in memcpy (dst0=<optimized out>, src0=0x100007f8c, length=32) at /usr/src/lib/libc/string/memcpy.c:97 ElementsProject#1 0x000008f73e838f60 in sqlite3VdbeMemSetStr () from /usr/local/lib/libsqlite3.so.37.12 ElementsProject#2 0x000008f73e83cb11 in bindText () from /usr/local/lib/libsqlite3.so.37.12 ElementsProject#3 0x000008f44bc91345 in db_sqlite3_query (stmt=0x8f6845bf028) at wallet/db_sqlite3.c:77 ElementsProject#4 0x000008f44bc91122 in db_sqlite3_exec (stmt=0x8f6845bf028) at wallet/db_sqlite3.c:110 ElementsProject#5 0x000008f44bcbb3b2 in db_exec_prepared_v2 (stmt=0x8f6845bf028) at ./wallet/db.c:2055 ElementsProject#6 0x000008f44bcc6890 in wallet_block_add (w=0x8f688b5bba8, b=0x7f7ffffca788) at ./wallet/wallet.c:3556 ElementsProject#7 0x000008f44bce2607 in test_wallet_outputs (ld=0x8f6a35a7828, ctx=0x8f6a35c0268) at wallet/test/run-wallet.c:1104 ElementsProject#8 0x000008f44bcddec0 in main (argc=1, argv=0x7f7ffffcaaf8) at wallet/test/run-wallet.c:1930 Fix by explicitely setting the whole structure to zero. [ Rebuilt generated files, too --RR ]

@rustyrussell

This will fix a crash that I caused on armv7 and by looking inside the coredump with gdb (by adding an assert on n that must be different from null) I get the following stacktrace ``` (gdb) bt \#0 0x00000000 in ?? () \#1 0x0043a038 in send_backtrace (why=0xbe9e3600 "FATAL SIGNAL 11") at common/daemon.c:36 \#2 0x0043a0ec in crashdump (sig=11) at common/daemon.c:46 \#3 <signal handler called> \#4 0x00406d04 in node_announcement (map=0x938ecc, nann_off=495146) at common/gossmap.c:586 \#5 0x00406fec in map_catchup (map=0x938ecc, num_rejected=0xbe9e3a40) at common/gossmap.c:643 \#6 0x004073a4 in load_gossip_store (map=0x938ecc, num_rejected=0xbe9e3a40) at common/gossmap.c:697 \ElementsProject#7 0x00408244 in gossmap_load (ctx=0x0, filename=0x4e16b8 "gossip_store", num_channel_updates_rejected=0xbe9e3a40) at common/gossmap.c:976 \ElementsProject#8 0x0041a548 in init (p=0x93831c, buf=0x9399d4 "\n\n{\"jsonrpc\":\"2.0\",\"id\":\"cln:init#25\",\"method\":\"init\",\"params\":{\"options\":{},\"configuration\":{\"lightning-dir\":\"/home/vincent/.lightning/testnet\",\"rpc-file\":\"lightning-rpc\",\"startup\":true,\"network\":\"te"..., config=0x939cdc) at plugins/topology.c:622 \ElementsProject#9 0x0041e5d0 in handle_init (cmd=0x938934, buf=0x9399d4 "\n\n{\"jsonrpc\":\"2.0\",\"id\":\"cln:init#25\",\"method\":\"init\",\"params\":{\"options\":{},\"configuration\":{\"lightning-dir\":\"/home/vincent/.lightning/testnet\",\"rpc-file\":\"lightning-rpc\",\"startup\":true,\"network\":\"te"..., params=0x939c8c) at plugins/libplugin.c:1208 \ElementsProject#10 0x0041fc04 in ld_command_handle (plugin=0x93831c, toks=0x939bec) at plugins/libplugin.c:1572 \ElementsProject#11 0x00420050 in ld_read_json_one (plugin=0x93831c) at plugins/libplugin.c:1667 \ElementsProject#12 0x004201bc in ld_read_json (conn=0x9391c4, plugin=0x93831c) at plugins/libplugin.c:1687 \ElementsProject#13 0x004cb82c in next_plan (conn=0x9391c4, plan=0x9391d8) at ccan/ccan/io/io.c:59 \ElementsProject#14 0x004cc67c in do_plan (conn=0x9391c4, plan=0x9391d8, idle_on_epipe=false) at ccan/ccan/io/io.c:407 \ElementsProject#15 0x004cc6dc in io_ready (conn=0x9391c4, pollflags=1) at ccan/ccan/io/io.c:417 \ElementsProject#16 0x004cf8cc in io_loop (timers=0x9383c4, expired=0xbe9e3ce4) at ccan/ccan/io/poll.c:453 \ElementsProject#17 0x00420af4 in plugin_main (argv=0xbe9e3eb4, init=0x41a46c <init>, restartability=PLUGIN_STATIC, init_rpc=true, features=0x0, commands=0x6167e8 <commands>, num_commands=4, notif_subs=0x0, num_notif_subs=0, hook_subs=0x0, num_hook_subs=0, notif_topics=0x0, num_notif_topics=0) at plugins/libplugin.c:1891 \ElementsProject#18 0x0041a6f8 in main (argc=1, argv=0xbe9e3eb4) at plugins/topology.c:679 ``` I do not know if this is a solution because I do not know when I can parse a node announcement for a node that it is not longer in the gossip map. So, I hope this is just usefult for @rustyrussell Changelog-Fixes: fixes `FATAL SIGNAL 11` on gossmap node announcement parsing. Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>

@rustyrussell

This will fix a crash that I caused on armv7 and by looking inside the coredump with gdb (by adding an assert on n that must be different from null) I get the following stacktrace ``` (gdb) bt \#0 0x00000000 in ?? () \#1 0x0043a038 in send_backtrace (why=0xbe9e3600 "FATAL SIGNAL 11") at common/daemon.c:36 \#2 0x0043a0ec in crashdump (sig=11) at common/daemon.c:46 \#3 <signal handler called> \#4 0x00406d04 in node_announcement (map=0x938ecc, nann_off=495146) at common/gossmap.c:586 \#5 0x00406fec in map_catchup (map=0x938ecc, num_rejected=0xbe9e3a40) at common/gossmap.c:643 \#6 0x004073a4 in load_gossip_store (map=0x938ecc, num_rejected=0xbe9e3a40) at common/gossmap.c:697 \ElementsProject#7 0x00408244 in gossmap_load (ctx=0x0, filename=0x4e16b8 "gossip_store", num_channel_updates_rejected=0xbe9e3a40) at common/gossmap.c:976 \ElementsProject#8 0x0041a548 in init (p=0x93831c, buf=0x9399d4 "\n\n{\"jsonrpc\":\"2.0\",\"id\":\"cln:init#25\",\"method\":\"init\",\"params\":{\"options\":{},\"configuration\":{\"lightning-dir\":\"/home/vincent/.lightning/testnet\",\"rpc-file\":\"lightning-rpc\",\"startup\":true,\"network\":\"te"..., config=0x939cdc) at plugins/topology.c:622 \ElementsProject#9 0x0041e5d0 in handle_init (cmd=0x938934, buf=0x9399d4 "\n\n{\"jsonrpc\":\"2.0\",\"id\":\"cln:init#25\",\"method\":\"init\",\"params\":{\"options\":{},\"configuration\":{\"lightning-dir\":\"/home/vincent/.lightning/testnet\",\"rpc-file\":\"lightning-rpc\",\"startup\":true,\"network\":\"te"..., params=0x939c8c) at plugins/libplugin.c:1208 \ElementsProject#10 0x0041fc04 in ld_command_handle (plugin=0x93831c, toks=0x939bec) at plugins/libplugin.c:1572 \ElementsProject#11 0x00420050 in ld_read_json_one (plugin=0x93831c) at plugins/libplugin.c:1667 \ElementsProject#12 0x004201bc in ld_read_json (conn=0x9391c4, plugin=0x93831c) at plugins/libplugin.c:1687 \ElementsProject#13 0x004cb82c in next_plan (conn=0x9391c4, plan=0x9391d8) at ccan/ccan/io/io.c:59 \ElementsProject#14 0x004cc67c in do_plan (conn=0x9391c4, plan=0x9391d8, idle_on_epipe=false) at ccan/ccan/io/io.c:407 \ElementsProject#15 0x004cc6dc in io_ready (conn=0x9391c4, pollflags=1) at ccan/ccan/io/io.c:417 \ElementsProject#16 0x004cf8cc in io_loop (timers=0x9383c4, expired=0xbe9e3ce4) at ccan/ccan/io/poll.c:453 \ElementsProject#17 0x00420af4 in plugin_main (argv=0xbe9e3eb4, init=0x41a46c <init>, restartability=PLUGIN_STATIC, init_rpc=true, features=0x0, commands=0x6167e8 <commands>, num_commands=4, notif_subs=0x0, num_notif_subs=0, hook_subs=0x0, num_hook_subs=0, notif_topics=0x0, num_notif_topics=0) at plugins/libplugin.c:1891 \ElementsProject#18 0x0041a6f8 in main (argc=1, argv=0xbe9e3eb4) at plugins/topology.c:679 ``` I do not know if this is a solution because I do not know when I can parse a node announcement for a node that it is not longer in the gossip map. So, I hope this is just usefult for @rustyrussell Changelog-Fix: fixes `FATAL SIGNAL 11` on gossmap node announcement parsing. Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>

@rustyrussell

This will fix a crash that I caused on armv7 and by looking inside the coredump with gdb (by adding an assert on n that must be different from null) I get the following stacktrace ``` (gdb) bt \#0 0x00000000 in ?? () \#1 0x0043a038 in send_backtrace (why=0xbe9e3600 "FATAL SIGNAL 11") at common/daemon.c:36 \#2 0x0043a0ec in crashdump (sig=11) at common/daemon.c:46 \#3 <signal handler called> \#4 0x00406d04 in node_announcement (map=0x938ecc, nann_off=495146) at common/gossmap.c:586 \#5 0x00406fec in map_catchup (map=0x938ecc, num_rejected=0xbe9e3a40) at common/gossmap.c:643 \#6 0x004073a4 in load_gossip_store (map=0x938ecc, num_rejected=0xbe9e3a40) at common/gossmap.c:697 \ElementsProject#7 0x00408244 in gossmap_load (ctx=0x0, filename=0x4e16b8 "gossip_store", num_channel_updates_rejected=0xbe9e3a40) at common/gossmap.c:976 \ElementsProject#8 0x0041a548 in init (p=0x93831c, buf=0x9399d4 "\n\n{\"jsonrpc\":\"2.0\",\"id\":\"cln:init#25\",\"method\":\"init\",\"params\":{\"options\":{},\"configuration\":{\"lightning-dir\":\"/home/vincent/.lightning/testnet\",\"rpc-file\":\"lightning-rpc\",\"startup\":true,\"network\":\"te"..., config=0x939cdc) at plugins/topology.c:622 \ElementsProject#9 0x0041e5d0 in handle_init (cmd=0x938934, buf=0x9399d4 "\n\n{\"jsonrpc\":\"2.0\",\"id\":\"cln:init#25\",\"method\":\"init\",\"params\":{\"options\":{},\"configuration\":{\"lightning-dir\":\"/home/vincent/.lightning/testnet\",\"rpc-file\":\"lightning-rpc\",\"startup\":true,\"network\":\"te"..., params=0x939c8c) at plugins/libplugin.c:1208 \ElementsProject#10 0x0041fc04 in ld_command_handle (plugin=0x93831c, toks=0x939bec) at plugins/libplugin.c:1572 \ElementsProject#11 0x00420050 in ld_read_json_one (plugin=0x93831c) at plugins/libplugin.c:1667 \ElementsProject#12 0x004201bc in ld_read_json (conn=0x9391c4, plugin=0x93831c) at plugins/libplugin.c:1687 \ElementsProject#13 0x004cb82c in next_plan (conn=0x9391c4, plan=0x9391d8) at ccan/ccan/io/io.c:59 \ElementsProject#14 0x004cc67c in do_plan (conn=0x9391c4, plan=0x9391d8, idle_on_epipe=false) at ccan/ccan/io/io.c:407 \ElementsProject#15 0x004cc6dc in io_ready (conn=0x9391c4, pollflags=1) at ccan/ccan/io/io.c:417 \ElementsProject#16 0x004cf8cc in io_loop (timers=0x9383c4, expired=0xbe9e3ce4) at ccan/ccan/io/poll.c:453 \ElementsProject#17 0x00420af4 in plugin_main (argv=0xbe9e3eb4, init=0x41a46c <init>, restartability=PLUGIN_STATIC, init_rpc=true, features=0x0, commands=0x6167e8 <commands>, num_commands=4, notif_subs=0x0, num_notif_subs=0, hook_subs=0x0, num_hook_subs=0, notif_topics=0x0, num_notif_topics=0) at plugins/libplugin.c:1891 \ElementsProject#18 0x0041a6f8 in main (argc=1, argv=0xbe9e3eb4) at plugins/topology.c:679 ``` I do not know if this is a solution because I do not know when I can parse a node announcement for a node that it is not longer in the gossip map. So, I hope this is just usefult for @rustyrussell Changelog-Fixed: fixes `FATAL SIGNAL 11` on gossmap node announcement parsing. Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>

@rustyrussell

This will fix a crash that I caused on armv7 and by looking inside the coredump with gdb (by adding an assert on n that must be different from null) I get the following stacktrace ``` (gdb) bt \#0 0x00000000 in ?? () \#1 0x0043a038 in send_backtrace (why=0xbe9e3600 "FATAL SIGNAL 11") at common/daemon.c:36 \#2 0x0043a0ec in crashdump (sig=11) at common/daemon.c:46 \#3 <signal handler called> \#4 0x00406d04 in node_announcement (map=0x938ecc, nann_off=495146) at common/gossmap.c:586 \#5 0x00406fec in map_catchup (map=0x938ecc, num_rejected=0xbe9e3a40) at common/gossmap.c:643 \#6 0x004073a4 in load_gossip_store (map=0x938ecc, num_rejected=0xbe9e3a40) at common/gossmap.c:697 \#7 0x00408244 in gossmap_load (ctx=0x0, filename=0x4e16b8 "gossip_store", num_channel_updates_rejected=0xbe9e3a40) at common/gossmap.c:976 \#8 0x0041a548 in init (p=0x93831c, buf=0x9399d4 "\n\n{\"jsonrpc\":\"2.0\",\"id\":\"cln:init#25\",\"method\":\"init\",\"params\":{\"options\":{},\"configuration\":{\"lightning-dir\":\"/home/vincent/.lightning/testnet\",\"rpc-file\":\"lightning-rpc\",\"startup\":true,\"network\":\"te"..., config=0x939cdc) at plugins/topology.c:622 \#9 0x0041e5d0 in handle_init (cmd=0x938934, buf=0x9399d4 "\n\n{\"jsonrpc\":\"2.0\",\"id\":\"cln:init#25\",\"method\":\"init\",\"params\":{\"options\":{},\"configuration\":{\"lightning-dir\":\"/home/vincent/.lightning/testnet\",\"rpc-file\":\"lightning-rpc\",\"startup\":true,\"network\":\"te"..., params=0x939c8c) at plugins/libplugin.c:1208 \#10 0x0041fc04 in ld_command_handle (plugin=0x93831c, toks=0x939bec) at plugins/libplugin.c:1572 \#11 0x00420050 in ld_read_json_one (plugin=0x93831c) at plugins/libplugin.c:1667 \#12 0x004201bc in ld_read_json (conn=0x9391c4, plugin=0x93831c) at plugins/libplugin.c:1687 \#13 0x004cb82c in next_plan (conn=0x9391c4, plan=0x9391d8) at ccan/ccan/io/io.c:59 \#14 0x004cc67c in do_plan (conn=0x9391c4, plan=0x9391d8, idle_on_epipe=false) at ccan/ccan/io/io.c:407 \#15 0x004cc6dc in io_ready (conn=0x9391c4, pollflags=1) at ccan/ccan/io/io.c:417 \#16 0x004cf8cc in io_loop (timers=0x9383c4, expired=0xbe9e3ce4) at ccan/ccan/io/poll.c:453 \#17 0x00420af4 in plugin_main (argv=0xbe9e3eb4, init=0x41a46c <init>, restartability=PLUGIN_STATIC, init_rpc=true, features=0x0, commands=0x6167e8 <commands>, num_commands=4, notif_subs=0x0, num_notif_subs=0, hook_subs=0x0, num_hook_subs=0, notif_topics=0x0, num_notif_topics=0) at plugins/libplugin.c:1891 \#18 0x0041a6f8 in main (argc=1, argv=0xbe9e3eb4) at plugins/topology.c:679 ``` I do not know if this is a solution because I do not know when I can parse a node announcement for a node that it is not longer in the gossip map. So, I hope this is just usefult for @rustyrussell Changelog-Fixed: fixes `FATAL SIGNAL 11` on gossmap node announcement parsing. Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>

The issue is that common_setup() wasn't called by the fuzz target, leaving secp256k1_ctx as NULL. UBSan error: $ UBSAN_OPTIONS="print_stacktrace=1:halt_on_error=1" \ ./fuzz-channel_id crash-1575b41ef09e62e4c09c165e6dc037a110b113f2 INFO: Running with entropic power schedule (0xFF, 100). INFO: Seed: 1153355603 INFO: Loaded 1 modules (25915 inline 8-bit counters): 25915 [0x563bae7ac3a8, 0x563bae7b28e3), INFO: Loaded 1 PC tables (25915 PCs): 25915 [0x563bae7b28e8,0x563bae817c98), ./fuzz-channel_id: Running 1 inputs 1 time(s) each. Running: crash-1575b41ef09e62e4c09c165e6dc037a110b113f2 bitcoin/pubkey.c:22:33: runtime error: null pointer passed as argument 1, which is declared to never be null external/libwally-core/src/secp256k1/include/secp256k1.h:373:3: note: nonnull attribute specified here #0 0x563bae41e3db in pubkey_from_der bitcoin/pubkey.c:19:7 #1 0x563bae4205e0 in fromwire_pubkey bitcoin/pubkey.c:111:7 #2 0x563bae46437c in run tests/fuzz/fuzz-channel_id.c:42:3 #3 0x563bae2f6016 in LLVMFuzzerTestOneInput tests/fuzz/libfuzz.c:23:2 #4 0x563bae20a450 in fuzzer::Fuzzer::ExecuteCallback(unsigned char const*, unsigned long) ElementsProject#5 0x563bae1f4c3f in fuzzer::RunOneTest(fuzzer::Fuzzer*, char const*, unsigned long) ElementsProject#6 0x563bae1fa6e6 in fuzzer::FuzzerDriver(int*, char***, int (*)(unsigned char const*, unsigned long)) ElementsProject#7 0x563bae223052 in main (tests/fuzz/fuzz-channel_id+0x181052) (BuildId: f7f56e14ffc06df54ab732d79ea922e773de1f25) ElementsProject#8 0x7fa7fa113082 in __libc_start_main ElementsProject#9 0x563bae1efbdd in _start SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior bitcoin/pubkey.c:22:33 in

The issue is that common_setup() wasn't called by the fuzz target, leaving secp256k1_ctx as NULL. UBSan error: $ UBSAN_OPTIONS="print_stacktrace=1:halt_on_error=1" \ ./fuzz-channel_id crash-1575b41ef09e62e4c09c165e6dc037a110b113f2 INFO: Running with entropic power schedule (0xFF, 100). INFO: Seed: 1153355603 INFO: Loaded 1 modules (25915 inline 8-bit counters): 25915 [0x563bae7ac3a8, 0x563bae7b28e3), INFO: Loaded 1 PC tables (25915 PCs): 25915 [0x563bae7b28e8,0x563bae817c98), ./fuzz-channel_id: Running 1 inputs 1 time(s) each. Running: crash-1575b41ef09e62e4c09c165e6dc037a110b113f2 bitcoin/pubkey.c:22:33: runtime error: null pointer passed as argument 1, which is declared to never be null external/libwally-core/src/secp256k1/include/secp256k1.h:373:3: note: nonnull attribute specified here #0 0x563bae41e3db in pubkey_from_der bitcoin/pubkey.c:19:7 #1 0x563bae4205e0 in fromwire_pubkey bitcoin/pubkey.c:111:7 #2 0x563bae46437c in run tests/fuzz/fuzz-channel_id.c:42:3 #3 0x563bae2f6016 in LLVMFuzzerTestOneInput tests/fuzz/libfuzz.c:23:2 #4 0x563bae20a450 in fuzzer::Fuzzer::ExecuteCallback(unsigned char const*, unsigned long) #5 0x563bae1f4c3f in fuzzer::RunOneTest(fuzzer::Fuzzer*, char const*, unsigned long) #6 0x563bae1fa6e6 in fuzzer::FuzzerDriver(int*, char***, int (*)(unsigned char const*, unsigned long)) #7 0x563bae223052 in main (tests/fuzz/fuzz-channel_id+0x181052) (BuildId: f7f56e14ffc06df54ab732d79ea922e773de1f25) #8 0x7fa7fa113082 in __libc_start_main #9 0x563bae1efbdd in _start SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior bitcoin/pubkey.c:22:33 in

@rustyrussell

This will fix a crash that I caused on armv7 and by looking inside the coredump with gdb (by adding an assert on n that must be different from null) I get the following stacktrace ``` (gdb) bt \#0 0x00000000 in ?? () \#1 0x0043a038 in send_backtrace (why=0xbe9e3600 "FATAL SIGNAL 11") at common/daemon.c:36 \#2 0x0043a0ec in crashdump (sig=11) at common/daemon.c:46 \#3 <signal handler called> \#4 0x00406d04 in node_announcement (map=0x938ecc, nann_off=495146) at common/gossmap.c:586 \#5 0x00406fec in map_catchup (map=0x938ecc, num_rejected=0xbe9e3a40) at common/gossmap.c:643 \ElementsProject#6 0x004073a4 in load_gossip_store (map=0x938ecc, num_rejected=0xbe9e3a40) at common/gossmap.c:697 \ElementsProject#7 0x00408244 in gossmap_load (ctx=0x0, filename=0x4e16b8 "gossip_store", num_channel_updates_rejected=0xbe9e3a40) at common/gossmap.c:976 \ElementsProject#8 0x0041a548 in init (p=0x93831c, buf=0x9399d4 "\n\n{\"jsonrpc\":\"2.0\",\"id\":\"cln:init#25\",\"method\":\"init\",\"params\":{\"options\":{},\"configuration\":{\"lightning-dir\":\"/home/vincent/.lightning/testnet\",\"rpc-file\":\"lightning-rpc\",\"startup\":true,\"network\":\"te"..., config=0x939cdc) at plugins/topology.c:622 \ElementsProject#9 0x0041e5d0 in handle_init (cmd=0x938934, buf=0x9399d4 "\n\n{\"jsonrpc\":\"2.0\",\"id\":\"cln:init#25\",\"method\":\"init\",\"params\":{\"options\":{},\"configuration\":{\"lightning-dir\":\"/home/vincent/.lightning/testnet\",\"rpc-file\":\"lightning-rpc\",\"startup\":true,\"network\":\"te"..., params=0x939c8c) at plugins/libplugin.c:1208 \ElementsProject#10 0x0041fc04 in ld_command_handle (plugin=0x93831c, toks=0x939bec) at plugins/libplugin.c:1572 \ElementsProject#11 0x00420050 in ld_read_json_one (plugin=0x93831c) at plugins/libplugin.c:1667 \ElementsProject#12 0x004201bc in ld_read_json (conn=0x9391c4, plugin=0x93831c) at plugins/libplugin.c:1687 \ElementsProject#13 0x004cb82c in next_plan (conn=0x9391c4, plan=0x9391d8) at ccan/ccan/io/io.c:59 \ElementsProject#14 0x004cc67c in do_plan (conn=0x9391c4, plan=0x9391d8, idle_on_epipe=false) at ccan/ccan/io/io.c:407 \ElementsProject#15 0x004cc6dc in io_ready (conn=0x9391c4, pollflags=1) at ccan/ccan/io/io.c:417 \ElementsProject#16 0x004cf8cc in io_loop (timers=0x9383c4, expired=0xbe9e3ce4) at ccan/ccan/io/poll.c:453 \ElementsProject#17 0x00420af4 in plugin_main (argv=0xbe9e3eb4, init=0x41a46c <init>, restartability=PLUGIN_STATIC, init_rpc=true, features=0x0, commands=0x6167e8 <commands>, num_commands=4, notif_subs=0x0, num_notif_subs=0, hook_subs=0x0, num_hook_subs=0, notif_topics=0x0, num_notif_topics=0) at plugins/libplugin.c:1891 \ElementsProject#18 0x0041a6f8 in main (argc=1, argv=0xbe9e3eb4) at plugins/topology.c:679 ``` I do not know if this is a solution because I do not know when I can parse a node announcement for a node that it is not longer in the gossip map. So, I hope this is just usefult for @rustyrussell Changelog-Fixed: fixes `FATAL SIGNAL 11` on gossmap node announcement parsing. Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>

The issue is that common_setup() wasn't called by the fuzz target, leaving secp256k1_ctx as NULL. UBSan error: $ UBSAN_OPTIONS="print_stacktrace=1:halt_on_error=1" \ ./fuzz-channel_id crash-1575b41ef09e62e4c09c165e6dc037a110b113f2 INFO: Running with entropic power schedule (0xFF, 100). INFO: Seed: 1153355603 INFO: Loaded 1 modules (25915 inline 8-bit counters): 25915 [0x563bae7ac3a8, 0x563bae7b28e3), INFO: Loaded 1 PC tables (25915 PCs): 25915 [0x563bae7b28e8,0x563bae817c98), ./fuzz-channel_id: Running 1 inputs 1 time(s) each. Running: crash-1575b41ef09e62e4c09c165e6dc037a110b113f2 bitcoin/pubkey.c:22:33: runtime error: null pointer passed as argument 1, which is declared to never be null external/libwally-core/src/secp256k1/include/secp256k1.h:373:3: note: nonnull attribute specified here #0 0x563bae41e3db in pubkey_from_der bitcoin/pubkey.c:19:7 #1 0x563bae4205e0 in fromwire_pubkey bitcoin/pubkey.c:111:7 #2 0x563bae46437c in run tests/fuzz/fuzz-channel_id.c:42:3 #3 0x563bae2f6016 in LLVMFuzzerTestOneInput tests/fuzz/libfuzz.c:23:2 #4 0x563bae20a450 in fuzzer::Fuzzer::ExecuteCallback(unsigned char const*, unsigned long) #5 0x563bae1f4c3f in fuzzer::RunOneTest(fuzzer::Fuzzer*, char const*, unsigned long) ElementsProject#6 0x563bae1fa6e6 in fuzzer::FuzzerDriver(int*, char***, int (*)(unsigned char const*, unsigned long)) ElementsProject#7 0x563bae223052 in main (tests/fuzz/fuzz-channel_id+0x181052) (BuildId: f7f56e14ffc06df54ab732d79ea922e773de1f25) ElementsProject#8 0x7fa7fa113082 in __libc_start_main ElementsProject#9 0x563bae1efbdd in _start SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior bitcoin/pubkey.c:22:33 in

It is possible for db_column_bytes() to return 0 and for db_column_blob() to return NULL even when db_column_is_null() returns false. We need to short circuit in this case. Detected by UBSan: db/bindings.c:479:12: runtime error: null pointer passed as argument 2, which is declared to never be null /usr/include/string.h:44:28: note: nonnull attribute specified here #0 0x95f117 in db_col_arr_ db/bindings.c:479:2 #1 0x95ef85 in db_col_channel_type db/bindings.c:459:32 #2 0x852c03 in wallet_stmt2channel wallet/wallet.c:1483:9 #3 0x81f396 in wallet_channels_load_active wallet/wallet.c:1749:23 #4 0x81f03d in wallet_init_channels wallet/wallet.c:1765:9 ElementsProject#5 0x72f1f9 in load_channels_from_wallet lightningd/peer_control.c:2257:7 ElementsProject#6 0x672856 in main lightningd/lightningd.c:1121:25

@rustyrussell

This will fix a crash that I caused on armv7 and by looking inside the coredump with gdb (by adding an assert on n that must be different from null) I get the following stacktrace ``` (gdb) bt \#0 0x00000000 in ?? () \#1 0x0043a038 in send_backtrace (why=0xbe9e3600 "FATAL SIGNAL 11") at common/daemon.c:36 \#2 0x0043a0ec in crashdump (sig=11) at common/daemon.c:46 \#3 <signal handler called> \ElementsProject#4 0x00406d04 in node_announcement (map=0x938ecc, nann_off=495146) at common/gossmap.c:586 \ElementsProject#5 0x00406fec in map_catchup (map=0x938ecc, num_rejected=0xbe9e3a40) at common/gossmap.c:643 \ElementsProject#6 0x004073a4 in load_gossip_store (map=0x938ecc, num_rejected=0xbe9e3a40) at common/gossmap.c:697 \ElementsProject#7 0x00408244 in gossmap_load (ctx=0x0, filename=0x4e16b8 "gossip_store", num_channel_updates_rejected=0xbe9e3a40) at common/gossmap.c:976 \ElementsProject#8 0x0041a548 in init (p=0x93831c, buf=0x9399d4 "\n\n{\"jsonrpc\":\"2.0\",\"id\":\"cln:init#25\",\"method\":\"init\",\"params\":{\"options\":{},\"configuration\":{\"lightning-dir\":\"/home/vincent/.lightning/testnet\",\"rpc-file\":\"lightning-rpc\",\"startup\":true,\"network\":\"te"..., config=0x939cdc) at plugins/topology.c:622 \ElementsProject#9 0x0041e5d0 in handle_init (cmd=0x938934, buf=0x9399d4 "\n\n{\"jsonrpc\":\"2.0\",\"id\":\"cln:init#25\",\"method\":\"init\",\"params\":{\"options\":{},\"configuration\":{\"lightning-dir\":\"/home/vincent/.lightning/testnet\",\"rpc-file\":\"lightning-rpc\",\"startup\":true,\"network\":\"te"..., params=0x939c8c) at plugins/libplugin.c:1208 \ElementsProject#10 0x0041fc04 in ld_command_handle (plugin=0x93831c, toks=0x939bec) at plugins/libplugin.c:1572 \ElementsProject#11 0x00420050 in ld_read_json_one (plugin=0x93831c) at plugins/libplugin.c:1667 \ElementsProject#12 0x004201bc in ld_read_json (conn=0x9391c4, plugin=0x93831c) at plugins/libplugin.c:1687 \ElementsProject#13 0x004cb82c in next_plan (conn=0x9391c4, plan=0x9391d8) at ccan/ccan/io/io.c:59 \ElementsProject#14 0x004cc67c in do_plan (conn=0x9391c4, plan=0x9391d8, idle_on_epipe=false) at ccan/ccan/io/io.c:407 \ElementsProject#15 0x004cc6dc in io_ready (conn=0x9391c4, pollflags=1) at ccan/ccan/io/io.c:417 \ElementsProject#16 0x004cf8cc in io_loop (timers=0x9383c4, expired=0xbe9e3ce4) at ccan/ccan/io/poll.c:453 \ElementsProject#17 0x00420af4 in plugin_main (argv=0xbe9e3eb4, init=0x41a46c <init>, restartability=PLUGIN_STATIC, init_rpc=true, features=0x0, commands=0x6167e8 <commands>, num_commands=4, notif_subs=0x0, num_notif_subs=0, hook_subs=0x0, num_hook_subs=0, notif_topics=0x0, num_notif_topics=0) at plugins/libplugin.c:1891 \ElementsProject#18 0x0041a6f8 in main (argc=1, argv=0xbe9e3eb4) at plugins/topology.c:679 ``` I do not know if this is a solution because I do not know when I can parse a node announcement for a node that it is not longer in the gossip map. So, I hope this is just usefult for @rustyrussell Changelog-Fixed: fixes `FATAL SIGNAL 11` on gossmap node announcement parsing. Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>

The issue is that common_setup() wasn't called by the fuzz target, leaving secp256k1_ctx as NULL. UBSan error: $ UBSAN_OPTIONS="print_stacktrace=1:halt_on_error=1" \ ./fuzz-channel_id crash-1575b41ef09e62e4c09c165e6dc037a110b113f2 INFO: Running with entropic power schedule (0xFF, 100). INFO: Seed: 1153355603 INFO: Loaded 1 modules (25915 inline 8-bit counters): 25915 [0x563bae7ac3a8, 0x563bae7b28e3), INFO: Loaded 1 PC tables (25915 PCs): 25915 [0x563bae7b28e8,0x563bae817c98), ./fuzz-channel_id: Running 1 inputs 1 time(s) each. Running: crash-1575b41ef09e62e4c09c165e6dc037a110b113f2 bitcoin/pubkey.c:22:33: runtime error: null pointer passed as argument 1, which is declared to never be null external/libwally-core/src/secp256k1/include/secp256k1.h:373:3: note: nonnull attribute specified here #0 0x563bae41e3db in pubkey_from_der bitcoin/pubkey.c:19:7 #1 0x563bae4205e0 in fromwire_pubkey bitcoin/pubkey.c:111:7 #2 0x563bae46437c in run tests/fuzz/fuzz-channel_id.c:42:3 #3 0x563bae2f6016 in LLVMFuzzerTestOneInput tests/fuzz/libfuzz.c:23:2 ElementsProject#4 0x563bae20a450 in fuzzer::Fuzzer::ExecuteCallback(unsigned char const*, unsigned long) ElementsProject#5 0x563bae1f4c3f in fuzzer::RunOneTest(fuzzer::Fuzzer*, char const*, unsigned long) ElementsProject#6 0x563bae1fa6e6 in fuzzer::FuzzerDriver(int*, char***, int (*)(unsigned char const*, unsigned long)) ElementsProject#7 0x563bae223052 in main (tests/fuzz/fuzz-channel_id+0x181052) (BuildId: f7f56e14ffc06df54ab732d79ea922e773de1f25) ElementsProject#8 0x7fa7fa113082 in __libc_start_main ElementsProject#9 0x563bae1efbdd in _start SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior bitcoin/pubkey.c:22:33 in

@rustyrussell

This will fix a crash that I caused on armv7 and by looking inside the coredump with gdb (by adding an assert on n that must be different from null) I get the following stacktrace ``` (gdb) bt \#0 0x00000000 in ?? () \#1 0x0043a038 in send_backtrace (why=0xbe9e3600 "FATAL SIGNAL 11") at common/daemon.c:36 \#2 0x0043a0ec in crashdump (sig=11) at common/daemon.c:46 \#3 <signal handler called> \#4 0x00406d04 in node_announcement (map=0x938ecc, nann_off=495146) at common/gossmap.c:586 \#5 0x00406fec in map_catchup (map=0x938ecc, num_rejected=0xbe9e3a40) at common/gossmap.c:643 \ElementsProject#6 0x004073a4 in load_gossip_store (map=0x938ecc, num_rejected=0xbe9e3a40) at common/gossmap.c:697 \ElementsProject#7 0x00408244 in gossmap_load (ctx=0x0, filename=0x4e16b8 "gossip_store", num_channel_updates_rejected=0xbe9e3a40) at common/gossmap.c:976 \ElementsProject#8 0x0041a548 in init (p=0x93831c, buf=0x9399d4 "\n\n{\"jsonrpc\":\"2.0\",\"id\":\"cln:init#25\",\"method\":\"init\",\"params\":{\"options\":{},\"configuration\":{\"lightning-dir\":\"/home/vincent/.lightning/testnet\",\"rpc-file\":\"lightning-rpc\",\"startup\":true,\"network\":\"te"..., config=0x939cdc) at plugins/topology.c:622 \ElementsProject#9 0x0041e5d0 in handle_init (cmd=0x938934, buf=0x9399d4 "\n\n{\"jsonrpc\":\"2.0\",\"id\":\"cln:init#25\",\"method\":\"init\",\"params\":{\"options\":{},\"configuration\":{\"lightning-dir\":\"/home/vincent/.lightning/testnet\",\"rpc-file\":\"lightning-rpc\",\"startup\":true,\"network\":\"te"..., params=0x939c8c) at plugins/libplugin.c:1208 \ElementsProject#10 0x0041fc04 in ld_command_handle (plugin=0x93831c, toks=0x939bec) at plugins/libplugin.c:1572 \ElementsProject#11 0x00420050 in ld_read_json_one (plugin=0x93831c) at plugins/libplugin.c:1667 \ElementsProject#12 0x004201bc in ld_read_json (conn=0x9391c4, plugin=0x93831c) at plugins/libplugin.c:1687 \ElementsProject#13 0x004cb82c in next_plan (conn=0x9391c4, plan=0x9391d8) at ccan/ccan/io/io.c:59 \ElementsProject#14 0x004cc67c in do_plan (conn=0x9391c4, plan=0x9391d8, idle_on_epipe=false) at ccan/ccan/io/io.c:407 \ElementsProject#15 0x004cc6dc in io_ready (conn=0x9391c4, pollflags=1) at ccan/ccan/io/io.c:417 \ElementsProject#16 0x004cf8cc in io_loop (timers=0x9383c4, expired=0xbe9e3ce4) at ccan/ccan/io/poll.c:453 \ElementsProject#17 0x00420af4 in plugin_main (argv=0xbe9e3eb4, init=0x41a46c <init>, restartability=PLUGIN_STATIC, init_rpc=true, features=0x0, commands=0x6167e8 <commands>, num_commands=4, notif_subs=0x0, num_notif_subs=0, hook_subs=0x0, num_hook_subs=0, notif_topics=0x0, num_notif_topics=0) at plugins/libplugin.c:1891 \ElementsProject#18 0x0041a6f8 in main (argc=1, argv=0xbe9e3eb4) at plugins/topology.c:679 ``` I do not know if this is a solution because I do not know when I can parse a node announcement for a node that it is not longer in the gossip map. So, I hope this is just usefult for @rustyrussell Changelog-Fixed: fixes `FATAL SIGNAL 11` on gossmap node announcement parsing. Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>

The issue is that common_setup() wasn't called by the fuzz target, leaving secp256k1_ctx as NULL. UBSan error: $ UBSAN_OPTIONS="print_stacktrace=1:halt_on_error=1" \ ./fuzz-channel_id crash-1575b41ef09e62e4c09c165e6dc037a110b113f2 INFO: Running with entropic power schedule (0xFF, 100). INFO: Seed: 1153355603 INFO: Loaded 1 modules (25915 inline 8-bit counters): 25915 [0x563bae7ac3a8, 0x563bae7b28e3), INFO: Loaded 1 PC tables (25915 PCs): 25915 [0x563bae7b28e8,0x563bae817c98), ./fuzz-channel_id: Running 1 inputs 1 time(s) each. Running: crash-1575b41ef09e62e4c09c165e6dc037a110b113f2 bitcoin/pubkey.c:22:33: runtime error: null pointer passed as argument 1, which is declared to never be null external/libwally-core/src/secp256k1/include/secp256k1.h:373:3: note: nonnull attribute specified here #0 0x563bae41e3db in pubkey_from_der bitcoin/pubkey.c:19:7 #1 0x563bae4205e0 in fromwire_pubkey bitcoin/pubkey.c:111:7 #2 0x563bae46437c in run tests/fuzz/fuzz-channel_id.c:42:3 #3 0x563bae2f6016 in LLVMFuzzerTestOneInput tests/fuzz/libfuzz.c:23:2 #4 0x563bae20a450 in fuzzer::Fuzzer::ExecuteCallback(unsigned char const*, unsigned long) #5 0x563bae1f4c3f in fuzzer::RunOneTest(fuzzer::Fuzzer*, char const*, unsigned long) ElementsProject#6 0x563bae1fa6e6 in fuzzer::FuzzerDriver(int*, char***, int (*)(unsigned char const*, unsigned long)) ElementsProject#7 0x563bae223052 in main (tests/fuzz/fuzz-channel_id+0x181052) (BuildId: f7f56e14ffc06df54ab732d79ea922e773de1f25) ElementsProject#8 0x7fa7fa113082 in __libc_start_main ElementsProject#9 0x563bae1efbdd in _start SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior bitcoin/pubkey.c:22:33 in

It is possible for db_column_bytes() to return 0 and for db_column_blob() to return NULL even when db_column_is_null() returns false. We need to short circuit in this case. Detected by UBSan: db/bindings.c:479:12: runtime error: null pointer passed as argument 2, which is declared to never be null /usr/include/string.h:44:28: note: nonnull attribute specified here #0 0x95f117 in db_col_arr_ db/bindings.c:479:2 #1 0x95ef85 in db_col_channel_type db/bindings.c:459:32 #2 0x852c03 in wallet_stmt2channel wallet/wallet.c:1483:9 #3 0x81f396 in wallet_channels_load_active wallet/wallet.c:1749:23 #4 0x81f03d in wallet_init_channels wallet/wallet.c:1765:9 ElementsProject#5 0x72f1f9 in load_channels_from_wallet lightningd/peer_control.c:2257:7 ElementsProject#6 0x672856 in main lightningd/lightningd.c:1121:25

It is possible for db_column_bytes() to return 0 and for db_column_blob() to return NULL even when db_column_is_null() returns false. We need to short circuit in this case. Detected by UBSan: db/bindings.c:479:12: runtime error: null pointer passed as argument 2, which is declared to never be null /usr/include/string.h:44:28: note: nonnull attribute specified here #0 0x95f117 in db_col_arr_ db/bindings.c:479:2 #1 0x95ef85 in db_col_channel_type db/bindings.c:459:32 #2 0x852c03 in wallet_stmt2channel wallet/wallet.c:1483:9 #3 0x81f396 in wallet_channels_load_active wallet/wallet.c:1749:23 #4 0x81f03d in wallet_init_channels wallet/wallet.c:1765:9 #5 0x72f1f9 in load_channels_from_wallet lightningd/peer_control.c:2257:7 #6 0x672856 in main lightningd/lightningd.c:1121:25

rustyrussell closed this as completed Sep 27, 2016

instagibbs mentioned this issue Jan 9, 2018

lightningd hanging after calling getpeers #541

Closed

sfmcnally mentioned this issue Feb 9, 2018

'dev-forget-channel' on channel in state ONCHAIND_THEIR_UNILATERAL crashes lightningd #946

Closed

rustyrussell mentioned this issue Feb 22, 2018

Funding channel causes crash due to "UNIQUE constraint failed: peers.node_id" #1029

Closed

Ulmo mentioned this issue Mar 3, 2018

Three crashes every time it sync'd with blockchain. #1157

Closed

arowser mentioned this issue Mar 23, 2018

Disable backtrace on mac #1258

Merged

robtex mentioned this issue Mar 25, 2018

lightning_channeld: Writing out status 65520: Broken pipe #1269

Closed

chandram47 pushed a commit to chandram47/lightning that referenced this issue Nov 23, 2018

Merge pull request ElementsProject#6 from sg777/master

94a0121

distributing the keys securely

darosior mentioned this issue Sep 7, 2020

Re-enable ASAN when building with clang #4013

Merged

ShahanaFarooqui mentioned this issue Nov 20, 2023

clnrest: Import sys in except clause explicitly #6883

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Don't use u32 locktime internally. #6

Don't use u32 locktime internally. #6

rustyrussell commented Oct 1, 2015

rustyrussell commented Sep 27, 2016

Don't use u32 locktime internally. #6

Don't use u32 locktime internally. #6

Comments

rustyrussell commented Oct 1, 2015

rustyrussell commented Sep 27, 2016