Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Minor issues in documentation #11

Closed
jnewbery opened this issue Oct 17, 2015 · 1 comment
Closed

Minor issues in documentation #11

jnewbery opened this issue Oct 17, 2015 · 1 comment

Comments

@jnewbery
Copy link

Hi Rusty,

I decided to finally lock myself away with your Reaching the Ground with Lightning paper and your earlier blog post series last night and spent some time digging into the details of Lightning. I love the work you're doing on this, and I'm happy that I'm finally beginning to get my head around lightning.

I have a few minor comments about the paper:

  1. in Figure 1: Figure 1 from the Lightning Network Draft 0.5
    • Commitment Close Tx 1A (CC1b) -- should be CC1a

    • Commitment Close Tx 1b (CC1a) -- should be CC1b

      (both of these are carried over from the Lightning Paper 0.5 and fixed in 0.5.9)

  2. in Figure 2: Figure 2 from the Lightning Network Draft 0.5
    • "Commitment Refund Tx1a (CR2a)" should be "Commitment Refund Tx2a (CR2a)"

    • The settlement/timeout transactions are named S1a/SD1a/T1a/TR1a. I believe these should be called S2a/SD2a/T2a/TR2a (since they're descendants of Commitment transaction 2a)

      (carried over from the Lightning Paper 0.5 and still in 0.5.9)

  3. Section 3.2: "These can be later balanced by the lightning network itself, or an atomic-swap an on-chain bitcoin transaction" should be "These can be later balanced by the lightning network itself, or an on-chain atomic-swap transaction."
  4. Section 3.2: "implementation splits fees where possible< and never allows either side" has an errant < character inserted (which doesn't appear in the .lyx file)
  5. The references seem to have been broken between v0.1 and v0.2. References are mis-numbered and in the wrong order.
  6. References to the Appendixes are broken ('The scripts for this can be found in 4' rather than 'The scripts for this can be found in appendix A' and 'as the commitment transaction in 4' instead of 'as the commitment transaction in appendix B')
  7. In v0.2, the conclusion can be updated: 'By using a dual anchor and escape transactions, channel establishment can also avoid new CHECKSIG flags, though it loses the important ability to outsoure the enforcement of channel contract terms.' to 'By using a rebalanced single anchor, channel establishment can also avoid new CHECKSIG flags, while retaining the important ability to outsource the enforcement of channel contract terms.'
  8. I'd recommend that the 'Escape transactions' and 'Fast escape transactions' in appendix A get moved to appendix B now that dual anchors are not part of the current design.

I'll open a pull request for 3 and 7. The others are either:

  • changes to .eps files. I'm afraid I don't know how to modify those.
  • errors in whatever tool you used to generate the .pdf from the source files.

I've also been following along on the Lightning-dev mailing list, and see that there's been a lot of movement on state machines and routing since v0.2 of the documentation. Are there plans to update the paper to include information on those aspects? I'm happy to lend a hand if you think that'd be useful.

Thanks,
John

@rustyrussell
Copy link
Contributor

John Newbery notifications@github.com writes:

Hi Rusty,

I decided to finally lock myself away with your Reaching the Ground with Lightning paper and your earlier blog post series last night and spent some time digging into the details of Lightning. I love the work you're doing on this, and I'm happy that I'm finally beginning to get my head around lightning.

Thanks for the proofreading, I'm sure others have spotted my mistakes
and not been so charitable with their time :)

I have a few minor comments about the paper:

  1. in Figure 1: Figure 1 from the Lightning Network Draft 0.5
    • Commitment Close Tx 1A (CC1b) -- should be CC1a

    • Commitment Close Tx 1b (CC1a) -- should be CC1b

      (both of these are carried over from the Lightning Paper 0.5 and fixed in 0.5.9)

Thanks! Yes, I have an eps of those figures only, so it'd be a PITA
to fix. But I've put in a note.

  1. in Figure 2: Figure 2 from the Lightning Network Draft 0.5
    • The settlement/timeout transactions are named S1a/SD1a/T1a/TR1a. I believe these should be called S2a/SD2a/T2a/TR2a (since they're descendants of Commitment transaction 2a)

      (carried over from the Lightning Paper 0.5 and still in 0.5.9)

Hmm, good point, but I don't refer to them anywhere. I've CC'd Joseph.

  1. Section 3.2: "These can be later balanced by the lightning network itself, or an atomic-swap an on-chain bitcoin transaction" should be "These can be later balanced by the lightning network itself, or an on-chain atomic-swap transaction."

OK, "an atomic swap to an onchain bitcoin transaction".

  1. Section 3.2: "implementation splits fees where possible< and never allows either side" has an errant < character inserted (which doesn't appear in the .lyx file)

Weird, that's not in the source. Fixed.

  1. The references seem to have been broken between v0.1 and v0.2. References are mis-numbered and in the wrong order.

Hmm, regenerated. They look better now.

  1. References to the Appendixes are broken ('The scripts for this can be found in 4' rather than 'The scripts for this can be found in appendix A' and 'as the commitment transaction in 4' instead of 'as the commitment transaction in appendix B')

Yep, fixed. Plus.

  1. In v0.2, the conclusion can be updated: 'By using a dual anchor and escape transactions, channel establishment can also avoid new CHECKSIG flags, though it loses the important ability to outsoure the enforcement of channel contract terms.' to 'By using a rebalanced single anchor, channel establishment can also avoid new CHECKSIG flags, while retaining the important ability to outsource the enforcement of channel contract terms.'

The first half is correct, the second half is incorrect: we still need
new CHECKSIG flags to outsource contract terms.

  1. I'd recommend that the 'Escape transactions' and 'Fast escape transactions' in appendix A get moved to appendix B now that dual anchors are not part of the current design.

Excellent point. Done.

I'll open a pull request for 3 and 7. The others are either:

  • changes to .eps files. I'm afraid I don't know how to modify those.
  • errors in whatever tool you used to generate the .pdf from the source files.

Oops, I should have applied that first, but I edited as I read this bug
report.

I've also been following along on the Lightning-dev mailing list, and see that there's been a lot of movement on state machines and routing since v0.2 of the documentation. Are there plans to update the paper to include information on those aspects? I'm happy to lend a hand if you think that'd be useful.

I think that the paper stands alone; there are many more areas to
document outside its scope however:

  1. The protocol used for onion routing.
  2. The cryptography used for inter-node communications.
  3. The protocol and state machine used for inter-node communications.
  4. The routing and fee protocols.

These are all in different stage of development though. The first two
should be getting some serious crypto review in the next few weeks. The
inter-node comms protocol seems fairly stable, but no doubt will change
as we actually try to interoperate. And the routing and fee protocols
are still being actively discussed.

But if you wanted to flesh out the very bare wiki, at least with ML
references, that would be a definite start!

Thanks,
Rusty.

chandram47 pushed a commit to chandram47/lightning that referenced this issue Nov 23, 2018
decoding the cards as separate nodes
rustyrussell referenced this issue in rustyrussell/lightning Jan 15, 2019
Don't do this:
  (gdb) bt
  #0  0x00007f37ae667c40 in ?? () from /lib/x86_64-linux-gnu/libz.so.1
  #1  0x00007f37ae668b38 in ?? () from /lib/x86_64-linux-gnu/libz.so.1
  #2  0x00007f37ae669907 in deflate () from /lib/x86_64-linux-gnu/libz.so.1
  #3  0x00007f37ae674c65 in compress2 () from /lib/x86_64-linux-gnu/libz.so.1
  #4  0x000000000040cfe3 in zencode_scids (ctx=0xc1f118, scids=0x2599bc49 "\a\325{", len=176320) at gossipd/gossipd.c:218
  #5  0x000000000040d0b3 in encode_short_channel_ids_end (encoded=0x7fff8f98d9f0, max_bytes=65490) at gossipd/gossipd.c:236
  #6  0x000000000040dd28 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17290511, number_of_blocks=8) at gossipd/gossipd.c:576
  #7  0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17290511, number_of_blocks=16) at gossipd/gossipd.c:595
  #8  0x000000000040ddee in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17290495, number_of_blocks=32) at gossipd/gossipd.c:596
  #9  0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17290495, number_of_blocks=64) at gossipd/gossipd.c:595
  #10 0x000000000040ddee in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17290431, number_of_blocks=128) at gossipd/gossipd.c:596
  #11 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17290431, number_of_blocks=256) at gossipd/gossipd.c:595
  #12 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17290431, number_of_blocks=512) at gossipd/gossipd.c:595
  #13 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17290431, number_of_blocks=1024) at gossipd/gossipd.c:595
  ElementsProject#14 0x000000000040ddee in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=2047) at gossipd/gossipd.c:596
  ElementsProject#15 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=4095) at gossipd/gossipd.c:595
  ElementsProject#16 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=8191) at gossipd/gossipd.c:595
  ElementsProject#17 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=16382) at gossipd/gossipd.c:595
  ElementsProject#18 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=32764) at gossipd/gossipd.c:595
  ElementsProject#19 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=65528) at gossipd/gossipd.c:595
  ElementsProject#20 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=131056) at gossipd/gossipd.c:595
  ElementsProject#21 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=262112) at gossipd/gossipd.c:595
  ElementsProject#22 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=524225) at gossipd/gossipd.c:595
  ElementsProject#23 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=1048450) at gossipd/gossipd.c:595
  ElementsProject#24 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=2096900) at gossipd/gossipd.c:595
  ElementsProject#25 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=4193801) at gossipd/gossipd.c:595
  ElementsProject#26 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=8387603) at gossipd/gossipd.c:595
  ElementsProject#27 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=16775207) at gossipd/gossipd.c:595
  ElementsProject#28 0x000000000040ddee in queue_channel_ranges (peer=0x3868fc8, first_blocknum=514201, number_of_blocks=33550414) at gossipd/gossipd.c:596
  ElementsProject#29 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=514201, number_of_blocks=67100829) at gossipd/gossipd.c:595
  ElementsProject#30 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=514201, number_of_blocks=134201659) at gossipd/gossipd.c:595
  ElementsProject#31 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=514201, number_of_blocks=268403318) at gossipd/gossipd.c:595
  ElementsProject#32 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=514201, number_of_blocks=536806636) at gossipd/gossipd.c:595
  ElementsProject#33 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=514201, number_of_blocks=1073613273) at gossipd/gossipd.c:595
  ElementsProject#34 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=514201, number_of_blocks=2147226547) at gossipd/gossipd.c:595
  ElementsProject#35 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=514201, number_of_blocks=4294453094) at gossipd/gossipd.c:595
  ElementsProject#36 0x000000000040df26 in handle_query_channel_range (peer=0x3868fc8, msg=0x37e0678 "\001\ao\342\214\n\266\361\263r\301\246\242F\256c\367O\223\036\203e\341Z\b\234h\326\031") at gossipd/gossipd.c:625

The cause was that converting a block number to an scid truncates it
at 24 bits.  When we look through the index from (truncated number) to
(real end number) we get every channel, which is too large to encode,
so we iterate again.

This fixes both that problem, and also the issue that we'd end up
dividing into many empty sections until we get to the highest block
number.  Instead, we just tack the empty blocks on to then end of the
final query.

Reported-by: George Vaccaro
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
rustyrussell referenced this issue in rustyrussell/lightning Jan 15, 2019
Don't do this:
  (gdb) bt
  #0  0x00007f37ae667c40 in ?? () from /lib/x86_64-linux-gnu/libz.so.1
  #1  0x00007f37ae668b38 in ?? () from /lib/x86_64-linux-gnu/libz.so.1
  #2  0x00007f37ae669907 in deflate () from /lib/x86_64-linux-gnu/libz.so.1
  #3  0x00007f37ae674c65 in compress2 () from /lib/x86_64-linux-gnu/libz.so.1
  #4  0x000000000040cfe3 in zencode_scids (ctx=0xc1f118, scids=0x2599bc49 "\a\325{", len=176320) at gossipd/gossipd.c:218
  #5  0x000000000040d0b3 in encode_short_channel_ids_end (encoded=0x7fff8f98d9f0, max_bytes=65490) at gossipd/gossipd.c:236
  #6  0x000000000040dd28 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17290511, number_of_blocks=8) at gossipd/gossipd.c:576
  #7  0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17290511, number_of_blocks=16) at gossipd/gossipd.c:595
  #8  0x000000000040ddee in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17290495, number_of_blocks=32) at gossipd/gossipd.c:596
  #9  0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17290495, number_of_blocks=64) at gossipd/gossipd.c:595
  #10 0x000000000040ddee in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17290431, number_of_blocks=128) at gossipd/gossipd.c:596
  #11 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17290431, number_of_blocks=256) at gossipd/gossipd.c:595
  #12 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17290431, number_of_blocks=512) at gossipd/gossipd.c:595
  #13 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17290431, number_of_blocks=1024) at gossipd/gossipd.c:595
  ElementsProject#14 0x000000000040ddee in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=2047) at gossipd/gossipd.c:596
  ElementsProject#15 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=4095) at gossipd/gossipd.c:595
  ElementsProject#16 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=8191) at gossipd/gossipd.c:595
  ElementsProject#17 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=16382) at gossipd/gossipd.c:595
  ElementsProject#18 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=32764) at gossipd/gossipd.c:595
  ElementsProject#19 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=65528) at gossipd/gossipd.c:595
  ElementsProject#20 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=131056) at gossipd/gossipd.c:595
  ElementsProject#21 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=262112) at gossipd/gossipd.c:595
  ElementsProject#22 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=524225) at gossipd/gossipd.c:595
  ElementsProject#23 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=1048450) at gossipd/gossipd.c:595
  ElementsProject#24 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=2096900) at gossipd/gossipd.c:595
  ElementsProject#25 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=4193801) at gossipd/gossipd.c:595
  ElementsProject#26 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=8387603) at gossipd/gossipd.c:595
  ElementsProject#27 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=16775207) at gossipd/gossipd.c:595
  ElementsProject#28 0x000000000040ddee in queue_channel_ranges (peer=0x3868fc8, first_blocknum=514201, number_of_blocks=33550414) at gossipd/gossipd.c:596
  ElementsProject#29 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=514201, number_of_blocks=67100829) at gossipd/gossipd.c:595
  ElementsProject#30 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=514201, number_of_blocks=134201659) at gossipd/gossipd.c:595
  ElementsProject#31 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=514201, number_of_blocks=268403318) at gossipd/gossipd.c:595
  ElementsProject#32 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=514201, number_of_blocks=536806636) at gossipd/gossipd.c:595
  ElementsProject#33 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=514201, number_of_blocks=1073613273) at gossipd/gossipd.c:595
  ElementsProject#34 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=514201, number_of_blocks=2147226547) at gossipd/gossipd.c:595
  ElementsProject#35 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=514201, number_of_blocks=4294453094) at gossipd/gossipd.c:595
  ElementsProject#36 0x000000000040df26 in handle_query_channel_range (peer=0x3868fc8, msg=0x37e0678 "\001\ao\342\214\n\266\361\263r\301\246\242F\256c\367O\223\036\203e\341Z\b\234h\326\031") at gossipd/gossipd.c:625

The cause was that converting a block number to an scid truncates it
at 24 bits.  When we look through the index from (truncated number) to
(real end number) we get every channel, which is too large to encode,
so we iterate again.

This fixes both that problem, and also the issue that we'd end up
dividing into many empty sections until we get to the highest block
number.  Instead, we just tack the empty blocks on to then end of the
final query.

(My initial version requested 0xFFFFFFFE blocks, but the dev code
which records what blocks were returned can't make a bitmap that big
on 32 bit).

Reported-by: George Vaccaro
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
cdecker referenced this issue in rustyrussell/lightning Jan 15, 2019
Don't do this:
  (gdb) bt
  #0  0x00007f37ae667c40 in ?? () from /lib/x86_64-linux-gnu/libz.so.1
  #1  0x00007f37ae668b38 in ?? () from /lib/x86_64-linux-gnu/libz.so.1
  #2  0x00007f37ae669907 in deflate () from /lib/x86_64-linux-gnu/libz.so.1
  #3  0x00007f37ae674c65 in compress2 () from /lib/x86_64-linux-gnu/libz.so.1
  #4  0x000000000040cfe3 in zencode_scids (ctx=0xc1f118, scids=0x2599bc49 "\a\325{", len=176320) at gossipd/gossipd.c:218
  #5  0x000000000040d0b3 in encode_short_channel_ids_end (encoded=0x7fff8f98d9f0, max_bytes=65490) at gossipd/gossipd.c:236
  #6  0x000000000040dd28 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17290511, number_of_blocks=8) at gossipd/gossipd.c:576
  #7  0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17290511, number_of_blocks=16) at gossipd/gossipd.c:595
  #8  0x000000000040ddee in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17290495, number_of_blocks=32) at gossipd/gossipd.c:596
  #9  0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17290495, number_of_blocks=64) at gossipd/gossipd.c:595
  #10 0x000000000040ddee in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17290431, number_of_blocks=128) at gossipd/gossipd.c:596
  #11 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17290431, number_of_blocks=256) at gossipd/gossipd.c:595
  #12 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17290431, number_of_blocks=512) at gossipd/gossipd.c:595
  #13 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17290431, number_of_blocks=1024) at gossipd/gossipd.c:595
  ElementsProject#14 0x000000000040ddee in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=2047) at gossipd/gossipd.c:596
  ElementsProject#15 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=4095) at gossipd/gossipd.c:595
  ElementsProject#16 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=8191) at gossipd/gossipd.c:595
  ElementsProject#17 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=16382) at gossipd/gossipd.c:595
  ElementsProject#18 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=32764) at gossipd/gossipd.c:595
  ElementsProject#19 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=65528) at gossipd/gossipd.c:595
  ElementsProject#20 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=131056) at gossipd/gossipd.c:595
  ElementsProject#21 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=262112) at gossipd/gossipd.c:595
  ElementsProject#22 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=524225) at gossipd/gossipd.c:595
  ElementsProject#23 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=1048450) at gossipd/gossipd.c:595
  ElementsProject#24 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=2096900) at gossipd/gossipd.c:595
  ElementsProject#25 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=4193801) at gossipd/gossipd.c:595
  ElementsProject#26 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=8387603) at gossipd/gossipd.c:595
  ElementsProject#27 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=16775207) at gossipd/gossipd.c:595
  ElementsProject#28 0x000000000040ddee in queue_channel_ranges (peer=0x3868fc8, first_blocknum=514201, number_of_blocks=33550414) at gossipd/gossipd.c:596
  ElementsProject#29 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=514201, number_of_blocks=67100829) at gossipd/gossipd.c:595
  ElementsProject#30 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=514201, number_of_blocks=134201659) at gossipd/gossipd.c:595
  ElementsProject#31 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=514201, number_of_blocks=268403318) at gossipd/gossipd.c:595
  ElementsProject#32 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=514201, number_of_blocks=536806636) at gossipd/gossipd.c:595
  ElementsProject#33 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=514201, number_of_blocks=1073613273) at gossipd/gossipd.c:595
  ElementsProject#34 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=514201, number_of_blocks=2147226547) at gossipd/gossipd.c:595
  ElementsProject#35 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=514201, number_of_blocks=4294453094) at gossipd/gossipd.c:595
  ElementsProject#36 0x000000000040df26 in handle_query_channel_range (peer=0x3868fc8, msg=0x37e0678 "\001\ao\342\214\n\266\361\263r\301\246\242F\256c\367O\223\036\203e\341Z\b\234h\326\031") at gossipd/gossipd.c:625

The cause was that converting a block number to an scid truncates it
at 24 bits.  When we look through the index from (truncated number) to
(real end number) we get every channel, which is too large to encode,
so we iterate again.

This fixes both that problem, and also the issue that we'd end up
dividing into many empty sections until we get to the highest block
number.  Instead, we just tack the empty blocks on to then end of the
final query.

(My initial version requested 0xFFFFFFFE blocks, but the dev code
which records what blocks were returned can't make a bitmap that big
on 32 bit).

Reported-by: George Vaccaro
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
cdecker referenced this issue in rustyrussell/lightning Jan 15, 2019
Don't do this:
  (gdb) bt
  #0  0x00007f37ae667c40 in ?? () from /lib/x86_64-linux-gnu/libz.so.1
  #1  0x00007f37ae668b38 in ?? () from /lib/x86_64-linux-gnu/libz.so.1
  #2  0x00007f37ae669907 in deflate () from /lib/x86_64-linux-gnu/libz.so.1
  #3  0x00007f37ae674c65 in compress2 () from /lib/x86_64-linux-gnu/libz.so.1
  #4  0x000000000040cfe3 in zencode_scids (ctx=0xc1f118, scids=0x2599bc49 "\a\325{", len=176320) at gossipd/gossipd.c:218
  #5  0x000000000040d0b3 in encode_short_channel_ids_end (encoded=0x7fff8f98d9f0, max_bytes=65490) at gossipd/gossipd.c:236
  #6  0x000000000040dd28 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17290511, number_of_blocks=8) at gossipd/gossipd.c:576
  #7  0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17290511, number_of_blocks=16) at gossipd/gossipd.c:595
  #8  0x000000000040ddee in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17290495, number_of_blocks=32) at gossipd/gossipd.c:596
  #9  0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17290495, number_of_blocks=64) at gossipd/gossipd.c:595
  #10 0x000000000040ddee in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17290431, number_of_blocks=128) at gossipd/gossipd.c:596
  #11 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17290431, number_of_blocks=256) at gossipd/gossipd.c:595
  #12 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17290431, number_of_blocks=512) at gossipd/gossipd.c:595
  #13 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17290431, number_of_blocks=1024) at gossipd/gossipd.c:595
  ElementsProject#14 0x000000000040ddee in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=2047) at gossipd/gossipd.c:596
  ElementsProject#15 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=4095) at gossipd/gossipd.c:595
  ElementsProject#16 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=8191) at gossipd/gossipd.c:595
  ElementsProject#17 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=16382) at gossipd/gossipd.c:595
  ElementsProject#18 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=32764) at gossipd/gossipd.c:595
  ElementsProject#19 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=65528) at gossipd/gossipd.c:595
  ElementsProject#20 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=131056) at gossipd/gossipd.c:595
  ElementsProject#21 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=262112) at gossipd/gossipd.c:595
  ElementsProject#22 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=524225) at gossipd/gossipd.c:595
  ElementsProject#23 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=1048450) at gossipd/gossipd.c:595
  ElementsProject#24 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=2096900) at gossipd/gossipd.c:595
  ElementsProject#25 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=4193801) at gossipd/gossipd.c:595
  ElementsProject#26 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=8387603) at gossipd/gossipd.c:595
  ElementsProject#27 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=16775207) at gossipd/gossipd.c:595
  ElementsProject#28 0x000000000040ddee in queue_channel_ranges (peer=0x3868fc8, first_blocknum=514201, number_of_blocks=33550414) at gossipd/gossipd.c:596
  ElementsProject#29 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=514201, number_of_blocks=67100829) at gossipd/gossipd.c:595
  ElementsProject#30 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=514201, number_of_blocks=134201659) at gossipd/gossipd.c:595
  ElementsProject#31 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=514201, number_of_blocks=268403318) at gossipd/gossipd.c:595
  ElementsProject#32 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=514201, number_of_blocks=536806636) at gossipd/gossipd.c:595
  ElementsProject#33 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=514201, number_of_blocks=1073613273) at gossipd/gossipd.c:595
  ElementsProject#34 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=514201, number_of_blocks=2147226547) at gossipd/gossipd.c:595
  ElementsProject#35 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=514201, number_of_blocks=4294453094) at gossipd/gossipd.c:595
  ElementsProject#36 0x000000000040df26 in handle_query_channel_range (peer=0x3868fc8, msg=0x37e0678 "\001\ao\342\214\n\266\361\263r\301\246\242F\256c\367O\223\036\203e\341Z\b\234h\326\031") at gossipd/gossipd.c:625

The cause was that converting a block number to an scid truncates it
at 24 bits.  When we look through the index from (truncated number) to
(real end number) we get every channel, which is too large to encode,
so we iterate again.

This fixes both that problem, and also the issue that we'd end up
dividing into many empty sections until we get to the highest block
number.  Instead, we just tack the empty blocks on to then end of the
final query.

(My initial version requested 0xFFFFFFFE blocks, but the dev code
which records what blocks were returned can't make a bitmap that big
on 32 bit).

Reported-by: George Vaccaro
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
niftynei pushed a commit that referenced this issue Jan 15, 2019
Don't do this:
  (gdb) bt
  #0  0x00007f37ae667c40 in ?? () from /lib/x86_64-linux-gnu/libz.so.1
  #1  0x00007f37ae668b38 in ?? () from /lib/x86_64-linux-gnu/libz.so.1
  #2  0x00007f37ae669907 in deflate () from /lib/x86_64-linux-gnu/libz.so.1
  #3  0x00007f37ae674c65 in compress2 () from /lib/x86_64-linux-gnu/libz.so.1
  #4  0x000000000040cfe3 in zencode_scids (ctx=0xc1f118, scids=0x2599bc49 "\a\325{", len=176320) at gossipd/gossipd.c:218
  #5  0x000000000040d0b3 in encode_short_channel_ids_end (encoded=0x7fff8f98d9f0, max_bytes=65490) at gossipd/gossipd.c:236
  #6  0x000000000040dd28 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17290511, number_of_blocks=8) at gossipd/gossipd.c:576
  #7  0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17290511, number_of_blocks=16) at gossipd/gossipd.c:595
  #8  0x000000000040ddee in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17290495, number_of_blocks=32) at gossipd/gossipd.c:596
  #9  0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17290495, number_of_blocks=64) at gossipd/gossipd.c:595
  #10 0x000000000040ddee in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17290431, number_of_blocks=128) at gossipd/gossipd.c:596
  #11 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17290431, number_of_blocks=256) at gossipd/gossipd.c:595
  #12 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17290431, number_of_blocks=512) at gossipd/gossipd.c:595
  #13 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17290431, number_of_blocks=1024) at gossipd/gossipd.c:595
  #14 0x000000000040ddee in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=2047) at gossipd/gossipd.c:596
  #15 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=4095) at gossipd/gossipd.c:595
  #16 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=8191) at gossipd/gossipd.c:595
  #17 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=16382) at gossipd/gossipd.c:595
  #18 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=32764) at gossipd/gossipd.c:595
  #19 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=65528) at gossipd/gossipd.c:595
  #20 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=131056) at gossipd/gossipd.c:595
  #21 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=262112) at gossipd/gossipd.c:595
  #22 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=524225) at gossipd/gossipd.c:595
  #23 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=1048450) at gossipd/gossipd.c:595
  #24 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=2096900) at gossipd/gossipd.c:595
  #25 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=4193801) at gossipd/gossipd.c:595
  #26 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=8387603) at gossipd/gossipd.c:595
  #27 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=16775207) at gossipd/gossipd.c:595
  #28 0x000000000040ddee in queue_channel_ranges (peer=0x3868fc8, first_blocknum=514201, number_of_blocks=33550414) at gossipd/gossipd.c:596
  #29 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=514201, number_of_blocks=67100829) at gossipd/gossipd.c:595
  #30 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=514201, number_of_blocks=134201659) at gossipd/gossipd.c:595
  #31 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=514201, number_of_blocks=268403318) at gossipd/gossipd.c:595
  #32 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=514201, number_of_blocks=536806636) at gossipd/gossipd.c:595
  #33 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=514201, number_of_blocks=1073613273) at gossipd/gossipd.c:595
  #34 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=514201, number_of_blocks=2147226547) at gossipd/gossipd.c:595
  #35 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=514201, number_of_blocks=4294453094) at gossipd/gossipd.c:595
  #36 0x000000000040df26 in handle_query_channel_range (peer=0x3868fc8, msg=0x37e0678 "\001\ao\342\214\n\266\361\263r\301\246\242F\256c\367O\223\036\203e\341Z\b\234h\326\031") at gossipd/gossipd.c:625

The cause was that converting a block number to an scid truncates it
at 24 bits.  When we look through the index from (truncated number) to
(real end number) we get every channel, which is too large to encode,
so we iterate again.

This fixes both that problem, and also the issue that we'd end up
dividing into many empty sections until we get to the highest block
number.  Instead, we just tack the empty blocks on to then end of the
final query.

(My initial version requested 0xFFFFFFFE blocks, but the dev code
which records what blocks were returned can't make a bitmap that big
on 32 bit).

Reported-by: George Vaccaro
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
NicolasDorier pushed a commit to btcpayserver/lightning that referenced this issue Jan 17, 2019
Don't do this:
  (gdb) bt
  #0  0x00007f37ae667c40 in ?? () from /lib/x86_64-linux-gnu/libz.so.1
  #1  0x00007f37ae668b38 in ?? () from /lib/x86_64-linux-gnu/libz.so.1
  #2  0x00007f37ae669907 in deflate () from /lib/x86_64-linux-gnu/libz.so.1
  #3  0x00007f37ae674c65 in compress2 () from /lib/x86_64-linux-gnu/libz.so.1
  #4  0x000000000040cfe3 in zencode_scids (ctx=0xc1f118, scids=0x2599bc49 "\a\325{", len=176320) at gossipd/gossipd.c:218
  ElementsProject#5  0x000000000040d0b3 in encode_short_channel_ids_end (encoded=0x7fff8f98d9f0, max_bytes=65490) at gossipd/gossipd.c:236
  ElementsProject#6  0x000000000040dd28 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17290511, number_of_blocks=8) at gossipd/gossipd.c:576
  ElementsProject#7  0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17290511, number_of_blocks=16) at gossipd/gossipd.c:595
  ElementsProject#8  0x000000000040ddee in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17290495, number_of_blocks=32) at gossipd/gossipd.c:596
  ElementsProject#9  0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17290495, number_of_blocks=64) at gossipd/gossipd.c:595
  ElementsProject#10 0x000000000040ddee in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17290431, number_of_blocks=128) at gossipd/gossipd.c:596
  ElementsProject#11 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17290431, number_of_blocks=256) at gossipd/gossipd.c:595
  ElementsProject#12 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17290431, number_of_blocks=512) at gossipd/gossipd.c:595
  ElementsProject#13 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17290431, number_of_blocks=1024) at gossipd/gossipd.c:595
  ElementsProject#14 0x000000000040ddee in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=2047) at gossipd/gossipd.c:596
  ElementsProject#15 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=4095) at gossipd/gossipd.c:595
  ElementsProject#16 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=8191) at gossipd/gossipd.c:595
  ElementsProject#17 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=16382) at gossipd/gossipd.c:595
  ElementsProject#18 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=32764) at gossipd/gossipd.c:595
  ElementsProject#19 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=65528) at gossipd/gossipd.c:595
  ElementsProject#20 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=131056) at gossipd/gossipd.c:595
  ElementsProject#21 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=262112) at gossipd/gossipd.c:595
  ElementsProject#22 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=524225) at gossipd/gossipd.c:595
  ElementsProject#23 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=1048450) at gossipd/gossipd.c:595
  ElementsProject#24 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=2096900) at gossipd/gossipd.c:595
  ElementsProject#25 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=4193801) at gossipd/gossipd.c:595
  ElementsProject#26 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=8387603) at gossipd/gossipd.c:595
  ElementsProject#27 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=16775207) at gossipd/gossipd.c:595
  ElementsProject#28 0x000000000040ddee in queue_channel_ranges (peer=0x3868fc8, first_blocknum=514201, number_of_blocks=33550414) at gossipd/gossipd.c:596
  ElementsProject#29 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=514201, number_of_blocks=67100829) at gossipd/gossipd.c:595
  ElementsProject#30 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=514201, number_of_blocks=134201659) at gossipd/gossipd.c:595
  ElementsProject#31 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=514201, number_of_blocks=268403318) at gossipd/gossipd.c:595
  ElementsProject#32 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=514201, number_of_blocks=536806636) at gossipd/gossipd.c:595
  ElementsProject#33 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=514201, number_of_blocks=1073613273) at gossipd/gossipd.c:595
  ElementsProject#34 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=514201, number_of_blocks=2147226547) at gossipd/gossipd.c:595
  ElementsProject#35 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=514201, number_of_blocks=4294453094) at gossipd/gossipd.c:595
  ElementsProject#36 0x000000000040df26 in handle_query_channel_range (peer=0x3868fc8, msg=0x37e0678 "\001\ao\342\214\n\266\361\263r\301\246\242F\256c\367O\223\036\203e\341Z\b\234h\326\031") at gossipd/gossipd.c:625

The cause was that converting a block number to an scid truncates it
at 24 bits.  When we look through the index from (truncated number) to
(real end number) we get every channel, which is too large to encode,
so we iterate again.

This fixes both that problem, and also the issue that we'd end up
dividing into many empty sections until we get to the highest block
number.  Instead, we just tack the empty blocks on to then end of the
final query.

(My initial version requested 0xFFFFFFFE blocks, but the dev code
which records what blocks were returned can't make a bitmap that big
on 32 bit).

Reported-by: George Vaccaro
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
danielsocials pushed a commit to lightningcn/lightning that referenced this issue Feb 15, 2019
Don't do this:
  (gdb) bt
  #0  0x00007f37ae667c40 in ?? () from /lib/x86_64-linux-gnu/libz.so.1
  ElementsProject#1  0x00007f37ae668b38 in ?? () from /lib/x86_64-linux-gnu/libz.so.1
  ElementsProject#2  0x00007f37ae669907 in deflate () from /lib/x86_64-linux-gnu/libz.so.1
  ElementsProject#3  0x00007f37ae674c65 in compress2 () from /lib/x86_64-linux-gnu/libz.so.1
  ElementsProject#4  0x000000000040cfe3 in zencode_scids (ctx=0xc1f118, scids=0x2599bc49 "\a\325{", len=176320) at gossipd/gossipd.c:218
  ElementsProject#5  0x000000000040d0b3 in encode_short_channel_ids_end (encoded=0x7fff8f98d9f0, max_bytes=65490) at gossipd/gossipd.c:236
  ElementsProject#6  0x000000000040dd28 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17290511, number_of_blocks=8) at gossipd/gossipd.c:576
  ElementsProject#7  0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17290511, number_of_blocks=16) at gossipd/gossipd.c:595
  ElementsProject#8  0x000000000040ddee in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17290495, number_of_blocks=32) at gossipd/gossipd.c:596
  ElementsProject#9  0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17290495, number_of_blocks=64) at gossipd/gossipd.c:595
  ElementsProject#10 0x000000000040ddee in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17290431, number_of_blocks=128) at gossipd/gossipd.c:596
  ElementsProject#11 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17290431, number_of_blocks=256) at gossipd/gossipd.c:595
  ElementsProject#12 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17290431, number_of_blocks=512) at gossipd/gossipd.c:595
  ElementsProject#13 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17290431, number_of_blocks=1024) at gossipd/gossipd.c:595
  ElementsProject#14 0x000000000040ddee in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=2047) at gossipd/gossipd.c:596
  ElementsProject#15 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=4095) at gossipd/gossipd.c:595
  ElementsProject#16 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=8191) at gossipd/gossipd.c:595
  ElementsProject#17 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=16382) at gossipd/gossipd.c:595
  ElementsProject#18 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=32764) at gossipd/gossipd.c:595
  ElementsProject#19 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=65528) at gossipd/gossipd.c:595
  ElementsProject#20 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=131056) at gossipd/gossipd.c:595
  ElementsProject#21 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=262112) at gossipd/gossipd.c:595
  ElementsProject#22 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=524225) at gossipd/gossipd.c:595
  ElementsProject#23 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=1048450) at gossipd/gossipd.c:595
  ElementsProject#24 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=2096900) at gossipd/gossipd.c:595
  ElementsProject#25 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=4193801) at gossipd/gossipd.c:595
  ElementsProject#26 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=8387603) at gossipd/gossipd.c:595
  ElementsProject#27 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=16775207) at gossipd/gossipd.c:595
  ElementsProject#28 0x000000000040ddee in queue_channel_ranges (peer=0x3868fc8, first_blocknum=514201, number_of_blocks=33550414) at gossipd/gossipd.c:596
  ElementsProject#29 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=514201, number_of_blocks=67100829) at gossipd/gossipd.c:595
  ElementsProject#30 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=514201, number_of_blocks=134201659) at gossipd/gossipd.c:595
  ElementsProject#31 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=514201, number_of_blocks=268403318) at gossipd/gossipd.c:595
  ElementsProject#32 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=514201, number_of_blocks=536806636) at gossipd/gossipd.c:595
  ElementsProject#33 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=514201, number_of_blocks=1073613273) at gossipd/gossipd.c:595
  ElementsProject#34 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=514201, number_of_blocks=2147226547) at gossipd/gossipd.c:595
  ElementsProject#35 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=514201, number_of_blocks=4294453094) at gossipd/gossipd.c:595
  ElementsProject#36 0x000000000040df26 in handle_query_channel_range (peer=0x3868fc8, msg=0x37e0678 "\001\ao\342\214\n\266\361\263r\301\246\242F\256c\367O\223\036\203e\341Z\b\234h\326\031") at gossipd/gossipd.c:625

The cause was that converting a block number to an scid truncates it
at 24 bits.  When we look through the index from (truncated number) to
(real end number) we get every channel, which is too large to encode,
so we iterate again.

This fixes both that problem, and also the issue that we'd end up
dividing into many empty sections until we get to the highest block
number.  Instead, we just tack the empty blocks on to then end of the
final query.

(My initial version requested 0xFFFFFFFE blocks, but the dev code
which records what blocks were returned can't make a bitmap that big
on 32 bit).

Reported-by: George Vaccaro
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
rustyrussell referenced this issue in rustyrussell/lightning Jun 12, 2019
Direct leak of 64 byte(s) in 1 object(s) allocated from:
    #0 0x7f4dc279163e in calloc (/lib/x86_64-linux-gnu/libasan.so.5+0x10c63e)
    #1 0x564ee8a24bb1 in htable_default_alloc ccan/ccan/htable/htable.c:19
    #2 0x564ee8a2551b in double_table ccan/ccan/htable/htable.c:226
    #3 0x564ee8a259e5 in htable_add_ ccan/ccan/htable/htable.c:331
    #4 0x564ee89a5300 in block_map_add lightningd/chaintopology.h:83
    #5 0x564ee89a6ece in add_tip lightningd/chaintopology.c:626
    #6 0x564ee89a72c3 in have_new_block lightningd/chaintopology.c:694
    #7 0x564ee89a3ab0 in process_rawblock lightningd/bitcoind.c:466
    #8 0x564ee89a2fb4 in bcli_finished lightningd/bitcoind.c:214
    #9 0x564ee8a284d6 in destroy_conn ccan/ccan/io/poll.c:244
    #10 0x564ee8a284f6 in destroy_conn_close_fd ccan/ccan/io/poll.c:250
    #11 0x564ee8a34a0d in notify ccan/ccan/tal/tal.c:235
    #12 0x564ee8a34efc in del_tree ccan/ccan/tal/tal.c:397
    #13 0x564ee8a35288 in tal_free ccan/ccan/tal/tal.c:481
    ElementsProject#14 0x564ee8a26cf5 in io_close ccan/ccan/io/io.c:450
    ElementsProject#15 0x564ee8a28c11 in io_loop ccan/ccan/io/poll.c:449
    ElementsProject#16 0x564ee89b3c3b in io_loop_with_timers lightningd/io_loop_with_timers.c:24
    ElementsProject#17 0x564ee89ba540 in main lightningd/lightningd.c:822
    ElementsProject#18 0x7f4dc2143b6a in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x26b6a)

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
rustyrussell referenced this issue in rustyrussell/lightning Jun 12, 2019
Direct leak of 32 byte(s) in 1 object(s) allocated from:
    #0 0x7f7678ee863e in calloc (/lib/x86_64-linux-gnu/libasan.so.5+0x10c63e)
    #1 0x55f8c7b0fce5 in htable_default_alloc ccan/ccan/htable/htable.c:19
    #2 0x55f8c7b1064f in double_table ccan/ccan/htable/htable.c:226
    #3 0x55f8c7b10b19 in htable_add_ ccan/ccan/htable/htable.c:331
    #4 0x55f8c7afac63 in scriptpubkeyset_add wallet/txfilter.c:30
    #5 0x55f8c7afafce in txfilter_add_scriptpubkey wallet/txfilter.c:77
    #6 0x55f8c7afb05f in txfilter_add_derkey wallet/txfilter.c:91
    #7 0x55f8c7aa4d67 in init_txfilter lightningd/lightningd.c:482
    #8 0x55f8c7aa52d8 in main lightningd/lightningd.c:721
    #9 0x7f767889ab6a in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x26b6a)

Direct leak of 16 byte(s) in 1 object(s) allocated from:
    #0 0x7f05f389563e in calloc (/lib/x86_64-linux-gnu/libasan.so.5+0x10c63e)
    #1 0x55cac1e6bc99 in htable_default_alloc ccan/ccan/htable/htable.c:19
    #2 0x55cac1e6c603 in double_table ccan/ccan/htable/htable.c:226
    #3 0x55cac1e6cacd in htable_add_ ccan/ccan/htable/htable.c:331
    #4 0x55cac1e56e48 in outpointset_add wallet/txfilter.c:61
    #5 0x55cac1e57162 in outpointfilter_add wallet/txfilter.c:116
    #6 0x55cac1e5ea3a in wallet_utxoset_add wallet/wallet.c:2365
    #7 0x55cac1deddc2 in topo_add_utxos lightningd/chaintopology.c:603
    #8 0x55cac1dedeac in add_tip lightningd/chaintopology.c:620
    #9 0x55cac1dee2de in have_new_block lightningd/chaintopology.c:694
    #10 0x55cac1deaab0 in process_rawblock lightningd/bitcoind.c:466
    #11 0x55cac1de9fb4 in bcli_finished lightningd/bitcoind.c:214
    #12 0x55cac1e6f5be in destroy_conn ccan/ccan/io/poll.c:244
    #13 0x55cac1e6f5de in destroy_conn_close_fd ccan/ccan/io/poll.c:250
    ElementsProject#14 0x55cac1e7baf5 in notify ccan/ccan/tal/tal.c:235
    ElementsProject#15 0x55cac1e7bfe4 in del_tree ccan/ccan/tal/tal.c:397
    ElementsProject#16 0x55cac1e7c370 in tal_free ccan/ccan/tal/tal.c:481
    ElementsProject#17 0x55cac1e6dddd in io_close ccan/ccan/io/io.c:450
    ElementsProject#18 0x55cac1e6fcf9 in io_loop ccan/ccan/io/poll.c:449
    ElementsProject#19 0x55cac1dfac66 in io_loop_with_timers lightningd/io_loop_with_timers.c:24
    ElementsProject#20 0x55cac1e0156b in main lightningd/lightningd.c:822
    ElementsProject#21 0x7f05f3247b6a in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x26b6a)

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
rustyrussell referenced this issue in rustyrussell/lightning Jun 12, 2019
Direct leak of 16 byte(s) in 1 object(s) allocated from:
    #0 0x7ff02889063e in calloc (/lib/x86_64-linux-gnu/libasan.so.5+0x10c63e)
    #1 0x555ce2ad8d2e in htable_default_alloc ccan/ccan/htable/htable.c:19
    #2 0x555ce2ad9698 in double_table ccan/ccan/htable/htable.c:226
    #3 0x555ce2ad9b62 in htable_add_ ccan/ccan/htable/htable.c:331
    #4 0x555ce2a638e4 in htlc_in_map_add lightningd/htlc_end.h:113
    #5 0x555ce2a63beb in connect_htlc_in lightningd/htlc_end.c:39
    #6 0x555ce2a85cbc in channel_added_their_htlc lightningd/peer_htlcs.c:1382
    #7 0x555ce2a860e1 in peer_got_commitsig lightningd/peer_htlcs.c:1466
    #8 0x555ce2a5db04 in channel_msg lightningd/channel_control.c:228
    #9 0x555ce2a8d393 in sd_msg_read lightningd/subd.c:474
    #10 0x555ce2ada157 in next_plan ccan/ccan/io/io.c:59
    #11 0x555ce2adacd4 in do_plan ccan/ccan/io/io.c:407
    #12 0x555ce2adad12 in io_ready ccan/ccan/io/io.c:417
    #13 0x555ce2adcd67 in io_loop ccan/ccan/io/poll.c:445
    ElementsProject#14 0x555ce2a67c66 in io_loop_with_timers lightningd/io_loop_with_timers.c:24
    ElementsProject#15 0x555ce2a6e56b in main lightningd/lightningd.c:822
    ElementsProject#16 0x7ff028242b6a in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x26b6a)

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
rustyrussell referenced this issue in rustyrussell/lightning Jun 12, 2019
Direct leak of 64 byte(s) in 1 object(s) allocated from:
    #0 0x7f4dc279163e in calloc (/lib/x86_64-linux-gnu/libasan.so.5+0x10c63e)
    #1 0x564ee8a24bb1 in htable_default_alloc ccan/ccan/htable/htable.c:19
    #2 0x564ee8a2551b in double_table ccan/ccan/htable/htable.c:226
    #3 0x564ee8a259e5 in htable_add_ ccan/ccan/htable/htable.c:331
    #4 0x564ee89a5300 in block_map_add lightningd/chaintopology.h:83
    #5 0x564ee89a6ece in add_tip lightningd/chaintopology.c:626
    #6 0x564ee89a72c3 in have_new_block lightningd/chaintopology.c:694
    #7 0x564ee89a3ab0 in process_rawblock lightningd/bitcoind.c:466
    #8 0x564ee89a2fb4 in bcli_finished lightningd/bitcoind.c:214
    #9 0x564ee8a284d6 in destroy_conn ccan/ccan/io/poll.c:244
    #10 0x564ee8a284f6 in destroy_conn_close_fd ccan/ccan/io/poll.c:250
    #11 0x564ee8a34a0d in notify ccan/ccan/tal/tal.c:235
    #12 0x564ee8a34efc in del_tree ccan/ccan/tal/tal.c:397
    #13 0x564ee8a35288 in tal_free ccan/ccan/tal/tal.c:481
    ElementsProject#14 0x564ee8a26cf5 in io_close ccan/ccan/io/io.c:450
    ElementsProject#15 0x564ee8a28c11 in io_loop ccan/ccan/io/poll.c:449
    ElementsProject#16 0x564ee89b3c3b in io_loop_with_timers lightningd/io_loop_with_timers.c:24
    ElementsProject#17 0x564ee89ba540 in main lightningd/lightningd.c:822
    ElementsProject#18 0x7f4dc2143b6a in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x26b6a)

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
rustyrussell referenced this issue in rustyrussell/lightning Jun 12, 2019
Direct leak of 32 byte(s) in 1 object(s) allocated from:
    #0 0x7f7678ee863e in calloc (/lib/x86_64-linux-gnu/libasan.so.5+0x10c63e)
    #1 0x55f8c7b0fce5 in htable_default_alloc ccan/ccan/htable/htable.c:19
    #2 0x55f8c7b1064f in double_table ccan/ccan/htable/htable.c:226
    #3 0x55f8c7b10b19 in htable_add_ ccan/ccan/htable/htable.c:331
    #4 0x55f8c7afac63 in scriptpubkeyset_add wallet/txfilter.c:30
    #5 0x55f8c7afafce in txfilter_add_scriptpubkey wallet/txfilter.c:77
    #6 0x55f8c7afb05f in txfilter_add_derkey wallet/txfilter.c:91
    #7 0x55f8c7aa4d67 in init_txfilter lightningd/lightningd.c:482
    #8 0x55f8c7aa52d8 in main lightningd/lightningd.c:721
    #9 0x7f767889ab6a in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x26b6a)

Direct leak of 16 byte(s) in 1 object(s) allocated from:
    #0 0x7f05f389563e in calloc (/lib/x86_64-linux-gnu/libasan.so.5+0x10c63e)
    #1 0x55cac1e6bc99 in htable_default_alloc ccan/ccan/htable/htable.c:19
    #2 0x55cac1e6c603 in double_table ccan/ccan/htable/htable.c:226
    #3 0x55cac1e6cacd in htable_add_ ccan/ccan/htable/htable.c:331
    #4 0x55cac1e56e48 in outpointset_add wallet/txfilter.c:61
    #5 0x55cac1e57162 in outpointfilter_add wallet/txfilter.c:116
    #6 0x55cac1e5ea3a in wallet_utxoset_add wallet/wallet.c:2365
    #7 0x55cac1deddc2 in topo_add_utxos lightningd/chaintopology.c:603
    #8 0x55cac1dedeac in add_tip lightningd/chaintopology.c:620
    #9 0x55cac1dee2de in have_new_block lightningd/chaintopology.c:694
    #10 0x55cac1deaab0 in process_rawblock lightningd/bitcoind.c:466
    #11 0x55cac1de9fb4 in bcli_finished lightningd/bitcoind.c:214
    #12 0x55cac1e6f5be in destroy_conn ccan/ccan/io/poll.c:244
    #13 0x55cac1e6f5de in destroy_conn_close_fd ccan/ccan/io/poll.c:250
    ElementsProject#14 0x55cac1e7baf5 in notify ccan/ccan/tal/tal.c:235
    ElementsProject#15 0x55cac1e7bfe4 in del_tree ccan/ccan/tal/tal.c:397
    ElementsProject#16 0x55cac1e7c370 in tal_free ccan/ccan/tal/tal.c:481
    ElementsProject#17 0x55cac1e6dddd in io_close ccan/ccan/io/io.c:450
    ElementsProject#18 0x55cac1e6fcf9 in io_loop ccan/ccan/io/poll.c:449
    ElementsProject#19 0x55cac1dfac66 in io_loop_with_timers lightningd/io_loop_with_timers.c:24
    ElementsProject#20 0x55cac1e0156b in main lightningd/lightningd.c:822
    ElementsProject#21 0x7f05f3247b6a in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x26b6a)

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
rustyrussell referenced this issue in rustyrussell/lightning Jun 12, 2019
Direct leak of 16 byte(s) in 1 object(s) allocated from:
    #0 0x7ff02889063e in calloc (/lib/x86_64-linux-gnu/libasan.so.5+0x10c63e)
    #1 0x555ce2ad8d2e in htable_default_alloc ccan/ccan/htable/htable.c:19
    #2 0x555ce2ad9698 in double_table ccan/ccan/htable/htable.c:226
    #3 0x555ce2ad9b62 in htable_add_ ccan/ccan/htable/htable.c:331
    #4 0x555ce2a638e4 in htlc_in_map_add lightningd/htlc_end.h:113
    #5 0x555ce2a63beb in connect_htlc_in lightningd/htlc_end.c:39
    #6 0x555ce2a85cbc in channel_added_their_htlc lightningd/peer_htlcs.c:1382
    #7 0x555ce2a860e1 in peer_got_commitsig lightningd/peer_htlcs.c:1466
    #8 0x555ce2a5db04 in channel_msg lightningd/channel_control.c:228
    #9 0x555ce2a8d393 in sd_msg_read lightningd/subd.c:474
    #10 0x555ce2ada157 in next_plan ccan/ccan/io/io.c:59
    #11 0x555ce2adacd4 in do_plan ccan/ccan/io/io.c:407
    #12 0x555ce2adad12 in io_ready ccan/ccan/io/io.c:417
    #13 0x555ce2adcd67 in io_loop ccan/ccan/io/poll.c:445
    ElementsProject#14 0x555ce2a67c66 in io_loop_with_timers lightningd/io_loop_with_timers.c:24
    ElementsProject#15 0x555ce2a6e56b in main lightningd/lightningd.c:822
    ElementsProject#16 0x7ff028242b6a in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x26b6a)

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
rustyrussell referenced this issue in rustyrussell/lightning Jun 12, 2019
Direct leak of 64 byte(s) in 1 object(s) allocated from:
    #0 0x7f4dc279163e in calloc (/lib/x86_64-linux-gnu/libasan.so.5+0x10c63e)
    #1 0x564ee8a24bb1 in htable_default_alloc ccan/ccan/htable/htable.c:19
    #2 0x564ee8a2551b in double_table ccan/ccan/htable/htable.c:226
    #3 0x564ee8a259e5 in htable_add_ ccan/ccan/htable/htable.c:331
    #4 0x564ee89a5300 in block_map_add lightningd/chaintopology.h:83
    #5 0x564ee89a6ece in add_tip lightningd/chaintopology.c:626
    #6 0x564ee89a72c3 in have_new_block lightningd/chaintopology.c:694
    #7 0x564ee89a3ab0 in process_rawblock lightningd/bitcoind.c:466
    #8 0x564ee89a2fb4 in bcli_finished lightningd/bitcoind.c:214
    #9 0x564ee8a284d6 in destroy_conn ccan/ccan/io/poll.c:244
    #10 0x564ee8a284f6 in destroy_conn_close_fd ccan/ccan/io/poll.c:250
    #11 0x564ee8a34a0d in notify ccan/ccan/tal/tal.c:235
    #12 0x564ee8a34efc in del_tree ccan/ccan/tal/tal.c:397
    #13 0x564ee8a35288 in tal_free ccan/ccan/tal/tal.c:481
    ElementsProject#14 0x564ee8a26cf5 in io_close ccan/ccan/io/io.c:450
    ElementsProject#15 0x564ee8a28c11 in io_loop ccan/ccan/io/poll.c:449
    ElementsProject#16 0x564ee89b3c3b in io_loop_with_timers lightningd/io_loop_with_timers.c:24
    ElementsProject#17 0x564ee89ba540 in main lightningd/lightningd.c:822
    ElementsProject#18 0x7f4dc2143b6a in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x26b6a)

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
rustyrussell referenced this issue in rustyrussell/lightning Jun 12, 2019
Direct leak of 32 byte(s) in 1 object(s) allocated from:
    #0 0x7f7678ee863e in calloc (/lib/x86_64-linux-gnu/libasan.so.5+0x10c63e)
    #1 0x55f8c7b0fce5 in htable_default_alloc ccan/ccan/htable/htable.c:19
    #2 0x55f8c7b1064f in double_table ccan/ccan/htable/htable.c:226
    #3 0x55f8c7b10b19 in htable_add_ ccan/ccan/htable/htable.c:331
    #4 0x55f8c7afac63 in scriptpubkeyset_add wallet/txfilter.c:30
    #5 0x55f8c7afafce in txfilter_add_scriptpubkey wallet/txfilter.c:77
    #6 0x55f8c7afb05f in txfilter_add_derkey wallet/txfilter.c:91
    #7 0x55f8c7aa4d67 in init_txfilter lightningd/lightningd.c:482
    #8 0x55f8c7aa52d8 in main lightningd/lightningd.c:721
    #9 0x7f767889ab6a in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x26b6a)

Direct leak of 16 byte(s) in 1 object(s) allocated from:
    #0 0x7f05f389563e in calloc (/lib/x86_64-linux-gnu/libasan.so.5+0x10c63e)
    #1 0x55cac1e6bc99 in htable_default_alloc ccan/ccan/htable/htable.c:19
    #2 0x55cac1e6c603 in double_table ccan/ccan/htable/htable.c:226
    #3 0x55cac1e6cacd in htable_add_ ccan/ccan/htable/htable.c:331
    #4 0x55cac1e56e48 in outpointset_add wallet/txfilter.c:61
    #5 0x55cac1e57162 in outpointfilter_add wallet/txfilter.c:116
    #6 0x55cac1e5ea3a in wallet_utxoset_add wallet/wallet.c:2365
    #7 0x55cac1deddc2 in topo_add_utxos lightningd/chaintopology.c:603
    #8 0x55cac1dedeac in add_tip lightningd/chaintopology.c:620
    #9 0x55cac1dee2de in have_new_block lightningd/chaintopology.c:694
    #10 0x55cac1deaab0 in process_rawblock lightningd/bitcoind.c:466
    #11 0x55cac1de9fb4 in bcli_finished lightningd/bitcoind.c:214
    #12 0x55cac1e6f5be in destroy_conn ccan/ccan/io/poll.c:244
    #13 0x55cac1e6f5de in destroy_conn_close_fd ccan/ccan/io/poll.c:250
    ElementsProject#14 0x55cac1e7baf5 in notify ccan/ccan/tal/tal.c:235
    ElementsProject#15 0x55cac1e7bfe4 in del_tree ccan/ccan/tal/tal.c:397
    ElementsProject#16 0x55cac1e7c370 in tal_free ccan/ccan/tal/tal.c:481
    ElementsProject#17 0x55cac1e6dddd in io_close ccan/ccan/io/io.c:450
    ElementsProject#18 0x55cac1e6fcf9 in io_loop ccan/ccan/io/poll.c:449
    ElementsProject#19 0x55cac1dfac66 in io_loop_with_timers lightningd/io_loop_with_timers.c:24
    ElementsProject#20 0x55cac1e0156b in main lightningd/lightningd.c:822
    ElementsProject#21 0x7f05f3247b6a in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x26b6a)

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
rustyrussell referenced this issue in rustyrussell/lightning Jun 12, 2019
Direct leak of 16 byte(s) in 1 object(s) allocated from:
    #0 0x7ff02889063e in calloc (/lib/x86_64-linux-gnu/libasan.so.5+0x10c63e)
    #1 0x555ce2ad8d2e in htable_default_alloc ccan/ccan/htable/htable.c:19
    #2 0x555ce2ad9698 in double_table ccan/ccan/htable/htable.c:226
    #3 0x555ce2ad9b62 in htable_add_ ccan/ccan/htable/htable.c:331
    #4 0x555ce2a638e4 in htlc_in_map_add lightningd/htlc_end.h:113
    #5 0x555ce2a63beb in connect_htlc_in lightningd/htlc_end.c:39
    #6 0x555ce2a85cbc in channel_added_their_htlc lightningd/peer_htlcs.c:1382
    #7 0x555ce2a860e1 in peer_got_commitsig lightningd/peer_htlcs.c:1466
    #8 0x555ce2a5db04 in channel_msg lightningd/channel_control.c:228
    #9 0x555ce2a8d393 in sd_msg_read lightningd/subd.c:474
    #10 0x555ce2ada157 in next_plan ccan/ccan/io/io.c:59
    #11 0x555ce2adacd4 in do_plan ccan/ccan/io/io.c:407
    #12 0x555ce2adad12 in io_ready ccan/ccan/io/io.c:417
    #13 0x555ce2adcd67 in io_loop ccan/ccan/io/poll.c:445
    ElementsProject#14 0x555ce2a67c66 in io_loop_with_timers lightningd/io_loop_with_timers.c:24
    ElementsProject#15 0x555ce2a6e56b in main lightningd/lightningd.c:822
    ElementsProject#16 0x7ff028242b6a in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x26b6a)

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
rustyrussell referenced this issue in rustyrussell/lightning Jun 14, 2019
Direct leak of 64 byte(s) in 1 object(s) allocated from:
    #0 0x7f4dc279163e in calloc (/lib/x86_64-linux-gnu/libasan.so.5+0x10c63e)
    #1 0x564ee8a24bb1 in htable_default_alloc ccan/ccan/htable/htable.c:19
    #2 0x564ee8a2551b in double_table ccan/ccan/htable/htable.c:226
    #3 0x564ee8a259e5 in htable_add_ ccan/ccan/htable/htable.c:331
    #4 0x564ee89a5300 in block_map_add lightningd/chaintopology.h:83
    #5 0x564ee89a6ece in add_tip lightningd/chaintopology.c:626
    #6 0x564ee89a72c3 in have_new_block lightningd/chaintopology.c:694
    #7 0x564ee89a3ab0 in process_rawblock lightningd/bitcoind.c:466
    #8 0x564ee89a2fb4 in bcli_finished lightningd/bitcoind.c:214
    #9 0x564ee8a284d6 in destroy_conn ccan/ccan/io/poll.c:244
    #10 0x564ee8a284f6 in destroy_conn_close_fd ccan/ccan/io/poll.c:250
    #11 0x564ee8a34a0d in notify ccan/ccan/tal/tal.c:235
    #12 0x564ee8a34efc in del_tree ccan/ccan/tal/tal.c:397
    #13 0x564ee8a35288 in tal_free ccan/ccan/tal/tal.c:481
    ElementsProject#14 0x564ee8a26cf5 in io_close ccan/ccan/io/io.c:450
    ElementsProject#15 0x564ee8a28c11 in io_loop ccan/ccan/io/poll.c:449
    ElementsProject#16 0x564ee89b3c3b in io_loop_with_timers lightningd/io_loop_with_timers.c:24
    ElementsProject#17 0x564ee89ba540 in main lightningd/lightningd.c:822
    ElementsProject#18 0x7f4dc2143b6a in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x26b6a)

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
rustyrussell referenced this issue in rustyrussell/lightning Jun 14, 2019
Direct leak of 32 byte(s) in 1 object(s) allocated from:
    #0 0x7f7678ee863e in calloc (/lib/x86_64-linux-gnu/libasan.so.5+0x10c63e)
    #1 0x55f8c7b0fce5 in htable_default_alloc ccan/ccan/htable/htable.c:19
    #2 0x55f8c7b1064f in double_table ccan/ccan/htable/htable.c:226
    #3 0x55f8c7b10b19 in htable_add_ ccan/ccan/htable/htable.c:331
    #4 0x55f8c7afac63 in scriptpubkeyset_add wallet/txfilter.c:30
    #5 0x55f8c7afafce in txfilter_add_scriptpubkey wallet/txfilter.c:77
    #6 0x55f8c7afb05f in txfilter_add_derkey wallet/txfilter.c:91
    #7 0x55f8c7aa4d67 in init_txfilter lightningd/lightningd.c:482
    #8 0x55f8c7aa52d8 in main lightningd/lightningd.c:721
    #9 0x7f767889ab6a in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x26b6a)

Direct leak of 16 byte(s) in 1 object(s) allocated from:
    #0 0x7f05f389563e in calloc (/lib/x86_64-linux-gnu/libasan.so.5+0x10c63e)
    #1 0x55cac1e6bc99 in htable_default_alloc ccan/ccan/htable/htable.c:19
    #2 0x55cac1e6c603 in double_table ccan/ccan/htable/htable.c:226
    #3 0x55cac1e6cacd in htable_add_ ccan/ccan/htable/htable.c:331
    #4 0x55cac1e56e48 in outpointset_add wallet/txfilter.c:61
    #5 0x55cac1e57162 in outpointfilter_add wallet/txfilter.c:116
    #6 0x55cac1e5ea3a in wallet_utxoset_add wallet/wallet.c:2365
    #7 0x55cac1deddc2 in topo_add_utxos lightningd/chaintopology.c:603
    #8 0x55cac1dedeac in add_tip lightningd/chaintopology.c:620
    #9 0x55cac1dee2de in have_new_block lightningd/chaintopology.c:694
    #10 0x55cac1deaab0 in process_rawblock lightningd/bitcoind.c:466
    #11 0x55cac1de9fb4 in bcli_finished lightningd/bitcoind.c:214
    #12 0x55cac1e6f5be in destroy_conn ccan/ccan/io/poll.c:244
    #13 0x55cac1e6f5de in destroy_conn_close_fd ccan/ccan/io/poll.c:250
    ElementsProject#14 0x55cac1e7baf5 in notify ccan/ccan/tal/tal.c:235
    ElementsProject#15 0x55cac1e7bfe4 in del_tree ccan/ccan/tal/tal.c:397
    ElementsProject#16 0x55cac1e7c370 in tal_free ccan/ccan/tal/tal.c:481
    ElementsProject#17 0x55cac1e6dddd in io_close ccan/ccan/io/io.c:450
    ElementsProject#18 0x55cac1e6fcf9 in io_loop ccan/ccan/io/poll.c:449
    ElementsProject#19 0x55cac1dfac66 in io_loop_with_timers lightningd/io_loop_with_timers.c:24
    ElementsProject#20 0x55cac1e0156b in main lightningd/lightningd.c:822
    ElementsProject#21 0x7f05f3247b6a in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x26b6a)

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
rustyrussell referenced this issue in rustyrussell/lightning Jun 14, 2019
Direct leak of 16 byte(s) in 1 object(s) allocated from:
    #0 0x7ff02889063e in calloc (/lib/x86_64-linux-gnu/libasan.so.5+0x10c63e)
    #1 0x555ce2ad8d2e in htable_default_alloc ccan/ccan/htable/htable.c:19
    #2 0x555ce2ad9698 in double_table ccan/ccan/htable/htable.c:226
    #3 0x555ce2ad9b62 in htable_add_ ccan/ccan/htable/htable.c:331
    #4 0x555ce2a638e4 in htlc_in_map_add lightningd/htlc_end.h:113
    #5 0x555ce2a63beb in connect_htlc_in lightningd/htlc_end.c:39
    #6 0x555ce2a85cbc in channel_added_their_htlc lightningd/peer_htlcs.c:1382
    #7 0x555ce2a860e1 in peer_got_commitsig lightningd/peer_htlcs.c:1466
    #8 0x555ce2a5db04 in channel_msg lightningd/channel_control.c:228
    #9 0x555ce2a8d393 in sd_msg_read lightningd/subd.c:474
    #10 0x555ce2ada157 in next_plan ccan/ccan/io/io.c:59
    #11 0x555ce2adacd4 in do_plan ccan/ccan/io/io.c:407
    #12 0x555ce2adad12 in io_ready ccan/ccan/io/io.c:417
    #13 0x555ce2adcd67 in io_loop ccan/ccan/io/poll.c:445
    ElementsProject#14 0x555ce2a67c66 in io_loop_with_timers lightningd/io_loop_with_timers.c:24
    ElementsProject#15 0x555ce2a6e56b in main lightningd/lightningd.c:822
    ElementsProject#16 0x7ff028242b6a in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x26b6a)

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
rustyrussell referenced this issue in rustyrussell/lightning Jun 14, 2019
Direct leak of 64 byte(s) in 1 object(s) allocated from:
    #0 0x7f4dc279163e in calloc (/lib/x86_64-linux-gnu/libasan.so.5+0x10c63e)
    #1 0x564ee8a24bb1 in htable_default_alloc ccan/ccan/htable/htable.c:19
    #2 0x564ee8a2551b in double_table ccan/ccan/htable/htable.c:226
    #3 0x564ee8a259e5 in htable_add_ ccan/ccan/htable/htable.c:331
    #4 0x564ee89a5300 in block_map_add lightningd/chaintopology.h:83
    #5 0x564ee89a6ece in add_tip lightningd/chaintopology.c:626
    #6 0x564ee89a72c3 in have_new_block lightningd/chaintopology.c:694
    #7 0x564ee89a3ab0 in process_rawblock lightningd/bitcoind.c:466
    #8 0x564ee89a2fb4 in bcli_finished lightningd/bitcoind.c:214
    #9 0x564ee8a284d6 in destroy_conn ccan/ccan/io/poll.c:244
    #10 0x564ee8a284f6 in destroy_conn_close_fd ccan/ccan/io/poll.c:250
    #11 0x564ee8a34a0d in notify ccan/ccan/tal/tal.c:235
    #12 0x564ee8a34efc in del_tree ccan/ccan/tal/tal.c:397
    #13 0x564ee8a35288 in tal_free ccan/ccan/tal/tal.c:481
    ElementsProject#14 0x564ee8a26cf5 in io_close ccan/ccan/io/io.c:450
    ElementsProject#15 0x564ee8a28c11 in io_loop ccan/ccan/io/poll.c:449
    ElementsProject#16 0x564ee89b3c3b in io_loop_with_timers lightningd/io_loop_with_timers.c:24
    ElementsProject#17 0x564ee89ba540 in main lightningd/lightningd.c:822
    ElementsProject#18 0x7f4dc2143b6a in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x26b6a)

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
rustyrussell referenced this issue in rustyrussell/lightning Jun 14, 2019
Direct leak of 32 byte(s) in 1 object(s) allocated from:
    #0 0x7f7678ee863e in calloc (/lib/x86_64-linux-gnu/libasan.so.5+0x10c63e)
    #1 0x55f8c7b0fce5 in htable_default_alloc ccan/ccan/htable/htable.c:19
    #2 0x55f8c7b1064f in double_table ccan/ccan/htable/htable.c:226
    #3 0x55f8c7b10b19 in htable_add_ ccan/ccan/htable/htable.c:331
    #4 0x55f8c7afac63 in scriptpubkeyset_add wallet/txfilter.c:30
    #5 0x55f8c7afafce in txfilter_add_scriptpubkey wallet/txfilter.c:77
    #6 0x55f8c7afb05f in txfilter_add_derkey wallet/txfilter.c:91
    #7 0x55f8c7aa4d67 in init_txfilter lightningd/lightningd.c:482
    #8 0x55f8c7aa52d8 in main lightningd/lightningd.c:721
    #9 0x7f767889ab6a in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x26b6a)

Direct leak of 16 byte(s) in 1 object(s) allocated from:
    #0 0x7f05f389563e in calloc (/lib/x86_64-linux-gnu/libasan.so.5+0x10c63e)
    #1 0x55cac1e6bc99 in htable_default_alloc ccan/ccan/htable/htable.c:19
    #2 0x55cac1e6c603 in double_table ccan/ccan/htable/htable.c:226
    #3 0x55cac1e6cacd in htable_add_ ccan/ccan/htable/htable.c:331
    #4 0x55cac1e56e48 in outpointset_add wallet/txfilter.c:61
    #5 0x55cac1e57162 in outpointfilter_add wallet/txfilter.c:116
    #6 0x55cac1e5ea3a in wallet_utxoset_add wallet/wallet.c:2365
    #7 0x55cac1deddc2 in topo_add_utxos lightningd/chaintopology.c:603
    #8 0x55cac1dedeac in add_tip lightningd/chaintopology.c:620
    #9 0x55cac1dee2de in have_new_block lightningd/chaintopology.c:694
    #10 0x55cac1deaab0 in process_rawblock lightningd/bitcoind.c:466
    #11 0x55cac1de9fb4 in bcli_finished lightningd/bitcoind.c:214
    #12 0x55cac1e6f5be in destroy_conn ccan/ccan/io/poll.c:244
    #13 0x55cac1e6f5de in destroy_conn_close_fd ccan/ccan/io/poll.c:250
    ElementsProject#14 0x55cac1e7baf5 in notify ccan/ccan/tal/tal.c:235
    ElementsProject#15 0x55cac1e7bfe4 in del_tree ccan/ccan/tal/tal.c:397
    ElementsProject#16 0x55cac1e7c370 in tal_free ccan/ccan/tal/tal.c:481
    ElementsProject#17 0x55cac1e6dddd in io_close ccan/ccan/io/io.c:450
    ElementsProject#18 0x55cac1e6fcf9 in io_loop ccan/ccan/io/poll.c:449
    ElementsProject#19 0x55cac1dfac66 in io_loop_with_timers lightningd/io_loop_with_timers.c:24
    ElementsProject#20 0x55cac1e0156b in main lightningd/lightningd.c:822
    ElementsProject#21 0x7f05f3247b6a in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x26b6a)

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
rustyrussell added a commit that referenced this issue Jun 30, 2019
Direct leak of 16 byte(s) in 1 object(s) allocated from:
    #0 0x7ff02889063e in calloc (/lib/x86_64-linux-gnu/libasan.so.5+0x10c63e)
    #1 0x555ce2ad8d2e in htable_default_alloc ccan/ccan/htable/htable.c:19
    #2 0x555ce2ad9698 in double_table ccan/ccan/htable/htable.c:226
    #3 0x555ce2ad9b62 in htable_add_ ccan/ccan/htable/htable.c:331
    #4 0x555ce2a638e4 in htlc_in_map_add lightningd/htlc_end.h:113
    #5 0x555ce2a63beb in connect_htlc_in lightningd/htlc_end.c:39
    #6 0x555ce2a85cbc in channel_added_their_htlc lightningd/peer_htlcs.c:1382
    #7 0x555ce2a860e1 in peer_got_commitsig lightningd/peer_htlcs.c:1466
    #8 0x555ce2a5db04 in channel_msg lightningd/channel_control.c:228
    #9 0x555ce2a8d393 in sd_msg_read lightningd/subd.c:474
    #10 0x555ce2ada157 in next_plan ccan/ccan/io/io.c:59
    #11 0x555ce2adacd4 in do_plan ccan/ccan/io/io.c:407
    #12 0x555ce2adad12 in io_ready ccan/ccan/io/io.c:417
    #13 0x555ce2adcd67 in io_loop ccan/ccan/io/poll.c:445
    #14 0x555ce2a67c66 in io_loop_with_timers lightningd/io_loop_with_timers.c:24
    #15 0x555ce2a6e56b in main lightningd/lightningd.c:822
    #16 0x7ff028242b6a in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x26b6a)

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
rustyrussell added a commit that referenced this issue Jun 30, 2019
Direct leak of 64 byte(s) in 1 object(s) allocated from:
    #0 0x7f4dc279163e in calloc (/lib/x86_64-linux-gnu/libasan.so.5+0x10c63e)
    #1 0x564ee8a24bb1 in htable_default_alloc ccan/ccan/htable/htable.c:19
    #2 0x564ee8a2551b in double_table ccan/ccan/htable/htable.c:226
    #3 0x564ee8a259e5 in htable_add_ ccan/ccan/htable/htable.c:331
    #4 0x564ee89a5300 in block_map_add lightningd/chaintopology.h:83
    #5 0x564ee89a6ece in add_tip lightningd/chaintopology.c:626
    #6 0x564ee89a72c3 in have_new_block lightningd/chaintopology.c:694
    #7 0x564ee89a3ab0 in process_rawblock lightningd/bitcoind.c:466
    #8 0x564ee89a2fb4 in bcli_finished lightningd/bitcoind.c:214
    #9 0x564ee8a284d6 in destroy_conn ccan/ccan/io/poll.c:244
    #10 0x564ee8a284f6 in destroy_conn_close_fd ccan/ccan/io/poll.c:250
    #11 0x564ee8a34a0d in notify ccan/ccan/tal/tal.c:235
    #12 0x564ee8a34efc in del_tree ccan/ccan/tal/tal.c:397
    #13 0x564ee8a35288 in tal_free ccan/ccan/tal/tal.c:481
    #14 0x564ee8a26cf5 in io_close ccan/ccan/io/io.c:450
    #15 0x564ee8a28c11 in io_loop ccan/ccan/io/poll.c:449
    #16 0x564ee89b3c3b in io_loop_with_timers lightningd/io_loop_with_timers.c:24
    #17 0x564ee89ba540 in main lightningd/lightningd.c:822
    #18 0x7f4dc2143b6a in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x26b6a)

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
rustyrussell added a commit that referenced this issue Jun 30, 2019
Direct leak of 32 byte(s) in 1 object(s) allocated from:
    #0 0x7f7678ee863e in calloc (/lib/x86_64-linux-gnu/libasan.so.5+0x10c63e)
    #1 0x55f8c7b0fce5 in htable_default_alloc ccan/ccan/htable/htable.c:19
    #2 0x55f8c7b1064f in double_table ccan/ccan/htable/htable.c:226
    #3 0x55f8c7b10b19 in htable_add_ ccan/ccan/htable/htable.c:331
    #4 0x55f8c7afac63 in scriptpubkeyset_add wallet/txfilter.c:30
    #5 0x55f8c7afafce in txfilter_add_scriptpubkey wallet/txfilter.c:77
    #6 0x55f8c7afb05f in txfilter_add_derkey wallet/txfilter.c:91
    #7 0x55f8c7aa4d67 in init_txfilter lightningd/lightningd.c:482
    #8 0x55f8c7aa52d8 in main lightningd/lightningd.c:721
    #9 0x7f767889ab6a in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x26b6a)

Direct leak of 16 byte(s) in 1 object(s) allocated from:
    #0 0x7f05f389563e in calloc (/lib/x86_64-linux-gnu/libasan.so.5+0x10c63e)
    #1 0x55cac1e6bc99 in htable_default_alloc ccan/ccan/htable/htable.c:19
    #2 0x55cac1e6c603 in double_table ccan/ccan/htable/htable.c:226
    #3 0x55cac1e6cacd in htable_add_ ccan/ccan/htable/htable.c:331
    #4 0x55cac1e56e48 in outpointset_add wallet/txfilter.c:61
    #5 0x55cac1e57162 in outpointfilter_add wallet/txfilter.c:116
    #6 0x55cac1e5ea3a in wallet_utxoset_add wallet/wallet.c:2365
    #7 0x55cac1deddc2 in topo_add_utxos lightningd/chaintopology.c:603
    #8 0x55cac1dedeac in add_tip lightningd/chaintopology.c:620
    #9 0x55cac1dee2de in have_new_block lightningd/chaintopology.c:694
    #10 0x55cac1deaab0 in process_rawblock lightningd/bitcoind.c:466
    #11 0x55cac1de9fb4 in bcli_finished lightningd/bitcoind.c:214
    #12 0x55cac1e6f5be in destroy_conn ccan/ccan/io/poll.c:244
    #13 0x55cac1e6f5de in destroy_conn_close_fd ccan/ccan/io/poll.c:250
    #14 0x55cac1e7baf5 in notify ccan/ccan/tal/tal.c:235
    #15 0x55cac1e7bfe4 in del_tree ccan/ccan/tal/tal.c:397
    #16 0x55cac1e7c370 in tal_free ccan/ccan/tal/tal.c:481
    #17 0x55cac1e6dddd in io_close ccan/ccan/io/io.c:450
    #18 0x55cac1e6fcf9 in io_loop ccan/ccan/io/poll.c:449
    #19 0x55cac1dfac66 in io_loop_with_timers lightningd/io_loop_with_timers.c:24
    #20 0x55cac1e0156b in main lightningd/lightningd.c:822
    #21 0x7f05f3247b6a in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x26b6a)

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
rustyrussell added a commit that referenced this issue Jun 30, 2019
Direct leak of 16 byte(s) in 1 object(s) allocated from:
    #0 0x7ff02889063e in calloc (/lib/x86_64-linux-gnu/libasan.so.5+0x10c63e)
    #1 0x555ce2ad8d2e in htable_default_alloc ccan/ccan/htable/htable.c:19
    #2 0x555ce2ad9698 in double_table ccan/ccan/htable/htable.c:226
    #3 0x555ce2ad9b62 in htable_add_ ccan/ccan/htable/htable.c:331
    #4 0x555ce2a638e4 in htlc_in_map_add lightningd/htlc_end.h:113
    #5 0x555ce2a63beb in connect_htlc_in lightningd/htlc_end.c:39
    #6 0x555ce2a85cbc in channel_added_their_htlc lightningd/peer_htlcs.c:1382
    #7 0x555ce2a860e1 in peer_got_commitsig lightningd/peer_htlcs.c:1466
    #8 0x555ce2a5db04 in channel_msg lightningd/channel_control.c:228
    #9 0x555ce2a8d393 in sd_msg_read lightningd/subd.c:474
    #10 0x555ce2ada157 in next_plan ccan/ccan/io/io.c:59
    #11 0x555ce2adacd4 in do_plan ccan/ccan/io/io.c:407
    #12 0x555ce2adad12 in io_ready ccan/ccan/io/io.c:417
    #13 0x555ce2adcd67 in io_loop ccan/ccan/io/poll.c:445
    #14 0x555ce2a67c66 in io_loop_with_timers lightningd/io_loop_with_timers.c:24
    #15 0x555ce2a6e56b in main lightningd/lightningd.c:822
    #16 0x7ff028242b6a in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x26b6a)

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
ulrichard pushed a commit to ulrichard/lightning that referenced this issue May 20, 2021
Changed "warmstart" to "coldstart" (inverted semantic).

Modified all satoshi variable names to have "_sat" suffixes.

Removed deprecated Phase 1 fields.

Added push_value to hsm_ready_channel.  Added sanity checking to hsm_ready_channel stub.

Fixed issues w/ rebase onto upstream/master.

Replaced tx->input_amounts and tx->output_witscripts with tx->psbt.

Fixed redeemscript confusion. (ElementsProject#11)

Fix remote-hsmd CI (ElementsProject#12)

* adding g++, protobuf, grpc++
* add libgrpc++-dev to .travis.yml
* wget remotesigner.proto
* added libprotobuf-dev
* fixed compilation warnings/errors
* use Get instead of [] in protobuf seq
* fixed include sort order
* added protobuf generated files to skip nogen checks
* fixed gensrc pattern
* added libstdc++6
* fixed python spacing
* added specified inclusion guards
* renamed c++ header extensions to hpp to avoid c checks
* guard satoshi amount calculations
* libstdc++-6-dev
* added `ln -s /usr/lib/x86_64-linux-gnu/libstdc++.so.6 /usr/lib/x86_64-linux-gnu/libstdc++.so`

Enable handle_sign_withdrawal_tx signing of PSBT. (ElementsProject#13)

* Converted debug logging to json format.
* Switched status_debug to STATUS_DEBUG macro.

Skip two integration tests which mutate channel_nonce.

fixed nit w/ run-one-test

Added NodeConfig::NATIVE to InitReq. (ElementsProject#14)

Converted remotesigner.PubKey from 64 byte raw to 33 byte compressed DER. (ElementsProject#15)

Updated to channel_nonce0. (ElementsProject#17)

Replaced option_static_remotekey w/ CommitmentType. (ElementsProject#18)

* Replaced option_static_remotekey w/ CommitmentType.

* Improved NOTES

Upgraded SignFundingTxRequest KeyIndex to KeyPath. (ElementsProject#19)

Adapted to CloseInfo moving into KeyLocator

TxOut was replaced in API w/ it's only field.

Adapted to remotesigner api-doc mods.

Set input to 0 explicitly.

SignCounterpartyHtlcToUs -> SignCounterpartyHtlcSweep SignJusticeTxToUs -> SignJusticeSweep SignDelayedPaymentToUs -> SignDelayedSweep (ElementsProject#20)
dongcarl added a commit to dongcarl/lightning that referenced this issue Nov 7, 2022
The existing description is incorrect. `createinvoice` doesn't actually
work when supplied with a custom-encoded bolt11 invoice without the
final 520 signature bits appended. If a users tries to do so, some of
their tagged fields will be incorrectly truncated.

`createinvoice` actually expects that the signatures are there, and it
simply ignores them.

See common/bolt11.c's bolt11_decode_nosig:

         /* BOLT ElementsProject#11:
          *
          * The data part of a Lightning invoice consists of multiple sections:
          *
          * 1. `timestamp`: seconds-since-1970 (35 bits, big-endian)
          * 1. zero or more tagged parts
          * 1. `signature`: Bitcoin-style signature of above (520 bits)
          */
         if (!pull_uint(&hu5, &data, &data_len, &b11->timestamp, 35))
                 return decode_fail(b11, fail, "Can't get 35-bit timestamp");

>        while (data_len > 520 / 5) {
                 const char *problem = NULL;
                 u64 type, data_length;
rustyrussell pushed a commit to dongcarl/lightning that referenced this issue Feb 6, 2023
The existing description is incorrect. `createinvoice` doesn't actually
work when supplied with a custom-encoded bolt11 invoice without the
final 520 signature bits appended. If a users tries to do so, some of
their tagged fields will be incorrectly truncated.

`createinvoice` actually expects that the signatures are there, and it
simply ignores them.

See common/bolt11.c's bolt11_decode_nosig:

         /* BOLT ElementsProject#11:
          *
          * The data part of a Lightning invoice consists of multiple sections:
          *
          * 1. `timestamp`: seconds-since-1970 (35 bits, big-endian)
          * 1. zero or more tagged parts
          * 1. `signature`: Bitcoin-style signature of above (520 bits)
          */
         if (!pull_uint(&hu5, &data, &data_len, &b11->timestamp, 35))
                 return decode_fail(b11, fail, "Can't get 35-bit timestamp");

>        while (data_len > 520 / 5) {
                 const char *problem = NULL;
                 u64 type, data_length;
dongcarl added a commit to dongcarl/lightning that referenced this issue Feb 6, 2023
The existing description is incorrect. `createinvoice` doesn't actually
work when supplied with a custom-encoded bolt11 invoice without the
final 520 signature bits appended. If a users tries to do so, some of
their tagged fields will be incorrectly truncated.

`createinvoice` actually expects that the signatures are there, and it
simply ignores them.

See common/bolt11.c's bolt11_decode_nosig:

         /* BOLT ElementsProject#11:
          *
          * The data part of a Lightning invoice consists of multiple sections:
          *
          * 1. `timestamp`: seconds-since-1970 (35 bits, big-endian)
          * 1. zero or more tagged parts
          * 1. `signature`: Bitcoin-style signature of above (520 bits)
          */
         if (!pull_uint(&hu5, &data, &data_len, &b11->timestamp, 35))
                 return decode_fail(b11, fail, "Can't get 35-bit timestamp");

>        while (data_len > 520 / 5) {
                 const char *problem = NULL;
                 u64 type, data_length;
endothermicdev pushed a commit that referenced this issue Feb 6, 2023
The existing description is incorrect. `createinvoice` doesn't actually
work when supplied with a custom-encoded bolt11 invoice without the
final 520 signature bits appended. If a users tries to do so, some of
their tagged fields will be incorrectly truncated.

`createinvoice` actually expects that the signatures are there, and it
simply ignores them.

See common/bolt11.c's bolt11_decode_nosig:

         /* BOLT #11:
          *
          * The data part of a Lightning invoice consists of multiple sections:
          *
          * 1. `timestamp`: seconds-since-1970 (35 bits, big-endian)
          * 1. zero or more tagged parts
          * 1. `signature`: Bitcoin-style signature of above (520 bits)
          */
         if (!pull_uint(&hu5, &data, &data_len, &b11->timestamp, 35))
                 return decode_fail(b11, fail, "Can't get 35-bit timestamp");

>        while (data_len > 520 / 5) {
                 const char *problem = NULL;
                 u64 type, data_length;
vincenzopalazzo added a commit to vincenzopalazzo/lightning that referenced this issue Feb 13, 2023
This will fix a crash that I caused on armv7
and by looking inside the coredump with gdb
(by adding an assert on n that must be
different from null) I get the following stacktrace

```
(gdb) bt
\#0  0x00000000 in ?? ()
\#1  0x0043a038 in send_backtrace (why=0xbe9e3600 "FATAL SIGNAL 11") at common/daemon.c:36
\#2  0x0043a0ec in crashdump (sig=11) at common/daemon.c:46
\#3  <signal handler called>
\#4  0x00406d04 in node_announcement (map=0x938ecc, nann_off=495146) at common/gossmap.c:586
\#5  0x00406fec in map_catchup (map=0x938ecc, num_rejected=0xbe9e3a40) at common/gossmap.c:643
\#6  0x004073a4 in load_gossip_store (map=0x938ecc, num_rejected=0xbe9e3a40) at common/gossmap.c:697
\ElementsProject#7  0x00408244 in gossmap_load (ctx=0x0, filename=0x4e16b8 "gossip_store", num_channel_updates_rejected=0xbe9e3a40) at common/gossmap.c:976
\ElementsProject#8  0x0041a548 in init (p=0x93831c, buf=0x9399d4 "\n\n{\"jsonrpc\":\"2.0\",\"id\":\"cln:init#25\",\"method\":\"init\",\"params\":{\"options\":{},\"configuration\":{\"lightning-dir\":\"/home/vincent/.lightning/testnet\",\"rpc-file\":\"lightning-rpc\",\"startup\":true,\"network\":\"te"..., config=0x939cdc) at plugins/topology.c:622
\ElementsProject#9  0x0041e5d0 in handle_init (cmd=0x938934, buf=0x9399d4 "\n\n{\"jsonrpc\":\"2.0\",\"id\":\"cln:init#25\",\"method\":\"init\",\"params\":{\"options\":{},\"configuration\":{\"lightning-dir\":\"/home/vincent/.lightning/testnet\",\"rpc-file\":\"lightning-rpc\",\"startup\":true,\"network\":\"te"..., params=0x939c8c)
    at plugins/libplugin.c:1208
\ElementsProject#10 0x0041fc04 in ld_command_handle (plugin=0x93831c, toks=0x939bec) at plugins/libplugin.c:1572
\ElementsProject#11 0x00420050 in ld_read_json_one (plugin=0x93831c) at plugins/libplugin.c:1667
\ElementsProject#12 0x004201bc in ld_read_json (conn=0x9391c4, plugin=0x93831c) at plugins/libplugin.c:1687
\ElementsProject#13 0x004cb82c in next_plan (conn=0x9391c4, plan=0x9391d8) at ccan/ccan/io/io.c:59
\ElementsProject#14 0x004cc67c in do_plan (conn=0x9391c4, plan=0x9391d8, idle_on_epipe=false) at ccan/ccan/io/io.c:407
\ElementsProject#15 0x004cc6dc in io_ready (conn=0x9391c4, pollflags=1) at ccan/ccan/io/io.c:417
\ElementsProject#16 0x004cf8cc in io_loop (timers=0x9383c4, expired=0xbe9e3ce4) at ccan/ccan/io/poll.c:453
\ElementsProject#17 0x00420af4 in plugin_main (argv=0xbe9e3eb4, init=0x41a46c <init>, restartability=PLUGIN_STATIC, init_rpc=true, features=0x0, commands=0x6167e8 <commands>, num_commands=4, notif_subs=0x0, num_notif_subs=0, hook_subs=0x0, num_hook_subs=0, notif_topics=0x0, num_notif_topics=0) at plugins/libplugin.c:1891
\ElementsProject#18 0x0041a6f8 in main (argc=1, argv=0xbe9e3eb4) at plugins/topology.c:679
```

I do not know if this is a solution because I do not know
when I can parse a node announcement for a node that
it is not longer in the gossip map.

So, I hope this is just usefult for @rustyrussell

Changelog-Fixes: fixes `FATAL SIGNAL 11` on gossmap node announcement parsing.

Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
vincenzopalazzo added a commit to vincenzopalazzo/lightning that referenced this issue Feb 13, 2023
This will fix a crash that I caused on armv7
and by looking inside the coredump with gdb
(by adding an assert on n that must be
different from null) I get the following stacktrace

```
(gdb) bt
\#0  0x00000000 in ?? ()
\#1  0x0043a038 in send_backtrace (why=0xbe9e3600 "FATAL SIGNAL 11") at common/daemon.c:36
\#2  0x0043a0ec in crashdump (sig=11) at common/daemon.c:46
\#3  <signal handler called>
\#4  0x00406d04 in node_announcement (map=0x938ecc, nann_off=495146) at common/gossmap.c:586
\#5  0x00406fec in map_catchup (map=0x938ecc, num_rejected=0xbe9e3a40) at common/gossmap.c:643
\#6  0x004073a4 in load_gossip_store (map=0x938ecc, num_rejected=0xbe9e3a40) at common/gossmap.c:697
\ElementsProject#7  0x00408244 in gossmap_load (ctx=0x0, filename=0x4e16b8 "gossip_store", num_channel_updates_rejected=0xbe9e3a40) at common/gossmap.c:976
\ElementsProject#8  0x0041a548 in init (p=0x93831c, buf=0x9399d4 "\n\n{\"jsonrpc\":\"2.0\",\"id\":\"cln:init#25\",\"method\":\"init\",\"params\":{\"options\":{},\"configuration\":{\"lightning-dir\":\"/home/vincent/.lightning/testnet\",\"rpc-file\":\"lightning-rpc\",\"startup\":true,\"network\":\"te"..., config=0x939cdc) at plugins/topology.c:622
\ElementsProject#9  0x0041e5d0 in handle_init (cmd=0x938934, buf=0x9399d4 "\n\n{\"jsonrpc\":\"2.0\",\"id\":\"cln:init#25\",\"method\":\"init\",\"params\":{\"options\":{},\"configuration\":{\"lightning-dir\":\"/home/vincent/.lightning/testnet\",\"rpc-file\":\"lightning-rpc\",\"startup\":true,\"network\":\"te"..., params=0x939c8c)
    at plugins/libplugin.c:1208
\ElementsProject#10 0x0041fc04 in ld_command_handle (plugin=0x93831c, toks=0x939bec) at plugins/libplugin.c:1572
\ElementsProject#11 0x00420050 in ld_read_json_one (plugin=0x93831c) at plugins/libplugin.c:1667
\ElementsProject#12 0x004201bc in ld_read_json (conn=0x9391c4, plugin=0x93831c) at plugins/libplugin.c:1687
\ElementsProject#13 0x004cb82c in next_plan (conn=0x9391c4, plan=0x9391d8) at ccan/ccan/io/io.c:59
\ElementsProject#14 0x004cc67c in do_plan (conn=0x9391c4, plan=0x9391d8, idle_on_epipe=false) at ccan/ccan/io/io.c:407
\ElementsProject#15 0x004cc6dc in io_ready (conn=0x9391c4, pollflags=1) at ccan/ccan/io/io.c:417
\ElementsProject#16 0x004cf8cc in io_loop (timers=0x9383c4, expired=0xbe9e3ce4) at ccan/ccan/io/poll.c:453
\ElementsProject#17 0x00420af4 in plugin_main (argv=0xbe9e3eb4, init=0x41a46c <init>, restartability=PLUGIN_STATIC, init_rpc=true, features=0x0, commands=0x6167e8 <commands>, num_commands=4, notif_subs=0x0, num_notif_subs=0, hook_subs=0x0, num_hook_subs=0, notif_topics=0x0, num_notif_topics=0) at plugins/libplugin.c:1891
\ElementsProject#18 0x0041a6f8 in main (argc=1, argv=0xbe9e3eb4) at plugins/topology.c:679
```

I do not know if this is a solution because I do not know
when I can parse a node announcement for a node that
it is not longer in the gossip map.

So, I hope this is just usefult for @rustyrussell

Changelog-Fix: fixes `FATAL SIGNAL 11` on gossmap node announcement parsing.

Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
vincenzopalazzo added a commit to vincenzopalazzo/lightning that referenced this issue Feb 13, 2023
This will fix a crash that I caused on armv7
and by looking inside the coredump with gdb
(by adding an assert on n that must be
different from null) I get the following stacktrace

```
(gdb) bt
\#0  0x00000000 in ?? ()
\#1  0x0043a038 in send_backtrace (why=0xbe9e3600 "FATAL SIGNAL 11") at common/daemon.c:36
\#2  0x0043a0ec in crashdump (sig=11) at common/daemon.c:46
\#3  <signal handler called>
\#4  0x00406d04 in node_announcement (map=0x938ecc, nann_off=495146) at common/gossmap.c:586
\#5  0x00406fec in map_catchup (map=0x938ecc, num_rejected=0xbe9e3a40) at common/gossmap.c:643
\#6  0x004073a4 in load_gossip_store (map=0x938ecc, num_rejected=0xbe9e3a40) at common/gossmap.c:697
\ElementsProject#7  0x00408244 in gossmap_load (ctx=0x0, filename=0x4e16b8 "gossip_store", num_channel_updates_rejected=0xbe9e3a40) at common/gossmap.c:976
\ElementsProject#8  0x0041a548 in init (p=0x93831c, buf=0x9399d4 "\n\n{\"jsonrpc\":\"2.0\",\"id\":\"cln:init#25\",\"method\":\"init\",\"params\":{\"options\":{},\"configuration\":{\"lightning-dir\":\"/home/vincent/.lightning/testnet\",\"rpc-file\":\"lightning-rpc\",\"startup\":true,\"network\":\"te"..., config=0x939cdc) at plugins/topology.c:622
\ElementsProject#9  0x0041e5d0 in handle_init (cmd=0x938934, buf=0x9399d4 "\n\n{\"jsonrpc\":\"2.0\",\"id\":\"cln:init#25\",\"method\":\"init\",\"params\":{\"options\":{},\"configuration\":{\"lightning-dir\":\"/home/vincent/.lightning/testnet\",\"rpc-file\":\"lightning-rpc\",\"startup\":true,\"network\":\"te"..., params=0x939c8c)
    at plugins/libplugin.c:1208
\ElementsProject#10 0x0041fc04 in ld_command_handle (plugin=0x93831c, toks=0x939bec) at plugins/libplugin.c:1572
\ElementsProject#11 0x00420050 in ld_read_json_one (plugin=0x93831c) at plugins/libplugin.c:1667
\ElementsProject#12 0x004201bc in ld_read_json (conn=0x9391c4, plugin=0x93831c) at plugins/libplugin.c:1687
\ElementsProject#13 0x004cb82c in next_plan (conn=0x9391c4, plan=0x9391d8) at ccan/ccan/io/io.c:59
\ElementsProject#14 0x004cc67c in do_plan (conn=0x9391c4, plan=0x9391d8, idle_on_epipe=false) at ccan/ccan/io/io.c:407
\ElementsProject#15 0x004cc6dc in io_ready (conn=0x9391c4, pollflags=1) at ccan/ccan/io/io.c:417
\ElementsProject#16 0x004cf8cc in io_loop (timers=0x9383c4, expired=0xbe9e3ce4) at ccan/ccan/io/poll.c:453
\ElementsProject#17 0x00420af4 in plugin_main (argv=0xbe9e3eb4, init=0x41a46c <init>, restartability=PLUGIN_STATIC, init_rpc=true, features=0x0, commands=0x6167e8 <commands>, num_commands=4, notif_subs=0x0, num_notif_subs=0, hook_subs=0x0, num_hook_subs=0, notif_topics=0x0, num_notif_topics=0) at plugins/libplugin.c:1891
\ElementsProject#18 0x0041a6f8 in main (argc=1, argv=0xbe9e3eb4) at plugins/topology.c:679
```

I do not know if this is a solution because I do not know
when I can parse a node announcement for a node that
it is not longer in the gossip map.

So, I hope this is just usefult for @rustyrussell

Changelog-Fixed: fixes `FATAL SIGNAL 11` on gossmap node announcement parsing.

Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
endothermicdev pushed a commit that referenced this issue Feb 13, 2023
This will fix a crash that I caused on armv7
and by looking inside the coredump with gdb
(by adding an assert on n that must be
different from null) I get the following stacktrace

```
(gdb) bt
\#0  0x00000000 in ?? ()
\#1  0x0043a038 in send_backtrace (why=0xbe9e3600 "FATAL SIGNAL 11") at common/daemon.c:36
\#2  0x0043a0ec in crashdump (sig=11) at common/daemon.c:46
\#3  <signal handler called>
\#4  0x00406d04 in node_announcement (map=0x938ecc, nann_off=495146) at common/gossmap.c:586
\#5  0x00406fec in map_catchup (map=0x938ecc, num_rejected=0xbe9e3a40) at common/gossmap.c:643
\#6  0x004073a4 in load_gossip_store (map=0x938ecc, num_rejected=0xbe9e3a40) at common/gossmap.c:697
\#7  0x00408244 in gossmap_load (ctx=0x0, filename=0x4e16b8 "gossip_store", num_channel_updates_rejected=0xbe9e3a40) at common/gossmap.c:976
\#8  0x0041a548 in init (p=0x93831c, buf=0x9399d4 "\n\n{\"jsonrpc\":\"2.0\",\"id\":\"cln:init#25\",\"method\":\"init\",\"params\":{\"options\":{},\"configuration\":{\"lightning-dir\":\"/home/vincent/.lightning/testnet\",\"rpc-file\":\"lightning-rpc\",\"startup\":true,\"network\":\"te"..., config=0x939cdc) at plugins/topology.c:622
\#9  0x0041e5d0 in handle_init (cmd=0x938934, buf=0x9399d4 "\n\n{\"jsonrpc\":\"2.0\",\"id\":\"cln:init#25\",\"method\":\"init\",\"params\":{\"options\":{},\"configuration\":{\"lightning-dir\":\"/home/vincent/.lightning/testnet\",\"rpc-file\":\"lightning-rpc\",\"startup\":true,\"network\":\"te"..., params=0x939c8c)
    at plugins/libplugin.c:1208
\#10 0x0041fc04 in ld_command_handle (plugin=0x93831c, toks=0x939bec) at plugins/libplugin.c:1572
\#11 0x00420050 in ld_read_json_one (plugin=0x93831c) at plugins/libplugin.c:1667
\#12 0x004201bc in ld_read_json (conn=0x9391c4, plugin=0x93831c) at plugins/libplugin.c:1687
\#13 0x004cb82c in next_plan (conn=0x9391c4, plan=0x9391d8) at ccan/ccan/io/io.c:59
\#14 0x004cc67c in do_plan (conn=0x9391c4, plan=0x9391d8, idle_on_epipe=false) at ccan/ccan/io/io.c:407
\#15 0x004cc6dc in io_ready (conn=0x9391c4, pollflags=1) at ccan/ccan/io/io.c:417
\#16 0x004cf8cc in io_loop (timers=0x9383c4, expired=0xbe9e3ce4) at ccan/ccan/io/poll.c:453
\#17 0x00420af4 in plugin_main (argv=0xbe9e3eb4, init=0x41a46c <init>, restartability=PLUGIN_STATIC, init_rpc=true, features=0x0, commands=0x6167e8 <commands>, num_commands=4, notif_subs=0x0, num_notif_subs=0, hook_subs=0x0, num_hook_subs=0, notif_topics=0x0, num_notif_topics=0) at plugins/libplugin.c:1891
\#18 0x0041a6f8 in main (argc=1, argv=0xbe9e3eb4) at plugins/topology.c:679
```

I do not know if this is a solution because I do not know
when I can parse a node announcement for a node that
it is not longer in the gossip map.

So, I hope this is just usefult for @rustyrussell

Changelog-Fixed: fixes `FATAL SIGNAL 11` on gossmap node announcement parsing.

Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
rustyrussell added a commit that referenced this issue Mar 14, 2023
It's still deprecated: we need the description since

1. This information is useful for any validation we want to do, such as
   the HSM, or runes.
2. We want this information in listpays so we can tell what we actually paid.
3. In general, we should never sign commitments to things we don't have!

I expect to have this information about payments *whatever the frontend* is,
which is why we deprecated (and then removed) this unintended use.  The spec
is pretty clear on this:

BOLT #11:
```
A reader:
...
  - MUST check that the SHA2 256-bit hash in the `h` field exactly matches the hashed
  description.
```

However, neither BTCPayServer nor lnbits updated despite the long deprecation
period, so revert 2afe7a1.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
vincenzopalazzo pushed a commit to vincenzopalazzo/lightning that referenced this issue Mar 23, 2023
It's still deprecated: we need the description since

1. This information is useful for any validation we want to do, such as
   the HSM, or runes.
2. We want this information in listpays so we can tell what we actually paid.
3. In general, we should never sign commitments to things we don't have!

I expect to have this information about payments *whatever the frontend* is,
which is why we deprecated (and then removed) this unintended use.  The spec
is pretty clear on this:

BOLT ElementsProject#11:
```
A reader:
...
  - MUST check that the SHA2 256-bit hash in the `h` field exactly matches the hashed
  description.
```

However, neither BTCPayServer nor lnbits updated despite the long deprecation
period, so revert 2afe7a1.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
morehouse added a commit to morehouse/lightning that referenced this issue Mar 24, 2023
pubkey_from_hexstr() was failing, which we didn't notice because we
weren't checking the return value. The problem was that we were passing
it a strlen that was half the actual length.

Relevant error:

  [libsecp256k1] illegal argument: !secp256k1_fe_is_zero(&ge->x)

  ==417723== ERROR: libFuzzer: deadly signal
    ElementsProject#7 0x7f5deaacc7fb in abort
    ElementsProject#8 0x51b0b0 in secp256k1_default_illegal_callback_fn secp256k1.c
    ElementsProject#9 0x51bd8e in secp256k1_ec_pubkey_serialize
    ElementsProject#10 0x4e235b in pubkey_to_der bitcoin/pubkey.c:29:7
    ElementsProject#11 0x4e2941 in pubkey_cmp bitcoin/pubkey.c:89:2
    ElementsProject#12 0x4e333d in bitcoin_redeem_2of2 bitcoin/script.c:144:6
    ElementsProject#13 0x4f1396 in run tests/fuzz/fuzz-close_tx.c:78:19
rustyrussell pushed a commit that referenced this issue Apr 3, 2023
pubkey_from_hexstr() was failing, which we didn't notice because we
weren't checking the return value. The problem was that we were passing
it a strlen that was half the actual length.

Relevant error:

  [libsecp256k1] illegal argument: !secp256k1_fe_is_zero(&ge->x)

  ==417723== ERROR: libFuzzer: deadly signal
    #7 0x7f5deaacc7fb in abort
    #8 0x51b0b0 in secp256k1_default_illegal_callback_fn secp256k1.c
    #9 0x51bd8e in secp256k1_ec_pubkey_serialize
    #10 0x4e235b in pubkey_to_der bitcoin/pubkey.c:29:7
    #11 0x4e2941 in pubkey_cmp bitcoin/pubkey.c:89:2
    #12 0x4e333d in bitcoin_redeem_2of2 bitcoin/script.c:144:6
    #13 0x4f1396 in run tests/fuzz/fuzz-close_tx.c:78:19
ddustin pushed a commit to ddustin/lightning that referenced this issue Apr 11, 2023
The existing description is incorrect. `createinvoice` doesn't actually
work when supplied with a custom-encoded bolt11 invoice without the
final 520 signature bits appended. If a users tries to do so, some of
their tagged fields will be incorrectly truncated.

`createinvoice` actually expects that the signatures are there, and it
simply ignores them.

See common/bolt11.c's bolt11_decode_nosig:

         /* BOLT ElementsProject#11:
          *
          * The data part of a Lightning invoice consists of multiple sections:
          *
          * 1. `timestamp`: seconds-since-1970 (35 bits, big-endian)
          * 1. zero or more tagged parts
          * 1. `signature`: Bitcoin-style signature of above (520 bits)
          */
         if (!pull_uint(&hu5, &data, &data_len, &b11->timestamp, 35))
                 return decode_fail(b11, fail, "Can't get 35-bit timestamp");

>        while (data_len > 520 / 5) {
                 const char *problem = NULL;
                 u64 type, data_length;
ddustin pushed a commit to ddustin/lightning that referenced this issue Apr 11, 2023
This will fix a crash that I caused on armv7
and by looking inside the coredump with gdb
(by adding an assert on n that must be
different from null) I get the following stacktrace

```
(gdb) bt
\#0  0x00000000 in ?? ()
\#1  0x0043a038 in send_backtrace (why=0xbe9e3600 "FATAL SIGNAL 11") at common/daemon.c:36
\#2  0x0043a0ec in crashdump (sig=11) at common/daemon.c:46
\#3  <signal handler called>
\#4  0x00406d04 in node_announcement (map=0x938ecc, nann_off=495146) at common/gossmap.c:586
\#5  0x00406fec in map_catchup (map=0x938ecc, num_rejected=0xbe9e3a40) at common/gossmap.c:643
\ElementsProject#6  0x004073a4 in load_gossip_store (map=0x938ecc, num_rejected=0xbe9e3a40) at common/gossmap.c:697
\ElementsProject#7  0x00408244 in gossmap_load (ctx=0x0, filename=0x4e16b8 "gossip_store", num_channel_updates_rejected=0xbe9e3a40) at common/gossmap.c:976
\ElementsProject#8  0x0041a548 in init (p=0x93831c, buf=0x9399d4 "\n\n{\"jsonrpc\":\"2.0\",\"id\":\"cln:init#25\",\"method\":\"init\",\"params\":{\"options\":{},\"configuration\":{\"lightning-dir\":\"/home/vincent/.lightning/testnet\",\"rpc-file\":\"lightning-rpc\",\"startup\":true,\"network\":\"te"..., config=0x939cdc) at plugins/topology.c:622
\ElementsProject#9  0x0041e5d0 in handle_init (cmd=0x938934, buf=0x9399d4 "\n\n{\"jsonrpc\":\"2.0\",\"id\":\"cln:init#25\",\"method\":\"init\",\"params\":{\"options\":{},\"configuration\":{\"lightning-dir\":\"/home/vincent/.lightning/testnet\",\"rpc-file\":\"lightning-rpc\",\"startup\":true,\"network\":\"te"..., params=0x939c8c)
    at plugins/libplugin.c:1208
\ElementsProject#10 0x0041fc04 in ld_command_handle (plugin=0x93831c, toks=0x939bec) at plugins/libplugin.c:1572
\ElementsProject#11 0x00420050 in ld_read_json_one (plugin=0x93831c) at plugins/libplugin.c:1667
\ElementsProject#12 0x004201bc in ld_read_json (conn=0x9391c4, plugin=0x93831c) at plugins/libplugin.c:1687
\ElementsProject#13 0x004cb82c in next_plan (conn=0x9391c4, plan=0x9391d8) at ccan/ccan/io/io.c:59
\ElementsProject#14 0x004cc67c in do_plan (conn=0x9391c4, plan=0x9391d8, idle_on_epipe=false) at ccan/ccan/io/io.c:407
\ElementsProject#15 0x004cc6dc in io_ready (conn=0x9391c4, pollflags=1) at ccan/ccan/io/io.c:417
\ElementsProject#16 0x004cf8cc in io_loop (timers=0x9383c4, expired=0xbe9e3ce4) at ccan/ccan/io/poll.c:453
\ElementsProject#17 0x00420af4 in plugin_main (argv=0xbe9e3eb4, init=0x41a46c <init>, restartability=PLUGIN_STATIC, init_rpc=true, features=0x0, commands=0x6167e8 <commands>, num_commands=4, notif_subs=0x0, num_notif_subs=0, hook_subs=0x0, num_hook_subs=0, notif_topics=0x0, num_notif_topics=0) at plugins/libplugin.c:1891
\ElementsProject#18 0x0041a6f8 in main (argc=1, argv=0xbe9e3eb4) at plugins/topology.c:679
```

I do not know if this is a solution because I do not know
when I can parse a node announcement for a node that
it is not longer in the gossip map.

So, I hope this is just usefult for @rustyrussell

Changelog-Fixed: fixes `FATAL SIGNAL 11` on gossmap node announcement parsing.

Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
ddustin pushed a commit to ddustin/lightning that referenced this issue Apr 11, 2023
It's still deprecated: we need the description since

1. This information is useful for any validation we want to do, such as
   the HSM, or runes.
2. We want this information in listpays so we can tell what we actually paid.
3. In general, we should never sign commitments to things we don't have!

I expect to have this information about payments *whatever the frontend* is,
which is why we deprecated (and then removed) this unintended use.  The spec
is pretty clear on this:

BOLT ElementsProject#11:
```
A reader:
...
  - MUST check that the SHA2 256-bit hash in the `h` field exactly matches the hashed
  description.
```

However, neither BTCPayServer nor lnbits updated despite the long deprecation
period, so revert 2afe7a1.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
ddustin pushed a commit to ddustin/lightning that referenced this issue Apr 11, 2023
pubkey_from_hexstr() was failing, which we didn't notice because we
weren't checking the return value. The problem was that we were passing
it a strlen that was half the actual length.

Relevant error:

  [libsecp256k1] illegal argument: !secp256k1_fe_is_zero(&ge->x)

  ==417723== ERROR: libFuzzer: deadly signal
    ElementsProject#7 0x7f5deaacc7fb in abort
    ElementsProject#8 0x51b0b0 in secp256k1_default_illegal_callback_fn secp256k1.c
    ElementsProject#9 0x51bd8e in secp256k1_ec_pubkey_serialize
    ElementsProject#10 0x4e235b in pubkey_to_der bitcoin/pubkey.c:29:7
    ElementsProject#11 0x4e2941 in pubkey_cmp bitcoin/pubkey.c:89:2
    ElementsProject#12 0x4e333d in bitcoin_redeem_2of2 bitcoin/script.c:144:6
    ElementsProject#13 0x4f1396 in run tests/fuzz/fuzz-close_tx.c:78:19
gkrizek pushed a commit to voltagecloud/lightning that referenced this issue Apr 26, 2023
The existing description is incorrect. `createinvoice` doesn't actually
work when supplied with a custom-encoded bolt11 invoice without the
final 520 signature bits appended. If a users tries to do so, some of
their tagged fields will be incorrectly truncated.

`createinvoice` actually expects that the signatures are there, and it
simply ignores them.

See common/bolt11.c's bolt11_decode_nosig:

         /* BOLT ElementsProject#11:
          *
          * The data part of a Lightning invoice consists of multiple sections:
          *
          * 1. `timestamp`: seconds-since-1970 (35 bits, big-endian)
          * 1. zero or more tagged parts
          * 1. `signature`: Bitcoin-style signature of above (520 bits)
          */
         if (!pull_uint(&hu5, &data, &data_len, &b11->timestamp, 35))
                 return decode_fail(b11, fail, "Can't get 35-bit timestamp");

>        while (data_len > 520 / 5) {
                 const char *problem = NULL;
                 u64 type, data_length;
gkrizek pushed a commit to voltagecloud/lightning that referenced this issue Apr 26, 2023
This will fix a crash that I caused on armv7
and by looking inside the coredump with gdb
(by adding an assert on n that must be
different from null) I get the following stacktrace

```
(gdb) bt
\#0  0x00000000 in ?? ()
\#1  0x0043a038 in send_backtrace (why=0xbe9e3600 "FATAL SIGNAL 11") at common/daemon.c:36
\#2  0x0043a0ec in crashdump (sig=11) at common/daemon.c:46
\#3  <signal handler called>
\ElementsProject#4  0x00406d04 in node_announcement (map=0x938ecc, nann_off=495146) at common/gossmap.c:586
\ElementsProject#5  0x00406fec in map_catchup (map=0x938ecc, num_rejected=0xbe9e3a40) at common/gossmap.c:643
\ElementsProject#6  0x004073a4 in load_gossip_store (map=0x938ecc, num_rejected=0xbe9e3a40) at common/gossmap.c:697
\ElementsProject#7  0x00408244 in gossmap_load (ctx=0x0, filename=0x4e16b8 "gossip_store", num_channel_updates_rejected=0xbe9e3a40) at common/gossmap.c:976
\ElementsProject#8  0x0041a548 in init (p=0x93831c, buf=0x9399d4 "\n\n{\"jsonrpc\":\"2.0\",\"id\":\"cln:init#25\",\"method\":\"init\",\"params\":{\"options\":{},\"configuration\":{\"lightning-dir\":\"/home/vincent/.lightning/testnet\",\"rpc-file\":\"lightning-rpc\",\"startup\":true,\"network\":\"te"..., config=0x939cdc) at plugins/topology.c:622
\ElementsProject#9  0x0041e5d0 in handle_init (cmd=0x938934, buf=0x9399d4 "\n\n{\"jsonrpc\":\"2.0\",\"id\":\"cln:init#25\",\"method\":\"init\",\"params\":{\"options\":{},\"configuration\":{\"lightning-dir\":\"/home/vincent/.lightning/testnet\",\"rpc-file\":\"lightning-rpc\",\"startup\":true,\"network\":\"te"..., params=0x939c8c)
    at plugins/libplugin.c:1208
\ElementsProject#10 0x0041fc04 in ld_command_handle (plugin=0x93831c, toks=0x939bec) at plugins/libplugin.c:1572
\ElementsProject#11 0x00420050 in ld_read_json_one (plugin=0x93831c) at plugins/libplugin.c:1667
\ElementsProject#12 0x004201bc in ld_read_json (conn=0x9391c4, plugin=0x93831c) at plugins/libplugin.c:1687
\ElementsProject#13 0x004cb82c in next_plan (conn=0x9391c4, plan=0x9391d8) at ccan/ccan/io/io.c:59
\ElementsProject#14 0x004cc67c in do_plan (conn=0x9391c4, plan=0x9391d8, idle_on_epipe=false) at ccan/ccan/io/io.c:407
\ElementsProject#15 0x004cc6dc in io_ready (conn=0x9391c4, pollflags=1) at ccan/ccan/io/io.c:417
\ElementsProject#16 0x004cf8cc in io_loop (timers=0x9383c4, expired=0xbe9e3ce4) at ccan/ccan/io/poll.c:453
\ElementsProject#17 0x00420af4 in plugin_main (argv=0xbe9e3eb4, init=0x41a46c <init>, restartability=PLUGIN_STATIC, init_rpc=true, features=0x0, commands=0x6167e8 <commands>, num_commands=4, notif_subs=0x0, num_notif_subs=0, hook_subs=0x0, num_hook_subs=0, notif_topics=0x0, num_notif_topics=0) at plugins/libplugin.c:1891
\ElementsProject#18 0x0041a6f8 in main (argc=1, argv=0xbe9e3eb4) at plugins/topology.c:679
```

I do not know if this is a solution because I do not know
when I can parse a node announcement for a node that
it is not longer in the gossip map.

So, I hope this is just usefult for @rustyrussell

Changelog-Fixed: fixes `FATAL SIGNAL 11` on gossmap node announcement parsing.

Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
gkrizek pushed a commit to voltagecloud/lightning that referenced this issue Apr 26, 2023
It's still deprecated: we need the description since

1. This information is useful for any validation we want to do, such as
   the HSM, or runes.
2. We want this information in listpays so we can tell what we actually paid.
3. In general, we should never sign commitments to things we don't have!

I expect to have this information about payments *whatever the frontend* is,
which is why we deprecated (and then removed) this unintended use.  The spec
is pretty clear on this:

BOLT ElementsProject#11:
```
A reader:
...
  - MUST check that the SHA2 256-bit hash in the `h` field exactly matches the hashed
  description.
```

However, neither BTCPayServer nor lnbits updated despite the long deprecation
period, so revert 2afe7a1.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
gkrizek pushed a commit to voltagecloud/lightning that referenced this issue Apr 26, 2023
pubkey_from_hexstr() was failing, which we didn't notice because we
weren't checking the return value. The problem was that we were passing
it a strlen that was half the actual length.

Relevant error:

  [libsecp256k1] illegal argument: !secp256k1_fe_is_zero(&ge->x)

  ==417723== ERROR: libFuzzer: deadly signal
    ElementsProject#7 0x7f5deaacc7fb in abort
    ElementsProject#8 0x51b0b0 in secp256k1_default_illegal_callback_fn secp256k1.c
    ElementsProject#9 0x51bd8e in secp256k1_ec_pubkey_serialize
    ElementsProject#10 0x4e235b in pubkey_to_der bitcoin/pubkey.c:29:7
    ElementsProject#11 0x4e2941 in pubkey_cmp bitcoin/pubkey.c:89:2
    ElementsProject#12 0x4e333d in bitcoin_redeem_2of2 bitcoin/script.c:144:6
    ElementsProject#13 0x4f1396 in run tests/fuzz/fuzz-close_tx.c:78:19
ddustin pushed a commit to ddustin/lightning that referenced this issue May 12, 2023
The existing description is incorrect. `createinvoice` doesn't actually
work when supplied with a custom-encoded bolt11 invoice without the
final 520 signature bits appended. If a users tries to do so, some of
their tagged fields will be incorrectly truncated.

`createinvoice` actually expects that the signatures are there, and it
simply ignores them.

See common/bolt11.c's bolt11_decode_nosig:

         /* BOLT ElementsProject#11:
          *
          * The data part of a Lightning invoice consists of multiple sections:
          *
          * 1. `timestamp`: seconds-since-1970 (35 bits, big-endian)
          * 1. zero or more tagged parts
          * 1. `signature`: Bitcoin-style signature of above (520 bits)
          */
         if (!pull_uint(&hu5, &data, &data_len, &b11->timestamp, 35))
                 return decode_fail(b11, fail, "Can't get 35-bit timestamp");

>        while (data_len > 520 / 5) {
                 const char *problem = NULL;
                 u64 type, data_length;
ddustin pushed a commit to ddustin/lightning that referenced this issue May 12, 2023
This will fix a crash that I caused on armv7
and by looking inside the coredump with gdb
(by adding an assert on n that must be
different from null) I get the following stacktrace

```
(gdb) bt
\#0  0x00000000 in ?? ()
\#1  0x0043a038 in send_backtrace (why=0xbe9e3600 "FATAL SIGNAL 11") at common/daemon.c:36
\#2  0x0043a0ec in crashdump (sig=11) at common/daemon.c:46
\#3  <signal handler called>
\#4  0x00406d04 in node_announcement (map=0x938ecc, nann_off=495146) at common/gossmap.c:586
\#5  0x00406fec in map_catchup (map=0x938ecc, num_rejected=0xbe9e3a40) at common/gossmap.c:643
\ElementsProject#6  0x004073a4 in load_gossip_store (map=0x938ecc, num_rejected=0xbe9e3a40) at common/gossmap.c:697
\ElementsProject#7  0x00408244 in gossmap_load (ctx=0x0, filename=0x4e16b8 "gossip_store", num_channel_updates_rejected=0xbe9e3a40) at common/gossmap.c:976
\ElementsProject#8  0x0041a548 in init (p=0x93831c, buf=0x9399d4 "\n\n{\"jsonrpc\":\"2.0\",\"id\":\"cln:init#25\",\"method\":\"init\",\"params\":{\"options\":{},\"configuration\":{\"lightning-dir\":\"/home/vincent/.lightning/testnet\",\"rpc-file\":\"lightning-rpc\",\"startup\":true,\"network\":\"te"..., config=0x939cdc) at plugins/topology.c:622
\ElementsProject#9  0x0041e5d0 in handle_init (cmd=0x938934, buf=0x9399d4 "\n\n{\"jsonrpc\":\"2.0\",\"id\":\"cln:init#25\",\"method\":\"init\",\"params\":{\"options\":{},\"configuration\":{\"lightning-dir\":\"/home/vincent/.lightning/testnet\",\"rpc-file\":\"lightning-rpc\",\"startup\":true,\"network\":\"te"..., params=0x939c8c)
    at plugins/libplugin.c:1208
\ElementsProject#10 0x0041fc04 in ld_command_handle (plugin=0x93831c, toks=0x939bec) at plugins/libplugin.c:1572
\ElementsProject#11 0x00420050 in ld_read_json_one (plugin=0x93831c) at plugins/libplugin.c:1667
\ElementsProject#12 0x004201bc in ld_read_json (conn=0x9391c4, plugin=0x93831c) at plugins/libplugin.c:1687
\ElementsProject#13 0x004cb82c in next_plan (conn=0x9391c4, plan=0x9391d8) at ccan/ccan/io/io.c:59
\ElementsProject#14 0x004cc67c in do_plan (conn=0x9391c4, plan=0x9391d8, idle_on_epipe=false) at ccan/ccan/io/io.c:407
\ElementsProject#15 0x004cc6dc in io_ready (conn=0x9391c4, pollflags=1) at ccan/ccan/io/io.c:417
\ElementsProject#16 0x004cf8cc in io_loop (timers=0x9383c4, expired=0xbe9e3ce4) at ccan/ccan/io/poll.c:453
\ElementsProject#17 0x00420af4 in plugin_main (argv=0xbe9e3eb4, init=0x41a46c <init>, restartability=PLUGIN_STATIC, init_rpc=true, features=0x0, commands=0x6167e8 <commands>, num_commands=4, notif_subs=0x0, num_notif_subs=0, hook_subs=0x0, num_hook_subs=0, notif_topics=0x0, num_notif_topics=0) at plugins/libplugin.c:1891
\ElementsProject#18 0x0041a6f8 in main (argc=1, argv=0xbe9e3eb4) at plugins/topology.c:679
```

I do not know if this is a solution because I do not know
when I can parse a node announcement for a node that
it is not longer in the gossip map.

So, I hope this is just usefult for @rustyrussell

Changelog-Fixed: fixes `FATAL SIGNAL 11` on gossmap node announcement parsing.

Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
ddustin pushed a commit to ddustin/lightning that referenced this issue May 12, 2023
It's still deprecated: we need the description since

1. This information is useful for any validation we want to do, such as
   the HSM, or runes.
2. We want this information in listpays so we can tell what we actually paid.
3. In general, we should never sign commitments to things we don't have!

I expect to have this information about payments *whatever the frontend* is,
which is why we deprecated (and then removed) this unintended use.  The spec
is pretty clear on this:

BOLT ElementsProject#11:
```
A reader:
...
  - MUST check that the SHA2 256-bit hash in the `h` field exactly matches the hashed
  description.
```

However, neither BTCPayServer nor lnbits updated despite the long deprecation
period, so revert 2afe7a1.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
ddustin pushed a commit to ddustin/lightning that referenced this issue May 12, 2023
pubkey_from_hexstr() was failing, which we didn't notice because we
weren't checking the return value. The problem was that we were passing
it a strlen that was half the actual length.

Relevant error:

  [libsecp256k1] illegal argument: !secp256k1_fe_is_zero(&ge->x)

  ==417723== ERROR: libFuzzer: deadly signal
    ElementsProject#7 0x7f5deaacc7fb in abort
    ElementsProject#8 0x51b0b0 in secp256k1_default_illegal_callback_fn secp256k1.c
    ElementsProject#9 0x51bd8e in secp256k1_ec_pubkey_serialize
    ElementsProject#10 0x4e235b in pubkey_to_der bitcoin/pubkey.c:29:7
    ElementsProject#11 0x4e2941 in pubkey_cmp bitcoin/pubkey.c:89:2
    ElementsProject#12 0x4e333d in bitcoin_redeem_2of2 bitcoin/script.c:144:6
    ElementsProject#13 0x4f1396 in run tests/fuzz/fuzz-close_tx.c:78:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants