diff --git a/_includes/references.md b/_includes/references.md index 33fa9e82e..23fb44d6f 100644 --- a/_includes/references.md +++ b/_includes/references.md @@ -22,6 +22,7 @@ [descriptor]: https://github.com/bitcoin/bitcoin/blob/master/doc/descriptors.md [eclair repo]: https://github.com/ACINQ/eclair [eltoo]: https://blockstream.com/eltoo.pdf +[erlay]: https://arxiv.org/pdf/1905.10518.pdf [hwi]: https://github.com/bitcoin-core/HWI [libminisketch]: https://github.com/sipa/minisketch [libsecp256k1]: https://github.com/bitcoin-core/secp256k1 diff --git a/_includes/specials/bech32/12-midway.md b/_includes/specials/bech32/12-midway.md new file mode 100644 index 000000000..79b9128b2 --- /dev/null +++ b/_includes/specials/bech32/12-midway.md @@ -0,0 +1,31 @@ +This segment marks half-way through our series about bech32, so we decided to +have some fun this week by describing some bech32-related trivia that's +interesting but not important enough for its own segment. + +- **How is bech32 pronounced?** Pieter Wuille, co-author of the proposal, uses a + soft "ch" so that the word sounds like ["besh thirty two"][wuille bech32]. + The name is a portmanteau that mixes the letters of the address's error + correction coding (BCH) into the name of its numeric base (base32). + Pronouncing it with a soft "ch" allows the first syllable of *bech32* to be + similar to Bitcoin's legacy address format, *base58*. We admit this extended + explanation ruins the joke, but it's a clever and amusing bit of wordplay. + +- **BCH has nothing to do with Bitcoin Cash's ticker code:** the name of the + BCH codes bech32 is based upon are an abbreviation for + [Bose-Chaudhuri-Hocquenghem][wikipedia bch], with Hocquenghem inventing this + type of cyclic codes in 1959 followed by Bose and Ray-Chaudhuri independently + rediscovering them in 1960. Moreover, the bech32 address format was announced + in March 2017, three months before the [first plans][] for what would later be + labeled Bitcoin Cash (which initially planned to use the ticker code *BCC*). + +- **Over ten CPU years consumed:** using existing information about + BCH codes, the authors of bech32 were able to find the set of codes that + provided the minimum amount of error detection they desired for Bitcoin + addresses. However, there were almost 160 thousand eligible codes in this + set, and the authors expected some of them to be better than others. To find the best + code among them, over 200 CPU cores and ["more than 10 years of computation time"][cpu time] was used. + +[wuille bech32]: https://youtu.be/NqiN9VFE4CU?t=1827 +[cpu time]: https://youtu.be/NqiN9VFE4CU?t=1329 +[wikipedia bch]: https://en.wikipedia.org/wiki/BCH_code +[first plans]:https://blog.bitmain.com/en/uahf-contingency-plan-uasf-bip148/ diff --git a/_posts/en/2019-03-19-bech32-sending-support.md b/_posts/en/2019-03-19-bech32-sending-support.md index bcb4ffefa..cc62e2544 100644 --- a/_posts/en/2019-03-19-bech32-sending-support.md +++ b/_posts/en/2019-03-19-bech32-sending-support.md @@ -92,5 +92,11 @@ on bech32 sending support, from March 19th to August 28th, 2019. {% include specials/bech32/11-only-bech32.md %} +## Bech32 trivia + +*Originally published in [Newsletter #49][].* + +{% include specials/bech32/12-midway.md %} + ## Footnotes {:.no_toc} diff --git a/_posts/en/newsletters/2019-06-05-newsletter.md b/_posts/en/newsletters/2019-06-05-newsletter.md new file mode 100644 index 000000000..264bafcab --- /dev/null +++ b/_posts/en/newsletters/2019-06-05-newsletter.md @@ -0,0 +1,234 @@ +--- +title: 'Bitcoin Optech Newsletter #49' +permalink: /en/newsletters/2019/06/05/ +name: 2019-06-05-newsletter +type: newsletter +layout: newsletter +lang: en +--- +This week's newsletter describes the proposed Erlay protocol that could +significantly reduce the overhead of relaying unconfirmed transactions between +nodes, summarizes an executive briefing by Bitrefill CEO Sergej Kotliar about +LN, and lists some recent changes to the COSHV proposal described last week. +Also included are our regular sections on bech32 sending support and notable +code changes in popular Bitcoin infrastructure projects. + +{% comment %}{% endcomment %} +{% include references.md %} + +## Action items + +*None this week.* + +## News + +- **Erlay proposed:** a new paper by Gleb Naumenko, Pieter Wuille, Gregory + Maxwell, Sasha Fedorova, and Ivan Beschastnikh describes an alternative +transaction relay protocol, [Erlay][]. Currently, when a node learns about a +new transaction that passes its relay policy, it announces the txid of that +transaction to all of its peers (except any peers that announced it +previously). This is a very simple and effective policy, but it's not +efficient---most of the announcements a node receives are announcements for +transactions it has already learned about from a different peer a few moments +before, a redundancy that wastes about 44% of all node bandwidth according to +the paper. + + The paper attempts to reduce that redundancy using the authors' Erlay +protocol which separates relaying into two phases, a fanout phase and a +reconciliation phase. In the fanout phase, a node will only directly announce +new transactions it has learned about to a maximum of eight peers selected from +the node's outbound peers. + + In the reconciliation phase, a node will periodically request from each of +its outbound peers a *sketch* of short transaction identifiers (short txids) +for all the new transactions that peer would normally announce to the node. +Sketches are created using the [libminisketch][] library described in +[Newsletter #26][] which implements a highly bandwidth efficient set +reconciliation technique based on error correction codes. Upon receipt of a +requested sketch, the node itself also generates a sketch of all the new +transactions that it would normally announced to its peer. The node +combines its sketch with the peer's sketch to produce a list of any differences +between the sketches. That diff contains the short txids of any transactions +that are in one set of new transactions but not the other. The node can then +request any transactions it doesn't have from that peer and proceed to querying +its next peer for that peer's sketch. This is repeated for a new peer once +each second, allowing the node to quickly receive any transactions it didn't +receive via a fanout announcement. This simple two-step protocol of fanout and +reconciliation describes the primary operation of Erlay; the remaining parts of +the protocol described and analyzed in the paper cover the optimal parameters +to use for the sketches and a series of fallback steps to take if a particular +attempt at set reconciliation fails. + + After describing the Erlay protocol, the paper analyzes its performance +using both a simulated network of 60,000 nodes (similar in number and use to +the current Bitcoin network) and a live set of 100 nodes spread internationally +across several datacenters. The most notable result is that Erlay reduces the +amount of bandwidth used to announce the existence of new transactions by 84%. + However, it does take about 80% (2.6 +seconds) longer for transactions to propagate to all of the nodes in the +network. It's still the case that Bitcoin transactions can only +be confirmed once every ten minutes on average, so a three second slowdown +seems like a reasonable price to pay for a major reduction in bandwidth. + + Having established that the protocol is a worthwhile efficiency +improvement, the paper considers the most important of its secondary aspects: +its effect on privacy. Currently, each Bitcoin Core node slightly delays +the relaying of transactions to +its peers; this makes it more difficult for spy nodes to use timing +correlations to guess that the first peer they receive a transaction from is +the peer that created the transaction. Multiple simulations were run testing +the effectiveness of the current relay protocol against the effectiveness of +Erlay for various amounts of spy nodes among network peers, from 5% spy nodes +to 60% spy nodes. In cases where the spy nodes were public nodes accepting +incoming connections, Erlay always performed as well or better than the current +protocol. In cases where the spy nodes were private nodes making connections +to honest nodes, Erlay sometimes performed better and sometimes performed +worse---but never more than 10% worse (and this worst case is an unlikely situation). The paper also notes that Erlay is compatible with the +proposed [BIP156][] Dandelion protocol for further improving the network's +resistance to spy nodes. + + Considering future possible changes to Bitcoin relay policy, the paper +notes that an increase in the number of outbound peers from 8 to 32 would only +increase the bandwidth used by a node to announce the existence of new +transactions by 32% with Erlay compared to 300% using the current protocol. + As described in the paragraph above about Erlay's two +phases, new transactions would still only be fanned out via direct announcements +to 8 peers, but nodes would perform set reconciliation with all 32 peers. +A four-fold improvement in relay connectivity improves the chance that +time-sensitive transactions for contract protocols like LN will make it +to miners quickly. + + In addition to libminisketch, the code changes necessary to implement Erlay +in Bitcoin Core amount to only 584 lines of code, and the CPU intensive part of +set reconciliation has been benchmarked to take less than a millisecond under +conditions that are worse than expected in practice. If no objections are +raised over the paper, Naumenko has announced his intention to write a BIP and +work to get Erlay included in a subsequent version of Bitcoin Core. The method +used for transaction relay is part of the P2P network protocol (not a consensus +rule), so the change could begin operation as soon as multiple users upgrade to +it, although we expect nodes will also include a backwards-compatible mode to +support peers who haven't upgraded. + +- **Presentation: Operating on Lightning:** Bitrefill CEO Sergej Kotliar gave a + presentation about LN for the Optech executive briefing last month. The +[video][kotliar ln] is now available. Kotliar begins by explaining that high +transaction fees during previous years had a significant effect on Bitrefill's +business, so they made a special effort to get really good at minimizing +fee-related expenses. The ability to receive LN payments supported that goal, +and he believes they were the first service on mainnet to sell real items for +LN payments. Today, LN payments represent about 5% of their sales, similar to +the amount of business they do using Ethereum. + + He believes it's important for businesses to start working on LN now. "In +Bitcoin we've gotten used to waiting for things [...] but making customers wait +an unknown amount of time creates a risk that the customer will go away." For +example, by the time a deposit clears at an exchange, the customer may no +longer be interested in making the trade that would've earned the exchange a +commission. Additionally, Bitrefill's experience with LN is that LN's improved +invoicing eliminates a number of different payment errors seen with onchain +bitcoin payments, including overpayments, underpayments, stuck transactions, +copy/paste errors, and other problems. Receiving payments over LN also +eliminates the need to consolidate UTXOs and reduces the need to rotate money +between hot and cold wallets. Eliminating all of these problems has the potential to significantly reduce customer support and backend expenses. + + In a particularly interesting section of his talk, Kotliar shows how perhaps as +much as 70% of current onchain payments are users moving money +from one exchange to another exchange (or even between different users of the +same exchange). If all that activity could be moved offchain using LN +payments, exchanges and their users could save a considerable amount of money and everyone in Bitcoin would benefit from the increase in available block space. + + Kotliar concluded his talk with a few short segments. He described what +software and services Bitrefill sees LN users using today and what he +expects them to be using in the near future. He then explained two of +Bitrefill's services for LN users (including businesses), [Thor][] and [Thor +Turbo][]. Finally, he briefly described several planned improvements to LN: +reusable addresses (see [Newsletter #30][]), splicing in and out (see +[#22][Newsletter #22]), and larger channels (also [#22][Newsletter +#22]). + + Overall, Kotliar made a compelling case that LN's faster speed, lower fees, +and improved invoicing means businesses that expect to remain competitive +serving Bitcoin customers in the near future should start working on +implementing LN support today. + +- **COSHV proposal replaced:** the author of the COSHV proposal we described + [last week][news48 coshv] has replaced it with a [similar + proposal][alt-coshv] under a different name. The new proposal checks more + than just the hash of a transaction's outputs---now the hash also includes the transaction's + version number, number of inputs, sequence numbers, and locktime. This + change eliminates concerns related to transaction malleability that would've + affected using the opcode with some types of contract protocols (such as + LN). Additionally, by hashing the + number of inputs allowed in the transaction, the original proposal's + restriction on just one input is lifted---however, the proposal warns that only + allowing a single input is still recommended to avoid unwanted interactions. + (Note: except for the new name, the changes don't affect the summary + of COSHV we wrote last week.) + +## Optech recommends + +As of this writing, the Bitcoin Core project currently has [over 300 open Pull +Requests (PRs).][core prs] Some of these are works in progress, but most of +the rest are waiting for developers to review them and either identify any +problems that need to be fixed or acknowledge that they're ready to be +merged. + +If you want to help Bitcoin Core improve by reviewing PRs but aren't +sure how to get started, we recommend you check out the [Bitcoin Core PR +Review Club][]. In a weekly IRC meeting, an experienced Bitcoin Core +contributor provides background information +about a selected PR and then answers live questions from new +contributors. Often they are assisted by other established Bitcoin Core +contributors, including sometimes the authors of the PRs being +discussed. + +Optech recommends attendance to any engineer who wants to get more +involved in the Bitcoin Core process. IRC meetings are held at 17:00 +UTC on Wednesdays. + +## Bech32 sending support + +*Week 12 of 24 in a [series][bech32 easy] about allowing the people +you pay to access all of segwit's benefits.* + +{% comment %}{% endcomment %} + +{% include specials/bech32/12-midway.md %} + +## Notable code and documentation changes + +*Notable changes this week in [Bitcoin Core][bitcoin core repo], +[LND][lnd repo], [C-Lightning][c-lightning repo], [Eclair][eclair repo], +[libsecp256k1][libsecp256k1 repo], and [Bitcoin Improvement Proposals +(BIPs)][bips repo].* + +- [Bitcoin Core #15741][] speeds up importing keys, addresses, and other + information into the wallet using the `importmulti` RPC by batching the + database updates rather than making them sequentially. In a test performed by + the PR author, this reduced the time to import 10,000 addresses from 465 + seconds to 4 seconds. + +- [LND #2985][] waits to relay gossip announcements until there are there are + at least ten of them to send and five seconds have elapsed since the previous + batch, reducing bandwidth overhead. This continues the work done by + LND and other implementations towards reducing the overhead of the + gossip protocol given the significant growth in the size of the LN + network over the past year. + +- [LND #2761][] switches to using a persistent state machine for routed + payments and to storing some additional state data for the machine, improving + the program's ability to correctly handle HTLCs that were unresolved when the + daemon was restarted. + +{% include linkers/issues.md issues="15741,2985,2761" %} +[bech32 easy]: {{news38}}#bech32-sending-support +[news48 coshv]: {{news48}}#proposed-transaction-output-commitments +[alt-coshv]: https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2019-June/016997.html +[kotliar ln]: https://www.youtube.com/watch?v=1UDD9PMFTds +[thor]: https://www.bitrefill.com/thor-lightning-network-channels/?hl=en +[thor turbo]: https://www.bitrefill.com/thor-turbo-channels/?hl=en +[Bitcoin Core PR Review Club]:https://bitcoin-core-review-club.github.io/ +[core prs]: https://github.com/bitcoin/bitcoin/pulls