Newsletter 96 (2020-05-06)

_includes/articles/suredbits-enterprise-ln.md

@@ -0,0 +1,59 @@
+{:.post-meta}
+*by [Roman Taranchenko][], Engineer at [Suredbits][]*
+
+After the first excitement of sending and, more importantly, receiving a
+payment over the Lightning Network has faded away, it’s always good to
+think about how to operate your node in a safe and reliable way.
+Failures almost always happen unexpectedly. How do you recover after a
+possible failure? How do you make backups reliable? How do you keep the
+seed in a secure location? Et cetera, et cetera…
+
+At [Suredbits][] we use Eclair for our nodes. Even though Eclair is
+pretty robust on its own, we took some steps to make it even more
+reliable---such as using PostgreSQL as a database backend [(using this PR
+)][db pr] and [AWS Secrets Manager][] to
+store private keys.
+
+Eclair has a built-in online backup feature, but it requires manual
+setup and script writing to automate, which is not really scalable and
+is error prone. Running PostgreSQL at AWS RDS allows us to automate
+backups and replication in a way that is familiar to many DevOps engineers
+and which makes restoring the database state easier.
+
+Using PostgreSQL as a remote database backend makes node failover
+simpler to implement because, if the node crashes for some reason,
+there’s no need to restore the database from a backup---all you need is
+to point a new Eclair instance to the correct database server. Here’s a
+[quick demo][failover demo] of an automated failover implemented with two Eclair
+instances plus AWS's RDS, ELB, and NAT Gateway.
+
+In the failover scenario depicted in the demo, we needed a secure way to
+allow the node’s seed for its private keys to be shared between the Eclair instances. Eclair
+stores the seed in a file on the local file system which should be
+backed up somewhere and restored when needed. The current Eclair implementation
+requires extra steps to do so in an automated fashion.  We instead use AWS Secrets
+Manager---an encrypted key/value store specifically designed to securely hold
+various kinds of secrets including database passwords and encryption
+keys. Now all you need to do to share the seed between the instances is to
+point them to the correct secrets location in the config file. Once
+configured, the instance can be stored as an AMI image which can re-imaged as
+many times as needed without manual configuration.
+
+The measures we took are just the first steps to building
+enterprise-grade Lightning nodes. There are still some more problems
+that need to be solved. For example, which Hardware Security Module
+(HSM) can be used for a Lightning node, or how to failover a Bitcoin
+Core node in a multi-instance setting. But we believe that our work is a
+solid base for scaling out Eclair and making it more fault-tolerant.
+
+For more information about this topic, see our [presentation][enterprise ln vid].
+
+*Disclaimer: Since private keys are involved, don't use third party cloud
+services without a thorough risk assessment.*
+
+[Roman Taranchenko]: https://github.com/rorp
+[suredbits]: https://suredbits.com
+[db pr]: https://github.com/ACINQ/eclair/pull/1249
+[aws secrets manager]: https://github.com/rorp/eclair/tree/aws_secretsmanager
+[failover demo]: https://youtu.be/L2DtolwS8ew
+[enterprise ln vid]: https://www.youtube.com/watch?v=tbwy9mJIrZE

_posts/en/2020-05-06-suredbits-enterprise-ln.md

@@ -0,0 +1,17 @@
+---
+title: 'Field Report: Running a Lightning node in an enterprise environment'
+permalink: /en/suredbits-enterprise-ln/
+name: 2020-05-06-suredbits-enterprise-ln
+slug: 2020-05-06-suredbits-enterprise-ln
+type: posts
+layout: post
+lang: en
+version: 1
+
+excerpt: >
+  A field report about how Suredbits uses Eclair, AWS, and a remote
+  PostgreSQL database to operate an LN node with fast failover
+  capability.
+
+---
+{% include articles/suredbits-enterprise-ln.md %}

_posts/en/newsletters/2020-05-06-newsletter.md

@@ -0,0 +1,240 @@
+---
+title: 'Bitcoin Optech Newsletter #96'
+permalink: /en/newsletters/2020/05/06/
+name: 2020-05-06-newsletter
+slug: 2020-05-06-newsletter
+type: newsletter
+layout: newsletter
+lang: en
+---
+This week's newsletter links to a discussion about using enhanced QR
+codes for communicating large transactions, includes a field report from
+Suredbits about building a high-availability LN node, and briefly
+summarizes several recently transcribed talks and conversations.  Also
+included are our regular sections with releases, release candidates, and
+notable code changes from popular Bitcoin infrastructure software.
+
+## Action items
+
+{:.center style="font-size: 1.5em"}
+<!-- $$\frac{50}{2^{\lfloor height/210000 \rfloor}}$$ -->
+![Happy halving!](/img/posts/2020-05-halving.png)
+
+## News
+
+- **QR codes for large transactions:** QR codes can practically contain
+  up to about 3 kilobytes of data, which is enough to accommodate
+  typical user transactions but far too small to contain the largest
+  transactions users can normally send.  Riccardo
+  Casatta and Christopher Allen each posted to the Bitcoin-Dev mailing
+  list ([1][casatta qr], [2][allen qr]) a request for discussion
+  hopefully leading to standardization of a method for visual
+  communication of [Partially Signed Bitcoin Transactions][topic psbt]
+  (PSBTs) and other potentially large data blobs related to Bitcoin
+  wallet interaction.  See the [previous discussion][qr old] in the
+  Specter DIY repository and [continue the discussion][qr new] in the
+  Airgapped Signing repository.
+
+## Field report: running a Lightning node in an enterprise environment
+
+{% include articles/suredbits-enterprise-ln.md %}
+
+## Recently transcribed talks and conversations
+
+*[Bitcoin Transcripts][] is the home for transcripts of technical
+Bitcoin presentations and discussions. In this new <!-- TODO: remove "new" next month --> monthly feature, we
+highlight a selection of the transcripts from the previous month.*
+
+- **Simplicity---Next-Generation Smart Contracting** Adam Back presented at a
+  Blockstream webinar on Simplicity, a next-generation, low-level
+  alternative to Bitcoin Script focused on provable security and
+  expressiveness. Back discussed how Simplicity, if enabled in Bitcoin
+  today, would allow developers to implement new functionality like
+  [SIGHASH_NOINPUT][topic sighash_noinput] without necessarily needing a soft fork. He also
+  displayed a [demo][simplicity demo] showing what you can do with
+  Simplicity today.  ([transcript][simplicity xs], [video][simplicity
+  vid], [slides][simplicity slides])
+
+- **Attacking Bitcoin Core** Amiti Uttarwar presented at LA BitDevs.
+  Uttarwar discussed how changes to Bitcoin’s peer-to-peer layer should
+  be assessed according to five goals: reliability, timeliness,
+  accessibility, privacy, and upgradability. She discussed the dangers of
+  network partitions and [eclipse attacks][topic eclipse attacks]---and then explained why
+  block-relay-only connections and anchor nodes are effective
+  mitigations.  ([transcript][attacking xs], [video][attacking vid])
+
+- **LND v0.10** Laolu Osuntokun, Joost Jager, and Oliver Gugger
+  presented in virtual reality at Reckless VR. Osuntokun presented Tor
+  and RPC enhancements in the latest release of LND plus a new channel
+  feature called [anchor outputs][topic anchor outputs] that addresses
+  the challenge of estimating onchain fees months in advance.
+  Jager discussed the challenges of [multipart payments][topic multipath
+  payments] including the splitting algorithm, what happens when the
+  shards of the payment arrive at different times, and strategies for
+  handling multipart payment failures. Gugger ended by discussing Partially
+  Signed Bitcoin Transaction ([PSBT][topic psbt]) channel funding and
+  the channel abstraction work that made this possible.
+  ([transcript][lnd10 xs], [video][lnd10 vid])
+
+- **Grokking Bitcoin** Kalle Rosenbaum participated in a Bitcoin developer meetup
+  and gave a presentation at London Bitcoin Devs.  The
+  meetup discussion focused on the role of Bitcoin technical
+  education, [BIP32][] HD wallets, and soft fork upgrades.  For the
+  presentation, Rosenbaum used the content in his book to discuss how
+  the segwit upgrade of 2017 addressed transaction malleability and
+  quadratic hashing.  ([Meetup transcript][grok xs2], [presentation
+  transcript][grok xs1], [presentation video][grok vid], [presentation
+  slides][grok slides])
+
+## Releases and release candidates
+
+*New releases and release candidates for popular Bitcoin infrastructure
+projects.  Please consider upgrading to new releases or helping to test
+release candidates.*
+
+- [C-Lightning 0.8.2][c-lightning 0.8.2] is a new version release
+  that adds support for opening channels of any size (using the
+  `--large-channels` configuration parameter), provides a keysend plugin
+  for receiving [spontaneous payments][topic spontaneous payments], and
+  contains several other new features and bug fixes. Additionally, both
+  novice and experienced users are encouraged to read the project's new
+  [FAQ][cl faq].
+
+- [LND 0.10.0-beta][lnd 0.10.0-beta] is a major version release that
+  adds support for sending [multipath payments][topic multipath
+  payments], funding channels using an external wallet via [Partially
+  Signed Bitcoin Transactions][topic psbt] (PSBTs), the ability to create
+  invoices larger than 0.043 BTC, and several other new features and bug
+  fixes.  Additionally, users may wish to read the new [Operational
+  Safety documentation][lnd op safety].
+
+- [Bitcoin Core 0.20.0rc1][bitcoin core 0.20.0] is a release candidate
+  for the next major version of Bitcoin Core.
+
+## Notable code and documentation changes
+
+*Notable changes this week in [Bitcoin Core][bitcoin core repo],
+[C-Lightning][c-lightning repo], [Eclair][eclair repo], [LND][lnd repo],
+[libsecp256k1][libsecp256k1 repo], [Rust-Lightning][rust-lightning repo],
+[Bitcoin Improvement Proposals (BIPs)][bips repo], and [Lightning
+BOLTs][bolts repo].*
+
+*Note: the commits to Bitcoin Core mentioned below apply to its master
+development branch and so those changes will likely not be released
+until version 0.21, about six months after the release of the upcoming
+version 0.20.*
+
+- [Bitcoin Core #16528][] allows the `createwallet` RPC to create a
+  wallet that uses [output script descriptors][topic descriptors] to
+  derive just the particular scriptPubKeys that the wallet uses to
+  receive payments.  This is a major improvement over the way legacy
+  wallets scan for payments by deriving every type of script handled by
+  the wallet for each public key in the wallet---a technique that dates
+  back to the original Bitcoin 0.1 release's support for receiving to
+  both P2PK and P2PKH scriptPubKeys.  Descriptor wallets should be more
+  efficient (because they don't need to scan for unused script types),
+  easier to upgrade to new types of script (e.g. for [taproot][topic
+  taproot]), and easier to use with external tools (e.g. multisig
+  wallets or [HWI][topic hwi]-compatible hardware wallets via
+  [partially-signed bitcoin transactions][topic psbt]).
+
+    By default, descriptor wallets use the popular [BIP32][] HD wallet
+    paths specified by BIPs [44][BIP44], [49][BIP49], and [84][BIP84]
+    rather than the non-standardized fully-hardened path used in legacy
+    Bitcoin Core HD wallets.  A number of wallet RPCs are unavailable
+    with descriptor wallets, either because they don't make sense with
+    descriptors or because developers are still adapting them to new
+    edge cases.  The merge of this PR early in the 0.21 development
+    cycle and the decision to make descriptor wallets a non-default
+    option will give the new features six months to mature before
+    their expected release.
+
+- [Bitcoin Core #18038][] improves privacy when initially broadcasting a
+  transaction by reducing the frequency that a wallet will attempt to resend
+  from about 30 minutes to about once a day. Previously, an entity surveilling
+  the network could see multiple broadcasts of the same transaction during these
+  resend periods from the same node and conclude that wallet was the originator.
+  By reducing the frequency of resending attempts from the wallet, the
+  originator of the transaction has less chance of being identified.
+
+  To ensure new transactions reach the network even without frequent
+  indiscriminate rebroadcast by the wallet, this PR also adds a type of
+  _unbroadcast_ transaction within the mempool. An unbroadcast transaction is
+  one that has been submitted locally via the wallet or RPC but which has not
+  yet been successfully relayed to peers on the network. Such unbroadcast
+  transactions remain in the mempool and will be rebroadcast every 10-15 minutes
+  until a peer has fetched the transaction by sending the node a `getdata` P2P
+  message for that transaction.
+
+- [BIPs #893][] makes several changes to the [BIP340][] specification of
+  [schnorr][topic schnorr signatures] pubkeys and signatures, with
+  related changes also being made to the [BIP341][] specification of
+  [taproot][topic taproot].  The major changes include:
+
+    - *Alternative x-only pubkey tiebreaker:* this changes which variant
+      of a public key to use when only its x-coordinate is known,
+      as previously discussed (see [Newsletter #83][news83 alt
+      tiebreaker]).
+
+    - *Updated nonce generation recommendations:* the suggested method
+      to use for generating a signature nonce has been updated to
+      prevent implementation-specific vulnerabilities (see previous
+      discussion in Newsletters [#83][news83 precomputed] and
+      [#91][news91 power analysis]).
+
+    - *Updated tagged hashes:* the tag prefixed to hash function input
+      for schnorr signatures has been updated to deliberately break
+      compatibility with the old draft specification.  Otherwise,
+      libraries written for the old code might sometimes generate
+      signatures valid under the new code, and sometimes might not,
+      leading to confusion.  This was also previously mentioned in
+      [Newsletter #83][news83 alt tiebreaker].
+
+- [BIPs #903][] simplifies the [BIP322][] specification of [generic
+  signed messages][topic generic signmessage] as previously proposed
+  (see [Newsletter #91][news91 bip322 update]), the change mainly
+  removes details that allowed signing the same message for multiple
+  scripts (addresses) in the same proof.
+
+- [BIPs #900][] updates the [BIP325][] specification of [signet][topic
+  signet] so that all signets use the same hardcoded genesis block
+  (block 0) but independent signets can be differentiated by their
+  [network magic][] (message start bytes).  In the updated protocol, the
+  message start bytes are the first four bytes of a hash digest of the
+  network's challenge script (the script used to determine whether a
+  block has a valid signature).  The change was [motivated][dorier
+  signet] by a desire to simplify the development of applications that
+  want to use multiple signets but which need to call libraries that
+  hardcode the genesis blocks for the networks they support.
+
+{% include references.md %}
+{% include linkers/issues.md issues="16528,18038,893,903,900" %}
+[bitcoin core 0.20.0]: https://bitcoincore.org/bin/bitcoin-core-0.20.0
+[lnd 0.10.0-beta]: https://github.com/lightningnetwork/lnd/releases/tag/v0.10.0-beta
+[c-lightning 0.8.2]: https://github.com/ElementsProject/lightning/releases/tag/v0.8.2
+[news83 alttiebreaker]: /en/newsletters/2020/02/05/#alternative-x-only-pubkey-tiebreaker
+[dorier signet]: https://github.com/bitcoin/bitcoin/pull/16411#issuecomment-577999888
+[cl faq]: https://github.com/ElementsProject/lightning/blob/master/doc/FAQ.md
+[news91 bip322 update]: /en/newsletters/2020/04/01/#proposed-update-to-bip322-generic-signmessage
+[casatta qr]: https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2020-April/017794.html
+[allen qr]: https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2020-April/017795.html
+[qr old]: https://github.com/cryptoadvance/specter-diy/issues/57
+[qr new]: https://github.com/BlockchainCommons/AirgappedSigning/issues/4
+[lnd op safety]: https://github.com/lightningnetwork/lnd/blob/master/docs/safety.md
+[news83 alt tiebreaker]: /en/newsletters/2020/02/05/#alternative-x-only-pubkey-tiebreaker
+[news83 precomputed]: /en/newsletters/2020/02/05/#safety-concerns-related-to-precomputed-public-keys-used-with-schnorr-signatures
+[news91 power analysis]: /en/newsletters/2020/04/01/#mitigating-differential-power-analysis-in-schnorr-signatures
+[network magic]: https://btcinformation.org/en/glossary/start-string
+[bitcoin transcripts]: https://twitter.com/btctranscripts
+[simplicity demo]: https://asciinema.org/a/rhIsJBixoB3k8yuFQQr2UGAQN
+[simplicity xs]: https://diyhpl.us/wiki/transcripts/blockstream-webinars/2020-04-08-adam-back-simplicity/
+[simplicity vid]: https://www.youtube.com/watch?v=RZNCk-nyx_A
+[simplicity slides]: https://docsend.com/view/svs27jr
+[attacking xs]: https://diyhpl.us/wiki/transcripts/la-bitdevs/2020-04-16-amiti-uttarwar-attacking-bitcoin-core/
+[attacking vid]: https://www.youtube.com/watch?v=8TaY730YlMg
+[lnd10 xs]: https://diyhpl.us/wiki/transcripts/vr-bitcoin/2020-04-18-laolu-joost-oliver-lnd0.10/
+[lnd10 vid]: https://www.youtube.com/watch?v=h34fUGuDjMg
+[grok xs1]: https://diyhpl.us/wiki/transcripts/london-bitcoin-devs/2020-04-29-kalle-rosenbaum-grokking-bitcoin/
+[grok vid]: https://www.youtube.com/watch?v=6tHnYyaw0qw
+[grok slides]: http://rosenbaum.se/ldnbitcoindev/drawing.sozi.html
+[grok xs2]: https://diyhpl.us/wiki/transcripts/london-bitcoin-devs/2020-04-22-socratic-seminar/