This release includes two migrations. The first migration upgrades existing payment attempts and results to support the new TLV onion format. The migration should look like this:
2019-09-26 15:37:31.794 [INF] CHDB: Checking for schema update: latest_version=11, min_upgrade_version=9, db_version=9 2019-09-26 15:37:31.794 [INF] CHDB: Performing database schema migration 2019-09-26 15:37:31.794 [INF] CHDB: Applying migration #10 2019-09-26 15:37:31.795 [INF] CHDB: Migration of route/hop serialization complete! 2019-09-26 15:37:31.795 [INF] CHDB: Migrating to new mission control store by clearing existing data 2019-09-26 15:37:31.795 [INF] CHDB: Migration to new mission control completed!
The second migration upgrades invoices to support tracking multiple HTLCs, rather than a single one. This increases the accuracy of invoice accounting within
lnd across restarts. Note that you cannot update if you have any pending hodl invoice. The migration should look like this:
2019-09-26 15:37:31.795 [INF] CHDB: Applying migration #11 2019-09-26 15:37:31.795 [INF] CHDB: Migrating invoices to new invoice format 2019-09-26 15:37:31.795 [INF] CHDB: Migration of invoices completed!
Once these migrations succeed, it will not be possible to return to a prior version of
Verifying the Release
In order to verify the release, you'll need to have
gpg2 installed on your system. Once you've obtained a copy (and hopefully verified that as well), you'll first need to import the keys that have signed this release if you haven't done so already:
curl https://keybase.io/roasbeef/pgp_keys.asc | gpg --import
Once you have the required PGP keys, you can verify the release (assuming
manifest-v0.8.0-beta.txt.sig are in the current directory) with:
gpg --verify manifest-v0.8.0-beta.txt.sig
You should see the following if the verification was successful:
gpg: assuming signed data in 'manifest-v0.8.0-beta.txt' gpg: Signature made Tue Oct 22 15:45:18 2019 PDT gpg: using RSA key 4AB7F8DA6FAEBB3B70B1F903BC13F65E2DC84465 gpg: Good signature from "Olaoluwa Osuntokun <firstname.lastname@example.org>" [ultimate]
That will verify the signature of the manifest file, which ensures integrity and authenticity of the archive you've downloaded locally containing the binaries. Next, depending on your operating system, you should then re-compute the
sha256 hash of the archive with
shasum -a 256 <filename>, compare it with the corresponding one in the manifest file, and ensure they match exactly.
Verifying the Release Binaries
As of this release, our release binaries are fully reproducible thanks to
go1.13! Third parties are now able to verify that the release binaries were produced properly without having to trust the release manager(s). See our reproducible builds guide for how this can be achieved. The release binaries are compiled with
go1.13.3, which is required by verifiers to arrive at the same ones. They include the following build tags:
watchtowerrpc. Note that these are already included in the release script, so they do not need to be provided.
Finally, you can also verify the tag itself with the following command:
git verify-tag v0.8.0-beta
Building the Contained Release
Users are able to rebuild the target release themselves without having to fetch any of the dependencies. In order to do so, assuming that
lnd-source-v0.8.0-beta.tar.gz are in the current directory, follow these steps:
tar -xvzf vendor.tar.gz tar -xvzf lnd-source-v0.8.0-beta.tar.gz GO111MODULE=on go install -v -mod=vendor -ldflags "-X github.com/lightningnetwork/lnd/build.Commit=v0.8.0-beta" ./cmd/lnd GO111MODULE=on go install -v -mod=vendor -ldflags "-X github.com/lightningnetwork/lnd/build.Commit=v0.8.0-beta" ./cmd/lncli
-mod=vendor flag tells the
go build command that it doesn't need to fetch the dependencies, and instead, they're all enclosed in the local vendor directory.
Additionally, it's now possible to use the enclosed
release.sh script to bundle a release for a specific system like so:
LNDBUILDSYS="linux-arm64 darwin-amd64" ./build/release/release.sh
Multi-Frame TLV Based Onion
In this release of
lnd, we’ll now by default signal adherence to the new multi-frame TLV based onion format used when routing HTLCs end to end within the network. Compared to the legacy format, this format is much more flexible as rather than using a handpacked encoding for each field, we’ll now transmit TLV (type-length-value) streams, which is essentially a key-value map. As a result, the onion format is now far more extensible, as we can easily add new types without triggering a network wide update. In many instances (such as AMP), only the sender and receiver need to know of the new information in the onion, streamlining upgrade paths.
This change isn’t exposed to developers yet, but will be in future versions, allowing them to send a small amount of bytes to the receiver alongside a payment. The largest impact of this new feature is to lay the groundwork for the AMP/MPP work that has begun. In the case of AMP, this new namespace in the onion will be used for things like sending a partial payment shard, the total amount of the unified set of payments, and so on. We’ll also leverage this space to roll out our spontaneous payment feature (keysend) in a future version of
Safu Commitments (
Another major protocol update that has shipped in this new version of
lnd is something we call “safu commitments”. This new commitment format represents a large step in making Lightning safer to use for end users. Before this change, if the remote party forced closed the channel on-chain, in order to sweep your funds (in the case of partial data loss, but SCB existence), then the party that lost data would need to obtain a random nonce from the other party in order to sweep their funds. This dependency at times made recovery very difficult, or even indeterminate, if the party that lost data was unable to obtain this per-state nonce from the remote party.
With this new commitment format, we’ve instead removed this nonce derivation from the commitment output. Instead, when the remote party force closes, the funds will go directly into one’s wallet. As a result, the only thing required to recover funds in the case of partial data loss now are one’s SCB and the on-chain event of the channel being confirmed on chain. A new SCB version (
v1) has been added to signal if the channel being backed up uses the old or new commitment format, allowing us to keep our recovery flow identical to the current one.
It’s important to note that in order to use this new commitment format, new channels must be opened. The SCBs stored on disk, as well as those that are sent over the various RPC calls, will be automatically updated to reflect if this new commitment is being used in the channels being backed up. A new field has been added to the
ListChannels output (
static_remote_key) that indicates if the channel is using the new safu commitment format or not.
TLV Serialization Library
This release also includes a new TLV serialization library, which implements the TLV format described in BOLT 01. The
tlv package produces encodings that are both forwards and backwards compatible, and will form the basis of making wire messages and onion payloads easily extensible fields for new features like MPP, AMP, or extended gossip queries.
BOLT 11 Feature Bits
lnd now supports the ability to generate and parse invoices with payment-specific feature bits. At the moment there are no such feature bits that are advertised, however this is a preliminary step in order to widely deploy the parsing logic for when this happens. The first candidate features for being advertised are likely for MPP or AMP, which can inform the sender as to whether they may pay a particular invoice using those payment methods. In theory, the sender can also require that an invoice be fulfilled with a particular payment type.
Indefinite Channel Reestablishment
Prior to 0.8.0,
lnd would give the remote peer 30 seconds to send a their
channel_reestablish message. Typically this duration was sufficient, however if the exchange was not completed in a timely manner
lnd would incorrectly escalate this into a protocol error, and cause strict implementations to force close the channel.
In one instance with a c-lightning node, this resulted in 40 channels being force closed, since the contention at startup prevented the remote node from promptly sending
channel_reestablish. In 0.7.2, c-lightning also added mitigations to avoid this situation, by spacing out reconnections to peers on startup and ignoring the escalated errors.
lnd will wait indefinitely over the span of a connection for the remote peer to send
channel_reestablish. As a result, these sporadic failures should not occur between updated nodes. Further, the
active flag will in
ListChannels is stricter and more accurate, which now only displays
true if the channel is sufficiently confirmed, has ever received
funding_locked, and has received
channel_reestablish on the current connection. Previously the
active boolean would display
true whenever the first two conditions had been met.
Chain Notifier Subserver
Subscriptions for confirmation and spend notifications now require a height hint to be provided, also known as the earliest height in the chain at which the event could have happened. This prevents
lnd from scanning blocks that are irrelevant to the subscription. Subscriptions for confirmation notifications now also require at least one confirmation. Previously, subscriptions without providing a number of confirmations would lead
lnd to deadlock.
Watchtower Client Subserver
Continuing our subserver saga, this release includes a new addition: the Watchtower Client subserver. This is the first subserver that is included in
lnd by default and does not require a build tag.
The subserver includes the following RPCs, along with a
lncli command for each:
- AddTower (
lncli wtclient add)
- RemoveTower (
lncli wtclient remove)
- ListTowers (
lncli wtclient towers)
- GetTowerInfo (
lncli wtclient tower)
- Stats (
lncli wtclient stats)
- Policy (
lncli wtclient policy)
These RPCs allow users to interact with the watchtower client of their
lnd node, allowing for modifications and information retrieval of any registered watchtowers and the policies used with them.
Note that due to the introduction of this interface, the
--wtclient.private-tower-uris flag has now been deprecated and will be removed in the next major release,
v0.9.0-beta. All that is required for the client to be active is
--wtclient.active, any setup that had been configured from before will carry over. If you're setting up a new watchtower or want to change the configuration, users will now need to so as stated in the watchtower documentation.
Gossip received as part of an initial sync is no longer forwarded to other peers as it’s assumed that they have been previously broadcast to the network. This results in a small decrease in outbound bandwidth usage.
lnd now rebroadcasts its node announcement to the network every 24 hours to ensure nodes have up-to-date information about them.
--ignore-historical-gossip-filters option has been added in 0.8.0 to reduce outbound bandwidth usage in syncing older lightning nodes. Some older nodes send gossip timestamp filters that request large portions of the graph on each connection; this flag will still allow new gossip messages to be forwarded that satisfy the timestamp, but forgo the initial dump which can rack up outgoing bandwidth.
Mission control, the subsystem in
lnd that drives the payment process and tracks past node performance, has been made persistent. It now retains historical payment results across restarts.
Further improvements have been made to mission control that make the payment process faster and more reliable. Routing nodes forward in a non-strict matter, meaning that if the requested channel isn’t available, the node may forward an HTLC over an alternative channel to the same next hop. Therefore, it is enough to try a single channel for each node pair. To speed up payments, we now track node performance based on node pairs instead of channels.
When a payment attempt fails, the sender receives a failure message from which information can be distilled that helps identifying the cause of the failure. This information can be used to steer around the problem for the next attempt. The interpretation of the failure message has been improved.
Previously, mission control only recorded payment failures. This prevented hitting the same failure twice, but ignored previously successful routes. It didn’t distinguish between a route that worked before and an untried route. We now favor successful routes over untried routes and thereby reduce the number of required payment attempts. The explore-exploit trade-off that this presents is controlled by the existing
routerrpc.attemptcost config parameter.
To enhance the transparency of the path finding process, the total route success probability is exposed on
A new RPC
BuildRoute has been added. This call targets advanced users who want to build their own routes without needing to keep a shadow copy of the graph. The call takes a list of hop pubkeys and transforms this into a fully specified route by looking up channel policies from the graph. It also allows the construction of circular routes.
Routing Node Enhancements
Nodes will now enforce a maximum channel commitment fee for channels in which they are initiators when attempting to update their commitment fee. The maximum enforced depends on a percentage (default of 50%) of a channel initiator’s balance. The percentage can be modified through the
--max-channel-fee-allocation CLI flag.
max_htlc channel policy parameter is now exposed on the RPC interface. This allows callers of
UpdateChannelPolicy to reduce the maximum htlc amount that is forwarded over a channel.
RPC Bug Fixes
RemoteChanReserveSat fields for waiting close channels are now properly set. These weren’t set previously, leading to the fields being blank until the channel moved to pending close.
Previously, using the
UpdateChannelPolicy RPC with a zero base fee and/or fee rate would lead to a desynchronization between the routing policy propagated throughout the network and the one within the channel state machine, leading to routing failures throughout the network. To work around this, users were required to restart their nodes, but this is no longer required as of #3439.
The REST endpoint for the
DescribeGraph RPC call (
v1/graph) has been updated to allow a larger response size due to the growing channel graph.
A bug in the ordering of channel updates for
getnodeinfo has been fixed that could cause the updates not to be sorted in order of increasing pubkey. Additionally, the
last_update time displayed will now properly compute the maximum of the each update's
last_update instead of inconsistently selecting one or the other.
Users who set the
perm field (or
lncli) prior to 0.8.0 may have noticed that
lnd would not automatically reconnect to peers if the connection was severed. This release fixes the bug by tracking whether or not the current connection was requested as permanent via the RPC, and restoring the original functionality of the
lncli Bug Fixes
Invoice HTLC tracking
ListInvoices now return a list of all HTLCs that pay to an invoice. This provides insight into the channel through which the invoice was paid, as well as acceptance time, resolution time, and the exact HTLC amount. The handling of HTLCs internally has also been improved, which fixed several existing consistency and accounting issues.
A potential probe vector that allowed attackers to find out the final destination of a payment via the
final_expiry_too_soon response has been eliminated. To not deprive the payer of information that is required for continuation of the payment process, the height at which the receiver accepted the HTLC is added to the failure message.
New OS/Arch Release Targets
0.8.0 will provide compiled binaries for 11 new architectures:
solaris-amd64 builds were newly enabled by
go1.13 while the others had been previously available but not included in the release targets. If you're interested in a target that is not listed, please file an issue!
lnd integration test suite is now also continuously being run with
bitcoind as the chain backend. All supported backends (
bitcoind) now enjoy the same test coverage, which ensures
lnd will behave the same regardless of which backend users choose. As of this release,
lnd is fully compatible with
bitcoind versions up to
The mobile APIs and build tools have been included in this release. This allows developers to start building mobile apps which integrate
lnd, either by checking out the latest release, or the master branch.
RBF Aware Transaction Broadcast
The wallet has been updated to understand RBF specific errors. For now, this handles a few edge cases with non-critical broadcast errors. In the future, this will pave the way for direct fee bumping through RBF in the wallet.
Channel Close Transaction Rebroadcast
To ensure proper confirmation of a channel close transaction,
lnd has been updated to rebroadcast it after restarts. It also now also aids remote nodes that have lost state by resending channel reestablishment messages for already closed channels.
Neutrino has steadily been improved, and this release contains several fixes and optimizations that make it stable enough for mainnet support. Keep in mind that it is still early, and stay craeful.
The full list of changes since
v0.7.1-beta can be found here:
Contributors (Alphabetical Order)
Johan T. Halseth
Juan Pablo Civile