Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Derive channel keys from the channel funding pubkey #1097

Merged
merged 17 commits into from Sep 23, 2019

Conversation

@sstone
Copy link
Member

commented Aug 9, 2019

This PR changes how we compute channel keys:

  • Each new channel is assigned a random BIP32 "funding key" path, which can be used to compute its funding privkey/pubkey pair.
  • All channel secrets/private keys are derived from a BIP32 path that can be deterministically computed from the channel's funding pubkey.

User funds can now be recovered from their seed, without using backups (we could say that "we use the bitcoin blockchain to backup channels").
If a user loses their channel backup but still has their BIP32 seed, they can use DLP to request peers they have active channels with to publish their current commitment transaction, extract their funding pubkey from its witness, re-compute their to-remote key and spend their output.

Compatibility with older versions of eclair is achieved by adding a version field to LocalParams using the same pattern as for channel versions (since legacy LocalParams also starts with a public key). It has been tested with the following scenarios:

  • new nodes can create and use channels with legacy nodes
  • legacy nodes can be upgraded. Channels created before the upgrade will have their LocalParam.version field set to 0, and new channels will have their LocalParam.version field set to 1.

For users, it means that this feature will only be available for new channels, they still need their backup data to recover legacy channels (note: for recovering channels with peers that support DLP, any backup will do and not just the last one)

Node: this is #1093 resurrected, it got corrupted when I forced-push changes :(

sstone added 2 commits Jul 31, 2019
We now generate a random funding key for each new channel, and use its public key to deterministically derive
all channel keys and secrets. This will let us easily recover funds using DLP even if we've lost everything but
our seed: we just need to connect to the node we had a channel with, ask them to publish their commit tx, and
once we see it on the blockchain we can extract our funding pubkey, recompute channel keys and spend our output.
@sstone

This comment has been minimized.

Copy link
Member Author

commented Aug 9, 2019

As pointed out in the old PR, this is not just useful when your peers support DLP (Data Loss Protect) and you can still connect to them and use DLP to get them to publish their commit tx.

If you've lost everything (except for you seed of course) you will still be able to scan the blockchain for commit tx that you can spend, which is good because it's very likely that your peers will eventually close their channels with you if they're never used.

It's also very easy to retrieve the ids of channels you've funded just by looking at your onchain wallet transactions.

And in most cases users should be able to remember/guess which nodes they opened a channel to.

@SomberNight

This comment has been minimized.

Copy link

commented Aug 14, 2019

If you've lost everything (except for you seed of course) you will still be able to scan the blockchain for commit tx that you can spend, which is good because it's very likely that your peers will eventually close their channels with you if they're never used.

If the remote closed with their commitment transaction, and you notice while scanning the blockchain, you probably won't have the per_commitment_point to derive the to_remote key. Or am I missing something? Are you assuming static_remotekey or some future feature?

@sstone

This comment has been minimized.

Copy link
Member Author

commented Aug 26, 2019

You're right it was wishful thinking (I want toRemoteKey based on the "remote" point so much that I keep forgetting it's not the case).

This change will also help with static_remotekey, especially if we keep the CSV delay (this is still an open point) but as is, it's better than what we currently have.

@codecov-io

This comment has been minimized.

Copy link

commented Aug 27, 2019

Codecov Report

Merging #1097 into master will increase coverage by 0.15%.
The diff coverage is 98.07%.

@@            Coverage Diff             @@
##           master    #1097      +/-   ##
==========================================
+ Coverage   83.63%   83.79%   +0.15%     
==========================================
  Files         103      104       +1     
  Lines        7743     7780      +37     
  Branches      328      322       -6     
==========================================
+ Hits         6476     6519      +43     
+ Misses       1267     1261       -6
Impacted Files Coverage Δ
...ain/scala/fr/acinq/eclair/wire/ChannelCodecs.scala 100% <100%> (ø) ⬆️
...c/main/scala/fr/acinq/eclair/channel/Helpers.scala 94.58% <100%> (+0.12%) ⬆️
...main/scala/fr/acinq/eclair/crypto/KeyManager.scala 100% <100%> (ø)
...n/scala/fr/acinq/eclair/channel/ChannelTypes.scala 100% <100%> (ø) ⬆️
...scala/fr/acinq/eclair/crypto/LocalKeyManager.scala 86.84% <100%> (-2.64%) ⬇️
...-core/src/main/scala/fr/acinq/eclair/io/Peer.scala 75.65% <100%> (+0.22%) ⬆️
...in/scala/fr/acinq/eclair/channel/Commitments.scala 88.38% <100%> (+0.17%) ⬆️
...c/main/scala/fr/acinq/eclair/channel/Channel.scala 84.67% <95.12%> (+0.18%) ⬆️
... and 4 more
@sstone sstone requested review from pm47 and t-bast Aug 27, 2019
sstone added 2 commits Aug 29, 2019
Copy link
Member

left a comment

LGTM

Copy link
Member

left a comment

Concept Ack, I think we could simply the changes made to LocalParams.

delayedPaymentBasepoint = keyManager.delayedPaymentPoint(localParams.channelKeyPath).publicKey,
htlcBasepoint = keyManager.htlcPoint(localParams.channelKeyPath).publicKey,
firstPerCommitmentPoint = keyManager.commitmentPoint(localParams.channelKeyPath, 0),
fundingPubkey = fundingPubKey.publicKey,

This comment has been minimized.

Copy link
@pm47

pm47 Aug 29, 2019

Member

I don't see the point of not creating fundingPubkey inline?

Suggested change
fundingPubkey = fundingPubKey.publicKey,
fundingPubkey = keyManager.fundingPublicKey(fundingKeyPath),
delayedPaymentBasepoint = keyManager.delayedPaymentPoint(localParams.channelKeyPath).publicKey,
htlcBasepoint = keyManager.htlcPoint(localParams.channelKeyPath).publicKey,
firstPerCommitmentPoint = keyManager.commitmentPoint(localParams.channelKeyPath, 0))
fundingPubkey = fundingPubKey.publicKey,

This comment has been minimized.

Copy link
@pm47

pm47 Aug 29, 2019

Member
Suggested change
fundingPubkey = fundingPubKey.publicKey,
fundingPubkey = keyManager.fundingPublicKey(fundingKeyPath),
sstone added 2 commits Sep 9, 2019
sstone added 5 commits Sep 16, 2019
This option is checked when we need to compute channel keys. For old channels it won't be set, and we always
set it for new ones.
This reverts commit 96507f5.
val ZEROES = ChannelVersion(bin"00000000000000000000000000000000")
val STANDARD = ZEROES
val USE_PUBKEY_KEYPATH_BIT = 0 // bit numbers start at 0
val USE_PUBKEY_KEYPATH = STANDARD | ChannelVersion(bin"00000000000000000000000000000001")

This comment has been minimized.

Copy link
@pm47

pm47 Sep 20, 2019

Member

Can't we use USE_PUBKEY_KEYPATH_BIT here?

sstone added 3 commits Sep 20, 2019
@pm47
pm47 approved these changes Sep 23, 2019
@sstone sstone merged commit ea77342 into master Sep 23, 2019
3 checks passed
3 checks passed
ci/semaphoreci/push: Build & Test The build passed on Semaphore 2.0.
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details
continuous-integration/travis-ci/push The Travis CI build passed
Details
@sstone sstone deleted the pubkey_channel_keypath branch Sep 23, 2019
sstone added a commit that referenced this pull request Oct 8, 2019
* Update list of commands in eclair-cli help (#1091)

* Add missing API endpoints to eclair-cli help

* Documentation update (#1092)

* Typed amounts (#1088)

* Route computation: fix fee check (#1101)

Fee check during route computation is:
- fee is below maximum value
- OR fee is below amout * maximum percentage

The second check was buggy and route computation would failed when fees we above maximum value but below maximum percentage of amount being paid.

* Publish transactions during transitions (#1089)

Follow up to #1082.

The goal is to be able to publish transactions only after we have
persisted the state. Otherwise we may run into corner cases like [1]
where a refund tx has been published, but we haven't kept track of it
and generate a different one (with different fees) the next time.

As a side effect, we can now remove the special case that we were
doing when publishing the funding tx, and remove the `store` function.

NB: the new `calling` transition method isn't restricted to publishing
transactions but that is the only use case for now.

[1] ACINQ/eclair-mobile#206

* Typed cltv expiry (#1104)

Untyped cltv expiry was confusing: delta and absolute expiries really need to be handled differently.
Even variable names were sometimes misleading.
Now the compiler will help us catch errors early.

* Extended queries optional (#899)

This is the implementation of lightningnetwork/lightning-rfc#557.

* Correctly handle multiple channel_range_replies

The scheme we use to keep tracks of channel queries with each peer would forget about
missing data when several channel_range_replies are sent back for a single channel_range_queries.

* RoutingSync: remove peer entry properly

* Remove peer entry on our sync map only when we've received
a `reply_short_channel_ids_end` message.
* Make routing sync test more explicit

* Do not send channel queries if we don't want to sync

* Router: clean our sync state when we (re)connect to a peer

We must clean up leftovers for the previous session and start the sync process again.

* Router: reset sync state on reconnection

When we're reconnected to a peer we will start a new sync process and should reset our sync
state with that peer.

* Extended Queries: use TLV format for optional data

Optional query extensions now use TLV instead of a custom format.
Flags are encoded as varint instead of bytes as originally proposed. With the current proposal they will all fit on a single byte, but will be
much easier to extends this way.

* TLV Stream: Implement a generic "get" method for TLV fields

If a have a TLV stream of type MyTLV which is a subtype of TLV, and MyTLV1 and MYTLV2 are both
subtypes of MyTLV then we can use stream.get[MyTLV1] to get the TLV record of type MYTLV1 (if any)
in our TLV stream.

* Channel range queries: send back node announcements if requested (#1108)

This PR adds support for sending back node announcements when replying to channel range queries:
- when explicitly requested (bit is set in the optional query flag)
- when query flags are not used and a channel announcement is sent (as per the BOLTs)

A new configuration option `request-node-announcements` has been added in the `router` section. If set to true, we
will request node announcements when we receive a channel id (through channel range queries) that we don't know of.
This is a setting that we will probably turn off on mobile devices.

* Rework router data structures (#902)

Instead of using two separate maps (for channels and channel_updates), we now use a single map, which groups channel+channel_updates. This is also true for data storage, resulting in the removal of the channel_updates table.

* Add more numeric utilities to MilliSatoshi (#1103)

Add comparisons and postfix operators.
Update most of the codebase to leverage those.

* Use unsigned comparison for 'maxHtlcValueInFlightMsat' (#1105)

* Add a sync whitelist (#954)

We will only sync with whilelisted peer. If the whitelist is empty then
we sync with everyone.

* Move http APIs to subproject eclair-node (#1102)

* Fix regression in `Commitments.availableForSend` (#1107)

We must consider `nextRemoteCommit` when applicable.

This is a regression caused in #784. The core bug only exists when we
have a pending unacked `commit_sig`, but since we only send the
`AvailableBalanceChanged` event when sending a signature (not when
receiving a revocation), actors relying on this event to know the
current available balance (e.g. the `Relayer`) will have a wrong
value in-between two outgoing sigs.

* Bolt4: remove final_expiry_too_soon error message (#1106)

It allowed probing attacks and the spec deprecated it in favor of IncorrectOrUnknownPaymentDetails.
Also add better support for unknown failure messages.

* Fix maven mirror (#1120)

* Use Long to back the UInt64 type (#1109)

* Define comparison operators between UInt64 and MilliSatoshi

* Implement Bolt 11 invoice feature bits (#1121)

lightningnetwork/lightning-rfc#656 introduced invoice feature bits as a pre-requisite for AMP and other advanced payment use-cases.

* Update docker build (#1123)

* Update docker base image to jdk11, update maven to 3.6.2 [ci skip]

* Reject expired invoices before payment flow starts (#1117)

* Made sync params configurable (#1124)

This allows us to choose smaller parameters for tests and reduce cpu
requirement during testing.

NB: The default value of 3500 for `reply_channel_range` was wrong. Theoretical max is ~2700.

* Activate support for variable-length onion (#1087)

This is now enabled by default.
We forward variable-length onions if we receive some.
We accept variable-length payments.
However for maximum compatibility with the network, we send payments using legacy payloads.

* Add Semaphore CI (#1125)

* Router computes network stats (#1116)

* Add comments and fix warnings in graph processing
* Add small feature to set the htlcMaximumMsat for routing hints (otherwise the graph processing algorithm used a minimum value which slightly reduced the benefits of those routing hints)
* Add the computation of network statistics to the router: this will be useful for multi-part payments to decide what thresholds should be used to split a payment

* Add monitoring with Kamon (disabled by default) (#1126)

For now:
- we only track some tasks (especially in the router, but not even
`node_announcement` and `channel_update`
- all db calls are monitored
- kamon is disabled by default

* Check funds in millisatoshi when sending/receiving an HTLC (#1128)

Instead of satoshi, which could introduce rounding errors.

Also, we check first the balance before the max-inflight amount, because
it makes more sense in terms of error management.

Co-Authored-By: Bastien Teinturier <31281497+t-bast@users.noreply.github.com>

* Don't hardcode the channel version (#1129)

Instead of hardcoding the channel version when we instantiate the
`Commitments` object, we rather define it when the channel is
instantiated. This is saner and prepares future usage.

* Removed Globals class (#1127)

This is a prerequisite to parallelization of tests.

* Make tests run in parallel (#1112)

There are two level of parallelization:
- between test suites (a suite = a test file)
- within a suite (depends on tests suites, some rely on sequential execution of tests, some don't)

* Add codecov integration to semaphore CI (#1134)

* Remove codecov integration from travis CI

* Drop support for Java 8 (#1135)

We already have Java 7 (for Android) and Java 11. Supporting Java 8
would require crossbuilding, which we are not doing (two recent PRs
broke the build on Java 8).

* Sphinx: accept invalid downstream errors (#1137)

When a downstream node sends us an onion error with an invalid length, we must forward the failure.
The recipient won't be able to extract the error but at least it knows the payment failed.

* Update string to match on bitcoind while it's indexing (#1138)

* Check for bitcoind's getrawtransaction availablilty during startup

* Peer: disable kamon

* Payment lifecycle refactoring (#1130)

* Unify payment events (no more duplication between payment types and events)
* Factorize DB and eventStream interactions: this paves the way for sub-payments that shouldn't be stored in the DB nor emit events.
* Add more fields to the payments DB:
  * bolt 11 invoice for sent payment
  * external id (for app developers)
  * parent id (AMP)
  * target node id
  * fees
  * route (if success)
  * failures (if failed)
* Re-work the PaymentsDb interface
* Clarify use of seconds / milliseconds in DB interfaces -> milliseconds everywhere
* Run SQL migrations inside transactions

* Improve error handling when we couldn't find all the channels for a supplied route in /sendtoroute API (#1142)

* Improve error handling when we couldn't find all the channels for a supplied route in /sendtoroute

* Handle fees increases when channel is OFFLINE (#1080)

* Add 'close-on-offline-feerate-mismatch' configuration to avoid closing offline channel when the feerate mismatch if over the threshold.

* Derive channel keys from the channel funding pubkey (#1097)

We now generate a random funding key for each new channel, and use its public key to deterministically derive all channel keys and secrets. This will let us easily recover funds using DLP even if we've lost everything but our seed: we just need to connect to the node we had a channel with, ask them to publish their commit tx, and once we see it on the blockchain we can extract our funding pubkey, recompute channel keys and spend our output.

* Add a "funding pubkey path" option to the channel version field

This option is checked when we need to compute channel keys. For old channels it won't be set, and we always set it for new ones.

* ChannelVersion: make sure that all bits are set to 0 for legacy channels

* ChannelVersion: USE_PUBKEY_KEYPATH is set by default

* Check if remote funder can handle an updated commit fee when sending HTLC (#1084)

If the sender of an htlc isn't the funder, then both sides will have to afford the payment:
- the sender needs to be able to afford the htlc amount
- the funder needs to be able to afford the greater commit tx fee incurred by the additional htlc output.

Fixes #1081.

Co-Authored-By: Pierre-Marie Padiou <pm47@users.noreply.github.com>

* Fix and expand channel keypath (#1147)

* Fix funding pubkey to channel key path computation

Channel key path is generated from 8 bytes computed from our funding pubkey, but we extracted 4 uint32 values instead of 2 (last 2 were always 0). We now use 128 bits to derive channel key paths.

* Add a channel key path compatibility test

This test will fail if we change the way we compute channel key paths, which would break existing channels.

* Use the same chain hash reference in all channel updates

To save memory, once we check that a channel_update's chain hash matches what
we expect we just replace it with a reference to our own chain hash.

* Commitments: take HTLC fee into account (#1152)

Our balance computation was slightly incorrect. If you want to know how much you can send (or receive), you need to take into account the fact that you'll add a new HTLC which adds weight to the commit tx (and thus adds fees).

* Android: add a spray-based API to eclair-node

This is a copy of the spray-based API developped by @araspitzu (akka-http does not
work for akka 2.3 which we use on the android branch)

* HTTP API: add type hints for payment status (#1150)

Cleans up the JSON payment status (easier to interpret for callers).

* Use "mock" Kamon library

Kamon does not work on Android and does not make much sense, so we replace
it with a basic Mock implementation that does nothing.

* Electrum: improve coin selection (fixes #1146) (#1149)

Our previous coin selection would sometimes fail when there was one wallet utxo and and low 
 feerate, because our first pass used a fee estimate that was too high and could sometimes not be met.

* Extend funding key path to 256 bits (#1154)

Our random funding key path is now 8 * 32 bits plus a 1' (funder) or 0' (fundee).
Channel key paths are computed from the sha256 of the funding public key (we take all 256 bits).

* Use bitcoin 0.18.1 in the test (#1148)

* Upgrade new unit tests to bitcoin 0.18.1 API (#1157)

We had 2 open PRs, one that added new tests using the 0.API, one that switched to 0.18.1, when they were merged the new tests failed since they had not been upgraded....

* Update netty dependency to 4.1.32 (#1160)

Also:
* explicitely set endpoint identification algorithm in strict mode
* force TLS protocols 1.2/1.3 in strict mode

Co-Authored-By: Bastien Teinturier <31281497+t-bast@users.noreply.github.com>

* Add execution time limit (#1161)

* Android: wipe channels table during db migration

We already wipe the updates table, and this make upgrading much simpler since we had different structures on
android vs mater.

* Activate extended channel range queries (#1165)
By default we now set the `gossip_queries_ex` feature bit.
We also change how we compare feature bits, and will use channel queries (or extended queries) only if the corresponding feature bit is set in both local and remote init messages.

* Use guava to compute CRC32C checksums (#1166)

CRC32C is not available in JDK 7 which we target on Android.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
6 participants
You can’t perform that action at this time.