Skip to content

Electrum integration for Bitcoin chain#3406

Merged
tomaszslabon merged 54 commits intomainfrom
bitcoin-electrum-integration
Dec 9, 2022
Merged

Electrum integration for Bitcoin chain#3406
tomaszslabon merged 54 commits intomainfrom
bitcoin-electrum-integration

Conversation

@nkuba
Copy link
Copy Markdown
Member

@nkuba nkuba commented Nov 15, 2022

This PR implements Bitcoin package integration with Electrum API.

There are a couple of servers implementations supporting the Electrum API:

A list of publicly exposed Electrum protocol servers for Bitcoin can be found here:

We added integration tests to verify the implementation works correctly with the most popular server implementations. To run the integration tests execute:

go test -v -tags=integration ./...

There are a couple of TODOs in the code regarding the improvements that will be handled in follow-up PRs.

Refs #3384
Refs #3369
Refs #3395

The intention of the function is to return the most recent block that
was mined (added to the chain) - also called a "tip".
Replacing `Current` with `Latest` seems accurate and should reduce the
confusion, as `current` may suggest the block that is currently being
mined.
For Bitcoin it seems more common to use the name `block height` over
the `block number`.
We need a `String()` function to serialize Hash automatically in the log
messages to a string. By default it will convert the Hash to a hex in
the InternalByteOrder.

This change required the previous `String` function to be renamed to
`Hex`.
The function can be used to reverse the bytes order in the slice.
Electrum client supports TCP and SSL connections to the Electrum server.
This enumeration will be used to handle the protocol types.
@nkuba nkuba self-assigned this Nov 15, 2022
@nkuba nkuba added this to the v2.0.0-m3 milestone Nov 15, 2022
@nkuba nkuba force-pushed the bitcoin-electrum-integration branch from b8e5ee1 to ea88f63 Compare November 15, 2022 22:52
The function initializes connection to a remote Electrum server.

TCP and SSL types of connections are supported.

Most of the servers run electrum protocol 1.4 version, hence we use it
for verification. If a server runs another version of the protocol a
warning will be issued.

A connection to an Electrum server may be dropped if no requests are
submitted, hence we implement a keep alive loop that will ping the
server to keep the connection alive.
@nkuba nkuba force-pushed the bitcoin-electrum-integration branch from ea88f63 to c51f12f Compare November 15, 2022 22:53
The function queries electrum server for a block header for a block at
the given height and then returns it in the format expected by the
bitcoin.Chain interface.
The function queries electrum server for a blockchain tip
and returns its' block height.
The function submits a serialized transaction to the chain.
The function queries electrum for a details of the specific transaction and
converts it in a format expected by the bitcoin.Chain interface.

The simpliest solution is to use `GetTransaction` function to query electrum
as it returns the transaction details in verbose format. Unfortunatelly it's
not compatible with Esplora/Electrs implementation of ElectrumX server.
(see: Blockstream/electrs#36)

As a workaround to cover integration with Esplora/Electrs we use `GetRawTransaction`
and deserialize the transaction.
The function returns number of confirmations for the given transaction.

As Esplora/Electrs doesn't support verbose transactions details in the
response, that would contain the confirmations number we need to
workaround it. (See: Blockstream/electrs#36)

The workaround:
1. Get the raw transaction
2. Deserialize the raw transaction
3. Find transaction block height by finding it in a history of
   transactions for the output script included in the transaction.
4. Get the latest block height
5. Calculate number of confirmations by subtracting the transaction
   block height from the latest block height and adding one.
The tests validate integration with a public electrum servers.
Mostly added TODOs and improved error handling.
The function was added to utils, so we can use it here.
@nkuba nkuba mentioned this pull request Nov 17, 2022
3 tasks
@nkuba nkuba marked this pull request as ready for review November 17, 2022 16:28
nkuba added 11 commits December 1, 2022 13:46
The function passes a context that should be used within it.
To reduce confusion we name the property `parentCtx`.
The `BitcoinConfig` struct is a container for configuration of different
implementations of the bitcoin integration. We cannot place it in
`electrum` package, as the package is a specific implementation.

The struct cannot be located insige the `pkg/bitcoin` package due to a
cyclic dependencies. We move it out to the main `config` package as it
seems the most clean approach. We can regiter more implementations (e.g.
Electrs).
The flags reflect all the configuration properties for electrum.

We introduced a custom flag for Protocol enum.
The implementation can be used in the maintainer already.
@nkuba nkuba force-pushed the bitcoin-electrum-integration branch from b42431c to a4c21e8 Compare December 2, 2022 14:25
Comment thread cmd/tbtc_maintainer.go Outdated
Comment thread pkg/bitcoin/electrum/config.go
Comment thread pkg/bitcoin/electrum/electrum_integration_test.go
nkuba added 5 commits December 5, 2022 13:52
The config properties reflect the default config of the client.
Wrappers defined in keep-common library require a parent context to be
passed to correctly initialize the context with timeout.
The change was introduced in keep-network/keep-common#109
@tomaszslabon tomaszslabon merged commit 36ed2e8 into main Dec 9, 2022
@tomaszslabon tomaszslabon deleted the bitcoin-electrum-integration branch December 9, 2022 14:50
lukasz-zimnoch added a commit that referenced this pull request Dec 19, 2022
In #3421 a Witness field
was added to the bitcoin. Transaction structure. Here we populate the
field with a value decoded from the transaction returned by electrum.

Depends on #3406
Depends on #3421
@pdyraga pdyraga modified the milestones: v2.0.0-m3, v2.0.0-m2 Jan 25, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants