Skip to content

protocol 1.6: minimal update to get broadcast_package out #6

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 9 commits into
base: master
Choose a base branch
from
+319 −53
Open

protocol 1.6: minimal update to get broadcast_package out #6

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
8e26c35
add new protocol version 1.6
SomberNight Feb 6, 2025
b774837
add submitpackage methods
f321x Feb 6, 2025
0466646
add note about estimatefee caching
SomberNight Feb 7, 2025
7b84d2a
add new method: mempool.get_info, and rm redundant relayfee method
SomberNight Oct 22, 2025
9657730
broadcast_package RPC: add notes about bitcoind version differences
SomberNight Oct 23, 2025
a4f158c
readthedocs: update version str
SomberNight Oct 23, 2025
e54c0e4
server.version: server must ignore extraneous args, to allow future ext
SomberNight Oct 30, 2025
b269a8f
broadcast_package RPC: explicitly state verbose=true is not stable
SomberNight Oct 31, 2025
919df4f
scripthash status calculation: add a detailed example
SomberNight Oct 31, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/conf.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@
import os
import sys
sys.path.insert(0, os.path.abspath('..'))
VERSION = "Electrum Protocol 1.4.x"
VERSION = "Electrum Protocol 1.6.x"

# -- Project information -----------------------------------------------------

Expand Down
84 changes: 68 additions & 16 deletions docs/protocol-basics.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -68,14 +68,12 @@ revision number.

A party to a connection will speak all protocol versions in a range,
say from `protocol_min` to `protocol_max`, which may be the same.
When a connection is made, both client and server must initially
assume the protocol to use is their own `protocol_min`.

The client should send a :func:`server.version` RPC call as early as
possible in order to negotiate the precise protocol version; see its
description for more detail. All responses received in the stream
from and including the server's response to this call will use its
negotiated protocol version.
The client must send a :func:`server.version` RPC call as the first
message on the wire, in order to negotiate the precise protocol
version; see its description for more detail.
All responses received in the stream from and including the server's
response to this call will use its negotiated protocol version.


.. _script hashes:
Expand Down Expand Up @@ -144,27 +142,81 @@ Status
To calculate the `status` of a :ref:`script hash <script hashes>` (or
address):

1. order confirmed transactions to the script hash by increasing
height (and position in the block if there are more than one in a
block)
1. Consider all transactions touching the script hash (both those spending
from it, and those funding it), both confirmed and unconfirmed (in mempool).

2. form a string that is the concatenation of strings
2. Order confirmed transactions by increasing height (and position in the
block if there are more than one in a block).

3. form a string that is the concatenation of strings
``"tx_hash:height:"`` for each transaction in order, where:

* ``tx_hash`` is the transaction hash in hexadecimal

* ``height`` is the height of the block it is in.

3. Next, with mempool transactions in any order, append a similar
string for those transactions, but where **height** is ``-1`` if the
transaction has at least one unconfirmed input, and ``0`` if all
inputs are confirmed.
4. For mempool transactions, we define **height** to be ``-1`` if the
transaction has at least one unconfirmed input, and ``0`` if all inputs are
confirmed.

5. Order mempool transactions by ``(-height, tx_hash)``, that is,
``0`` height txs come before ``-1`` height txs, and secondarily the
txid (in network byteorder) is used to arrive at a canonical ordering.
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note that the byteorder used for the sorting is not the human-readable txid byteorder but the network one.
I chose this because it fits the existing internals of electrumx much nicer.

However in cculianu/electrum-cash-protocol#5 (comment) @cculianu expressed frustration about this choice.

I guess I don't mind either way though.
Any strong opinions?

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I appreciate your asking others for feedback on this.

the network one.

Just to be annoying -- I'll smartly say that it can be argued the "network" order is only useful if you speak the p2p protocol. Otherwise it can be argued the true network order is the RPC order. :P

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Any strong opinions?

Network byteorder makes sense to me for such an internal behavior, IMO. But, the most important thing is that it's well defined.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added a detailed example in 919df4f, to try to avoid any misunderstanding.


6. Next, with mempool transactions in the specified order, append a similar
string.

4. The :dfn:`status` of the script hash is the :func:`sha256` hash of the
7. The :dfn:`status` of the script hash is the :func:`sha256` hash of the
full string expressed as a hexadecimal string, or :const:`null` if the
string is empty because there are no transactions.


**Status Example**

Consider the following BTC regtest address: `bcrt1qwu2d7rjn9yxalg7cjw6phqzk77lf3fmsf57wq4`,
which has script hash: `76bbdc72fdc74fbb0b049d21b0f80045f2a227cde1cd57a00925a437638984b5`.

Example RPC traffic::

<-- ('blockchain.scripthash.subscribe', ['76bbdc72fdc74fbb0b049d21b0f80045f2a227cde1cd57a00925a437638984b5']) (id: 15)
--> 3b3b8f1bf926bde2e8e21d80b7d23afbef8f8ef697e7e9204683d9a29801b671 (id: 15)
<-- ('blockchain.scripthash.get_history', ['76bbdc72fdc74fbb0b049d21b0f80045f2a227cde1cd57a00925a437638984b5']) (id: 16)
--> [
{'tx_hash': '5c75485491ef13ca39c06a6c985de5223ac34a365fe26d461e567521f87bb378', 'height': 1860},
{'tx_hash': '9a2808f0cebab7fd71033b2974ab476098f8a6deb972b72fed60878a44badeda', 'height': 1861},
{'tx_hash': '07ec1e1dd23e337d55c7e21e892cca4d328cef5d6eddf29e59587e9056e891f7', 'height': 1861},
{'tx_hash': '98edd636bf8142d9ec0b88de3c8dbd3eb3da9bcfb02f7496f21408d8c6be58f2', 'height': 0, 'fee': 200},
{'tx_hash': '73fe76fe862413b8ff0f74b72c65c21fa1229a07d0625d994028363aa7cd58f7', 'height': 0, 'fee': 200},
{'tx_hash': 'b870b52ab5ecc3c3133ef1b9fff54d8d49c9c3ab7aa4236bf4c8d734d4d3b231', 'height': -1, 'fee': 110}
] (id: 16)
<-- ('blockchain.transaction.get', ['5c75485491ef13ca39c06a6c985de5223ac34a365fe26d461e567521f87bb378']) (id: 17)
<-- ('blockchain.transaction.get', ['9a2808f0cebab7fd71033b2974ab476098f8a6deb972b72fed60878a44badeda']) (id: 18)
<-- ('blockchain.transaction.get', ['07ec1e1dd23e337d55c7e21e892cca4d328cef5d6eddf29e59587e9056e891f7']) (id: 19)
<-- ('blockchain.transaction.get', ['98edd636bf8142d9ec0b88de3c8dbd3eb3da9bcfb02f7496f21408d8c6be58f2']) (id: 20)
<-- ('blockchain.transaction.get', ['73fe76fe862413b8ff0f74b72c65c21fa1229a07d0625d994028363aa7cd58f7']) (id: 21)
<-- ('blockchain.transaction.get', ['b870b52ab5ecc3c3133ef1b9fff54d8d49c9c3ab7aa4236bf4c8d734d4d3b231']) (id: 22)
--> 02000000000101887e2ea83a1fbface451aa3b2ba556757039bfe266d8b0c03383b2a6bc306a0b0000000000fdffffff0200e1f505000000001600147714df0e53290ddfa3d893b41b8056f7be98a77038b23f710000000016001496ea3197df375c6a62188cf9119aee984c0cc6150247304402206e1bbfca238613dbb5c877007273ddc708a5bdf0436e8e9df8cff4860937a64402200d86cc261234f5d448dd005c28a7bfc29d86b4d3004c76f5440aa662ba65479f01210242c23a20435cd52192581537b2017288b8cdf3fef349100c11a125d9529017e143070000 (id: 17)
--> 0200000000010178b37bf82175561e466de25f364ac33a22e55d986c6ac039ca13ef915448755c0100000000fdffffff0200e1f505000000001600147714df0e53290ddfa3d893b41b8056f7be98a77070d0496b00000000160014e5ec5a8eff827f8ab906f94026e83a7404b8bf860247304402200a6a2c21d2829d65e5439e3153a6b157e231942d3fa9b9d90b2567b1eec11851022036e4797a67bb1bbd1e2239ab51905db07bb5a31a3ac63b0eadbb8e5e27baf63d012103a23dcf3edc18180b0d82b0be8eb1c30ae61597e99e298246e05109b7ba89763744070000 (id: 18)
--> 02000000000101887e2ea83a1fbface451aa3b2ba556757039bfe266d8b0c03383b2a6bc306a0b0100000000fdffffff0200e1f505000000001600147714df0e53290ddfa3d893b41b8056f7be98a770c4b8004d3a000000160014692a676d16f83641a6518e9154d2b43a8465ed440247304402206578cda616dd7425b7f7cc35a67892361168275ac8a8ca0c6a3e1eefd595e9de02200466d0eb0a4ed4351ae806793151f5bfdae44904507efe4225a67f36ef2de76c0121030644479c1b8c106e2f0a495fe18dc45572b3413bda2db919c1fcf0f77f43241c44070000 (id: 19)
--> 02000000000101dadeba448a8760ed2fb772b9dea6f8986047ab74293b0371fdb7bacef008289a0100000000fdffffff0200e1f505000000001600147714df0e53290ddfa3d893b41b8056f7be98a770a8ee536500000000160014783997038ebfd28edef06a1e088e9fd905c48d61024730440220426b0570147156e19cd4c6eff02f49e11747dc74eaad0299d884740c0f05958302200c93e46a3db4686d576943c88274ddb43afbcd4c491867642c1a9c3fba8ca3bf012103e0e2b67a3e8244900fa916decb6a9d45787ebe15b882c7a065cd0151d298fb4e45070000 (id: 20)
--> 02000000000101f791e856907e58599ef2dd6e5def8c324dca2c891ee2c7557d333ed21d1eec070100000000fdffffff0200e1f505000000001600147714df0e53290ddfa3d893b41b8056f7be98a770fcd60a473a000000160014641837eaea0de6ed898e30ff6e093c566d790521024730440220143065381913b92b572b6fc6362f89517f0d411d19d23ba025f43885736fdf7602201176d0f5284c65023baa06d760810085e8d7e0835b9a64a8046ab3c883499357012102be2e344c2f49afa8d734a2f79a09827b038d2f8f6bfa77d5a2e077c22a3fd80a45070000 (id: 21)
--> 02000000000101f258bec6d80814f296742fb0cf9bdab33ebd8d3cde880becd94281bf36d6ed980000000000fdffffff0192e0f505000000001600147714df0e53290ddfa3d893b41b8056f7be98a770024730440220569e38b9034c5855b866a35a16d949d16b7b544aff9d6b02b6d2ca7ebbd9c3b2022074a7d230241335436a2df1380a4d7c5442424a9a56a02124f4a0b495e9de24b50121027ba4d3ee6471a985307a37d09eb9a0a73c1a31b57616fe3e53a3d6d45400251945070000 (id: 22)
<-- ('blockchain.transaction.get_merkle', ['5c75485491ef13ca39c06a6c985de5223ac34a365fe26d461e567521f87bb378', 1860]) (id: 23)
<-- ('blockchain.transaction.get_merkle', ['9a2808f0cebab7fd71033b2974ab476098f8a6deb972b72fed60878a44badeda', 1861]) (id: 24)
<-- ('blockchain.transaction.get_merkle', ['07ec1e1dd23e337d55c7e21e892cca4d328cef5d6eddf29e59587e9056e891f7', 1861]) (id: 25)
--> {'block_height': 1860, 'merkle': ['7df6810d7ac97c43cbfc2bdc6be10e26cd54ad34ab5e563fd8bde7f86ff36975'], 'pos': 1} (id: 23)
--> {'block_height': 1861, 'merkle': ['ac7d7a5d8932f565a2cf9cb103881bfc3bf7aa2f56ccbcdeb31e02c524ddfd0b', 'f7d333193f2e5c48c236db7c3eafd5d03f387a71c4f9b6056617faefcf3d44cf'], 'pos': 1} (id: 24)
--> {'block_height': 1861, 'merkle': ['07ec1e1dd23e337d55c7e21e892cca4d328cef5d6eddf29e59587e9056e891f7', '5441fd37bd00b55ff5f9ea1a8956d2f3ade17c7fb7a363e8000b3e7cb40ea135'], 'pos': 2} (id: 25)

So, mined txs are first ordered by block height, but block 1861 needs a tie-breaker:
the two txs there are ordered based on position in the block.

The mempool txs are first ordered based on inverse-"height", but height 0 needs a tie-breaker:
those two txs are ordered based on network-endian txid (`f2` comes before `f7`).

Finally, the status is `3b3b8f1bf926bde2e8e21d80b7d23afbef8f8ef697e7e9204683d9a29801b671`.


Block Headers
-------------

Expand Down
43 changes: 42 additions & 1 deletion docs/protocol-changes.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
Protocol Changes
================

This documents lists changes made by protocol version.
This document lists changes made by protocol version.

Version 1.0
===========
Expand Down Expand Up @@ -175,3 +175,44 @@ New methods
-----------

* :func:`blockchain.name.get_value_proof` to resolve a name (with proof). Name index coins (e.g. Namecoin) only.


Version 1.5
===========

(this version number was skipped, no corresponding protocol is defined)
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note that I skipped 1.5.x as Fulcrum and https://github.com/cculianu/electrum-cash-protocol uses these version numbers.
However I now realise this is nevertheless somewhat problematic:
if a client wants to support protocol version 1.4 and 1.6 but not 1.5.x, they are in quite a pickle.
If the client sends server.version("electrum/4.6.2", ["1.4", "1.6"]) to Fulcrum 2.0, the server will select protocol version 1.5.x.

By design of server.version, clients must support a continuous range of versions.
OTOH, servers are free to pick-and-choose any particular protocol version they want and can skip interim versions.

From a quick read of "electrum-cash-protocol", I think all the changes in the interim versions are backwards compatible. So maybe this is all ~fine... Nevertheless, the situation is unfortunate.



.. _version 1.6:

Version 1.6
===========

Changes
-------

* Breaking change for the version negotiation: we now mandate that
the :func:`server.version` message must be the first message sent.
That is, version negotiation must happen before any other messages.
* Also for :func:`server.version`, the server must tolerate and ignore
extraneous arguments, to allow for extensions in future protocol versions.
* The status of a scripthash has its definition tightened in a
backwards-compatible way: mempool txs now have a canonical ordering
defined for the calculation (previously their order was undefined).
* :func:`blockchain.scripthash.get_mempool` previously did not define
an order for mempool transactions. We now mandate a specific ordering.
* Optional *mode* argument added to :func:`blockchain.estimatefee`.
* :func:`blockchain.block.headers` now returns headers as a list,
instead of a single concatenated hex string.

New methods
-----------

* :func:`blockchain.transaction.broadcast_package` to broadcast a package of transactions (submitpackage).
* :func:`mempool.get_info` to get more detailed and general relayfee info.

Removed methods
---------------

* :func:`blockchain.relayfee` is removed. The `minrelaytxfee` field
of the new :func:`mempool.get_info` RPC is a direct replacement.
Loading