Compressed Bitcoin Transactions

[TomBriar]

This is a draft PR and reference implementation for a Compressed Bitcoin Transaction, As stated in the doc the main application for this is steganography, satellite/radio broadcast, and other low bandwidth channels with a high CPU availability on decompression.

doc: Added Compressed Transaction Schema Documentation
util: Added a variable length bitstream encoder
script: Added the rest of the IsPayTo Script functions, And Split ExtractDestination into two functions for ease of use
node: Added a vector based coin lookup
primitives: Added Compressed Transaction
validation: Added OutPoint Compression methods to the chainstate class
rpc: Added RPC endpoints for compression and decompression of Transactions
test: Added Tests for the RPC and new Primitive
fuzz: Added a fuzz test for Compressed Transactions

Link to first post on the bitcoin mailing list:
https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2023-August/021924.html

[DrahtBot]

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Code Coverage

For detailed information about the code coverage, see the test coverage report.

Reviews

See the guideline for information on the review process.

Type	Reviewers
Concept ACK	1440000bytes

If your review is incorrectly listed, please react with 👎 to this comment and the bot will ignore it on the next update.

Conflicts

Reviewers, this pull request conflicts with the following ones:

• #29280 (Implement OP_CHECKTEMPLATEVERIFY by reardencode)
• #29221 (Implement 64 bit arithmetic op codes in the Script interpreter by Christewart)
• #29050 (Add OP_TXHASH and OP_CHECKTXHASHVERIFY opcodes by stevenroose)
• #28930 (wallet: Add scan_utxo option to getbalances RPC by BrandonOdiwuor)

If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

[1440000bytes]

Concept ACK

25-50% savings look good and compressed raw transactions can help in some cases as mentioned in the PR description.

Related Q&A: https://bitcoin.stackexchange.com/questions/98886/compress-transaction-hex-string (still wont fit in one text message but will need less text messages)

[achow101]

Is this intended to be used for storage, p2p relay, or something else? If for our storage, I would be interested in seeing benchmarks as this would have an effect on how quickly things can be pulled from disk to be relayed. If for relay, then I think there should first be a BIP that describes the protocol for using compression and how that is negotiated. And if neither of those, then I don't think it's within the scope of Bitcoin Core. It seems to be a pretty significant maintenance burden for something that, as implemented, is only used by specific new RPCs.

[maflcko]

And if neither of those, then I don't think it's within the scope of Bitcoin Core. It seems to be a pretty significant maintenance burden for something that, as implemented, is only used by specific new RPCs.

My understanding is that it requires the chainstate, so this is written inside of Bitcoin Core. However, I am also wondering if existing RPCs can be used to query the chainstate, or if the RPC interface could be extended, if needed. Putting this piece of code directly in the satellite broadcast software seems easier and preferable.

[maflcko]

Closing for now due to lack of replies to questions. Please reply below, and leave a comment if this should be reopened.