[Stacks 2.1] Generic Bitcoin Data Import

[jcnelson]

@pavitthrap and I were talking this week, and she posed a very interesting question -- why not just have a generic Bitcoin memo burnchain operation? Such an operation would allow for miners and users alike to broadcast signals to one another via Bitcoin in a non-consensus-critical manner, such that all Stacks nodes would pick them up and report them. This would enable large-scale coordination between users to do things like execute soft forks or ratify SIPs.

Turns out, there's also another major use-case: Bitcoin NFTs. The delta between a generic memo burnchain operation and supporting Bitcoin NFTs -- or generally, any Clarity contract that needs to read a lot of Bitcoin data -- is to expose these burnchain transactions into Clarity directly.

We can do this today by requiring the user to supply a Bitcoin transaction, Merkle proof, and block header, in a contract-call, and then require the contract to verify the proof and parse the transaction itself. This is an expensive and tedious operation, but it is possible today. Can we do better?

The .data Contract

To make this really cheap and easy to use, what if instead we made it so that any Bitcoin transaction with this specially-crafted OP_RETURN could be copied into a well-known smart contract by miners? We already do some very similar things with TransferStxOp and StackStxOp -- a miner must process these on-Bitcoin operations and (for StackStxOp) translate them into contract-calls to be run against .pox. What if we did something like this?

Contract Implementation Sketch

The .data contract would be a boot code contract. It would be instantiated at the 2.1 epoch boundary, along with .pox-2 and .costs-3. It would contain:

• A data map that maps a (buff 32) (a Bitcoin txid) to a tuple that encodes a BurnchainTransaction struct.

• A private function that could insert data into this map. Miners would call this when processing the burnchain operation

• A public function to query this map, given the Bitcoin txid.

The burnchain operation would not place any requirements on the 77-byte scratch space that we allocate. It would just be copied verbatim.

Block-processing Changes

Because this is a burnchain operation we recognize -- i.e. its OP_RETURN starts with X2 and it has a dedicated opcode (d), it would automatically be picked up by the burnchain indexer and stored in the burnchain DB. We would treat these data operations the same way we treat TransferStxOp and StackStxOp:

• We'd store a copy in a table in the sortition DB, in a dedicated table. These operations would be stored here when processing a burnchain state transition. We'd add a getter that would allow a caller to load up all such operations for a particular sortition.

• We'd update the StacksChainState::get_stacking_and_transfer_burn_ops() function to also load up these data operations

• We'd add a StacksChainState::process_data_ops() function that works a lot like StacksChainState::process_stacking_ops() -- it would just call the private "store this data" function in .data with the Clarity tuple representation of the burnchain op.

Applications

The effect of these changes is that anyone who sends a Bitcoin transaction with an OP_RETURN that starts with X2d would see it materialize in the .data contract, in all Stacks forks. Then, any Clarity contract could query this .data contract for a Clarity representation of this Bitcoin transaction, without having to go through all the trouble of parsing and verifying a Bitcoin transaction.

This would significantly lower the barrier-to-entry for Bitcoin-centric applications, like Bitcoin NFTs. For example, a Bitcoin NFT project could enable a user to list, unlist, and buy an NFT controlled by a Bitcoin address simply by sending a well-formed Bitcoin transaction. For example, the last 77 bytes of the OP_RETURN could be used to encode the NFT ID and an opcode indicating "list", "unlist", or "buy" to indicate what the user wants to do. The first input to the transaction indicates the owner -- the user's public key and signature information would already be parsed and available in the .data contract as part of the decoded scriptSig. The first non-data output could be the BTC payment for a "buy" operation.

Because the Bitcoin transaction is already parsed and loaded into Clarity this way, the user no longer needs to mirror the Bitcoin transaction and Merkle proof to the NFT contract in Stacks. Instead, the NFT marketplace owner would just watch the .data contract and call into the NFT marketplace contract to "complete" any on-Bitcoin transactions meant to interact with it (i.e. by passing in the txid to the Stacks-side implementations of "list", "unlist", or "buy"). Moreover, because the Bitcoin transaction already identifies the principal acting on the NFT, the NFT marketplace could be designed so that anyone could do this call completion.

Implementation Complexity

Because we already have the means to do all of the things needed to make this happen, I don't think this is a very high-lift project. We just need to clone and tweak the machinery that handles StackStxOp for this purpose. I think this would significantly help the Bitcoin NFT project, as well as any project that finds itself interacting with Bitcoin regularly (including e.g. Magic STX, catamaran swaps, and so on).

[muneeb-ali]

Love this idea! It'd be great to see if there are engineers at any entities (outside of people already helping with 2.1) who can help move this forward this week.

[whoabuddy]

Noting that on today's engineering call it was discussed that the work required for this would be very similar to the work being done here for adding delegate-stx BTC wire format in #3339.

The BTC wire format for STX-on-BTC transactions is also defined for stacking or transferring STX as part of SIP-007.

[donpdonp]

I can start a draft PR that has the rust side bits such as adding StacksChainState::process_data_ops() that Jude mentioned. The clarity side is completely new to me, especially the .data 'boot code contract'. Someone familiar with that area would probably need to pick it from up there.

[saralab]

Deprioritizing for 2.1

[jcnelson]

We'll make the call on November 15.

[hstove]

I think this is a great idea.

I'd imagine there is still a need for apps that need the full transaction proof capabilities, because so many wallets can't create a BTC transaction to an OP_RETURN. Are there ideas for how we could include similar capabilities for "vanilla" Bitcoin transactions?

[jcnelson]

Sure -- there already exists bitcoin.clar which can parse Bitcoin transactions on-chain. It will need to get revamped with 2.1 features, but it works today. Trouble is, it's slow, and it requires the caller to supply a Bitcoin block header and Merkle proof. This could be supplied by a dedicated service, but AFAIK no one's put in the effort yet.