Feature request: allow hashing functions to hash other Clarity types

[MarvinJanssen]

Is your feature request related to a problem? Please describe.
The ability to hash any Clarity value could open up a lot of DeFi/DeX potential. This suggestion is to have a counterpart to Ethereum's EIP712. The hashes can be signed by the user and then passed to off-chains apps to be used in contract calls later.

Some use-cases:

• Simple address ownership proofs without having to hit the chain.
• Signing orders for off-chain order books.
• Creation of pseudo-transactions that can be stored and broadcast later without having to worry about nonces.
• Limitless signature fun.

Describe the solution you'd like
CVs could just be serialised to the SIP wire format and then hashed. We might also want to add the chain ID to prevent collisions across forks. A super quick example can be found in this branch: https://github.com/MarvinJanssen/clarity-repl/tree/feat/hash-structured-data.

Working examples:

Javascript:

const {
	listCV,
	uintCV,
	tupleCV,
	bufferCVFromString,
	serializeCV
	} = require('@stacks/transactions');

const {createHash} = require('crypto');

const sha256CV = input => createHash('sha256').update(serializeCV(input)).digest('hex');

let list = listCV([uintCV(3), uintCV(4), uintCV(5)]);

console.log(sha256CV(list)); // -> 6f3a852f0d216d0012c109b67fd021e34a8873d4c788c69ea79926562d38e1e0

let tuple = tupleCV({num: uintCV(5), buff: bufferCVFromString("buffer string")});

console.log(sha256CV(tuple)); // -> 9aa9471d690ba4844b1282ae8c9387b175557c71f61293660f4a72bd2361299e

Clarity:

(sha256 (list u3 u4 u5))
;; -> 0x6f3a852f0d216d0012c109b67fd021e34a8873d4c788c69ea79926562d38e1e0

(sha256 {num: u5, buff: 0x62756666657220737472696e67})
;; -> 0x9aa9471d690ba4844b1282ae8c9387b175557c71f61293660f4a72bd2361299e

buff, uint, and int types remain as is to be backwards compatible.

Additional context
Provides a solution to: leather-wallet/extension#1051

This is what signing CVs could look like in the Web Extension:

[jcnelson]

I thought about this a bit more. I think we can achieve this if we simply provide a way to serialize a Clarity structure into a buff on-chain.

Calculating a hash over data can be done today, for the most part. You can write Clarity code that will calculate the hash of a piece of structured Clarity data, provided you represent your Clarity data as things that are hashable. For example, you could do this:

(define-private (hash-16-fold (next-item uint) (hashbuf (buff 32)))
    (let (
        (num-hash (sha256 next-item))
        (combined-hash (concat num-hash hashbuf))
    )
    (sha256 combined-hash)))

(define-read-only (hash-16 (nums (list 16 uint)))
    (fold hash-16-fold nums 0x0000000000000000000000000000000000000000000000000000000000000000))

This would let you deterministically calculate a hash of a list of up to 16 uints by calculating a rolling sha256 over them. You could to this for strings and buffs too, as well as for structured tuples containing them. But you'd need to declare all the hashing methods for all structured data up front, and your client library (which lives off chain) would need to be aware of these hashing methods. We also don't have methods for hashing principals.

If we could simply serialize any Clarity datum to a buff, then I think we achieve the same ends.

[MarvinJanssen]

Yep I like that too: the ability to turn any CV into a buff. It will also help for token standards where you for example want to return a URL for (get-token-uri (token-id uint) ...) in the form of https://mydomain.com/token/token-id-form-here. However, I assume we don't want to be able to do the opposite (deserialising a buff into all CV types). Also, how does this work for trait references? I can imagine wanting to hash a tuple containing a trait ref principal.

I know we can do the above, but I think it's very verbose to fold over a list or something like that. Also you'd want the ability to hash principals, etc., as well.

Still, even if we were to have a (to-buff ...) for any CV, it would be nice if you can also just pass them into the hashing functions directly while we're at it.

[jcnelson]

@lgalabru brought up an interesting question. If the purpose of this feature is to implement something like meta-transactions (or making it possible to commit to transaction payloads that don't hit the chain until necessary), then why not just use sponsored transactions? The user would generate a sponsored transaction for a contract-call (with all the serialized arguments), sign it, and then give it to the DEX or whatever intermediary to hold onto until it needs to actually be relayed. If the user wants to cancel the sponsored transaction later, she only needs to send a separate transaction with a conflicting nonce (thereby invalidating the sponsored transaction). Would that serve the examples you gave?

[MarvinJanssen]

I thought about that before and see a couple of issues (correct me if I'm wrong):

1. Users cannot use their wallet for anything else until the "meta-transaction" is either relayed or canceled by replacing it.
2. Those meta-transactions are easily invalidated by accident if a user decides to interact with a different app / smart contract using the same wallet.
3. They cannot be fulfilled out of order. So a user can submit multiple delayed transactions but it requires a lot of user intervention to let some go through and cancel others.
4. More advanced order matching that involves moving multiple different tokens owned by different users is likely impossible or at the least extremely tedious.

I could probably think of more reasons. Happy to jump on a call to elaborate and discuss this. The proposed feature also enables more than just the ability to commit to transaction payloads that don't hit the chain until necessary, like signing proofs for purely off-chain usage.

[lgalabru]

Just to be clear, I think I'm supportive for augmenting Clarity with the features you presented.
That being said, I'm curious, and would love to see how can sponsored transactions be addressing the issues you'd like to address. And if they have some limitations today, how can we improve their design to make them more usable?
Looking at the limitations you're raising, they almost seems like features to me, as in, as a user, having the ability to cancel some stale "meta transactions" is great (but yes, the UX in the wallet should be very clear).
Note that introducing meta-transactions via signed buffers in Stacks will also have their own limitations, like the fact that by being "asynchronous", token activities can't be secured by post-conditions.

[MarvinJanssen]

Totally agree that, depending on the situation, the issues listed above are considered features. A sponsored transaction with post conditions can do a lot and is likely all that is needed for some applications. But still, it is too fragile and one-off for apps that demand lots of these per user.

Making true meta-transactions a thing (which would be amazing, and such discussion would warrant a new issue) will open a huge can of worms that requires a lot of research. E.g.: dealing with the origin nonce issue (or not), introducing post conditions that assess block-height, possibly even have the ability for multiple users to co-sign a transaction, each adding their own post conditions. (So two users could agree to sign a transaction that swaps two of their tokens by adding post conditions.) Sure, some of that can be done with escrow contracts, but then you're hitting the chain a lot again. I have a model in mind that would become possible if the feature request in the OP is implemented. Also, depending on the model, some asynchronous token activities could still be secured by some post-conditions for the user broadcasting the transaction.

[garyng2000]

agree with OP, there must be a way to sign/execute/endorse a transaction that is not time(thus nonce/order) dependent. I have 10 BTC and want to send it to 10 person. They are completely independent activities. Similarly why would my NFT transfer depends on my other BTC transfer or vice-versa.

[gregorycoppola]

So, are we going to implement this with the API exactly as @MarvinJanssen requested?

Or, are we going to implement serialize and deserialize to buff and then, if the user wants, they can sign the buff?

@jcnelson

[MarvinJanssen]

Given the design principles of Clarity, I would prefer the ability to pass different types straight into hashing functions. One thing I love about Clarity is that I am not juggling byte buffers around. A case can be made for both though.

[kantai]

I'm much more in favor of adding something like serialize / deserialize functions, rather than extending the type admission of the hash functions. That's generally more in line with Clarity's APIs: it provides the most flexibility (i.e., it allows other uses of the same thing) and it limits the amount of "special" treatment in the hashing functions.

[LNow]

@kantai something like this?

;; @param value: any valid Clarity value
;; @returns buff
(serialize value)

;; @param input: buff
;; @param type: TypeDefinition
;; @returns (response Value uint) or (optional Value), and value is the type defined in type argument
(deserialize input type)

[MarvinJanssen]

Looks perfect to me.

[kantai]

Yeah, I think that'd be great. I think I'd opt for the (optional ...) return type.

[jcnelson]

This is now implemented in next via #3132.