Support for CashTokens

[mr-zwets]

The Cashtokens upgrade adds 6 new introspection opcodes, these would map cleanly to the following 6 new cashscript expressions.
These are just a suggestion but might be helpful for other devs thinking already about new contracts enabled by the cashtokens upgrade

bytes tx.inputs[i].tokenCategory

bytes tx.inputs[i].nftCommitment

int tx.inputs[i].tokenAmount

bytes tx.outputs[i].tokenCategory

bytes tx.outputs[i].nftCommitment

int tx.outputs[i].tokenAmount

[rkalis]

Nice suggestions! Those opcode mappings make sense to me. Here are some notes from our call today.

Compiler TODOs:

• Update grammar file (grammar/CashScript.g4)
• Add new UnaryOperator values (ast/Operator.ts)
• Add new IntrospectionOp / OP_UNKNOWN mappings (utils/src/script.ts)
• Add UnarypOp mapping to compileUnaryOp() (generation/utils.ts)
• Add test cases (at least one test case in test/generation/fixtures.ts)

SDK TODOs:

• Update electrum-cash / other network providers (and remove the ones that don't support cashtokens)
  • Update underlying libraries and expose their cashtokens support
• Update Utxo and Recipient typings to match new formats
• Add functionality to add token outputs (same as .to(address, amount) but for tokens and NFTs)
• Add functionality to do token UTXO selection in transaction.setInputsAndOutputs()
• Add checks that we don't send tokens to non-token cashaddresses

Docs TODOs:

• Update all docs for new CashScript functionality
• Update / checks docs for new BCH functionality (potentially unrelated to CashScript changes)
  • https://blog.bitjson.com/bitcoin-cash-upgrade-2023/
• Release + migration notes

[mr-zwets]

Started working on the PR in my "CashTokens" branch. I suggest we first just upgrade the cashscript compiler & language documentation for the new functionality so new contracts can already be compiled for chipnet. Then afterwards upgrade the SDK for Cashtokens and add support for P2SH32 with a new expression LockingBytecodeP2SH32 and support in the SDK.

Compiler TODOs:

• Update grammar file (grammar/CashScript.g4)
• Add new UnaryOperator values (ast/Operator.ts)
• Add new IntrospectionOp / OP_UNKNOWN mappings (utils/src/script.ts)
• Add UnarypOp mapping to compileUnaryOp() (generation/utils.ts)
• Add test cases (at least one test case in test/generation/fixtures.ts
• Add type-checking for the new ops. (src/semantic/TypeCheckTraversal.ts in visitUnaryOp)
• updated AST test cases and add failure cases

Docs TODOs:

• Update all docs for new CashScript functionality
• Release + migration notes

[rkalis]

Great work so far @mr-zwets! Let me get to those AST test cases.

[rkalis]

AST test cases are added. I think we're probably almost ready on the compiler side so we can launch 0.8.0-next.0 this week and then work on the SDK between now and May after some other libraries have also added more support for CashTokens.

[mr-zwets]

I changed the type of tokenCategory from bytes32 back to bytes, as it can just return 0 or the tokenCategory concatenated together with the NFT's capability (mutable or minting).

I also added LockingBytecodeP2SH32 already because it fits better together with this first step of adding the new features to the compiler and only afterwards to the SDK. The migration notes now also make mention of the removal of the network options staging and testnet. So everything looks finished to me for release!

[rkalis]

Ok, in 0.8.0-next.0 we completed all the compiler TODOs. Today @mr-zwets and I had another call to refine the SDK TODOs.

Network functionality (depends on libs like electrum-cash):

• Update electrum-cash / other network providers (and remove the ones that don't support cashtokens)
  • Update underlying libraries and expose their cashtokens support

Transaction building functionality:

• Update Utxo and Recipient typings to match new formats
• Add functionality to add token outputs (same as .to(address, amount) but for tokens and NFTs)
  • Add third parameter tokenDetails object containing fields for tokenCategory, tokenAmount, nftCommitment, capability
• Add functionality to do token UTXO selection in transaction.setInputsAndOutputs()
  • We need to keep track of all the required token amounts by summing all transaction outputs
  • We need to keep track of all the available token amounts by summing selected inputs
  • We need to add change outputs for all tokens that are included in the transaction (and have change)
  • How do we select which UTXOs we'll use, do we e.g. add a getTokenUtxos() function or add a tokenCategory parameter to getUtxos()?
    • We will look at this later
• Do we need to add separate minChange and withoutChange functionality per token?
  • We'll keep the minChange setting global for now and revisit later
• Add checks that we don't send tokens to non-token cashaddresses
  • We do this in the transaction.to() function

Transaction signing functionality (depends on v2 of Libauth):

• In transaction.build() we need to add token data to the outputs (will need to check the required format)
• utils.ts>createSigHashPreimage()we'll need to make some changes to sourceOutput.

Depends on both transaction signing and transaction building:

• Add test file similar to P2PKH.test.ts (simple contract) but with transactions that send tokens (manual UTXO selection for now)

[cculianu]

FWIW I'm adding additional API support to Fulcrum

blockchain.*.listunspent will spit out additional info for UTXOs that have token_data on them. Basically something like this will do in an additional optional token_data key in the results dict for each UTXO

{ 
  "height": 126184,
  "token_data": {
    "amount": "1000000", 
    "category": "8fd6a2f713beaa5907a776b8b3060cddd1c6ff0588554c2364698ae271321ce9",
    "nft": {"capability": "minting", "commitment":"f00fd00fb33f"}
  },
  "tx_hash": "87489c43bae69c297bbaf65276573b0001c20c647a3d54d2842a4425ff87bacc",
  "tx_pos":1,
  "value":1000000
}

The listunspent and get_balance APIs will have an optional second argument to control inclusion or filtering of token UTXOs from their results. The question for you guys is -- what should the default be if unspecified?

I was partial to including all UTXOs, including tokens, to all results if unspecified.

I encourage you to join the discussion on telegram to hash this out: https://t.me/electroncashserver

[mr-zwets]

Just for completeness I want to mention that the move to Bigint #140 is also a prerequisite for adding CashTokens and that it has been decided to use P2SH32 addresses for CashScript smart contracts for security reasons to prevent collision attacks.

So together with the ESM changes necessary to change to alpha version of libauth v2, this issue contains everything necessary to upgrade CashScript for the May 15 network upgrade.

[mr-zwets]

progress

• BigInt everywhere
• Updated interfaces
• Add functionality to add tokens to outputs
• Add checks for sending tokens to token aware addresses
• Rework fee calculation & dust amount
• Add checks for token amounts & nfts in inputs when doing manual UTXO selection
• Add automatic change for token amounts & nfts in inputs when doing manual UTXO selection
• Upgrade to ESM only for Libauth v2
• Change to P2SH32 for contracts by default, keep P2SH20 as option
• add .tokenAddress to contract
• Add tests for P2SH32 and new address format
• Account for Tokens in fee & dust calculation
• Rework transaction signing for tokens
• Add tests for token functionality
• Enabling SIGHASH_UTXOS by default
• Implement automatic UTXO selection for fungible tokens
• Add test for automatic UTXO selection fungible tokens
• Implement automatic UTXO selection for NFTs
• Add tests for automatic UTXO selection NFTs
• Allow for token genesis transactions
• Add test for token genesis transactions
• Update all the examples
• Add .experimentalFromP2PKH to the docs
• Split up tests for better performance
• Add note about manual NFT UTXO selection to the docs

[rkalis]

Note: "old style" covenants don't work any more with the upgrade to P2SH32, because they use OutputP2SH to send money back into the contract, which won't work any more if the SDK only supports P2SH32.

I see 3 options:

1. Drop SDK support for "old style" covenants (compiler already dropped support for this in v0.7)
2. Make it possible to get the P2SH20 address for a contract from the SDK
3. Backport OutputP2SH32 to v0.6

I think option (1) probably makes the most sense. But I'd like to make sure that there are no legitimate use cases for why someone would want to use an "old style" covenant (that they'd need to compile with cashc v0.6), but then use it in a modern version of the SDK.

I don't think there are any cases where this make sense. But would love it if I can get some other eyes on that @mr-zwets @emergent-reasons.

[emergent-reasons]

Appreciate you asking. (1) is fine with GP.

[mr-zwets]

We decided to keep P2SH20 as an option when initiating a new contract which means old style covenants can still be used.

new Contract(
  artifact: Artifact,
  constructorArgs: Argument[],
  options? : {
    provider: NetworkProvider,
    addressType: 'p2sh20' | 'p2sh32',
  }
)

[rkalis]

For now we've decided not to implement automated UTXO selection for NFTs. So if you want to use NFTs with the CashScript SDK, you have to manually select your UTXOS. If we hear about any strong use cases that require this after launch, we can consider adding it after all.

[rkalis]

We're all ready to launch this after CashTokens goes live: https://twitter.com/CashScriptBCH/status/1651177916212948992. We'll keep this issue open until it's released in production.

[rkalis]

This is released in production and CashTokens is live. 🥳