multi: add support for ZCash #1570

buck54321 · 2022-04-07T13:04:34Z

Implement block deserialization, tx deserialization/serialization
and input signing for ZCash and generalize those functions in the
client and server btc packages.

Implemenation notes
  1. zcashd does not support encrypted wallets. No passwords allowed.
  2. After starting the harness, it takes a few minutes for beta to
     get caught up.
  3. zcashd can take a very long time to get it's fee estimates primed.

buck54321 · 2022-04-07T16:06:28Z

Might want to keep an eye on https://github.com/zcash/lightwalletd too for a possible built-in light wallet. It doesn't appear to be suitable as is, but not certain.

chappjc · 2022-04-25T19:05:06Z

Will review shorty. Wanna rebase away the first two commits that are already merged (the first one without the ExchangeWalletFullNode override)?

martonp

Looks good, I was able to do a swap on simnet. Haven't tested with testnet yet. For some reason the dex client is constantly sending getbestblockhash, getnetworkinfo, and getblockchaininfo calls to the zcash client when I'm running on simnet. It seems a bit excessive.. there are at least 2-3 calls every second.

client/asset/btc/btc.go

martonp · 2022-05-06T13:42:58Z

client/asset/btc/rpcclient.go

+	serializeTx              func(*wire.MsgTx) ([]byte, error)
+	deserializeBlock         func([]byte) (*wire.MsgBlock, error)
+	hashTx                   func(*wire.MsgTx) *chainhash.Hash
+	numericGetRawTxRPC       bool


Could use a doc saying this rpc method has different args in zcash.

dex/networks/zec/block.go

dex/networks/zec/tx.go

client/asset/zec/sign.go

dex/testing/zec/harness.sh

chappjc

Some thoughts on sign.go to start

client/asset/zec/sign.go

chappjc · 2022-05-11T19:41:00Z

client/asset/zec/sign.go

+//   https://github.com/zcash/zips/blob/main/zip-0243.rst#specification
+//   which extends https://zips.z.cash/zip-0143#specification
+// See also implementation @
+//	https://github.com/iqoption/zecutil/blob/master/sign.go


This sign.go file needs a license comment at the top. It's unfortunate that zecutil completely threw away bchutil's license, and double unfortunate that libzec-go is gpl. I think with some minor rewriting and using our own code style and idioms to make this properly ours we're good though. It's a short function

client/asset/zec/sign.go

chappjc

dex/networks/zec

dex/networks/zec/addr.go

dex/networks/zec/block.go

dex/networks/zec/tx.go

go.mod

dex/networks/zec/params.go

dex/networks/zec/block_test.go

client/asset/btc/btc.go

buck54321 · 2022-05-14T13:58:24Z

Updated for NU5. I expect that ZCash won't be live on DEX when the upgrade goes live on May 31st, but we're ready-ish for the upgrade if it is. For a ZCash upgrade, an operator should be expected to shut down for the upgrade, since old version txs won't be accepted on the new chain, per this upgrade guide.

I'm syncing testnet and then I'll search for some NU5 txs to use for tests.

dex/networks/zec/tx.go

buck54321 · 2022-05-15T15:09:43Z

For some reason the dex client is constantly sending getbestblockhash, getnetworkinfo, and getblockchaininfo calls to the zcash client when I'm running on simnet. It seems a bit excessive.. there are at least 2-3 calls every second.

I put some feelers in, and it seems to me like the calls to getbestblockhash are our normal block polling. I'm not sure why we're seeing two log messages for each in the zcashd/bitcoind terminals. Might be worth looking into further, but I don't see any issues on the client side.

The getnetworkinfo is coming from our monitorPeers loop.

I'm not seeing a lot of getblockchaininfo, but that would be expected with a new block.

I do see some estimatesmartfee pretty regularly from (*Core).tick → (*Core).cacheRedemptionFeeSuggestion. Related: #1515.

chappjc

Looks great! 🚀 🔒 🎆

Just need a resolve.

JoeGruffins

Wow. Thats a lot of zcash. My eyes started to glaze over at tx.go

I've not been able to trade so far with :

2022-05-19 23:23:46.860 [ERR] CORE: notify: |ERROR| (order) Swap send error - Error encountered sending a swap output(s) worth 1.00000000 ZEC on order 117411dd - Order: 117411dd464a5b5f40e3fc40c32f5d45cf0fbc88d1fa38fca69ab9ec668b1090
2022-05-19 23:23:46.860 [ERR] CORE: 127.0.0.1:17273 tick: {swapMatches order 117411dd464a5b5f40e3fc40c32f5d45cf0fbc88d1fa38fca69ab9ec668b1090 - {error sending swap transaction: signing error: tx signing error: rawrequest (signrawtransaction) error: -22: TX decode failed, raw tx: 0500000001ce67eab2739d1db57d16dcc8457cd0f1028c424459a5052ef4d65ca59cf841150100000000ffffffff0100e1f5050000000017a914c83a74347ff867cade02a48197b39345b8635b018700000000}}

I also attempted to get the core simnet tests running but they failed. Can do in another pr though.

Noticed a couple things maybe unrelated to this pr.

If I start to make a ZEC wallet but I put in a password, it errors, which is fine. But if I close that box then I see this for the wallet:

And I can click on that box to unlock the wallet that isnt there.

Also, I was not able to pay the fee with bitcoin for the simnet tests. May be a problem with the simnet tests.

$ zcashd --version
Zcash Daemon version v5.0.0

dex/testing/zec/harness.sh

JoeGruffins · 2022-05-19T09:47:38Z

client/asset/zec/sign.go

+
+func validateSigHashType(v txscript.SigHashType) bool {
+	return v == txscript.SigHashAll || v == txscript.SigHashNone || v == txscript.SigHashSingle ||
+		v == txscript.SigHashAnyOneCanPay || v == 0x82 || v == 0x83


What are these last two hex?

I can't seem to figure it out, but I'm basing it on

Consensus rule: [NU5 onward] Any SIGHASH type encoding used in a version 5 transaction MUST be the
canonical encoding of one of the defined SIGHASH types, i.e. one of 0x01, 0x02, 0x03, 0x81, 0x82, or 0x83.

from https://zips.z.cash/protocol/protocol.pdf

Would that be the bit-or of SIGHASH_ANYONECANPAY with either SIGHASH_NONE or SIGHASH_SINGLE?

/** Signature hash types/flags */ enum { SIGHASH_ALL = 1, SIGHASH_NONE = 2, SIGHASH_SINGLE = 3, SIGHASH_ANYONECANPAY = 0x80, };

Now that I say that, I think we're missing 0x81 too (SIGHASH_ALL | SIGHASH_ANYONECANPAY)?

Oh, and just SigHashAnyOneCanPay isn't valid it seems. It's gotta be combined with one of the all/none/single options: https://github.com/zcash/zcash/blob/a45c2a2d4e3d387bf2dcb2e5808db2f6f7008799/src/script/interpreter.cpp#L1237-L1257

dex/networks/btc/clone.go

chappjc · 2022-05-19T14:57:56Z

Oh! zcash 5.0 was released a week ago. https://github.com/zcash/zcash/releases/tag/v5.0.0 I've been working with 4.6

go.mod

chappjc · 2022-05-19T18:45:29Z

How about https://zcashblockexplorer.com/ for the mainnet explorer? No external JS, displays fine with no JS, seems to be reference deployment of https://github.com/nighthawk-apps/zcash-explorer, has an onion address, NU5 tx and UA support

I don't see a good testnet option though. Maybe https://sochain.com/testnet/zec

dex/networks/zec/tx.go

chappjc · 2022-05-22T14:37:02Z

#1597 made some conflicts

JoeGruffins

This server log seems off: Blocks are too empty to calculate %!d(string=zec) median fees. Using no-competition rate.

Also, I can't seem to get refunds to happen. Is client checking the locktime correctly?

It looks like will need a couple more wallets for simnet tests just like doge. Would you like to add or shall I?

JoeGruffins · 2022-05-25T07:51:03Z

dex/networks/zec/tx.go

+// Tx can only produce tx hashes for unshielded transactions. Tx can only create
+// signature hashes for unshielded version 5 transactions.


Maybe first Tx can only produce tx hashes for unshielded transactions. is redundant.

JoeGruffins · 2022-05-25T08:08:11Z

dex/networks/zec/tx.go

-	ConsensusBranchHeartwood  = [4]byte{0x0B, 0x23, 0xB9, 0xF5} // 903000
-	ConsensusBranchCanopy     = [4]byte{0xA6, 0x75, 0xFF, 0xE9} // 1046400, tesnet: 1028500
-	ConsensusBranchNU5        = [4]byte{0xB4, 0xD0, 0xD6, 0xC2} // 1687104, testnet: 1842420
+	ConsensusBranchNU5 = [4]byte{0xB4, 0xD0, 0xD6, 0xC2} // 1687104, testnet: 1842420


Exported bytes probably better as a constant so that they cannot be mutated.

JoeGruffins · 2022-05-25T08:13:11Z

dex/networks/zec/tx.go

+	copy(b[:32], prevoutsDigest[:])
+	copy(b[32:64], seqDigest[:])
+	copy(b[64:], outputsDigest[:])
+	return blake2bHash(b, []byte("ZTxIdTranspaHash"))


Could make some of these strings that are reused constants to avoid typos. I don't see any typos currently.

chappjc

OK. Just one silly conflict with the new txscript.NewTxSigHashes function signature. Also @JoeGruffins review

chappjc · 2022-05-31T16:26:49Z

dex/testing/dcrdex/harness.sh

+            "base": "ZEC_simnet",
+            "quote": "BTC_simnet",
+            "lotSize": 100000000,
+            "rateStep": 100000,


This rate step is too high to reflect realistic rates. No a big deal on simnet, just not realistic.
Currently it's 0.00293146 BTC/ZEC, but I can only use 0.00x

chappjc · 2022-05-31T16:52:43Z

Also, I can't seem to get refunds to happen. Is client checking the locktime correctly?

@JoeGruffins is right. It can never refund. It seems the reason is that the getblockheader response does not include MedianTime, so it is always treated as 0 in (*baseWallet).LocktimeExpired and thus cannot ever be after contract expiry.

$  ./alpha getblockheader 0b5d2de16ab1f74674f4ad52f2096d90a293c61b9e552c49433a33fafc9541a9
{
  "hash": "0b5d2de16ab1f74674f4ad52f2096d90a293c61b9e552c49433a33fafc9541a9",
  "confirmations": 1,
  "height": 427,
  "version": 4,
  "merkleroot": "0abafddb3bd8053a633e64a50e7876b663371c328e7f31036524a23dccf5fb4e",
  "finalsaplingroot": "3e49b5f954aa9d3545bc6c37744661eea48d7c34e3000d82b7f0010c30f4c2fb",
  "time": 1654015360,
  "nonce": "00005ab9fbe1195127e17d82a2929c18e0c5037e44836c4a2d3ccdf7ef2c002e",
  "solution": "0c6203f9b1fd8888ab255da63a4435b577c910339676e1a30d45cc3cba2fd9f4b45e95b6",
  "bits": "200f0f0f",
  "difficulty": 1,
  "chainwork": "0000000000000000000000000000000000000000000000000000000000001c6c",
  "previousblockhash": "05a282582126e141821e885428d852e5756a471bacc122aec282392fcca4cc2b"
}

chappjc · 2022-05-31T17:02:54Z

I suppose we'll just have to pull the last 11 block headers to compute mediantime. The only RPCs I can see that might give it to us is getblockdeltas, which requires some crazy switches like -insightexplorer, or getblocktemplate->mintime-1, and it's probably a terrible idea to use mining functions.

BTW, refunds do work if I hack the expiry check:

2022-05-31 12:05:21.005 [DBG] CORE: Refundable match 5010ba73c1a55f3b5a7bebcbf08072e7c72a3f286ef3714af7da9ba5ffb471d9 for order 4d6f2b0b33d6040b943f020b5ce43930c978ff8ad4f606ef468ec5295b28d166 (Maker)
2022-05-31 12:05:21.005 [INF] CORE: Refunding zec contract 94565e2e3933e0a3c1d081b352bc3114e468eb72fe6b66a6a96e83f55a4258d8:0 for match 5010ba73c1a55f3b5a7bebcbf08072e7c72a3f286ef3714af7da9ba5ffb471d9 (no valid counterswap received from Taker)
2022-05-31 12:05:21.006 [TRC] CORE[zec]: Failed to get fee rate: fee could not be estimated
2022-05-31 12:05:21.007 [TRC] CORE[zec]: no 2-conf feeRate available: fee could not be estimated
2022-05-31 12:05:21.007 [TRC] CORE[zec]: feeRateWithFallback using caller's suggestion for fee rate, 10. Local estimate unavailable
2022-05-31 12:05:21.011 [WRN] CORE: notify: |WARNING| (order) Matches Refunded - Refunded 1.00000000 ZEC on order 4d6f2b0b - Order: 4d6f2b0b33d6040b943f020b5ce43930c978ff8ad4f606ef468ec5295b28d166

Implement block deserialization, tx deserialization/serialization and input signing for ZCash and generalize those functions in the client and server btc packages. Implemenation notes 1. zcashd does not support encrypted wallets. No passwords allowed. 2. After starting the harness, it takes a few minutes for beta to get caught up. 3. zcashd can take a very long time to get it's fee estimates primed.

All txs we create and sign must be version 5. Use new SIGHASH algos from ZIP-244. Move SIGHASH stuff to methods of dexzec.Tx. Add live test to scan testnet blocks looking for deserialization errors.

chappjc

Latest updates look good. Refunds work now.

[DBG] CORE: Refundable match 4092328257640b9120a1384debe25b6b185acec21b24f6e1711cd4a1d47649d6 for order 72f905d3dfc7a504b7fff74e79d80318295aa8f472107d19c9c65d9e52d25dd5 (Maker)
[INF] CORE: Refunding zec contract bd12551965edd38fbc1aa1c8cfd5fdec42f48bae92bff6b15f4494adf36db9d6:0 for match 4092328257640b9120a1384debe25b6b185acec21b24f6e1711cd4a1d47649d6 (no valid counterswap received from Taker)

buck54321 force-pushed the zec branch 2 times, most recently from 324335e to 0d2d75b Compare April 7, 2022 14:08

buck54321 force-pushed the zec branch 3 times, most recently from b563c4c to e999610 Compare April 10, 2022 04:36

buck54321 mentioned this pull request Apr 22, 2022

doge client: blockDeserializer #1596

Merged

buck54321 force-pushed the zec branch from e999610 to ea94ca0 Compare April 25, 2022 11:50

buck54321 force-pushed the zec branch from ea94ca0 to e03d543 Compare April 25, 2022 19:27

martonp reviewed May 8, 2022

View reviewed changes

buck54321 mentioned this pull request May 11, 2022

bch: switch to testnet4 and use custom address stringer #1606

Merged

chappjc reviewed May 11, 2022

View reviewed changes

chappjc requested changes May 12, 2022

View reviewed changes

dex/networks/zec/params.go Outdated Show resolved Hide resolved

dex/networks/zec/block_test.go Outdated Show resolved Hide resolved

client/asset/btc/btc.go Outdated Show resolved Hide resolved

buck54321 force-pushed the zec branch from 87c6af1 to 69cbf21 Compare May 14, 2022 13:53

chappjc reviewed May 14, 2022

View reviewed changes

dex/networks/zec/tx.go Show resolved Hide resolved

buck54321 force-pushed the zec branch 2 times, most recently from 9d20bcc to 2003811 Compare May 15, 2022 10:09

chappjc approved these changes May 17, 2022

View reviewed changes

JoeGruffins reviewed May 19, 2022

View reviewed changes

buck54321 force-pushed the zec branch from e507d57 to e6d296f Compare May 19, 2022 16:01

chappjc reviewed May 19, 2022

View reviewed changes

go.mod Outdated Show resolved Hide resolved

buck54321 force-pushed the zec branch from 3d2c1e9 to 35dd74f Compare May 20, 2022 21:08

chappjc requested changes May 21, 2022

View reviewed changes

dex/networks/zec/tx.go Outdated Show resolved Hide resolved

dex/networks/zec/tx.go Outdated Show resolved Hide resolved

dex/networks/zec/tx.go Outdated Show resolved Hide resolved

dex/networks/zec/tx.go Show resolved Hide resolved

buck54321 force-pushed the zec branch from 35dd74f to 977d9ed Compare May 25, 2022 02:42

chappjc added this to the 0.5 milestone May 25, 2022

JoeGruffins reviewed May 26, 2022

View reviewed changes

chappjc approved these changes May 30, 2022

View reviewed changes

buck54321 force-pushed the zec branch from 977d9ed to 15df5a5 Compare May 31, 2022 02:24

chappjc reviewed May 31, 2022

View reviewed changes

buck54321 force-pushed the zec branch 2 times, most recently from 6fa91e1 to 8dbf2c8 Compare June 11, 2022 00:53

buck54321 added 5 commits June 10, 2022 21:11

finish v5 upgrade

eef4c36

All txs we create and sign must be version 5. Use new SIGHASH algos from ZIP-244. Move SIGHASH stuff to methods of dexzec.Tx. Add live test to scan testnet blocks looking for deserialization errors.

Adapt median fee calcs and ShieldedIO for zec

0a20769

review followup

fc341d6

manual median times. more nodes

6bab295

buck54321 force-pushed the zec branch from 8dbf2c8 to ccf546d Compare June 11, 2022 04:10

pass core harness tests

ae584e6

buck54321 force-pushed the zec branch from ccf546d to ae584e6 Compare June 11, 2022 04:23

chappjc approved these changes Jun 13, 2022

View reviewed changes

chappjc merged commit f6cccaa into decred:master Jun 13, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

multi: add support for ZCash #1570

multi: add support for ZCash #1570

buck54321 commented Apr 7, 2022 •

edited

buck54321 commented Apr 7, 2022

chappjc commented Apr 25, 2022

martonp left a comment

martonp May 6, 2022

chappjc left a comment

chappjc May 11, 2022 •

edited

chappjc left a comment

buck54321 commented May 14, 2022

buck54321 commented May 15, 2022

chappjc left a comment

JoeGruffins left a comment •

edited

JoeGruffins May 19, 2022

buck54321 May 19, 2022

chappjc May 19, 2022 •

edited

chappjc May 19, 2022

chappjc commented May 19, 2022

chappjc commented May 19, 2022 •

edited

chappjc commented May 22, 2022

JoeGruffins left a comment

JoeGruffins May 25, 2022

JoeGruffins May 25, 2022

JoeGruffins May 25, 2022

chappjc left a comment •

edited

chappjc May 31, 2022 •

edited

chappjc commented May 31, 2022

chappjc commented May 31, 2022 •

edited

chappjc left a comment

		// Tx can only produce tx hashes for unshielded transactions. Tx can only create
		// signature hashes for unshielded version 5 transactions.

multi: add support for ZCash #1570

multi: add support for ZCash #1570

Conversation

buck54321 commented Apr 7, 2022 • edited

buck54321 commented Apr 7, 2022

chappjc commented Apr 25, 2022

martonp left a comment

Choose a reason for hiding this comment

martonp May 6, 2022

Choose a reason for hiding this comment

chappjc left a comment

Choose a reason for hiding this comment

chappjc May 11, 2022 • edited

Choose a reason for hiding this comment

chappjc left a comment

Choose a reason for hiding this comment

buck54321 commented May 14, 2022

buck54321 commented May 15, 2022

chappjc left a comment

Choose a reason for hiding this comment

JoeGruffins left a comment • edited

Choose a reason for hiding this comment

JoeGruffins May 19, 2022

Choose a reason for hiding this comment

buck54321 May 19, 2022

Choose a reason for hiding this comment

chappjc May 19, 2022 • edited

Choose a reason for hiding this comment

chappjc May 19, 2022

Choose a reason for hiding this comment

chappjc commented May 19, 2022

chappjc commented May 19, 2022 • edited

chappjc commented May 22, 2022

JoeGruffins left a comment

Choose a reason for hiding this comment

JoeGruffins May 25, 2022

Choose a reason for hiding this comment

JoeGruffins May 25, 2022

Choose a reason for hiding this comment

JoeGruffins May 25, 2022

Choose a reason for hiding this comment

chappjc left a comment • edited

Choose a reason for hiding this comment

chappjc May 31, 2022 • edited

Choose a reason for hiding this comment

chappjc commented May 31, 2022

chappjc commented May 31, 2022 • edited

chappjc left a comment

Choose a reason for hiding this comment

buck54321 commented Apr 7, 2022 •

edited

chappjc May 11, 2022 •

edited

JoeGruffins left a comment •

edited

chappjc May 19, 2022 •

edited

chappjc commented May 19, 2022 •

edited

chappjc left a comment •

edited

chappjc May 31, 2022 •

edited

chappjc commented May 31, 2022 •

edited