Support Koblitz-based and Keccak256-based custom witness verification (…

…#3209)

* Native: extend CryptoLib's verifyWithECDsa with hasher parameter

A port of
nspcc-dev/neo-go@1e2b438.

This commit contains minor protocol extension needed for custom
Koblitz-based verification scripts (an alternative to
#3205).

Replace native CryptoLib's verifyWithECDsa `curve` parameter by
`curveHash` parameter which is a enum over supported pairs of named
curves and hash functions. NamedCurve enum mark as deprecated and
replaced by NamedCurveHash with compatible behaviour.

Even though this change is a compatible extension of the protocol, it
changes the genesis state due to parameter renaming (CryptoLib's
manifest is changed). But we're going to resync chain in 3.7 release
anyway, so it's not a big deal.

Also, we need to check mainnet and testnet compatibility in case if
anyone has ever called verifyWithECDsa with 24 or 25 `curve` value.

Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>

* SmartContract: add extension to ScriptBuilder for System.Contract.Call

Group the set of common operations required to emit
System.Contract.Call appcall.

Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>

* Native: add an example of custom Koblitz signature verification

Koblitz-based and Keccak-based transaction witness verification for
single signature and multisignature ported from
nspcc-dev/neo-go#3425.

An alternative to #3205.

Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>

* SmartContract: make multisig koblitz easier to parse

1. Make prologue be exactly the same as regular CheckMultisig.
2. But instead of "SYSCALL System.Crypto.CheckMultisig" do INITSLOT and K check.
3. This makes all of the code from INITSLOT below be independent of N/M, so
   one can parse the script beginning in the same way CheckMultisig is parsed and
   then just compare the rest of it with some known-good blob.
4. The script becomes a tiny bit larger now, but properties above are too good.

Ported from
nspcc-dev/neo-go@34ee294.

Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>

* SmartContract: use ABORT in Koblitz multisig

Make the script a bit shorter. ABORTMSG would cost a bit more.

Ported from
nspcc-dev/neo-go@fb16891.
Ref.
nspcc-dev/neo-go#3425 (comment).

Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>

* SmartContract: reduce callflag scope for Koblitz verification scripts

All flag is too wide. A port of
nspcc-dev/neo-go@fe292f3.

Ref.
nspcc-dev/neo-go#3425 (comment).

Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>

* Native: add tests for CryptoLib's verifyWithECDsa

No functional changes, just add more unit-tests.

Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>

* Native: update NamedCurveHash values for Keccak256 hasher

Use 122 and 123 respectively for secp256k1Keccak256 and
secp256r1Keccak256.

Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>

* SmartContract: move EmitAppCallNoArgs to the testing code

We're not going to implement custom Koblitz witness generation at the
core, and thus, the only user of this API is testing code.

Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>

* Apply suggestions from code review

clean ut lines

* fix names

* Cryptography: cache ECDomainParameters for Secp256r1 and Secp256k1

Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>

* Update tests/Neo.UnitTests/SmartContract/Native/UT_CryptoLib.cs

* Update tests/Neo.UnitTests/SmartContract/Native/UT_CryptoLib.cs

* Update tests/Neo.UnitTests/SmartContract/Native/UT_CryptoLib.cs

* Update tests/Neo.UnitTests/SmartContract/Native/UT_CryptoLib.cs

* Update tests/Neo.UnitTests/SmartContract/Native/UT_CryptoLib.cs

* Update tests/Neo.UnitTests/SmartContract/Native/UT_CryptoLib.cs

* Update tests/Neo.UnitTests/SmartContract/Native/UT_CryptoLib.cs

* Update tests/Neo.UnitTests/SmartContract/Native/UT_CryptoLib.cs

* Update tests/Neo.UnitTests/SmartContract/Native/UT_CryptoLib.cs

* Update tests/Neo.UnitTests/SmartContract/Native/UT_CryptoLib.cs

* Update tests/Neo.UnitTests/SmartContract/Native/UT_CryptoLib.cs

* Update tests/Neo.UnitTests/SmartContract/Native/UT_CryptoLib.cs

* Update tests/Neo.UnitTests/SmartContract/Native/UT_CryptoLib.cs

* Update tests/Neo.UnitTests/SmartContract/Native/UT_CryptoLib.cs

* Update tests/Neo.UnitTests/SmartContract/Native/UT_CryptoLib.cs

---------

Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
Co-authored-by: Shargon <shargon@gmail.com>
Co-authored-by: Jimmy <jinghui@wayne.edu>

src/Neo/Cryptography/Crypto.cs

@@ -28,6 +28,17 @@ public static class Crypto
         private static readonly bool IsOSX = RuntimeInformation.IsOSPlatform(OSPlatform.OSX);
         private static readonly ECCurve secP256k1 = ECCurve.CreateFromFriendlyName("secP256k1");
         private static readonly X9ECParameters bouncySecp256k1 = Org.BouncyCastle.Asn1.Sec.SecNamedCurves.GetByName("secp256k1");
+        private static readonly X9ECParameters bouncySecp256r1 = Org.BouncyCastle.Asn1.Sec.SecNamedCurves.GetByName("secp256r1");
+
+        /// <summary>
+        /// Holds domain parameters for Secp256r1 elliptic curve.
+        /// </summary>
+        private static readonly ECDomainParameters secp256r1DomainParams = new ECDomainParameters(bouncySecp256r1.Curve, bouncySecp256r1.G, bouncySecp256r1.N, bouncySecp256r1.H);
+
+        /// <summary>
+        /// Holds domain parameters for Secp256k1 elliptic curve.
+        /// </summary>
+        private static readonly ECDomainParameters secp256k1DomainParams = new ECDomainParameters(bouncySecp256k1.Curve, bouncySecp256k1.G, bouncySecp256k1.N, bouncySecp256k1.H);
 
         /// <summary>
         /// Calculates the 160-bit hash value of the specified message.
@@ -50,22 +61,30 @@ public static byte[] Hash256(ReadOnlySpan<byte> message)
         }
 
         /// <summary>
-        /// Signs the specified message using the ECDSA algorithm.
+        /// Signs the specified message using the ECDSA algorithm and specified hash algorithm.
         /// </summary>
         /// <param name="message">The message to be signed.</param>
         /// <param name="priKey">The private key to be used.</param>
         /// <param name="ecCurve">The <see cref="ECC.ECCurve"/> curve of the signature, default is <see cref="ECC.ECCurve.Secp256r1"/>.</param>
+        /// <param name="hasher">The hash algorithm to hash the message, default is SHA256.</param>
         /// <returns>The ECDSA signature for the specified message.</returns>
-        public static byte[] Sign(byte[] message, byte[] priKey, ECC.ECCurve ecCurve = null)
+        public static byte[] Sign(byte[] message, byte[] priKey, ECC.ECCurve ecCurve = null, Hasher hasher = Hasher.SHA256)
         {
-            if (IsOSX && ecCurve == ECC.ECCurve.Secp256k1)
+            if (hasher == Hasher.Keccak256 || (IsOSX && ecCurve == ECC.ECCurve.Secp256k1))
             {
-                var domain = new ECDomainParameters(bouncySecp256k1.Curve, bouncySecp256k1.G, bouncySecp256k1.N, bouncySecp256k1.H);
+                var domain =
+                    ecCurve == null || ecCurve == ECC.ECCurve.Secp256r1 ? secp256r1DomainParams :
+                    ecCurve == ECC.ECCurve.Secp256k1 ? secp256k1DomainParams :
+                    throw new NotSupportedException(nameof(ecCurve));
                 var signer = new Org.BouncyCastle.Crypto.Signers.ECDsaSigner();
                 var privateKey = new BigInteger(1, priKey);
                 var priKeyParameters = new ECPrivateKeyParameters(privateKey, domain);
                 signer.Init(true, priKeyParameters);
-                var signature = signer.GenerateSignature(message.Sha256());
+                var messageHash =
+                    hasher == Hasher.SHA256 ? message.Sha256() :
+                    hasher == Hasher.Keccak256 ? message.Keccak256() :
+                    throw new NotSupportedException(nameof(hasher));
+                var signature = signer.GenerateSignature(messageHash);
 
                 var signatureBytes = new byte[64];
                 var rBytes = signature[0].ToByteArrayUnsigned();
@@ -87,24 +106,35 @@ public static byte[] Sign(byte[] message, byte[] priKey, ECC.ECCurve ecCurve = n
                 Curve = curve,
                 D = priKey,
             });
-            return ecdsa.SignData(message, HashAlgorithmName.SHA256);
+            var hashAlg =
+                hasher == Hasher.SHA256 ? HashAlgorithmName.SHA256 :
+                throw new NotSupportedException(nameof(hasher));
+            return ecdsa.SignData(message, hashAlg);
         }
 
         /// <summary>
-        /// Verifies that a digital signature is appropriate for the provided key and message.
+        /// Verifies that a digital signature is appropriate for the provided key, message and hash algorithm.
         /// </summary>
         /// <param name="message">The signed message.</param>
         /// <param name="signature">The signature to be verified.</param>
         /// <param name="pubkey">The public key to be used.</param>
+        /// <param name="hasher">The hash algorithm to be used to hash the message, the default is SHA256.</param>
         /// <returns><see langword="true"/> if the signature is valid; otherwise, <see langword="false"/>.</returns>
-        public static bool VerifySignature(ReadOnlySpan<byte> message, ReadOnlySpan<byte> signature, ECC.ECPoint pubkey)
+        public static bool VerifySignature(ReadOnlySpan<byte> message, ReadOnlySpan<byte> signature, ECC.ECPoint pubkey, Hasher hasher = Hasher.SHA256)
         {
             if (signature.Length != 64) return false;
 
-            if (IsOSX && pubkey.Curve == ECC.ECCurve.Secp256k1)
+            if (hasher == Hasher.Keccak256 || (IsOSX && pubkey.Curve == ECC.ECCurve.Secp256k1))
             {
-                var domain = new ECDomainParameters(bouncySecp256k1.Curve, bouncySecp256k1.G, bouncySecp256k1.N, bouncySecp256k1.H);
-                var point = bouncySecp256k1.Curve.CreatePoint(
+                var domain =
+                    pubkey.Curve == ECC.ECCurve.Secp256r1 ? secp256r1DomainParams :
+                    pubkey.Curve == ECC.ECCurve.Secp256k1 ? secp256k1DomainParams :
+                    throw new NotSupportedException(nameof(pubkey.Curve));
+                var curve =
+                    pubkey.Curve == ECC.ECCurve.Secp256r1 ? bouncySecp256r1.Curve :
+                    bouncySecp256k1.Curve;
+
+                var point = curve.CreatePoint(
                     new BigInteger(pubkey.X.Value.ToString()),
                     new BigInteger(pubkey.Y.Value.ToString()));
                 var pubKey = new ECPublicKeyParameters("ECDSA", point, domain);
@@ -115,11 +145,19 @@ public static bool VerifySignature(ReadOnlySpan<byte> message, ReadOnlySpan<byte
                 var r = new BigInteger(1, sig, 0, 32);
                 var s = new BigInteger(1, sig, 32, 32);
 
-                return signer.VerifySignature(message.Sha256(), r, s);
+                var messageHash =
+                    hasher == Hasher.SHA256 ? message.Sha256() :
+                    hasher == Hasher.Keccak256 ? message.Keccak256() :
+                    throw new NotSupportedException(nameof(hasher));
+
+                return signer.VerifySignature(messageHash, r, s);
             }
 
             var ecdsa = CreateECDsa(pubkey);
-            return ecdsa.VerifyData(message, signature, HashAlgorithmName.SHA256);
+            var hashAlg =
+                hasher == Hasher.SHA256 ? HashAlgorithmName.SHA256 :
+                throw new NotSupportedException(nameof(hasher));
+            return ecdsa.VerifyData(message, signature, hashAlg);
         }
 
         /// <summary>
@@ -153,16 +191,17 @@ public static ECDsa CreateECDsa(ECC.ECPoint pubkey)
         }
 
         /// <summary>
-        /// Verifies that a digital signature is appropriate for the provided key and message.
+        /// Verifies that a digital signature is appropriate for the provided key, curve, message and hasher.
         /// </summary>
         /// <param name="message">The signed message.</param>
         /// <param name="signature">The signature to be verified.</param>
         /// <param name="pubkey">The public key to be used.</param>
         /// <param name="curve">The curve to be used by the ECDSA algorithm.</param>
+        /// <param name="hasher">The hash algorithm to be used hash the message, the default is SHA256.</param>
         /// <returns><see langword="true"/> if the signature is valid; otherwise, <see langword="false"/>.</returns>
-        public static bool VerifySignature(ReadOnlySpan<byte> message, ReadOnlySpan<byte> signature, ReadOnlySpan<byte> pubkey, ECC.ECCurve curve)
+        public static bool VerifySignature(ReadOnlySpan<byte> message, ReadOnlySpan<byte> signature, ReadOnlySpan<byte> pubkey, ECC.ECCurve curve, Hasher hasher = Hasher.SHA256)
         {
-            return VerifySignature(message, signature, ECC.ECPoint.DecodePoint(pubkey, curve));
+            return VerifySignature(message, signature, ECC.ECPoint.DecodePoint(pubkey, curve), hasher);
         }
     }
 }

src/Neo/Cryptography/Hasher.cs

@@ -0,0 +1,29 @@
+// Copyright (C) 2015-2024 The Neo Project.
+//
+// Hasher.cs file belongs to the neo project and is free
+// software distributed under the MIT software license, see the
+// accompanying file LICENSE in the main directory of the
+// repository or http://www.opensource.org/licenses/mit-license.php
+// for more details.
+//
+// Redistribution and use in source and binary forms with or without
+// modifications are permitted.
+
+namespace Neo.Cryptography
+{
+    /// <summary>
+    /// Represents hash function identifiers supported by ECDSA message signature and verification.
+    /// </summary>
+    public enum Hasher : byte
+    {
+        /// <summary>
+        /// The SHA256 hash algorithm.
+        /// </summary>
+        SHA256 = 0x00,
+
+        /// <summary>
+        /// The Keccak256 hash algorithm.
+        /// </summary>
+        Keccak256 = 0x01,
+    }
+}

src/Neo/Cryptography/Helper.cs

@@ -12,6 +12,7 @@
 using Neo.IO;
 using Neo.Network.P2P.Payloads;
 using Neo.Wallets;
+using Org.BouncyCastle.Crypto.Digests;
 using Org.BouncyCastle.Crypto.Engines;
 using Org.BouncyCastle.Crypto.Modes;
 using Org.BouncyCastle.Crypto.Parameters;
@@ -153,6 +154,40 @@ public static byte[] Sha256(this Span<byte> value)
             return Sha256((ReadOnlySpan<byte>)value);
         }
 
+        /// <summary>
+        /// Computes the hash value for the specified byte array using the keccak256 algorithm.
+        /// </summary>
+        /// <param name="value">The input to compute the hash code for.</param>
+        /// <returns>The computed hash code.</returns>
+        public static byte[] Keccak256(this byte[] value)
+        {
+            KeccakDigest keccak = new(256);
+            keccak.BlockUpdate(value, 0, value.Length);
+            byte[] result = new byte[keccak.GetDigestSize()];
+            keccak.DoFinal(result, 0);
+            return result;
+        }
+
+        /// <summary>
+        /// Computes the hash value for the specified byte array using the keccak256 algorithm.
+        /// </summary>
+        /// <param name="value">The input to compute the hash code for.</param>
+        /// <returns>The computed hash code.</returns>
+        public static byte[] Keccak256(this ReadOnlySpan<byte> value)
+        {
+            return Keccak256(value.ToArray());
+        }
+
+        /// <summary>
+        /// Computes the hash value for the specified byte array using the keccak256 algorithm.
+        /// </summary>
+        /// <param name="value">The input to compute the hash code for.</param>
+        /// <returns>The computed hash code.</returns>
+        public static byte[] Keccak256(this Span<byte> value)
+        {
+            return Keccak256(value.ToArray());
+        }
+
         public static byte[] AES256Encrypt(this byte[] plainData, byte[] key, byte[] nonce, byte[] associatedData = null)
         {
             if (nonce.Length != 12) throw new ArgumentOutOfRangeException(nameof(nonce));

src/Neo/SmartContract/Native/CryptoLib.cs

@@ -11,7 +11,6 @@
 
 using Neo.Cryptography;
 using Neo.Cryptography.ECC;
-using Org.BouncyCastle.Crypto.Digests;
 using System;
 using System.Collections.Generic;
 
@@ -22,10 +21,12 @@ namespace Neo.SmartContract.Native
     /// </summary>
     public sealed partial class CryptoLib : NativeContract
     {
-        private static readonly Dictionary<NamedCurve, ECCurve> curves = new()
+        private static readonly Dictionary<NamedCurveHash, (ECCurve Curve, Hasher Hasher)> s_curves = new()
         {
-            [NamedCurve.secp256k1] = ECCurve.Secp256k1,
-            [NamedCurve.secp256r1] = ECCurve.Secp256r1
+            [NamedCurveHash.secp256k1SHA256] = (ECCurve.Secp256k1, Hasher.SHA256),
+            [NamedCurveHash.secp256r1SHA256] = (ECCurve.Secp256r1, Hasher.SHA256),
+            [NamedCurveHash.secp256k1Keccak256] = (ECCurve.Secp256k1, Hasher.Keccak256),
+            [NamedCurveHash.secp256r1Keccak256] = (ECCurve.Secp256r1, Hasher.Keccak256),
         };
 
         internal CryptoLib() : base() { }
@@ -73,11 +74,7 @@ public static byte[] Murmur32(byte[] data, uint seed)
         [ContractMethod(Hardfork.HF_Cockatrice, CpuFee = 1 << 15)]
         public static byte[] Keccak256(byte[] data)
         {
-            KeccakDigest keccak = new(256);
-            keccak.BlockUpdate(data, 0, data.Length);
-            byte[] result = new byte[keccak.GetDigestSize()];
-            keccak.DoFinal(result, 0);
-            return result;
+            return data.Keccak256();
         }
 
         /// <summary>
@@ -86,14 +83,15 @@ public static byte[] Keccak256(byte[] data)
         /// <param name="message">The signed message.</param>
         /// <param name="pubkey">The public key to be used.</param>
         /// <param name="signature">The signature to be verified.</param>
-        /// <param name="curve">The curve to be used by the ECDSA algorithm.</param>
+        /// <param name="curveHash">A pair of the curve to be used by the ECDSA algorithm and the hasher function to be used to hash message.</param>
         /// <returns><see langword="true"/> if the signature is valid; otherwise, <see langword="false"/>.</returns>
         [ContractMethod(CpuFee = 1 << 15)]
-        public static bool VerifyWithECDsa(byte[] message, byte[] pubkey, byte[] signature, NamedCurve curve)
+        public static bool VerifyWithECDsa(byte[] message, byte[] pubkey, byte[] signature, NamedCurveHash curveHash)
         {
             try
             {
-                return Crypto.VerifySignature(message, signature, pubkey, curves[curve]);
+                var ch = s_curves[curveHash];
+                return Crypto.VerifySignature(message, signature, pubkey, ch.Curve, ch.Hasher);
             }
             catch (ArgumentException)
             {

src/Neo/SmartContract/Native/NamedCurve.cs

@@ -9,24 +9,30 @@
 // Redistribution and use in source and binary forms with or without
 // modifications are permitted.
 
+using System;
+
 namespace Neo.SmartContract.Native
 {
     /// <summary>
-    /// Represents the named curve used in ECDSA.
+    /// Represents the named curve used in ECDSA. This enum is obsolete
+    /// and will be removed in future versions. Please, use an extended <see cref="NamedCurveHash"/> instead.
     /// </summary>
     /// <remarks>
     /// https://tools.ietf.org/html/rfc4492#section-5.1.1
     /// </remarks>
+    [Obsolete("NamedCurve enum is obsolete and will be removed in future versions. Please, use an extended NamedCurveHash instead.")]
     public enum NamedCurve : byte
     {
         /// <summary>
         /// The secp256k1 curve.
         /// </summary>
+        [Obsolete("secp256k1 value is obsolete and will be removed in future versions. Please, use NamedCurveHash.secp256k1SHA256 for compatible behaviour.")]
         secp256k1 = 22,
 
         /// <summary>
         /// The secp256r1 curve, which known as prime256v1 or nistP-256.
         /// </summary>
+        [Obsolete("secp256r1 value is obsolete and will be removed in future versions. Please, use NamedCurveHash.secp256r1SHA256 for compatible behaviour.")]
         secp256r1 = 23
     }
 }

src/Neo/SmartContract/Native/NamedCurveHash.cs

@@ -0,0 +1,40 @@
+// Copyright (C) 2015-2024 The Neo Project.
+//
+// NamedCurveHash.cs file belongs to the neo project and is free
+// software distributed under the MIT software license, see the
+// accompanying file LICENSE in the main directory of the
+// repository or http://www.opensource.org/licenses/mit-license.php
+// for more details.
+//
+// Redistribution and use in source and binary forms with or without
+// modifications are permitted.
+
+namespace Neo.SmartContract.Native
+{
+    /// <summary>
+    /// Represents a pair of the named curve used in ECDSA and a hash algorithm used to hash message.
+    /// This is a compatible extension of an obsolete <see cref="NamedCurve"/> enum.
+    /// </summary>
+    public enum NamedCurveHash : byte
+    {
+        /// <summary>
+        /// The secp256k1 curve and SHA256 hash algorithm.
+        /// </summary>
+        secp256k1SHA256 = 22,
+
+        /// <summary>
+        /// The secp256r1 curve, which known as prime256v1 or nistP-256, and SHA256 hash algorithm.
+        /// </summary>
+        secp256r1SHA256 = 23,
+
+        /// <summary>
+        /// The secp256k1 curve and Keccak256 hash algorithm.
+        /// </summary>
+        secp256k1Keccak256 = 122,
+
+        /// <summary>
+        /// The secp256r1 curve, which known as prime256v1 or nistP-256, and Keccak256 hash algorithm.
+        /// </summary>
+        secp256r1Keccak256 = 123
+    }
+}