From 3a989054b91a8bb563b66094d8f40f9791d624ce Mon Sep 17 00:00:00 2001 From: Shargon Date: Fri, 17 Apr 2020 14:59:02 +0200 Subject: [PATCH] Oracle policy contract (#1445) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * first commit * format * Simplify code * add UT and add some feature * Fix bug * Add summary and fix bug * format * format * little change * little change * little change * change Fee & Fix bug * Optimize * add config class * FiX UT * Format * Fix UT * Fix format and Optimize * Add some UT * Fix bug and add UT * Update src/neo/SmartContract/Native/Oracle/OraclePolicyContract.cs Co-Authored-By: Luchuan * little change * format * Update src/neo/SmartContract/Native/Oracle/OraclePolicyContract.cs Co-Authored-By: Shargon * add check * change validator * Add double initialization check * add UT * Add UT * change UT * fill UT * up ut and fix NeoToken.GetValidators * remove console * reset NEO.getValidators * fix GetOracleValidators * Rename variable * Update src/neo/SmartContract/Native/Oracle/OracleContract.cs Co-Authored-By: Erik Zhang * Update src/neo/SmartContract/Native/Oracle/OracleContract.cs Co-Authored-By: Erik Zhang * Fix bug * change logic * Fix name * change to const * Add double initialization check and fix ut * optimize Co-authored-by: Shargon Co-authored-by: Luchuan Co-authored-by: Luchuan Co-authored-by: Erik Zhang Oracle Syscall (#1399) * Plugins from List to array * Move false to init * Fix UT * Refactor * UT * Change HTTP to HTTPS * UT Syscall * Remove Version * Remove HTTP from the syscall name * Check schema * Rename * Update src/neo/Oracle/OracleRequest.cs Co-Authored-By: Erik Zhang * Fix abstract * Rename * Remove Body * Change order of members * Rename Neo.Oracle.Get Co-authored-by: Vitor Nazário Coelho Co-authored-by: erikzhang Co-authored-by: Belane Oracle Syscall (#1399) * Plugins from List to array * Move false to init * Fix UT * Refactor * UT * Change HTTP to HTTPS * UT Syscall * Remove Version * Remove HTTP from the syscall name * Check schema * Rename * Update src/neo/Oracle/OracleRequest.cs Co-Authored-By: Erik Zhang * Fix abstract * Rename * Remove Body * Change order of members * Rename Neo.Oracle.Get Co-authored-by: Vitor Nazário Coelho Co-authored-by: erikzhang Co-authored-by: Belane Rename OracleResult->OracleResponse (#1516) Removed hash from the response Oracle Syscall (#1399) * Plugins from List to array * Move false to init * Fix UT * Refactor * UT * Change HTTP to HTTPS * UT Syscall * Remove Version * Remove HTTP from the syscall name * Check schema * Rename * Update src/neo/Oracle/OracleRequest.cs Co-Authored-By: Erik Zhang * Fix abstract * Rename * Remove Body * Change order of members * Rename Neo.Oracle.Get Co-authored-by: Vitor Nazário Coelho Co-authored-by: erikzhang Co-authored-by: Belane Rename OracleResult->OracleResponse (#1516) Removed hash from the response Oracle Service (#1517) * Oracle Service * Fix fee * Optimize load configuration * Fix akka message * Optimize * dotnet format * Fix sort * Oracle Service * Fix fee * Optimize load configuration * Fix akka message * Optimize * dotnet format * Fix sort * Apply rename * Small changes * Add UT * Fix Start Stop * dotnet-format * Migrate to HTTPS test * Change AKKA message * Tommo recomendations * Add comunication between OracleTx and UserTx * Advance of signature * Fix UT * dotnet-format * Advance of signature Fix UT dotnet-format * Fix UT * Remove duplica method * Reduce timeout time * Add log for fix UT * Add filters * Change port to 443 * Oracle Service * Fix fee * Optimize load configuration * Fix akka message * Optimize * dotnet format * Fix sort * Oracle Service * Fix fee * Optimize load configuration * Fix akka message * Optimize * dotnet format * Fix sort * Apply rename * Small changes * Add UT * Fix Start Stop * dotnet-format * Migrate to HTTPS test * Change AKKA message * Tommo recomendations * Add comunication between OracleTx and UserTx * Advance of signature * Fix UT * dotnet-format * Fix UT * Remove duplica method * Reduce timeout time * Add log for fix UT * Add filters * Change port to 443 * Merge changes * Change SSL Protocol * Merge remote-tracking branch 'neo-project/oracle-service' into oracle-service * Add UT certificate * Clean code * Add UT fix GetOracleMultiSigAddress (#1538) * fix GetOracleMultiSigAddress * First lowercase Co-authored-by: Shargon Oracle: Tx comunication (#1540) * Tx comunication * Some fixes Transaction Version (#1539) Oracle: Allow wallets to add assert for the oracle response (#1541) * Add assert for each oracle request * Change to goto * Improve wallet asserts * Optimize * Fix URL type * dotnet format * Oracle: Tx comunication (#1540) * Tx comunication * Some fixes * Optimize OracleExecutionCache * Add assert for each oracle request * Change to goto * Improve wallet asserts * Optimize * Fix URL type * dotnet format * Optimize OracleExecutionCache * MakeTransaction UT Oracle: Set the right Tx version (#1542) * Set the right Tx version * Remove Error types from TX and VM * Neo.Oracle.Get now only return the result or null Oracle: P2P Response Signature Message (#1553) * Oracle Response Signature P2P * caches * rename private fields * remove comments * remove more comments and fix size * Clean code Co-authored-by: Shargon Oracle: OracleService (#1555) * Start pool * Prepare for neo-node * Fix double call * Prepare pool * Fix * Join oracle pool with oracle service * dotnet-format * Try to fix UT * UT pass * Fix UT * Fix p2p message * Unify collections * Relay p2p message * Clean code and fixes * Send tx to OracleService * RequestTx will wait for ResponseTx * Rename * Remove supervisor * Allow to put request, and response in the same block * Check the sender of OracleResponses * Remove OracleResponse message in OracleService * Organize code * Fix typo * Remove task count TODO * Remove Thread-safe TODO * Remove TODO * ResponseItem changes * Read the oracle contract on my response Receive oraclePayload * Clean code * Add Stop message * Save only one response for PublicKey/RequestTx Improve sorting response pool * Improve sort * Group sort methods in the same region * Ask for the request TX if I don't have it * Reorder code * Rename * First oracle TX Oracle: Remove some TODOs (#1574) * Remove some todos * Remove one method * Reuse ManualWitness * dotnet format * Allow both tx in the same block without change order or fee * Fix p2p message serialization * Fix p2p * Fixes and IInventory * Fix akka message * Fix ut Fix conflicts Remove double call --- .gitignore | 1 + src/neo/Consensus/ConsensusContext.cs | 2 +- src/neo/Consensus/ConsensusService.cs | 2 +- src/neo/Ledger/Blockchain.cs | 24 +- src/neo/Ledger/MemoryPool.cs | 48 +- src/neo/Ledger/PoolItem.cs | 49 ++ src/neo/Ledger/SortedBlockingCollection.cs | 84 ++ src/neo/Ledger/SortedConcurrentDictionary.cs | 210 +++++ src/neo/NeoSystem.cs | 18 + src/neo/Network/P2P/MessageCommand.cs | 4 +- src/neo/Network/P2P/Payloads/InventoryType.cs | 3 +- src/neo/Network/P2P/Payloads/OraclePayload.cs | 139 +++ .../P2P/Payloads/OracleResponseSignature.cs | 110 +++ src/neo/Network/P2P/Payloads/Transaction.cs | 61 +- .../P2P/Payloads/TransactionVersion.cs | 9 + src/neo/Network/P2P/ProtocolHandler.cs | 451 ++++++++++ .../Network/P2P/RemoteNode.ProtocolHandler.cs | 12 +- src/neo/Network/P2P/TaskManager.cs | 4 +- src/neo/Oracle/ManualWitness.cs | 33 + src/neo/Oracle/OracleExecutionCache.cs | 173 ++++ src/neo/Oracle/OracleFilter.cs | 15 + src/neo/Oracle/OracleRequest.cs | 36 + src/neo/Oracle/OracleRequestType.cs | 7 + src/neo/Oracle/OracleResponse.cs | 154 ++++ src/neo/Oracle/OracleResultError.cs | 40 + src/neo/Oracle/OracleService.cs | 816 ++++++++++++++++++ src/neo/Oracle/OracleWalletBehaviour.cs | 20 + src/neo/Oracle/Protocols/Https/HttpMethod.cs | 7 + .../Protocols/Https/OracleHttpsProtocol.cs | 188 ++++ .../Protocols/Https/OracleHttpsRequest.cs | 60 ++ src/neo/SmartContract/ApplicationEngine.cs | 13 +- .../SmartContract/InteropService.Oracle.cs | 119 +++ .../SmartContract/Native/NativeContract.cs | 2 + .../SmartContract/Native/Oracle/HttpConfig.cs | 7 + .../Native/Oracle/OracleContract.cs | 298 +++++++ src/neo/Wallets/Wallet.cs | 87 +- src/neo/neo.csproj | 4 +- .../Nep5NativeContractExtensions.cs | 32 +- tests/neo.UnitTests/Ledger/UT_MemoryPool.cs | 26 +- .../Protocols/Https/UT_OracleHTTPRequest.cs | 44 + .../Protocols/Https/UT_OracleHttpsProtocol.cs | 60 ++ .../Oracle/UT_OracleExecutionCache.cs | 143 +++ .../neo.UnitTests/Oracle/UT_OracleResponse.cs | 53 ++ .../neo.UnitTests/Oracle/UT_OracleService.cs | 420 +++++++++ .../Native/Oracle/UT_OracleContract.cs | 355 ++++++++ .../Native/Tokens/UT_NeoToken.cs | 5 +- .../SmartContract/Native/UT_PolicyContract.cs | 24 +- .../SmartContract/UT_Syscalls.cs | 6 +- tests/neo.UnitTests/UT-cert.pfx | Bin 0 -> 2469 bytes tests/neo.UnitTests/neo.UnitTests.csproj | 7 +- 50 files changed, 4388 insertions(+), 97 deletions(-) create mode 100644 src/neo/Ledger/SortedBlockingCollection.cs create mode 100644 src/neo/Ledger/SortedConcurrentDictionary.cs create mode 100644 src/neo/Network/P2P/Payloads/OraclePayload.cs create mode 100644 src/neo/Network/P2P/Payloads/OracleResponseSignature.cs create mode 100644 src/neo/Network/P2P/Payloads/TransactionVersion.cs create mode 100644 src/neo/Network/P2P/ProtocolHandler.cs create mode 100644 src/neo/Oracle/ManualWitness.cs create mode 100644 src/neo/Oracle/OracleExecutionCache.cs create mode 100644 src/neo/Oracle/OracleFilter.cs create mode 100644 src/neo/Oracle/OracleRequest.cs create mode 100644 src/neo/Oracle/OracleRequestType.cs create mode 100644 src/neo/Oracle/OracleResponse.cs create mode 100644 src/neo/Oracle/OracleResultError.cs create mode 100644 src/neo/Oracle/OracleService.cs create mode 100644 src/neo/Oracle/OracleWalletBehaviour.cs create mode 100644 src/neo/Oracle/Protocols/Https/HttpMethod.cs create mode 100644 src/neo/Oracle/Protocols/Https/OracleHttpsProtocol.cs create mode 100644 src/neo/Oracle/Protocols/Https/OracleHttpsRequest.cs create mode 100644 src/neo/SmartContract/InteropService.Oracle.cs create mode 100644 src/neo/SmartContract/Native/Oracle/HttpConfig.cs create mode 100644 src/neo/SmartContract/Native/Oracle/OracleContract.cs create mode 100644 tests/neo.UnitTests/Oracle/Protocols/Https/UT_OracleHTTPRequest.cs create mode 100644 tests/neo.UnitTests/Oracle/Protocols/Https/UT_OracleHttpsProtocol.cs create mode 100644 tests/neo.UnitTests/Oracle/UT_OracleExecutionCache.cs create mode 100644 tests/neo.UnitTests/Oracle/UT_OracleResponse.cs create mode 100644 tests/neo.UnitTests/Oracle/UT_OracleService.cs create mode 100644 tests/neo.UnitTests/SmartContract/Native/Oracle/UT_OracleContract.cs create mode 100644 tests/neo.UnitTests/UT-cert.pfx diff --git a/.gitignore b/.gitignore index cb49b769de..6f485d660d 100644 --- a/.gitignore +++ b/.gitignore @@ -192,6 +192,7 @@ ClientBin/ *.dbmdl *.dbproj.schemaview *.pfx +!UT-cert.pfx *.publishsettings node_modules/ orleans.codegen.cs diff --git a/src/neo/Consensus/ConsensusContext.cs b/src/neo/Consensus/ConsensusContext.cs index f7f500f3dc..05328df0e6 100644 --- a/src/neo/Consensus/ConsensusContext.cs +++ b/src/neo/Consensus/ConsensusContext.cs @@ -283,7 +283,7 @@ public ConsensusPayload MakePrepareRequest() Span buffer = stackalloc byte[sizeof(ulong)]; random.NextBytes(buffer); Block.ConsensusData.Nonce = BitConverter.ToUInt64(buffer); - EnsureMaxBlockSize(Blockchain.Singleton.MemPool.GetSortedVerifiedTransactions()); + EnsureMaxBlockSize(Blockchain.Singleton.MemPool.GetSortedVerifiedTransactions(Snapshot)); Block.Timestamp = Math.Max(TimeProvider.Current.UtcNow.ToTimestampMS(), PrevHeader.Timestamp + 1); return PreparationPayloads[MyIndex] = MakeSignedPayload(new PrepareRequest diff --git a/src/neo/Consensus/ConsensusService.cs b/src/neo/Consensus/ConsensusService.cs index 9e59ba7283..c1ce19775f 100644 --- a/src/neo/Consensus/ConsensusService.cs +++ b/src/neo/Consensus/ConsensusService.cs @@ -443,7 +443,7 @@ private void OnPrepareRequestReceived(ConsensusPayload payload, PrepareRequest m return; } - Dictionary mempoolVerified = Blockchain.Singleton.MemPool.GetVerifiedTransactions().ToDictionary(p => p.Hash); + Dictionary mempoolVerified = Blockchain.Singleton.MemPool.GetVerifiedTransactions(context.Snapshot).ToDictionary(p => p.Hash); List unverified = new List(); foreach (UInt256 hash in context.TransactionHashes) { diff --git a/src/neo/Ledger/Blockchain.cs b/src/neo/Ledger/Blockchain.cs index fd76d1ebca..90a060c9d5 100644 --- a/src/neo/Ledger/Blockchain.cs +++ b/src/neo/Ledger/Blockchain.cs @@ -64,7 +64,7 @@ public class RelayResult { public IInventory Inventory; public VerifyResult Resu private uint stored_header_count = 0; private readonly Dictionary block_cache = new Dictionary(); private readonly Dictionary> block_cache_unverified = new Dictionary>(); - internal readonly RelayCache ConsensusRelayCache = new RelayCache(100); + internal readonly RelayCache RelayCache = new RelayCache(100); private SnapshotView currentSnapshot; public IStore Store { get; } @@ -301,6 +301,7 @@ private void OnInventory(IInventory inventory, bool relay = true) Block block => OnNewBlock(block), Transaction transaction => OnNewTransaction(transaction), ConsensusPayload payload => OnNewConsensus(payload), + OraclePayload payload => OnNewOracle(payload), _ => VerifyResult.Unknown } }; @@ -393,7 +394,15 @@ private VerifyResult OnNewConsensus(ConsensusPayload payload) { if (!payload.Verify(currentSnapshot)) return VerifyResult.Invalid; system.Consensus?.Tell(payload); - ConsensusRelayCache.Add(payload); + RelayCache.Add(payload); + return VerifyResult.Succeed; + } + + private VerifyResult OnNewOracle(OraclePayload payload) + { + if (!payload.Verify(currentSnapshot)) return VerifyResult.Invalid; + system.Oracle?.Tell(payload); + RelayCache.Add(payload); return VerifyResult.Succeed; } @@ -425,6 +434,14 @@ private VerifyResult OnNewTransaction(Transaction transaction) VerifyResult reason = transaction.Verify(currentSnapshot, MemPool.SendersFeeMonitor.GetSenderFee(transaction.Sender)); if (reason != VerifyResult.Succeed) return reason; if (!MemPool.TryAdd(transaction.Hash, transaction)) return VerifyResult.OutOfMemory; + + if (transaction.Version == TransactionVersion.OracleRequest) + { + // Oracle Service only need the OracleRequests + + system.Oracle?.Tell(transaction); + } + return VerifyResult.Succeed; } @@ -463,6 +480,9 @@ protected override void OnReceive(object message) case ConsensusPayload payload: OnInventory(payload); break; + case OraclePayload oracle: + OnInventory(oracle); + break; case Idle _: if (MemPool.ReVerifyTopUnverifiedTransactionsIfNeeded(MaxTxToReverifyPerIdle, currentSnapshot)) Self.Tell(Idle.Instance, ActorRefs.NoSender); diff --git a/src/neo/Ledger/MemoryPool.cs b/src/neo/Ledger/MemoryPool.cs index 62466d65e8..6bd0f0dee4 100644 --- a/src/neo/Ledger/MemoryPool.cs +++ b/src/neo/Ledger/MemoryPool.cs @@ -27,7 +27,6 @@ public class MemoryPool : IReadOnlyCollection private readonly NeoSystem _system; - // /// /// Guarantees consistency of the pool data structures. /// @@ -169,17 +168,35 @@ public IEnumerator GetEnumerator() IEnumerator IEnumerable.GetEnumerator() => GetEnumerator(); - public IEnumerable GetVerifiedTransactions() + public IEnumerable GetVerifiedTransactions(StoreView snapshot) { + Transaction[] ret; + var oracle = new PoolItem.DelayState(); + _txRwLock.EnterReadLock(); try { - return _unsortedTransactions.Select(p => p.Value.Tx).ToArray(); + ret = _unsortedTransactions + .Where(u => u.Value.IsReady(snapshot, oracle)) + .Select(p => p.Value.Tx) + .ToArray(); } finally { _txRwLock.ExitReadLock(); } + + // Fetch transactions + + foreach (var tx in ret) + { + yield return tx; + } + foreach (var delayed in oracle.Delayed) + { + if (oracle.Allowed.Contains(delayed.Hash)) + yield return delayed; + } } public void GetVerifiedAndUnverifiedTransactions(out IEnumerable verifiedTransactions, @@ -197,17 +214,36 @@ public IEnumerable GetVerifiedTransactions() } } - public IEnumerable GetSortedVerifiedTransactions() + public IEnumerable GetSortedVerifiedTransactions(StoreView snapshot) { + Transaction[] ret; + var oracle = new PoolItem.DelayState(); + _txRwLock.EnterReadLock(); try { - return _sortedTransactions.Reverse().Select(p => p.Tx).ToArray(); + ret = _sortedTransactions + .Reverse() + .Where(u => u.IsReady(snapshot, oracle)) + .Select(p => p.Tx) + .ToArray(); } finally { _txRwLock.ExitReadLock(); } + + // Fetch transactions + + foreach (var tx in ret) + { + yield return tx; + } + foreach (var delayed in oracle.Delayed) + { + if (oracle.Allowed.Contains(delayed.Hash)) + yield return delayed; + } } [MethodImpl(MethodImplOptions.AggressiveInlining)] @@ -239,7 +275,7 @@ private PoolItem GetLowestFeeTransaction(out Dictionary unsor } finally { - unsortedTxPool = Object.ReferenceEquals(sortedPool, _unverifiedSortedTransactions) + unsortedTxPool = ReferenceEquals(sortedPool, _unverifiedSortedTransactions) ? _unverifiedTransactions : _unsortedTransactions; } } diff --git a/src/neo/Ledger/PoolItem.cs b/src/neo/Ledger/PoolItem.cs index 45316d04db..3f5608eaa6 100644 --- a/src/neo/Ledger/PoolItem.cs +++ b/src/neo/Ledger/PoolItem.cs @@ -1,5 +1,8 @@ using Neo.Network.P2P.Payloads; +using Neo.Persistence; +using Neo.SmartContract.Native; using System; +using System.Collections.Generic; namespace Neo.Ledger { @@ -33,6 +36,52 @@ internal PoolItem(Transaction tx) LastBroadcastTimestamp = Timestamp; } + internal class DelayState + { + public HashSet Allowed = new HashSet(); + public HashSet Delayed = new HashSet(); + } + + public bool IsReady(StoreView snapshot, DelayState state) + { + switch (Tx.Version) + { + case TransactionVersion.OracleRequest: + { + if (state.Allowed.Remove(Tx.Hash)) + { + // The response was already fetched, we can put request and response in the same block + + return true; + } + else + { + if (NativeContract.Oracle.ContainsResponse(snapshot, Tx.Hash)) + { + // The response it's waiting to be consumed (block+n) + + return true; + } + else + { + // If the response it's in the pool it's located after the request + // We save the request in order to put after the response + + state.Delayed.Add(Tx); + return false; + } + } + } + case TransactionVersion.OracleResponse: + { + state.Allowed.Add(Tx.OracleRequestTx); + break; + } + } + + return true; + } + public int CompareTo(Transaction otherTx) { if (otherTx == null) return 1; diff --git a/src/neo/Ledger/SortedBlockingCollection.cs b/src/neo/Ledger/SortedBlockingCollection.cs new file mode 100644 index 0000000000..519dca18bd --- /dev/null +++ b/src/neo/Ledger/SortedBlockingCollection.cs @@ -0,0 +1,84 @@ +using System.Collections.Concurrent; +using System.Collections.Generic; +using System.Threading; + +namespace Neo.Ledger +{ + public class SortedBlockingCollection + { + /// + /// _oracleTasks will consume from this pool + /// + private readonly BlockingCollection _asyncPool = new BlockingCollection(); + + /// + /// Queue + /// + private readonly SortedConcurrentDictionary _queue; + + /// + /// Constructor + /// + /// Comparer + /// Capacity + public SortedBlockingCollection(IComparer> comparer, int capacity) + { + _queue = new SortedConcurrentDictionary(comparer, capacity); + } + + /// + /// Add entry + /// + /// Key + /// Value + public void Add(TKey key, TValue value) + { + if (_queue.TryAdd(key, value) && _asyncPool.Count <= 0) + { + Pop(); + } + } + + /// + /// Clear + /// + public void Clear() + { + _queue.Clear(); + + while (_asyncPool.Count > 0) + { + _asyncPool.TryTake(out _); + } + } + + /// + /// Get consuming enumerable + /// + /// Token + public IEnumerable GetConsumingEnumerable(CancellationToken token) + { + foreach (var entry in _asyncPool.GetConsumingEnumerable(token)) + { + // Prepare other item in _asyncPool + + Pop(); + + // Iterate items + + yield return entry; + } + } + + /// + /// Move one item from the sorted queue to _asyncPool, this will ensure that the threads process the entries according to the priority + /// + private void Pop() + { + if (_queue.TryPop(out var entry)) + { + _asyncPool.Add(entry); + } + } + } +} diff --git a/src/neo/Ledger/SortedConcurrentDictionary.cs b/src/neo/Ledger/SortedConcurrentDictionary.cs new file mode 100644 index 0000000000..5174f340d9 --- /dev/null +++ b/src/neo/Ledger/SortedConcurrentDictionary.cs @@ -0,0 +1,210 @@ +using System; +using System.Collections; +using System.Collections.Generic; +using System.Threading; + +namespace Neo.Ledger +{ + public class SortedConcurrentDictionary : IEnumerable> + { + private int _isDirty = 0; + + private readonly Dictionary _keys; + private readonly List> _sortedValues; + private readonly IComparer> _comparer; + + public event EventHandler> OnTrimEnd; + + private readonly object _lock = new object(); + + /// + /// Count + /// + public int Count + { + get + { + lock (_lock) + { + return _sortedValues.Count; + } + } + } + + /// + /// Capacity + /// + public int Capacity { get; } + + /// + /// Constructor + /// + /// Comparer + /// Capacity + public SortedConcurrentDictionary(IComparer> comparer, int capacity) + { + Capacity = Math.Max(1, capacity); + + _comparer = comparer; + _sortedValues = new List>(); + _keys = new Dictionary(); + } + + /// + /// Try to get a value + /// + /// Key + /// Value + /// True if was found + public bool TryGetValue(TKey key, out TValue value) + { + lock (_lock) + { + return _keys.TryGetValue(key, out value); + } + } + + /// + /// Try to get value and add new one if not found + /// + /// Key + /// Value + /// New value if it was not found + /// True if was getted or added + public bool TryGetValue(TKey key, out TValue value, TValue add) + { + lock (_lock) + { + if (!_keys.TryGetValue(key, out value)) + { + value = default; + return TryAdd(key, add); + } + } + + return true; + } + + + public bool TryRemove(TKey key, out TValue value) + { + lock (_lock) + { + if (_keys.Remove(key, out value)) + { + _sortedValues.RemoveAll(u => u.Key.Equals(key)); + return true; + } + } + + return false; + } + + public bool TryAdd(TKey key, TValue value) + { + lock (_lock) + { + if (_keys.TryAdd(key, value)) + { + Interlocked.Exchange(ref _isDirty, 0x01); + _sortedValues.Add(new KeyValuePair(key, value)); + + if (_sortedValues.Count > Capacity) + { + // Trim the last element (sorted) + + Sort(); + + var index = _sortedValues.Count - 1; + var last = _sortedValues[index]; + + if (_keys.Remove(last.Key)) + { + _sortedValues.RemoveAt(index); + + // Call the event + + OnTrimEnd?.Invoke(this, last); + } + } + + return true; + } + } + + return false; + } + + public void Set(TKey key, TValue value) + { + lock (_lock) + { + TryRemove(key, out _); + TryAdd(key, value); + } + } + + public void Clear() + { + lock (_lock) + { + _sortedValues.Clear(); + _keys.Clear(); + } + } + + public bool TryPop(out TValue value) + { + lock (_lock) + { + if (_sortedValues.Count > 0) + { + Sort(); + + var entry = _sortedValues[0]; + _sortedValues.RemoveAt(0); + _keys.Remove(entry.Key); + + value = entry.Value; + return true; + } + } + + value = default; + return false; + } + + #region Get sorted list + + /// + /// Sort (thread not safe) + /// + private void Sort() + { + if (_comparer != null && Interlocked.Exchange(ref _isDirty, 0x00) == 0x01) + { + _sortedValues.Sort(_comparer); + } + } + + public IEnumerator> GetEnumerator() + { + KeyValuePair[] array; + + lock (_lock) + { + Sort(); + array = _sortedValues.ToArray(); + } + + return (IEnumerator>)array.GetEnumerator(); + } + + IEnumerator IEnumerable.GetEnumerator() + { + return this.GetEnumerator(); + } + + #endregion + } +} diff --git a/src/neo/NeoSystem.cs b/src/neo/NeoSystem.cs index 301df2c5d6..e4b689ed76 100644 --- a/src/neo/NeoSystem.cs +++ b/src/neo/NeoSystem.cs @@ -2,6 +2,7 @@ using Neo.Consensus; using Neo.Ledger; using Neo.Network.P2P; +using Neo.Oracle; using Neo.Persistence; using Neo.Plugins; using Neo.Wallets; @@ -21,6 +22,7 @@ public class NeoSystem : IDisposable public IActorRef LocalNode { get; } internal IActorRef TaskManager { get; } public IActorRef Consensus { get; private set; } + public IActorRef Oracle { get; private set; } private readonly IStore store; private ChannelsConfig start_message = null; @@ -70,10 +72,26 @@ internal void ResumeNodeStartup() public void StartConsensus(Wallet wallet, IStore consensus_store = null, bool ignoreRecoveryLogs = false) { + if (Consensus != null) return; Consensus = ActorSystem.ActorOf(ConsensusService.Props(this.LocalNode, this.TaskManager, consensus_store ?? store, wallet)); Consensus.Tell(new ConsensusService.Start { IgnoreRecoveryLogs = ignoreRecoveryLogs }, Blockchain); } + public void StartOracle(Wallet wallet, byte numberOfTasks = 4) + { + if (Oracle != null) return; + if (numberOfTasks == 0) throw new ArgumentException("The task count must be greater than 0"); + Oracle = ActorSystem.ActorOf(OracleService.Props(this, this.LocalNode, wallet)); + Oracle.Tell(new OracleService.StartMessage() { NumberOfTasks = numberOfTasks }, Blockchain); + } + + public void StopOracle() + { + if (Oracle == null) return; + Oracle.Tell(new OracleService.StopMessage(), Blockchain); + Oracle = null; + } + public void StartNode(ChannelsConfig config) { start_message = config; diff --git a/src/neo/Network/P2P/MessageCommand.cs b/src/neo/Network/P2P/MessageCommand.cs index ccddacfba2..34d39497f6 100644 --- a/src/neo/Network/P2P/MessageCommand.cs +++ b/src/neo/Network/P2P/MessageCommand.cs @@ -40,6 +40,8 @@ public enum MessageCommand : byte Block = 0x2c, [ReflectionCache(typeof(ConsensusPayload))] Consensus = 0x2d, + [ReflectionCache(typeof(OraclePayload))] + Oracle = 0x2e, Reject = 0x2f, //SPV protocol @@ -52,6 +54,6 @@ public enum MessageCommand : byte MerkleBlock = 0x38, //others - Alert = 0x40, + Alert = 0x40 } } diff --git a/src/neo/Network/P2P/Payloads/InventoryType.cs b/src/neo/Network/P2P/Payloads/InventoryType.cs index 0a1b831d12..6bc5d7bd26 100644 --- a/src/neo/Network/P2P/Payloads/InventoryType.cs +++ b/src/neo/Network/P2P/Payloads/InventoryType.cs @@ -4,6 +4,7 @@ public enum InventoryType : byte { TX = MessageCommand.Transaction, Block = MessageCommand.Block, - Consensus = MessageCommand.Consensus + Consensus = MessageCommand.Consensus, + Oracle = MessageCommand.Oracle, } } diff --git a/src/neo/Network/P2P/Payloads/OraclePayload.cs b/src/neo/Network/P2P/Payloads/OraclePayload.cs new file mode 100644 index 0000000000..0b18cdf10a --- /dev/null +++ b/src/neo/Network/P2P/Payloads/OraclePayload.cs @@ -0,0 +1,139 @@ +using Neo.Cryptography; +using Neo.Cryptography.ECC; +using Neo.IO; +using Neo.Persistence; +using Neo.SmartContract; +using Neo.SmartContract.Native; +using System; +using System.IO; +using System.Linq; + +namespace Neo.Network.P2P.Payloads +{ + public class OraclePayload : IInventory + { + private byte[] _data; + public byte[] Data + { + get => _data; + set { _data = value; _hash = null; _size = 0; } + } + + private ECPoint _oraclePub; + public ECPoint OraclePub + { + get => _oraclePub; + set { _oraclePub = value; _hash = null; _size = 0; } + } + + private Witness _witness; + public Witness Witness + { + get => _witness; + set { _witness = value; _hash = null; _size = 0; } + } + + private int _size; + public int Size + { + get + { + if (_size == 0) + { + _size = Data.GetVarSize() + //Data + OraclePub.Size + //Oracle Public key + Witness.Size; //Witness + } + return _size; + } + } + + private UInt256 _hash = null; + public UInt256 Hash + { + get + { + if (_hash == null) + { + _hash = new UInt256(Crypto.Hash256(this.GetHashData())); + } + return _hash; + } + } + + Witness[] IVerifiable.Witnesses + { + get => new[] { Witness }; + set + { + if (value.Length != 1) throw new ArgumentException(); + Witness = value[0]; + } + } + + private OracleResponseSignature _deserializedOracleSignature = null; + public OracleResponseSignature OracleSignature + { + get + { + if (_deserializedOracleSignature is null) + _deserializedOracleSignature = OracleResponseSignature.DeserializeFrom(Data); + return _deserializedOracleSignature; + } + internal set + { + if (!ReferenceEquals(_deserializedOracleSignature, value)) + { + _deserializedOracleSignature = value; + Data = value?.ToArray(); + } + } + } + + public InventoryType InventoryType => InventoryType.Oracle; + + public OracleResponseSignature GetDeserializedOracleSignature() + { + return OracleSignature; + } + + void ISerializable.Deserialize(BinaryReader reader) + { + ((IVerifiable)this).DeserializeUnsigned(reader); + + var witness = reader.ReadSerializableArray(1); + if (witness.Length != 1) throw new FormatException(); + Witness = witness[0]; + } + void IVerifiable.DeserializeUnsigned(BinaryReader reader) + { + Data = reader.ReadVarBytes(Transaction.MaxTransactionSize); + OraclePub = reader.ReadSerializable(); + } + + public virtual void Serialize(BinaryWriter writer) + { + ((IVerifiable)this).SerializeUnsigned(writer); + writer.Write(new Witness[] { Witness }); + } + + void IVerifiable.SerializeUnsigned(BinaryWriter writer) + { + writer.WriteVarBytes(Data); + writer.Write(OraclePub); + } + + UInt160[] IVerifiable.GetScriptHashesForVerifying(StoreView snapshot) + { + return new[] { Contract.CreateSignatureRedeemScript(OraclePub).ToScriptHash() }; + } + + public bool Verify(StoreView snapshot) + { + ECPoint[] validators = NativeContract.Oracle.GetOracleValidators(snapshot); + if (!validators.Any(u => u.Equals(OraclePub))) + return false; + return this.VerifyWitnesses(snapshot, 0_02000000); + } + } +} diff --git a/src/neo/Network/P2P/Payloads/OracleResponseSignature.cs b/src/neo/Network/P2P/Payloads/OracleResponseSignature.cs new file mode 100644 index 0000000000..0369284f33 --- /dev/null +++ b/src/neo/Network/P2P/Payloads/OracleResponseSignature.cs @@ -0,0 +1,110 @@ +using Neo.Cryptography; +using Neo.IO; +using System; +using System.IO; +using System.Linq; +using System.Text; + +namespace Neo.Network.P2P.Payloads +{ + public class OracleResponseSignature : ISerializable + { + private const byte ResponseSignatureType = 0x01; + + private UInt256 _transactionRequestHash; + public UInt256 TransactionRequestHash + { + get => _transactionRequestHash; + set { _transactionRequestHash = value; _hash = null; _size = 0; } + } + + private UInt160 _oracleExecutionCacheHash; + public UInt160 OracleExecutionCacheHash + { + get => _oracleExecutionCacheHash; + set { _oracleExecutionCacheHash = value; _hash = null; _size = 0; } + } + + /// + /// Signature for the oracle response tx for this public key + /// + private byte[] _signature; + public byte[] Signature + { + get => _signature; + set + { + if (value.Length != 64) throw new ArgumentException(); + _signature = value; + _hash = null; + _size = 0; + } + } + + private int _size; + public int Size + { + get + { + if (_size == 0) + { + _size = sizeof(byte) + //Type + UInt256.Length + //Transaction Hash + UInt160.Length + //OracleExecutionCache Hash + Signature.Length; //Oracle Validator Signature + } + return _size; + } + } + + private UInt256 _hash = null; + public UInt256 Hash + { + get + { + if (_hash == null) + { + _hash = new UInt256(Crypto.Hash256(this.ToArray())); + } + return _hash; + } + } + + public virtual void Deserialize(BinaryReader reader) + { + if (reader.ReadByte() != ResponseSignatureType) throw new FormatException(); + DeserializeWithoutType(reader); + } + + private void DeserializeWithoutType(BinaryReader reader) + { + TransactionRequestHash = reader.ReadSerializable(); + OracleExecutionCacheHash = reader.ReadSerializable(); + Signature = reader.ReadFixedBytes(64); + } + + public static OracleResponseSignature DeserializeFrom(byte[] data) + { + switch (data[0]) + { + case ResponseSignatureType: + { + using BinaryReader reader = new BinaryReader(new MemoryStream(data, 1, data.Length - 1), Encoding.UTF8, false); + + var ret = new OracleResponseSignature(); + ret.DeserializeWithoutType(reader); + return ret; + } + default: throw new FormatException(); + } + } + + public virtual void Serialize(BinaryWriter writer) + { + writer.Write(ResponseSignatureType); + writer.Write(TransactionRequestHash); + writer.Write(OracleExecutionCacheHash); + writer.Write(Signature); + } + } +} diff --git a/src/neo/Network/P2P/Payloads/Transaction.cs b/src/neo/Network/P2P/Payloads/Transaction.cs index 6035ac2285..51c29dea9f 100644 --- a/src/neo/Network/P2P/Payloads/Transaction.cs +++ b/src/neo/Network/P2P/Payloads/Transaction.cs @@ -30,7 +30,7 @@ public class Transaction : IEquatable, IInventory, IInteroperable /// private const int MaxCosigners = 16; - private byte version; + private TransactionVersion version; private uint nonce; private UInt160 sender; private long sysfee; @@ -40,9 +40,10 @@ public class Transaction : IEquatable, IInventory, IInteroperable private Cosigner[] cosigners; private byte[] script; private Witness[] witnesses; + private UInt256 oracleRequestTx; public const int HeaderSize = - sizeof(byte) + //Version + sizeof(TransactionVersion) + //Version sizeof(uint) + //Nonce 20 + //Sender sizeof(long) + //SystemFee @@ -61,6 +62,12 @@ public Cosigner[] Cosigners set { cosigners = value; _hash = null; _size = 0; } } + public UInt256 OracleRequestTx + { + get => oracleRequestTx; + set { oracleRequestTx = value; _hash = null; _size = 0; } + } + /// /// The NetworkFee for the transaction divided by its Size. /// Note that this property must be used with care. Getting the value of this property multiple times will return the same result. The value of this property can only be obtained after the transaction has been completely built (no longer modified). @@ -121,6 +128,11 @@ public int Size Cosigners.GetVarSize() + //Cosigners Script.GetVarSize() + //Script Witnesses.GetVarSize(); //Witnesses + + if (Version == TransactionVersion.OracleResponse) + { + _size += UInt256.Length; + } } return _size; } @@ -141,7 +153,7 @@ public uint ValidUntilBlock set { validUntilBlock = value; _hash = null; } } - public byte Version + public TransactionVersion Version { get => version; set { version = value; _hash = null; } @@ -166,8 +178,8 @@ void ISerializable.Deserialize(BinaryReader reader) public void DeserializeUnsigned(BinaryReader reader) { - Version = reader.ReadByte(); - if (Version > 0) throw new FormatException(); + Version = (TransactionVersion)reader.ReadByte(); + if (!Enum.IsDefined(typeof(TransactionVersion), Version)) throw new FormatException(); Nonce = reader.ReadUInt32(); Sender = reader.ReadSerializable(); SystemFee = reader.ReadInt64(); @@ -181,6 +193,7 @@ public void DeserializeUnsigned(BinaryReader reader) if (Cosigners.Select(u => u.Account).Distinct().Count() != Cosigners.Length) throw new FormatException(); Script = reader.ReadVarBytes(ushort.MaxValue); if (Script.Length == 0) throw new FormatException(); + OracleRequestTx = Version == TransactionVersion.OracleResponse ? reader.ReadSerializable() : null; } public bool Equals(Transaction other) @@ -215,7 +228,7 @@ void ISerializable.Serialize(BinaryWriter writer) void IVerifiable.SerializeUnsigned(BinaryWriter writer) { - writer.Write(Version); + writer.Write((byte)Version); writer.Write(Nonce); writer.Write(Sender); writer.Write(SystemFee); @@ -224,6 +237,10 @@ void IVerifiable.SerializeUnsigned(BinaryWriter writer) writer.Write(Attributes); writer.Write(Cosigners); writer.WriteVarBytes(Script); + if (Version == TransactionVersion.OracleResponse) + { + writer.Write(OracleRequestTx); + } } public JObject ToJson() @@ -231,7 +248,7 @@ public JObject ToJson() JObject json = new JObject(); json["hash"] = Hash.ToString(); json["size"] = Size; - json["version"] = Version; + json["version"] = (byte)Version; json["nonce"] = Nonce; json["sender"] = Sender.ToAddress(); json["sys_fee"] = SystemFee.ToString(); @@ -241,13 +258,18 @@ public JObject ToJson() json["cosigners"] = Cosigners.Select(p => p.ToJson()).ToArray(); json["script"] = Convert.ToBase64String(Script); json["witnesses"] = Witnesses.Select(p => p.ToJson()).ToArray(); + if (Version == TransactionVersion.OracleResponse) + { + json["oracle_response_tx"] = OracleRequestTx.ToString(); + } return json; } public static Transaction FromJson(JObject json) { Transaction tx = new Transaction(); - tx.Version = byte.Parse(json["version"].AsString()); + tx.Version = (TransactionVersion)byte.Parse(json["version"].AsString()); + if (!Enum.IsDefined(typeof(TransactionVersion), tx.Version)) throw new FormatException(); tx.Nonce = uint.Parse(json["nonce"].AsString()); tx.Sender = json["sender"].AsString().ToScriptHash(); tx.SystemFee = long.Parse(json["sys_fee"].AsString()); @@ -257,6 +279,14 @@ public static Transaction FromJson(JObject json) tx.Cosigners = ((JArray)json["cosigners"]).Select(p => Cosigner.FromJson(p)).ToArray(); tx.Script = Convert.FromBase64String(json["script"].AsString()); tx.Witnesses = ((JArray)json["witnesses"]).Select(p => Witness.FromJson(p)).ToArray(); + if (tx.Version == TransactionVersion.OracleResponse) + { + tx.OracleRequestTx = UInt256.Parse(json["oracle_response_tx"].AsString()); + } + else + { + tx.OracleRequestTx = null; + } return tx; } @@ -293,6 +323,21 @@ public virtual VerifyResult Verify(StoreView snapshot, BigInteger totalSenderFee long net_fee = NetworkFee - size * NativeContract.Policy.GetFeePerByte(snapshot); if (net_fee < 0) return VerifyResult.InsufficientFunds; if (!this.VerifyWitnesses(snapshot, net_fee)) return VerifyResult.Invalid; + + if (Version == TransactionVersion.OracleResponse) + { + // Oracle response only can be signed by oracle nodes + + var hashes = GetScriptHashesForVerifying(snapshot); + + if (hashes.Length != 1 || + hashes[0] != NativeContract.Oracle.GetOracleMultiSigAddress(snapshot) || + hashes[0] != Sender) + { + return VerifyResult.Invalid; + } + } + return VerifyResult.Succeed; } diff --git a/src/neo/Network/P2P/Payloads/TransactionVersion.cs b/src/neo/Network/P2P/Payloads/TransactionVersion.cs new file mode 100644 index 0000000000..3b5c110c78 --- /dev/null +++ b/src/neo/Network/P2P/Payloads/TransactionVersion.cs @@ -0,0 +1,9 @@ +namespace Neo.Network.P2P.Payloads +{ + public enum TransactionVersion : byte + { + Transaction = 0x00, + OracleRequest = 0x01, + OracleResponse = 0x02 + } +} diff --git a/src/neo/Network/P2P/ProtocolHandler.cs b/src/neo/Network/P2P/ProtocolHandler.cs new file mode 100644 index 0000000000..32944db68a --- /dev/null +++ b/src/neo/Network/P2P/ProtocolHandler.cs @@ -0,0 +1,451 @@ +using Akka.Actor; +using Akka.Configuration; +using Neo.Cryptography; +using Neo.IO; +using Neo.IO.Actors; +using Neo.IO.Caching; +using Neo.Ledger; +using Neo.Network.P2P.Payloads; +using Neo.Persistence; +using Neo.Plugins; +using System; +using System.Collections; +using System.Collections.Generic; +using System.Collections.ObjectModel; +using System.Linq; +using System.Net; + +namespace Neo.Network.P2P +{ + internal class ProtocolHandler : UntypedActor + { + public class SetFilter { public BloomFilter Filter; } + internal class Timer { } + + private class PendingKnownHashesCollection : KeyedCollection + { + protected override UInt256 GetKeyForItem((UInt256, DateTime) item) + { + return item.Item1; + } + } + + private readonly NeoSystem system; + private readonly PendingKnownHashesCollection pendingKnownHashes; + private readonly HashSetCache knownHashes; + private readonly HashSetCache sentHashes; + private VersionPayload version; + private bool verack = false; + private BloomFilter bloom_filter; + + private static readonly TimeSpan TimerInterval = TimeSpan.FromSeconds(30); + private static readonly TimeSpan PendingTimeout = TimeSpan.FromMinutes(1); + + private readonly ICancelable timer = Context.System.Scheduler.ScheduleTellRepeatedlyCancelable(TimerInterval, TimerInterval, Context.Self, new Timer(), ActorRefs.NoSender); + + public ProtocolHandler(NeoSystem system) + { + this.system = system; + this.pendingKnownHashes = new PendingKnownHashesCollection(); + this.knownHashes = new HashSetCache(Blockchain.Singleton.MemPool.Capacity * 2 / 5); + this.sentHashes = new HashSetCache(Blockchain.Singleton.MemPool.Capacity * 2 / 5); + } + + protected override void OnReceive(object message) + { + switch (message) + { + case Message msg: + OnMessage(msg); + break; + case Timer _: + OnTimer(); + break; + } + } + + private void OnMessage(Message msg) + { + foreach (IP2PPlugin plugin in Plugin.P2PPlugins) + if (!plugin.OnP2PMessage(msg)) + return; + if (version == null) + { + if (msg.Command != MessageCommand.Version) + throw new ProtocolViolationException(); + OnVersionMessageReceived((VersionPayload)msg.Payload); + return; + } + if (!verack) + { + if (msg.Command != MessageCommand.Verack) + throw new ProtocolViolationException(); + OnVerackMessageReceived(); + return; + } + switch (msg.Command) + { + case MessageCommand.Addr: + OnAddrMessageReceived((AddrPayload)msg.Payload); + break; + case MessageCommand.Block: + OnInventoryReceived((Block)msg.Payload); + break; + case MessageCommand.Consensus: + OnInventoryReceived((ConsensusPayload)msg.Payload); + break; + case MessageCommand.Oracle: + OnInventoryReceived((OraclePayload)msg.Payload); + break; + case MessageCommand.FilterAdd: + OnFilterAddMessageReceived((FilterAddPayload)msg.Payload); + break; + case MessageCommand.FilterClear: + OnFilterClearMessageReceived(); + break; + case MessageCommand.FilterLoad: + OnFilterLoadMessageReceived((FilterLoadPayload)msg.Payload); + break; + case MessageCommand.GetAddr: + OnGetAddrMessageReceived(); + break; + case MessageCommand.GetBlocks: + OnGetBlocksMessageReceived((GetBlocksPayload)msg.Payload); + break; + case MessageCommand.GetBlockData: + OnGetBlockDataMessageReceived((GetBlockDataPayload)msg.Payload); + break; + case MessageCommand.GetData: + OnGetDataMessageReceived((InvPayload)msg.Payload); + break; + case MessageCommand.GetHeaders: + OnGetHeadersMessageReceived((GetBlocksPayload)msg.Payload); + break; + case MessageCommand.Headers: + OnHeadersMessageReceived((HeadersPayload)msg.Payload); + break; + case MessageCommand.Inv: + OnInvMessageReceived((InvPayload)msg.Payload); + break; + case MessageCommand.Mempool: + OnMemPoolMessageReceived(); + break; + case MessageCommand.Ping: + OnPingMessageReceived((PingPayload)msg.Payload); + break; + case MessageCommand.Pong: + OnPongMessageReceived((PingPayload)msg.Payload); + break; + case MessageCommand.Transaction: + if (msg.Payload.Size <= Transaction.MaxTransactionSize) + OnInventoryReceived((Transaction)msg.Payload); + break; + case MessageCommand.Verack: + case MessageCommand.Version: + throw new ProtocolViolationException(); + case MessageCommand.Alert: + case MessageCommand.MerkleBlock: + case MessageCommand.NotFound: + case MessageCommand.Reject: + default: break; + } + } + + private void OnAddrMessageReceived(AddrPayload payload) + { + system.LocalNode.Tell(new Peer.Peers + { + EndPoints = payload.AddressList.Select(p => p.EndPoint).Where(p => p.Port > 0) + }); + } + + private void OnFilterAddMessageReceived(FilterAddPayload payload) + { + if (bloom_filter != null) + bloom_filter.Add(payload.Data); + } + + private void OnFilterClearMessageReceived() + { + bloom_filter = null; + Context.Parent.Tell(new SetFilter { Filter = null }); + } + + private void OnFilterLoadMessageReceived(FilterLoadPayload payload) + { + bloom_filter = new BloomFilter(payload.Filter.Length * 8, payload.K, payload.Tweak, payload.Filter); + Context.Parent.Tell(new SetFilter { Filter = bloom_filter }); + } + + /// + /// Will be triggered when a MessageCommand.GetAddr message is received. + /// Randomly select nodes from the local RemoteNodes and tells to RemoteNode actors a MessageCommand.Addr message. + /// The message contains a list of networkAddresses from those selected random peers. + /// + private void OnGetAddrMessageReceived() + { + Random rand = new Random(); + IEnumerable peers = LocalNode.Singleton.RemoteNodes.Values + .Where(p => p.ListenerTcpPort > 0) + .GroupBy(p => p.Remote.Address, (k, g) => g.First()) + .OrderBy(p => rand.Next()) + .Take(AddrPayload.MaxCountToSend); + NetworkAddressWithTime[] networkAddresses = peers.Select(p => NetworkAddressWithTime.Create(p.Listener.Address, p.Version.Timestamp, p.Version.Capabilities)).ToArray(); + if (networkAddresses.Length == 0) return; + Context.Parent.Tell(Message.Create(MessageCommand.Addr, AddrPayload.Create(networkAddresses))); + } + + /// + /// Will be triggered when a MessageCommand.GetBlocks message is received. + /// Tell the specified number of blocks' hashes starting with the requested HashStart until payload.Count or MaxHashesCount + /// Responses are sent to RemoteNode actor as MessageCommand.Inv Message. + /// + /// A GetBlocksPayload including start block Hash and number of blocks requested. + private void OnGetBlocksMessageReceived(GetBlocksPayload payload) + { + UInt256 hash = payload.HashStart; + // The default value of payload.Count is -1 + int count = payload.Count < 0 || payload.Count > InvPayload.MaxHashesCount ? InvPayload.MaxHashesCount : payload.Count; + TrimmedBlock state = Blockchain.Singleton.View.Blocks.TryGet(hash); + if (state == null) return; + List hashes = new List(); + for (uint i = 1; i <= count; i++) + { + uint index = state.Index + i; + if (index > Blockchain.Singleton.Height) + break; + hash = Blockchain.Singleton.GetBlockHash(index); + if (hash == null) break; + hashes.Add(hash); + } + if (hashes.Count == 0) return; + Context.Parent.Tell(Message.Create(MessageCommand.Inv, InvPayload.Create(InventoryType.Block, hashes.ToArray()))); + } + + private void OnGetBlockDataMessageReceived(GetBlockDataPayload payload) + { + for (uint i = payload.IndexStart, max = payload.IndexStart + payload.Count; i < max; i++) + { + Block block = Blockchain.Singleton.GetBlock(i); + if (block == null) + break; + + if (bloom_filter == null) + { + Context.Parent.Tell(Message.Create(MessageCommand.Block, block)); + } + else + { + BitArray flags = new BitArray(block.Transactions.Select(p => bloom_filter.Test(p)).ToArray()); + Context.Parent.Tell(Message.Create(MessageCommand.MerkleBlock, MerkleBlockPayload.Create(block, flags))); + } + } + } + + /// + /// Will be triggered when a MessageCommand.GetData message is received. + /// The payload includes an array of hash values. + /// For different payload.Type (Tx, Block, Consensus), get the corresponding (Txs, Blocks, Consensus) and tell them to RemoteNode actor. + /// + /// The payload containing the requested information. + private void OnGetDataMessageReceived(InvPayload payload) + { + UInt256[] hashes = payload.Hashes.Where(p => sentHashes.Add(p)).ToArray(); + foreach (UInt256 hash in hashes) + { + switch (payload.Type) + { + case InventoryType.TX: + Transaction tx = Blockchain.Singleton.GetTransaction(hash); + if (tx != null) + Context.Parent.Tell(Message.Create(MessageCommand.Transaction, tx)); + break; + case InventoryType.Block: + Block block = Blockchain.Singleton.GetBlock(hash); + if (block != null) + { + if (bloom_filter == null) + { + Context.Parent.Tell(Message.Create(MessageCommand.Block, block)); + } + else + { + BitArray flags = new BitArray(block.Transactions.Select(p => bloom_filter.Test(p)).ToArray()); + Context.Parent.Tell(Message.Create(MessageCommand.MerkleBlock, MerkleBlockPayload.Create(block, flags))); + } + } + break; + case InventoryType.Consensus: + if (Blockchain.Singleton.RelayCache.TryGet(hash, out IInventory inventoryConsensus)) + Context.Parent.Tell(Message.Create(MessageCommand.Consensus, inventoryConsensus)); + break; + case InventoryType.Oracle: + if (Blockchain.Singleton.RelayCache.TryGet(hash, out IInventory inventoryOracle)) + Context.Parent.Tell(Message.Create(MessageCommand.Oracle, inventoryOracle)); + break; + } + } + } + + /// + /// Will be triggered when a MessageCommand.GetHeaders message is received. + /// Tell the specified number of blocks' headers starting with the requested HashStart to RemoteNode actor. + /// A limit set by HeadersPayload.MaxHeadersCount is also applied to the number of requested Headers, namely payload.Count. + /// + /// A GetBlocksPayload including start block Hash and number of blocks' headers requested. + private void OnGetHeadersMessageReceived(GetBlocksPayload payload) + { + UInt256 hash = payload.HashStart; + int count = payload.Count < 0 || payload.Count > HeadersPayload.MaxHeadersCount ? HeadersPayload.MaxHeadersCount : payload.Count; + DataCache cache = Blockchain.Singleton.View.Blocks; + TrimmedBlock state = cache.TryGet(hash); + if (state == null) return; + List
headers = new List
(); + for (uint i = 1; i <= count; i++) + { + uint index = state.Index + i; + hash = Blockchain.Singleton.GetBlockHash(index); + if (hash == null) break; + Header header = cache.TryGet(hash)?.Header; + if (header == null) break; + headers.Add(header); + } + if (headers.Count == 0) return; + Context.Parent.Tell(Message.Create(MessageCommand.Headers, HeadersPayload.Create(headers.ToArray()))); + } + + private void OnHeadersMessageReceived(HeadersPayload payload) + { + if (payload.Headers.Length == 0) return; + system.Blockchain.Tell(payload.Headers, Context.Parent); + } + + private void OnInventoryReceived(IInventory inventory) + { + system.TaskManager.Tell(new TaskManager.TaskCompleted { Hash = inventory.Hash }, Context.Parent); + system.LocalNode.Tell(new LocalNode.Relay { Inventory = inventory }); + pendingKnownHashes.Remove(inventory.Hash); + knownHashes.Add(inventory.Hash); + } + + private void OnInvMessageReceived(InvPayload payload) + { + UInt256[] hashes = payload.Hashes.Where(p => !pendingKnownHashes.Contains(p) && !knownHashes.Contains(p) && !sentHashes.Contains(p)).ToArray(); + if (hashes.Length == 0) return; + switch (payload.Type) + { + case InventoryType.Block: + using (SnapshotView snapshot = Blockchain.Singleton.GetSnapshot()) + hashes = hashes.Where(p => !snapshot.ContainsBlock(p)).ToArray(); + break; + case InventoryType.TX: + using (SnapshotView snapshot = Blockchain.Singleton.GetSnapshot()) + hashes = hashes.Where(p => !snapshot.ContainsTransaction(p)).ToArray(); + break; + } + if (hashes.Length == 0) return; + foreach (UInt256 hash in hashes) + pendingKnownHashes.Add((hash, DateTime.UtcNow)); + system.TaskManager.Tell(new TaskManager.NewTasks { Payload = InvPayload.Create(payload.Type, hashes) }, Context.Parent); + } + + private void OnMemPoolMessageReceived() + { + using var snapshot = Blockchain.Singleton.GetSnapshot(); + foreach (InvPayload payload in InvPayload.CreateGroup(InventoryType.TX, Blockchain.Singleton.MemPool.GetVerifiedTransactions(snapshot).Select(p => p.Hash).ToArray())) + Context.Parent.Tell(Message.Create(MessageCommand.Inv, payload)); + } + + private void OnPingMessageReceived(PingPayload payload) + { + Context.Parent.Tell(payload); + Context.Parent.Tell(Message.Create(MessageCommand.Pong, PingPayload.Create(Blockchain.Singleton.Height, payload.Nonce))); + } + + private void OnPongMessageReceived(PingPayload payload) + { + Context.Parent.Tell(payload); + } + + private void OnVerackMessageReceived() + { + verack = true; + Context.Parent.Tell(MessageCommand.Verack); + } + + private void OnVersionMessageReceived(VersionPayload payload) + { + version = payload; + Context.Parent.Tell(payload); + } + + private void OnTimer() + { + RefreshPendingKnownHashes(); + } + + protected override void PostStop() + { + timer.CancelIfNotNull(); + base.PostStop(); + } + + private void RefreshPendingKnownHashes() + { + while (pendingKnownHashes.Count > 0) + { + var (_, time) = pendingKnownHashes[0]; + if (DateTime.UtcNow - time <= PendingTimeout) + break; + pendingKnownHashes.RemoveAt(0); + } + } + + public static Props Props(NeoSystem system) + { + return Akka.Actor.Props.Create(() => new ProtocolHandler(system)).WithMailbox("protocol-handler-mailbox"); + } + } + + internal class ProtocolHandlerMailbox : PriorityMailbox + { + public ProtocolHandlerMailbox(Settings settings, Config config) + : base(settings, config) + { + } + + internal protected override bool IsHighPriority(object message) + { + if (!(message is Message msg)) return false; + switch (msg.Command) + { + case MessageCommand.Consensus: + case MessageCommand.FilterAdd: + case MessageCommand.FilterClear: + case MessageCommand.FilterLoad: + case MessageCommand.Verack: + case MessageCommand.Version: + case MessageCommand.Alert: + return true; + default: + return false; + } + } + + internal protected override bool ShallDrop(object message, IEnumerable queue) + { + if (message is ProtocolHandler.Timer) return false; + if (!(message is Message msg)) return true; + switch (msg.Command) + { + case MessageCommand.GetAddr: + case MessageCommand.GetBlocks: + case MessageCommand.GetHeaders: + case MessageCommand.Mempool: + return queue.OfType().Any(p => p.Command == msg.Command); + default: + return false; + } + } + } +} diff --git a/src/neo/Network/P2P/RemoteNode.ProtocolHandler.cs b/src/neo/Network/P2P/RemoteNode.ProtocolHandler.cs index 91b3921326..373772cbf8 100644 --- a/src/neo/Network/P2P/RemoteNode.ProtocolHandler.cs +++ b/src/neo/Network/P2P/RemoteNode.ProtocolHandler.cs @@ -67,6 +67,9 @@ private void OnMessage(Message msg) case MessageCommand.Consensus: OnInventoryReceived((ConsensusPayload)msg.Payload); break; + case MessageCommand.Oracle: + OnInventoryReceived((OraclePayload)msg.Payload); + break; case MessageCommand.FilterAdd: OnFilterAddMessageReceived((FilterAddPayload)msg.Payload); break; @@ -243,9 +246,13 @@ private void OnGetDataMessageReceived(InvPayload payload) } break; case InventoryType.Consensus: - if (Blockchain.Singleton.ConsensusRelayCache.TryGet(hash, out IInventory inventoryConsensus)) + if (Blockchain.Singleton.RelayCache.TryGet(hash, out IInventory inventoryConsensus)) EnqueueMessage(Message.Create(MessageCommand.Consensus, inventoryConsensus)); break; + case InventoryType.Oracle: + if (Blockchain.Singleton.RelayCache.TryGet(hash, out IInventory inventoryOracle)) + EnqueueMessage(Message.Create(MessageCommand.Oracle, inventoryOracle)); + break; } } } @@ -314,7 +321,8 @@ private void OnInvMessageReceived(InvPayload payload) private void OnMemPoolMessageReceived() { - foreach (InvPayload payload in InvPayload.CreateGroup(InventoryType.TX, Blockchain.Singleton.MemPool.GetVerifiedTransactions().Select(p => p.Hash).ToArray())) + using var snapshot = Blockchain.Singleton.GetSnapshot(); + foreach (InvPayload payload in InvPayload.CreateGroup(InventoryType.TX, Blockchain.Singleton.MemPool.GetVerifiedTransactions(snapshot).Select(p => p.Hash).ToArray())) EnqueueMessage(Message.Create(MessageCommand.Inv, payload)); } diff --git a/src/neo/Network/P2P/TaskManager.cs b/src/neo/Network/P2P/TaskManager.cs index 73fa49441a..e7af4cce0b 100644 --- a/src/neo/Network/P2P/TaskManager.cs +++ b/src/neo/Network/P2P/TaskManager.cs @@ -292,7 +292,9 @@ internal protected override bool IsHighPriority(object message) case TaskManager.RestartTasks _: return true; case TaskManager.NewTasks tasks: - if (tasks.Payload.Type == InventoryType.Block || tasks.Payload.Type == InventoryType.Consensus) + if (tasks.Payload.Type == InventoryType.Block || + tasks.Payload.Type == InventoryType.Oracle || + tasks.Payload.Type == InventoryType.Consensus) return true; return false; default: diff --git a/src/neo/Oracle/ManualWitness.cs b/src/neo/Oracle/ManualWitness.cs new file mode 100644 index 0000000000..1110636d6b --- /dev/null +++ b/src/neo/Oracle/ManualWitness.cs @@ -0,0 +1,33 @@ +using Neo.Network.P2P.Payloads; +using Neo.Persistence; +using System; +using System.IO; + +namespace Neo.Oracle +{ + internal class ManualWitness : IVerifiable + { + public Witness[] Witnesses + { + get => throw new NotImplementedException(); + set => throw new NotImplementedException(); + } + public int Size => throw new NotImplementedException(); + public void Deserialize(BinaryReader reader) => throw new NotImplementedException(); + public void DeserializeUnsigned(BinaryReader reader) => throw new NotImplementedException(); + public void Serialize(BinaryWriter writer) => throw new NotImplementedException(); + public void SerializeUnsigned(BinaryWriter writer) => throw new NotImplementedException(); + + private readonly UInt160[] _hashes; + + public ManualWitness(params UInt160[] hashes) + { + _hashes = hashes ?? Array.Empty(); + } + + public UInt160[] GetScriptHashesForVerifying(StoreView snapshot) + { + return _hashes; + } + } +} diff --git a/src/neo/Oracle/OracleExecutionCache.cs b/src/neo/Oracle/OracleExecutionCache.cs new file mode 100644 index 0000000000..b498da50aa --- /dev/null +++ b/src/neo/Oracle/OracleExecutionCache.cs @@ -0,0 +1,173 @@ +using Neo.IO; +using Neo.SmartContract; +using System; +using System.Collections; +using System.Collections.Generic; +using System.IO; +using System.Linq; + +namespace Neo.Oracle +{ + public class OracleExecutionCache : IEnumerable>, ISerializable + { + /// + /// Results (OracleRequest.Hash/OracleResponse) + /// + private readonly Dictionary _cache = new Dictionary(); + + /// + /// Engine + /// + private readonly Func _oracle; + + /// + /// Count + /// + public int Count => _cache.Count; + + /// + /// Filter Cost + /// + public long FilterCost { get; private set; } + + /// + /// Responses + /// + public OracleResponse[] Responses => _cache.Values.ToArray(); + + public int Size => IO.Helper.GetVarSize(Count) + _cache.Values.Sum(u => u.Size); + + private UInt160 _hash; + + /// + /// Hash + /// + public UInt160 Hash + { + get + { + if (_hash != null) return _hash; + + using (var stream = new MemoryStream()) + { + foreach (var entry in _cache) + { + // Request Hash + stream.Write(entry.Key.ToArray()); + + // Response Hash + stream.Write(entry.Value.Hash.ToArray()); + } + + _hash = stream.ToArray().ToScriptHash(); + } + + return _hash; + } + } + + /// + /// Constructor for oracles + /// + /// Oracle Engine + public OracleExecutionCache(Func oracle = null) : this() + { + _oracle = oracle; + } + + /// + /// Constructor for ISerializable + /// + public OracleExecutionCache() + { + _hash = null; + FilterCost = 0; + } + + /// + /// Clear + /// + public void Clear() + { + _cache.Clear(); + _hash = null; + FilterCost = 0; + } + + /// + /// Constructor for cached results + /// + /// Results + public OracleExecutionCache(params OracleResponse[] results) + { + FilterCost = 0; + + _hash = null; + _oracle = null; + + foreach (var result in results) + { + _cache[result.RequestHash] = result; + FilterCost += result.FilterCost; + } + } + + /// + /// Get Oracle result + /// + /// Request + /// Result + /// + public bool TryGet(OracleRequest request, out OracleResponse result) + { + if (_cache.TryGetValue(request.Hash, out result)) + { + return true; + } + + // Not found inside the cache, invoke it + + result = _oracle?.Invoke(request); + + if (result != null) + { + _cache[request.Hash] = result; + return true; + } + + // Without oracle logic + + return false; + } + + public IEnumerator> GetEnumerator() + { + return _cache.GetEnumerator(); + } + + IEnumerator IEnumerable.GetEnumerator() + { + return _cache.GetEnumerator(); + } + + public void Serialize(BinaryWriter writer) + { + writer.Write(_cache.Values.ToArray()); + } + + public void Deserialize(BinaryReader reader) + { + FilterCost = 0; + _hash = null; + + var entries = reader.ReadSerializableArray(byte.MaxValue); + _cache.Clear(); + + foreach (var result in entries) + { + _cache[result.RequestHash] = result; + FilterCost += result.FilterCost; + } + } + } +} diff --git a/src/neo/Oracle/OracleFilter.cs b/src/neo/Oracle/OracleFilter.cs new file mode 100644 index 0000000000..ae35d5727c --- /dev/null +++ b/src/neo/Oracle/OracleFilter.cs @@ -0,0 +1,15 @@ +namespace Neo.Oracle +{ + public class OracleFilter + { + /// + /// Contract Hash + /// + public UInt160 ContractHash; + + /// + /// You need a specific method for your filters + /// + public string FilterMethod; + } +} diff --git a/src/neo/Oracle/OracleRequest.cs b/src/neo/Oracle/OracleRequest.cs new file mode 100644 index 0000000000..691b735b82 --- /dev/null +++ b/src/neo/Oracle/OracleRequest.cs @@ -0,0 +1,36 @@ +using Neo.Cryptography; + +namespace Neo.Oracle +{ + public abstract class OracleRequest + { + private UInt160 _hash; + + /// + /// Type + /// + public abstract OracleRequestType Type { get; } + + /// + /// Hash + /// + public UInt160 Hash + { + get + { + if (_hash == null) + { + _hash = new UInt160(Crypto.Hash160(GetHashData())); + } + + return _hash; + } + } + + /// + /// This method serialize the parts of the class that should be taken into account for compute the Hash + /// + /// Serialized data + protected abstract byte[] GetHashData(); + } +} diff --git a/src/neo/Oracle/OracleRequestType.cs b/src/neo/Oracle/OracleRequestType.cs new file mode 100644 index 0000000000..b7e0d08215 --- /dev/null +++ b/src/neo/Oracle/OracleRequestType.cs @@ -0,0 +1,7 @@ +namespace Neo.Oracle +{ + public enum OracleRequestType : byte + { + HTTPS = 0x01, + } +} diff --git a/src/neo/Oracle/OracleResponse.cs b/src/neo/Oracle/OracleResponse.cs new file mode 100644 index 0000000000..10b3ebbfe1 --- /dev/null +++ b/src/neo/Oracle/OracleResponse.cs @@ -0,0 +1,154 @@ +using Neo.Cryptography; +using Neo.IO; +using Neo.Network.P2P; +using Neo.Network.P2P.Payloads; +using Neo.Persistence; +using System; +using System.IO; +using System.Text; + +namespace Neo.Oracle +{ + public class OracleResponse : IVerifiable + { + private UInt160 _hash; + + /// + /// Request hash + /// + public UInt160 RequestHash { get; set; } + + /// + /// Result + /// + public byte[] Result { get; set; } + + /// + /// Error + /// + public bool Error => Result == null; + + /// + /// Filter cost paid by Oracle and must be claimed to the user + /// + public long FilterCost { get; set; } + + /// + /// Hash + /// + public UInt160 Hash + { + get + { + if (_hash == null) + { + _hash = new UInt160(Crypto.Hash160(this.GetHashData())); + } + + return _hash; + } + } + + public int Size => UInt160.Length + sizeof(byte) + Result.GetVarSize() + sizeof(long); + + public Witness[] Witnesses + { + get => throw new NotImplementedException(); + set => throw new NotImplementedException(); + } + + /// + /// Create error result + /// + /// Request Id + /// Error + /// Gas cost + /// OracleResult + public static OracleResponse CreateError(UInt160 requestHash, OracleResultError error, long filterCost = 0) + { + // TODO: We should log the error if we want, but in order to reduce the indeterminism, we will only say that the download was unsuccessful + + return CreateResult(requestHash, (byte[])null, filterCost); + } + + /// + /// Create result + /// + /// Request Hash + /// Result + /// Gas cost + /// OracleResult + public static OracleResponse CreateResult(UInt160 requestHash, string result, long filterCost) + { + return CreateResult(requestHash, Encoding.UTF8.GetBytes(result), filterCost); + } + + /// + /// Create result + /// + /// Request Id + /// Result + /// Gas cost + /// OracleResult + public static OracleResponse CreateResult(UInt160 requestHash, byte[] result, long filterCost) + { + return new OracleResponse() + { + RequestHash = requestHash, + Result = result, + FilterCost = filterCost + }; + } + + public void SerializeUnsigned(BinaryWriter writer) + { + writer.Write(RequestHash); + writer.Write(FilterCost); + + if (Result != null) + { + writer.Write((byte)0x01); + writer.WriteVarBytes(Result); + } + else + { + // Error result + + writer.Write((byte)0x00); + } + } + + public void Serialize(BinaryWriter writer) + { + SerializeUnsigned(writer); + } + + public void DeserializeUnsigned(BinaryReader reader) + { + RequestHash = reader.ReadSerializable(); + FilterCost = reader.ReadInt64(); + if (FilterCost < 0) throw new FormatException(nameof(FilterCost)); + + if (reader.ReadByte() == 0x01) + { + Result = reader.ReadVarBytes(ushort.MaxValue); + } + else + { + // Error result + + Result = null; + } + } + + public void Deserialize(BinaryReader reader) + { + DeserializeUnsigned(reader); + } + + public UInt160[] GetScriptHashesForVerifying(StoreView snapshot) + { + return new UInt160[] { new UInt160(Crypto.Hash160(this.GetHashData())) }; + } + } +} diff --git a/src/neo/Oracle/OracleResultError.cs b/src/neo/Oracle/OracleResultError.cs new file mode 100644 index 0000000000..19a9646fef --- /dev/null +++ b/src/neo/Oracle/OracleResultError.cs @@ -0,0 +1,40 @@ +namespace Neo.Oracle +{ + public enum OracleResultError : byte + { + /// + /// There was no errors + /// + None = 0x00, + + /// + /// Timeout + /// + Timeout = 0x01, + + /// + /// There was an error with the server + /// + ServerError = 0x02, + + /// + /// There was an error with the policy + /// + PolicyError = 0x03, + + /// + /// There was an error with the protocol + /// + ProtocolError = 0x04, + + /// + /// There was an error with the filter + /// + FilterError = 0x05, + + /// + /// Unrecognized format + /// + ResponseError = 0x06 + } +} diff --git a/src/neo/Oracle/OracleService.cs b/src/neo/Oracle/OracleService.cs new file mode 100644 index 0000000000..02acfd0f3b --- /dev/null +++ b/src/neo/Oracle/OracleService.cs @@ -0,0 +1,816 @@ +using Akka.Actor; +using Neo.Cryptography.ECC; +using Neo.IO; +using Neo.Ledger; +using Neo.Network.P2P; +using Neo.Network.P2P.Payloads; +using Neo.Oracle.Protocols.Https; +using Neo.Persistence; +using Neo.SmartContract; +using Neo.SmartContract.Native; +using Neo.VM; +using Neo.Wallets; +using System; +using System.Collections; +using System.Collections.Generic; +using System.Linq; +using System.Text; +using System.Threading; +using System.Threading.Tasks; + +namespace Neo.Oracle +{ + public class OracleService : UntypedActor + { + #region Sub classes + + internal class StartMessage { public byte NumberOfTasks = 4; } + internal class StopMessage { } + + private class RequestItem : PoolItem + { + // Request + + public readonly Transaction RequestTransaction; + + // Proposal + + private ResponseItem Proposal; + private ContractParametersContext ResponseContext; + + public Transaction ResponseTransaction => Proposal?.Tx; + + public bool IsCompleted => ResponseContext?.Completed == true; + + public RequestItem(Transaction requestTx) : base(requestTx) + { + RequestTransaction = requestTx; + } + + public bool AddSignature(ResponseItem response) + { + if (response.TransactionRequestHash != RequestTransaction.Hash) + { + return false; + } + + if (Proposal == null) + { + if (!response.IsMine) + { + return false; + } + + // Oracle service could attach the real TX + + Proposal = response; + ResponseContext = new ContractParametersContext(response.Tx); + } + else + { + if (response.ResultHash != Proposal.ResultHash) + { + // Unexpected result + + return false; + } + } + + if (ResponseContext.AddSignature(Proposal.Contract, response.OraclePub, response.Signature) == true) + { + if (ResponseContext.Completed) + { + // Append the witness to the response TX + + Proposal.Tx.Witnesses = ResponseContext.GetWitnesses(); + } + return true; + } + + return false; + } + } + + private class ResponseCollection : IEnumerable + { + public readonly DateTime Timestamp; + + private readonly SortedConcurrentDictionary _items; + + public int Count => _items.Count; + + public int MineCount { get; private set; } + + public ResponseCollection(ResponseItem item) + { + Timestamp = item.Timestamp; + _items = new SortedConcurrentDictionary + ( + Comparer>.Create(Sort), 1_000 + ); + + Add(item); + } + + public bool Add(ResponseItem item) + { + // Prevent duplicate messages using the publicKey as key + + if (_items.TryGetValue(item.OraclePub, out var prev)) + { + // If it's new, replace it + + if (prev.Timestamp > item.Timestamp) return false; + + if (!prev.IsMine && item.IsMine) MineCount++; + else if (prev.IsMine && !item.IsMine) MineCount--; + + _items.Set(item.OraclePub, item); + return true; + } + + if (_items.TryAdd(item.OraclePub, item)) + { + if (item.IsMine) MineCount++; + return true; + } + + return false; + } + + public IEnumerator GetEnumerator() + { + return (IEnumerator)_items.Select(u => u.Value).ToArray().GetEnumerator(); + } + + IEnumerator IEnumerable.GetEnumerator() + { + return this.GetEnumerator(); + } + } + + private class ResponseItem : PoolItem + { + private readonly OraclePayload Msg; + private readonly OracleResponseSignature Data; + + public readonly Contract Contract; + public ECPoint OraclePub => Msg.OraclePub; + public UInt256 MsgHash => Msg.Hash; + public byte[] Signature => Data.Signature; + public UInt160 ResultHash => Data.OracleExecutionCacheHash; + public UInt256 TransactionRequestHash => Data.TransactionRequestHash; + public bool IsMine { get; } + + public ResponseItem(OraclePayload payload, Contract contract = null, Transaction responseTx = null) : base(responseTx) + { + IsMine = responseTx != null && contract != null; + Contract = contract; + Msg = payload; + Data = payload.OracleSignature; + } + + public bool Verify(StoreView snapshot) + { + return Msg.Verify(snapshot); + } + } + + #endregion + + #region Protocols + + /// + /// HTTPS protocol + /// + internal static OracleHttpsProtocol HTTPSProtocol { get; } = new OracleHttpsProtocol(); + + #endregion + + // TODO: Fees + + private const long MaxGasFilter = 10_000_000; + + private long _isStarted = 0; + private Contract _lastContract; + private readonly NeoSystem _system; + private readonly IActorRef _localNode; + private CancellationTokenSource _cancel; + private readonly (Contract Contract, KeyPair Key)[] _accounts; + private readonly Func _snapshotFactory; + + /// + /// Number of threads for processing the oracle + /// + private Task[] _oracleTasks; + + /// + /// Sorted Queue for oracle tasks + /// + private readonly SortedBlockingCollection _processingQueue; + + /// + /// Oracle + /// + public Func Oracle { get; } + + /// + /// Pending user Transactions + /// + private readonly SortedConcurrentDictionary _pendingRequests; + + /// + /// Pending oracle response Transactions + /// + private readonly SortedConcurrentDictionary _pendingResponses; + + /// + /// Total maximum capacity of transactions the pool can hold. + /// + public int PendingCapacity => _pendingRequests.Capacity; + + /// + /// Total requests in the pool. + /// + public int PendingRequestCount => _pendingRequests.Count; + + /// + /// Total responses in the pool. + /// + public int PendingResponseCount => _pendingResponses.Count; + + /// + /// Is started + /// + public bool IsStarted => Interlocked.Read(ref _isStarted) == 1; + + /// + /// Constructor + /// + /// System + /// Local node + /// Wallet + /// Snapshot factory + /// Capacity + public OracleService(NeoSystem system, IActorRef localNode, Wallet wallet, Func snapshotFactory, int capacity) + { + Oracle = Process; + _system = system; + _localNode = localNode; + _snapshotFactory = snapshotFactory ?? new Func(() => Blockchain.Singleton.GetSnapshot()); + + // Find oracle account + + using var snapshot = _snapshotFactory(); + var oracles = NativeContract.Oracle.GetOracleValidators(snapshot) + .Select(u => Contract.CreateSignatureRedeemScript(u).ToScriptHash()); + + _accounts = wallet?.GetAccounts() + .Where(u => u.HasKey && !u.Lock && oracles.Contains(u.ScriptHash)) + .Select(u => (u.Contract, u.GetKey())) + .ToArray(); + + if (_accounts.Length == 0) + { + throw new ArgumentException("The wallet doesn't have any of the expected accounts"); + } + + // Create queue for pending request that should be processed + + _processingQueue = new SortedBlockingCollection + ( + Comparer>.Create(SortEnqueuedRequest), capacity + ); + + // Create internal collections for pending request/responses + + _pendingRequests = new SortedConcurrentDictionary + ( + Comparer>.Create(SortRequest), capacity + ); + _pendingResponses = new SortedConcurrentDictionary + ( + Comparer>.Create(SortResponse), capacity + ); + } + + /// + /// Receive AKKA Messages + /// + protected override void OnReceive(object message) + { + switch (message) + { + case StartMessage start: + { + Start(start.NumberOfTasks); + break; + } + case StopMessage _: + { + Stop(); + break; + } + case OraclePayload msg: + { + using var snapshot = _snapshotFactory(); + TryAddOracleResponse(snapshot, new ResponseItem(msg)); + break; + } + case Transaction tx: + { + // We only need to take care about the requests + + if (tx.Version == TransactionVersion.OracleRequest) + { + // If it's an OracleRequest and it's new, tell it to OracleService + + if (_pendingRequests.TryAdd(tx.Hash, new RequestItem(tx))) + { + using var snapshot = _snapshotFactory(); + + ReverifyPendingResponses(snapshot, tx.Hash); + + // Add it to the oracle processing queue + + _processingQueue.Add(tx.Hash, tx); + } + } + + break; + } + } + } + + /// + /// Start oracle + /// + /// Number of tasks + public void Start(byte numberOfTasks = 4) + { + if (Interlocked.Exchange(ref _isStarted, 1) != 0) return; + + // Create tasks + + Log($"OnStart: tasks={numberOfTasks}"); + + _cancel = new CancellationTokenSource(); + _oracleTasks = new Task[numberOfTasks]; + + for (int x = 0; x < _oracleTasks.Length; x++) + { + _oracleTasks[x] = new Task(() => + { + foreach (var tx in _processingQueue.GetConsumingEnumerable(_cancel.Token)) + { + ProcessRequestTransaction(tx); + } + }, + _cancel.Token); + } + + // Start tasks + + foreach (var task in _oracleTasks) task.Start(); + } + + /// + /// Stop oracle + /// + public void Stop() + { + if (Interlocked.Exchange(ref _isStarted, 0) != 1) return; + + Log("OnStop"); + + _cancel.Cancel(); + + for (int x = 0; x < _oracleTasks.Length; x++) + { + try { _oracleTasks[x].Wait(); } catch { } + try { _oracleTasks[x].Dispose(); } catch { } + } + + _cancel.Dispose(); + _cancel = null; + _oracleTasks = null; + + // Clean queue + + _processingQueue.Clear(); + _pendingRequests.Clear(); + _pendingResponses.Clear(); + } + + /// + /// Log + /// + /// Message + /// Log level + private static void Log(string message, LogLevel level = LogLevel.Info) + { + Utility.Log(nameof(OracleService), level, message); + } + + /// + /// Process request transaction + /// + /// Transaction + private void ProcessRequestTransaction(Transaction tx) + { + Log($"Process request tx: hash={tx.Hash}"); + + var oracle = new OracleExecutionCache(Process); + using var snapshot = _snapshotFactory(); + using (var engine = new ApplicationEngine(TriggerType.Application, tx, snapshot, tx.SystemFee, false, oracle)) + { + engine.LoadScript(tx.Script); + + if (engine.Execute() != VMState.HALT) + { + // If the request TX will FAULT we can save space removing the downloaded data + // But user paid for it, maybe it will not fault during OnPerists + + // oracle.Clear(); + } + } + + // Check the oracle contract + + var contract = NativeContract.Oracle.GetOracleMultiSigContract(snapshot); + + // Check the cached contract + + if (_lastContract?.ScriptHash != contract.ScriptHash) + { + // Reduce the memory load using the same Contract class + + _lastContract = contract; + } + + // Create deterministic oracle response + + var responseTx = CreateResponseTransaction(snapshot, oracle, contract, tx); + + Log($"Generated response tx: requestHash={tx.Hash} responseHash={responseTx.Hash}"); + + foreach (var account in _accounts) + { + // Sign the transaction + + var signatureTx = responseTx.Sign(account.Key); + + // Create the payload + + var response = new OraclePayload() + { + OraclePub = account.Key.PublicKey, + OracleSignature = new OracleResponseSignature() + { + OracleExecutionCacheHash = oracle.Hash, + Signature = signatureTx, + TransactionRequestHash = tx.Hash + } + }; + + var signatureMsg = response.Sign(account.Key); + var signPayload = new ContractParametersContext(response); + + if (signPayload.AddSignature(account.Contract, response.OraclePub, signatureMsg) && signPayload.Completed) + { + response.Witness = signPayload.GetWitnesses()[0]; + + if (TryAddOracleResponse(snapshot, new ResponseItem(response, contract, responseTx))) + { + // Send my signature by P2P + + Log($"Send oracle signature: oracle={response.OraclePub.ToString()} request={tx.Hash} response={response.Hash}"); + + _localNode.Tell(new LocalNode.SendDirectly { Inventory = response }); + } + } + } + } + + /// + /// Create Oracle response transaction + /// We need to create a deterministic TX for this result/oracleRequest + /// + /// Snapshot + /// Oracle + /// Contract + /// Request Hash + /// Transaction + private static Transaction CreateResponseTransaction(SnapshotView snapshot, OracleExecutionCache oracle, Contract contract, Transaction requestTx) + { + using ScriptBuilder script = new ScriptBuilder(); + script.EmitAppCall(NativeContract.Oracle.Hash, "setOracleResponse", requestTx.Hash, IO.Helper.ToArray(oracle)); + + // Calculate system fee + + long systemFee; + using (var engine = ApplicationEngine.Run + ( + script.ToArray(), snapshot.Clone(), + new ManualWitness(contract.ScriptHash), testMode: true + )) + { + if (engine.State != VMState.HALT) + { + // This should never happend + + throw new ApplicationException(); + } + + systemFee = engine.GasConsumed; + } + + // Generate tx + + var tx = new Transaction() + { + Version = TransactionVersion.OracleResponse, + ValidUntilBlock = requestTx.ValidUntilBlock, + Attributes = new TransactionAttribute[0], + OracleRequestTx = requestTx.Hash, + Sender = contract.ScriptHash, + Witnesses = new Witness[0], + Script = script.ToArray(), + NetworkFee = 0, + Nonce = requestTx.Nonce, + SystemFee = systemFee, + Cosigners = new Cosigner[] + { + new Cosigner() + { + Account = contract.ScriptHash, + AllowedContracts = new UInt160[]{ NativeContract.Oracle.Hash }, + Scopes = WitnessScope.CustomContracts + } + } + }; + + // Calculate network fee + + int size = tx.Size; + + tx.NetworkFee += Wallet.CalculateNetworkFee(contract.Script, ref size); + tx.NetworkFee += size * NativeContract.Policy.GetFeePerByte(snapshot); + + return tx; + } + + /// + /// Try add oracle response payload + /// + /// Snapshot + /// Response + /// True if it was added + private bool TryAddOracleResponse(StoreView snapshot, ResponseItem response) + { + if (!response.Verify(snapshot)) + { + Log($"Received wrong signed payload: oracle={response.OraclePub.ToString()} request={response.TransactionRequestHash} response={response.MsgHash}", LogLevel.Error); + + return false; + } + + if (!response.IsMine) + { + Log($"Received oracle signature: oracle={response.OraclePub.ToString()} request={response.TransactionRequestHash} response={response.MsgHash}"); + } + + // Find the request tx + + if (_pendingRequests.TryGetValue(response.TransactionRequestHash, out var request)) + { + // Append the signature if it's possible + + if (request.AddSignature(response)) + { + if (request.IsCompleted) + { + Log($"Send response tx: oracle={response.OraclePub.ToString()} hash={request.ResponseTransaction.Hash}"); + + // Done! Send to mem pool + + _pendingRequests.TryRemove(response.TransactionRequestHash, out _); + _pendingResponses.TryRemove(response.TransactionRequestHash, out _); + _localNode.Tell(new LocalNode.Relay { Inventory = request.ResponseTransaction }); + + // Request should be already there, but it could be removed because the mempool was full during the process + + _localNode.Tell(new LocalNode.Relay { Inventory = request.RequestTransaction }); + } + + return true; + } + } + else + { + // Ask for the request tx because it's not in my pool + + _localNode.Tell(Message.Create(MessageCommand.GetData, InvPayload.Create(InventoryType.TX, response.TransactionRequestHash))); + } + + // Save this payload for check it later + + if (_pendingResponses.TryGetValue(response.TransactionRequestHash, out var collection, new ResponseCollection(response))) + { + if (collection != null) + { + // It was getted + + return collection.Add(response); + } + + // It was added + + return true; + } + + return false; + } + + /// + /// Reverify pending responses + /// + /// Snapshot + /// Request transaction hash + private void ReverifyPendingResponses(StoreView snapshot, UInt256 requestTx) + { + // If the response is pending, we should process it now + + if (_pendingResponses.TryRemove(requestTx, out var collection)) + { + // Order by Transaction + + foreach (var entry in collection) + { + TryAddOracleResponse(snapshot, entry); + } + } + } + + #region Sorts + + private static int Sort(KeyValuePair a, KeyValuePair b) + { + // Sort by if it's mine or not + + int av = a.Value.IsMine ? 1 : 0; + int bv = b.Value.IsMine ? 1 : 0; + int ret = av.CompareTo(bv); + + if (ret != 0) return ret; + + // Sort by time + + return a.Value.Timestamp.CompareTo(b.Value.Timestamp); + } + + private static int SortRequest(KeyValuePair a, KeyValuePair b) + { + return a.Value.CompareTo(b.Value); + } + + private static int SortEnqueuedRequest(KeyValuePair a, KeyValuePair b) + { + var otherTx = b.Value; + if (otherTx == null) return 1; + + // Fees sorted ascending + + var tx = a.Value; + int ret = tx.FeePerByte.CompareTo(otherTx.FeePerByte); + if (ret != 0) return ret; + ret = tx.NetworkFee.CompareTo(otherTx.NetworkFee); + if (ret != 0) return ret; + + // Transaction hash sorted descending + + return otherTx.Hash.CompareTo(tx.Hash); + } + + private static int SortResponse(KeyValuePair a, KeyValuePair b) + { + // Sort by number of signatures + + var comp = a.Value.Count.CompareTo(b.Value.Count); + if (comp != 0) return comp; + + // Sort by if has my signature or not + + comp = a.Value.MineCount.CompareTo(b.Value.MineCount); + if (comp != 0) return comp; + + // Sort by age + + return a.Value.Timestamp.CompareTo(b.Value.Timestamp); + } + + #endregion + + #region Public Static methods + + /// + /// Process oracle request + /// + /// Request + /// Return Oracle response + public static OracleResponse Process(OracleRequest request) + { + try + { + return request switch + { + OracleHttpsRequest https => HTTPSProtocol.Process(https), + _ => OracleResponse.CreateError(request.Hash, OracleResultError.ProtocolError), + }; + } + catch + { + return OracleResponse.CreateError(request.Hash, OracleResultError.ServerError); + } + } + + /// + /// Filter response + /// + /// Input + /// Filter + /// Result + /// True if was filtered + public static bool FilterResponse(string input, OracleFilter filter, out string result, out long gasCost) + { + if (filter == null) + { + result = input; + gasCost = 0; + return true; + } + + if (FilterResponse(Encoding.UTF8.GetBytes(input), filter, out var bufferResult, out gasCost)) + { + result = Encoding.UTF8.GetString(bufferResult); + return true; + } + + result = null; + return false; + } + + /// + /// Filter response + /// + /// Input + /// Filter + /// Result + /// Gas cost + /// True if was filtered + public static bool FilterResponse(byte[] input, OracleFilter filter, out byte[] result, out long gasCost) + { + if (filter == null) + { + result = input; + gasCost = 0; + return true; + } + + // Prepare the execution + + using ScriptBuilder script = new ScriptBuilder(); + script.EmitSysCall(InteropService.Contract.CallEx, filter.ContractHash, filter.FilterMethod, input, (byte)CallFlags.None); + + // Execute + + using var engine = new ApplicationEngine(TriggerType.Application, null, null, MaxGasFilter, false, null); + + engine.LoadScript(script.ToArray(), CallFlags.None); + + if (engine.Execute() != VMState.HALT || engine.ResultStack.Count != 1) + { + result = null; + gasCost = engine.GasConsumed; + return false; + } + + // Extract the filtered item + + result = engine.ResultStack.Pop().GetSpan().ToArray(); + gasCost = engine.GasConsumed; + return true; + } + + #endregion + + #region Akka + + public static Props Props(NeoSystem system, IActorRef localNode, Wallet wallet) + { + return Akka.Actor.Props.Create(() => new OracleService(system, localNode, wallet, null, ProtocolSettings.Default.MemoryPoolMaxTransactions)); + } + + #endregion + } +} diff --git a/src/neo/Oracle/OracleWalletBehaviour.cs b/src/neo/Oracle/OracleWalletBehaviour.cs new file mode 100644 index 0000000000..08adac91f9 --- /dev/null +++ b/src/neo/Oracle/OracleWalletBehaviour.cs @@ -0,0 +1,20 @@ +namespace Neo.Oracle +{ + public enum OracleWalletBehaviour + { + /// + /// If an Oracle syscall was found, the tx will fault + /// + WithoutOracle, + + /// + /// If an Oracle syscall was found, the tx will be relayed without any check (The gas cost could be more if the result it's different) + /// + OracleWithoutAssert, + + /// + /// If an Oracle syscall was found, it will be added an asert at the begining of the script + /// + OracleWithAssert + } +} diff --git a/src/neo/Oracle/Protocols/Https/HttpMethod.cs b/src/neo/Oracle/Protocols/Https/HttpMethod.cs new file mode 100644 index 0000000000..5ce500b4fb --- /dev/null +++ b/src/neo/Oracle/Protocols/Https/HttpMethod.cs @@ -0,0 +1,7 @@ +namespace Neo.Oracle.Protocols.Https +{ + public enum HttpMethod : byte + { + GET = 0x00 + } +} diff --git a/src/neo/Oracle/Protocols/Https/OracleHttpsProtocol.cs b/src/neo/Oracle/Protocols/Https/OracleHttpsProtocol.cs new file mode 100644 index 0000000000..99cbd8846f --- /dev/null +++ b/src/neo/Oracle/Protocols/Https/OracleHttpsProtocol.cs @@ -0,0 +1,188 @@ +using Neo.Ledger; +using Neo.SmartContract.Native; +using Neo.SmartContract.Native.Oracle; +using System; +using System.Net; +using System.Net.Http; +using System.Threading; +using System.Threading.Tasks; + +namespace Neo.Oracle.Protocols.Https +{ + internal class OracleHttpsProtocol + { + private long _lastHeight = -1; + + /// + /// Timeout + /// + public TimeSpan TimeOut { get; internal set; } + + /// + /// Allow private host + /// + public bool AllowPrivateHost { get; internal set; } = false; + + /// + /// Constructor + /// + public OracleHttpsProtocol() + { + LoadConfig(); + } + + /// + /// Load config + /// + private void LoadConfig() + { + // Check if it's the same + + var height = Blockchain.Singleton.Height; + if (Interlocked.Exchange(ref _lastHeight, height) == height) + { + return; + } + + // Load the configuration + + ushort seconds; + using (var snapshot = Blockchain.Singleton.GetSnapshot()) + { + seconds = (ushort)NativeContract.Oracle.GetConfig(snapshot, HttpConfig.Timeout).ToBigInteger(); + } + + TimeOut = TimeSpan.FromMilliseconds(seconds); + } + + // + /// Process HTTP oracle request + /// + /// Request + /// Oracle result + public OracleResponse Process(OracleHttpsRequest request) + { + LoadConfig(); + + if (!AllowPrivateHost && IsInternal(Dns.GetHostEntry(request.URL.Host))) + { + // Don't allow private host in order to prevent SSRF + + return OracleResponse.CreateError(request.Hash, OracleResultError.PolicyError); + } + + Task result; + using var handler = new HttpClientHandler + { + // TODO: Accept all certificates + ServerCertificateCustomValidationCallback = HttpClientHandler.DangerousAcceptAnyServerCertificateValidator + }; + using var client = new HttpClient(handler); + + switch (request.Method) + { + case HttpMethod.GET: + { + result = client.GetAsync(request.URL); + break; + } + //case HttpMethod.POST: + // { + // result = client.PostAsync(httpRequest.URL, new ByteArrayContent(httpRequest.Body)); + // break; + // } + //case HttpMethod.PUT: + // { + // result = client.PutAsync(httpRequest.URL, new ByteArrayContent(httpRequest.Body)); + // break; + // } + //case HttpMethod.DELETE: + // { + // result = client.DeleteAsync(httpRequest.URL); + // break; + // } + default: + { + return OracleResponse.CreateError(request.Hash, OracleResultError.PolicyError); + } + } + + if (!result.Wait(TimeOut)) + { + // Timeout + + return OracleResponse.CreateError(request.Hash, OracleResultError.Timeout); + } + + if (!result.Result.IsSuccessStatusCode) + { + // Error with response + + return OracleResponse.CreateError(request.Hash, OracleResultError.ResponseError); + } + + string ret; + var taskRet = result.Result.Content.ReadAsStringAsync(); + + if (!taskRet.Wait(TimeOut)) + { + // Timeout + + return OracleResponse.CreateError(request.Hash, OracleResultError.Timeout); + } + else + { + // Good response + + ret = taskRet.Result; + } + + // Filter + + if (!OracleService.FilterResponse(ret, request.Filter, out string filteredStr, out var gasCost)) + { + return OracleResponse.CreateError(request.Hash, OracleResultError.FilterError, gasCost); + } + + return OracleResponse.CreateResult(request.Hash, filteredStr, gasCost); + } + + internal static bool IsInternal(IPHostEntry entry) + { + foreach (var ip in entry.AddressList) + { + if (IsInternal(ip)) return true; + } + + return false; + } + + /// + /// ::1 - IPv6 loopback + /// 10.0.0.0 - 10.255.255.255 (10/8 prefix) + /// 127.0.0.0 - 127.255.255.255 (127/8 prefix) + /// 172.16.0.0 - 172.31.255.255 (172.16/12 prefix) + /// 192.168.0.0 - 192.168.255.255 (192.168/16 prefix) + /// + /// Address + /// True if it was an internal address + internal static bool IsInternal(IPAddress ipAddress) + { + if (IPAddress.IsLoopback(ipAddress)) return true; + if (IPAddress.Broadcast.Equals(ipAddress)) return true; + if (IPAddress.Any.Equals(ipAddress)) return true; + if (IPAddress.IPv6Any.Equals(ipAddress)) return true; + if (IPAddress.IPv6Loopback.Equals(ipAddress)) return true; + + var ip = ipAddress.GetAddressBytes(); + switch (ip[0]) + { + case 10: + case 127: return true; + case 172: return ip[1] >= 16 && ip[1] < 32; + case 192: return ip[1] == 168; + default: return false; + } + } + } +} diff --git a/src/neo/Oracle/Protocols/Https/OracleHttpsRequest.cs b/src/neo/Oracle/Protocols/Https/OracleHttpsRequest.cs new file mode 100644 index 0000000000..941dbee795 --- /dev/null +++ b/src/neo/Oracle/Protocols/Https/OracleHttpsRequest.cs @@ -0,0 +1,60 @@ +using Neo.IO; +using System; +using System.IO; +using System.Text; + +namespace Neo.Oracle.Protocols.Https +{ + public class OracleHttpsRequest : OracleRequest + { + /// + /// Type + /// + public override OracleRequestType Type => OracleRequestType.HTTPS; + + /// + /// HTTP Methods + /// + public HttpMethod Method { get; set; } + + /// + /// URL + /// + public Uri URL { get; set; } + + /// + /// Filter + /// + public OracleFilter Filter { get; set; } + + /// + /// Get hash data + /// + /// Hash data + protected override byte[] GetHashData() + { + using (var stream = new MemoryStream()) + using (var writer = new BinaryWriter(stream, Encoding.UTF8, true)) + { + writer.Write((byte)Type); + writer.Write((byte)Method); + writer.WriteVarString(URL.ToString()); + + if (Filter != null) + { + writer.Write(0x01); + writer.Write(Filter.ContractHash); + writer.WriteVarString(Filter.FilterMethod); + } + else + { + writer.Write(0x00); + } + + writer.Flush(); + + return stream.ToArray(); + } + } + } +} diff --git a/src/neo/SmartContract/ApplicationEngine.cs b/src/neo/SmartContract/ApplicationEngine.cs index fc11594a03..39648363d2 100644 --- a/src/neo/SmartContract/ApplicationEngine.cs +++ b/src/neo/SmartContract/ApplicationEngine.cs @@ -1,5 +1,6 @@ using Neo.Ledger; using Neo.Network.P2P.Payloads; +using Neo.Oracle; using Neo.Persistence; using Neo.VM; using Neo.VM.Types; @@ -24,6 +25,7 @@ public partial class ApplicationEngine : ExecutionEngine public TriggerType Trigger { get; } public IVerifiable ScriptContainer { get; } public StoreView Snapshot { get; } + public OracleExecutionCache OracleCache { get; internal set; } public long GasConsumed { get; private set; } = 0; public long GasLeft => testMode ? -1 : gas_amount - GasConsumed; @@ -33,13 +35,14 @@ public partial class ApplicationEngine : ExecutionEngine public IReadOnlyList Notifications => notifications; internal Dictionary InvocationCounter { get; } = new Dictionary(); - public ApplicationEngine(TriggerType trigger, IVerifiable container, StoreView snapshot, long gas, bool testMode = false) + public ApplicationEngine(TriggerType trigger, IVerifiable container, StoreView snapshot, long gas, bool testMode = false, OracleExecutionCache oracleCache = null) { this.gas_amount = GasFree + gas; this.testMode = testMode; this.Trigger = trigger; this.ScriptContainer = container; this.Snapshot = snapshot; + this.OracleCache = oracleCache; } internal T AddDisposable(T disposable) where T : IDisposable @@ -114,20 +117,20 @@ private static Block CreateDummyBlock(StoreView snapshot) } public static ApplicationEngine Run(byte[] script, StoreView snapshot, - IVerifiable container = null, Block persistingBlock = null, int offset = 0, bool testMode = false, long extraGAS = default) + IVerifiable container = null, Block persistingBlock = null, int offset = 0, bool testMode = false, long extraGAS = default, OracleExecutionCache oracle = null) { snapshot.PersistingBlock = persistingBlock ?? snapshot.PersistingBlock ?? CreateDummyBlock(snapshot); - ApplicationEngine engine = new ApplicationEngine(TriggerType.Application, container, snapshot, extraGAS, testMode); + ApplicationEngine engine = new ApplicationEngine(TriggerType.Application, container, snapshot, extraGAS, testMode, oracle); engine.LoadScript(script).InstructionPointer = offset; engine.Execute(); return engine; } - public static ApplicationEngine Run(byte[] script, IVerifiable container = null, Block persistingBlock = null, int offset = 0, bool testMode = false, long extraGAS = default) + public static ApplicationEngine Run(byte[] script, IVerifiable container = null, Block persistingBlock = null, int offset = 0, bool testMode = false, long extraGAS = default, OracleExecutionCache oracle = null) { using (SnapshotView snapshot = Blockchain.Singleton.GetSnapshot()) { - return Run(script, snapshot, container, persistingBlock, offset, testMode, extraGAS); + return Run(script, snapshot, container, persistingBlock, offset, testMode, extraGAS, oracle); } } diff --git a/src/neo/SmartContract/InteropService.Oracle.cs b/src/neo/SmartContract/InteropService.Oracle.cs new file mode 100644 index 0000000000..9f6b1cfc6e --- /dev/null +++ b/src/neo/SmartContract/InteropService.Oracle.cs @@ -0,0 +1,119 @@ +using Neo.IO; +using Neo.Network.P2P.Payloads; +using Neo.Oracle; +using Neo.Oracle.Protocols.Https; +using Neo.SmartContract.Native; +using Neo.VM.Types; +using System; +using System.Text; + +namespace Neo.SmartContract +{ + partial class InteropService + { + public static class Oracle + { + public static readonly uint Neo_Oracle_Get = Register("Neo.Oracle.Get", Oracle_Get, 0, TriggerType.Application, CallFlags.None); + public static readonly uint Neo_Oracle_Hash = Register("Neo.Oracle.Hash", Oracle_Hash, 0, TriggerType.Application, CallFlags.None); + + /// + /// Oracle get the hash of the current OracleFlow [Request/Response] + /// + private static bool Oracle_Hash(ApplicationEngine engine) + { + if (engine.OracleCache == null) + { + engine.Push(StackItem.Null); + } + else + { + engine.Push(engine.OracleCache.Hash.ToArray()); + } + + return true; + } + + /// + /// Oracle Get + /// string url, [UInt160 filter], [string filterMethod] + /// + private static bool Oracle_Get(ApplicationEngine engine) + { + if (engine.OracleCache == null) + { + // We should enter here only during OnPersist with the OracleRequestTx + + if (engine.ScriptContainer is Transaction tx) + { + // Read Oracle Response + + engine.OracleCache = NativeContract.Oracle.ConsumeOracleResponse(engine.Snapshot, tx.Hash); + + // If it doesn't exist, fault + + if (engine.OracleCache == null) + { + return false; + } + } + else + { + return false; + } + } + if (!engine.TryPop(out string urlItem) || !Uri.TryCreate(urlItem, UriKind.Absolute, out var url)) return false; + if (!engine.TryPop(out StackItem filterContractItem)) return false; + if (!engine.TryPop(out StackItem filterMethodItem)) return false; + + // Create filter + + OracleFilter filter = null; + + if (!filterContractItem.IsNull) + { + if (filterContractItem is PrimitiveType filterContract && + filterMethodItem is PrimitiveType filterMethod) + { + filter = new OracleFilter() + { + ContractHash = new UInt160(filterContract.Span), + FilterMethod = Encoding.UTF8.GetString(filterMethod.Span) + }; + } + else + { + return false; + } + } + + // Create request + + OracleRequest request; + switch (url.Scheme.ToLowerInvariant()) + { + case "https": + { + request = new OracleHttpsRequest() + { + Method = HttpMethod.GET, + URL = url, + Filter = filter + }; + break; + } + default: return false; + } + + // Execute the oracle request + + if (engine.OracleCache.TryGet(request, out var response)) + { + engine.Push(response.Result ?? StackItem.Null); + return true; + } + + return false; + } + } + } +} diff --git a/src/neo/SmartContract/Native/NativeContract.cs b/src/neo/SmartContract/Native/NativeContract.cs index 7e402ac00a..f501404ff8 100644 --- a/src/neo/SmartContract/Native/NativeContract.cs +++ b/src/neo/SmartContract/Native/NativeContract.cs @@ -4,6 +4,7 @@ using Neo.Ledger; using Neo.Persistence; using Neo.SmartContract.Manifest; +using Neo.SmartContract.Native.Oracle; using Neo.SmartContract.Native.Tokens; using Neo.VM; using Neo.VM.Types; @@ -25,6 +26,7 @@ public abstract class NativeContract public static NeoToken NEO { get; } = new NeoToken(); public static GasToken GAS { get; } = new GasToken(); public static PolicyContract Policy { get; } = new PolicyContract(); + public static OracleContract Oracle { get; } = new OracleContract(); public abstract string ServiceName { get; } public uint ServiceHash { get; } diff --git a/src/neo/SmartContract/Native/Oracle/HttpConfig.cs b/src/neo/SmartContract/Native/Oracle/HttpConfig.cs new file mode 100644 index 0000000000..877467c0f7 --- /dev/null +++ b/src/neo/SmartContract/Native/Oracle/HttpConfig.cs @@ -0,0 +1,7 @@ +namespace Neo.SmartContract.Native.Oracle +{ + public class HttpConfig + { + public const string Timeout = "HttpTimeout"; + } +} diff --git a/src/neo/SmartContract/Native/Oracle/OracleContract.cs b/src/neo/SmartContract/Native/Oracle/OracleContract.cs new file mode 100644 index 0000000000..6170d8d7d3 --- /dev/null +++ b/src/neo/SmartContract/Native/Oracle/OracleContract.cs @@ -0,0 +1,298 @@ +using Neo.Cryptography.ECC; +using Neo.IO; +using Neo.Ledger; +using Neo.Oracle; +using Neo.Persistence; +using Neo.SmartContract.Manifest; +using Neo.VM; +using Neo.VM.Types; +using System; +using System.Collections.Generic; +using System.Linq; +using System.Numerics; +using System.Text; +using Array = Neo.VM.Types.Array; + +namespace Neo.SmartContract.Native.Oracle +{ + public sealed class OracleContract : NativeContract + { + public override string ServiceName => "Neo.Native.Oracle"; + + public override int Id => -4; + + internal const byte Prefix_Validator = 24; + internal const byte Prefix_Config = 25; + internal const byte Prefix_PerRequestFee = 26; + internal const byte Prefix_OracleResponse = 27; + + public OracleContract() + { + Manifest.Features = ContractFeatures.HasStorage; + } + + internal override bool Initialize(ApplicationEngine engine) + { + if (!base.Initialize(engine)) return false; + if (GetPerRequestFee(engine.Snapshot) != 0) return false; + + engine.Snapshot.Storages.Add(CreateStorageKey(Prefix_Config, Encoding.UTF8.GetBytes(HttpConfig.Timeout)), new StorageItem + { + Value = new ByteString(BitConverter.GetBytes(5000)).GetSpan().ToArray() + }); + engine.Snapshot.Storages.Add(CreateStorageKey(Prefix_PerRequestFee), new StorageItem + { + Value = BitConverter.GetBytes(1000) + }); + return true; + } + + /// + /// Set Oracle Response Only + /// + [ContractMethod(0_03000000, ContractParameterType.Boolean, ParameterTypes = new[] { ContractParameterType.ByteArray, ContractParameterType.ByteArray }, ParameterNames = new[] { "transactionHash", "oracleResponse" })] + private StackItem SetOracleResponse(ApplicationEngine engine, Array args) + { + if (args.Count != 2) return false; + + UInt160 account = GetOracleMultiSigAddress(engine.Snapshot); + if (!InteropService.Runtime.CheckWitnessInternal(engine, account)) return false; + + // This only can be called by the oracle's multi signature + + var txHash = args[0].GetSpan().AsSerializable(); + var response = args[1].GetSpan().AsSerializable(); + + StorageItem storage = engine.Snapshot.Storages.GetAndChange(CreateStorageKey(Prefix_OracleResponse, txHash.ToArray()), () => new StorageItem()); + storage.Value = IO.Helper.ToArray(response); + return false; + } + + /// + /// Check if the response it's already stored + /// + /// Snapshot + /// Transaction Hash + public bool ContainsResponse(StoreView snapshot, UInt256 txHash) + { + StorageKey key = CreateStorageKey(Prefix_OracleResponse, txHash.ToArray()); + return snapshot.Storages.TryGet(key) != null; + } + + /// + /// Consume Oracle Response + /// + /// Snapshot + /// Transaction Hash + public OracleExecutionCache ConsumeOracleResponse(StoreView snapshot, UInt256 txHash) + { + StorageKey key = CreateStorageKey(Prefix_OracleResponse, txHash.ToArray()); + StorageItem storage = snapshot.Storages.TryGet(key); + if (storage == null) return null; + + OracleExecutionCache ret = storage.Value.AsSerializable(); + + // It should be cached by the ApplicationEngine so we can save space removing it + + snapshot.Storages.Delete(key); + return ret; + } + + /// + /// Consensus node can delegate third party to operate Oracle nodes + /// + /// VM + /// Parameter Array + /// Returns true if the execution is successful, otherwise returns false + [ContractMethod(0_03000000, ContractParameterType.Boolean, ParameterTypes = new[] { ContractParameterType.ByteArray, ContractParameterType.ByteArray }, ParameterNames = new[] { "consignorPubKey", "consigneePubKey" })] + private StackItem DelegateOracleValidator(ApplicationEngine engine, Array args) + { + StoreView snapshot = engine.Snapshot; + ECPoint consignorPubKey = args[0].GetSpan().AsSerializable(); + ECPoint consigneePubKey = args[1].GetSpan().AsSerializable(); + ECPoint[] cnPubKeys = NEO.GetValidators(snapshot); + if (!cnPubKeys.Contains(consignorPubKey)) return false; + UInt160 account = Contract.CreateSignatureRedeemScript(consignorPubKey).ToScriptHash(); + if (!InteropService.Runtime.CheckWitnessInternal(engine, account)) return false; + StorageKey key = CreateStorageKey(Prefix_Validator, consignorPubKey); + StorageItem item = snapshot.Storages.GetAndChange(key, () => new StorageItem()); + item.Value = consigneePubKey.ToArray(); + + byte[] prefixKey = StorageKey.CreateSearchPrefix(Id, new[] { Prefix_Validator }); + List delegatedOracleValidators = snapshot.Storages.Find(prefixKey).Select(p => + ( + p.Key.Key.AsSerializable(1) + )).ToList(); + foreach (var validator in delegatedOracleValidators) + { + if (!cnPubKeys.Contains(validator)) + { + snapshot.Storages.Delete(CreateStorageKey(Prefix_Validator, validator)); + } + } + return true; + } + + /// + /// Get current authorized Oracle validator. + /// + /// VM + /// Parameter Array + /// Authorized Oracle validator + [ContractMethod(0_01000000, ContractParameterType.Array)] + private StackItem GetOracleValidators(ApplicationEngine engine, Array args) + { + return new Array(engine.ReferenceCounter, GetOracleValidators(engine.Snapshot).Select(p => (StackItem)p.ToArray())); + } + + /// + /// Get current authorized Oracle validator + /// + /// snapshot + /// Authorized Oracle validator + public ECPoint[] GetOracleValidators(StoreView snapshot) + { + ECPoint[] cnPubKeys = NEO.GetValidators(snapshot); + ECPoint[] oraclePubKeys = new ECPoint[cnPubKeys.Length]; + System.Array.Copy(cnPubKeys, oraclePubKeys, cnPubKeys.Length); + for (int index = 0; index < oraclePubKeys.Length; index++) + { + var oraclePubKey = oraclePubKeys[index]; + StorageKey key = CreateStorageKey(Prefix_Validator, oraclePubKey); + ECPoint delegatePubKey = snapshot.Storages.TryGet(key)?.Value.AsSerializable(); + if (delegatePubKey != null) { oraclePubKeys[index] = delegatePubKey; } + } + return oraclePubKeys.Distinct().ToArray(); + } + + /// + /// Get number of current authorized Oracle validator + /// + /// VM + /// Parameter Array + /// The number of authorized Oracle validator + [ContractMethod(0_01000000, ContractParameterType.Integer)] + private StackItem GetOracleValidatorsCount(ApplicationEngine engine, Array args) + { + return GetOracleValidatorsCount(engine.Snapshot); + } + + /// + /// Get number of current authorized Oracle validator + /// + /// snapshot + /// The number of authorized Oracle validator + public BigInteger GetOracleValidatorsCount(StoreView snapshot) + { + return GetOracleValidators(snapshot).Length; + } + + /// + /// Create a Oracle multisignature contract + /// + /// snapshot + /// Oracle multisignature address + public Contract GetOracleMultiSigContract(StoreView snapshot) + { + ECPoint[] oracleValidators = GetOracleValidators(snapshot); + return Contract.CreateMultiSigContract(oracleValidators.Length - (oracleValidators.Length - 1) / 3, oracleValidators); + } + + /// + /// Create a Oracle multisignature address + /// + /// snapshot + /// Oracle multisignature address + public UInt160 GetOracleMultiSigAddress(StoreView snapshot) + { + return GetOracleMultiSigContract(snapshot).ScriptHash; + } + + /// + /// Set HttpConfig + /// + /// VM + /// Parameter Array + /// Returns true if the execution is successful, otherwise returns false + [ContractMethod(0_03000000, ContractParameterType.Boolean, ParameterTypes = new[] { ContractParameterType.String, ContractParameterType.ByteArray }, ParameterNames = new[] { "configKey", "configValue" })] + private StackItem SetConfig(ApplicationEngine engine, Array args) + { + StoreView snapshot = engine.Snapshot; + UInt160 account = GetOracleMultiSigAddress(snapshot); + if (!InteropService.Runtime.CheckWitnessInternal(engine, account)) return false; + string key = args[0].GetString(); + ByteString value = args[1].GetSpan().ToArray(); + StorageItem storage = snapshot.Storages.GetAndChange(CreateStorageKey(Prefix_Config, Encoding.UTF8.GetBytes(key))); + storage.Value = value.GetSpan().ToArray(); + return true; + } + + /// + /// Get HttpConfig + /// + /// VM + /// value + [ContractMethod(0_01000000, ContractParameterType.Array, ParameterTypes = new[] { ContractParameterType.String }, ParameterNames = new[] { "configKey" })] + private StackItem GetConfig(ApplicationEngine engine, Array args) + { + StoreView snapshot = engine.Snapshot; + string key = args[0].GetString(); + return GetConfig(snapshot, key); + } + + /// + /// Get HttpConfig + /// + /// snapshot + /// key + /// value + public ByteString GetConfig(StoreView snapshot, string key) + { + StorageItem storage = snapshot.Storages.GetAndChange(CreateStorageKey(Prefix_Config, Encoding.UTF8.GetBytes(key))); + return storage.Value; + } + + /// + /// Set PerRequestFee + /// + /// VM + /// Parameter Array + /// Returns true if the execution is successful, otherwise returns false + [ContractMethod(0_03000000, ContractParameterType.Boolean, ParameterTypes = new[] { ContractParameterType.Integer }, ParameterNames = new[] { "fee" })] + private StackItem SetPerRequestFee(ApplicationEngine engine, Array args) + { + StoreView snapshot = engine.Snapshot; + UInt160 account = GetOracleMultiSigAddress(snapshot); + if (!InteropService.Runtime.CheckWitnessInternal(engine, account)) return false; + int perRequestFee = (int)args[0].GetBigInteger(); + if (perRequestFee <= 0) return false; + StorageItem storage = snapshot.Storages.GetAndChange(CreateStorageKey(Prefix_PerRequestFee)); + storage.Value = BitConverter.GetBytes(perRequestFee); + return true; + } + + /// + /// Get PerRequestFee + /// + /// VM + /// Parameter Array + /// Value + [ContractMethod(0_01000000, ContractParameterType.Integer, SafeMethod = true)] + private StackItem GetPerRequestFee(ApplicationEngine engine, Array args) + { + return new Integer(GetPerRequestFee(engine.Snapshot)); + } + + /// + /// Get PerRequestFee + /// + /// snapshot + /// Value + public int GetPerRequestFee(StoreView snapshot) + { + StorageItem storage = snapshot.Storages.TryGet(CreateStorageKey(Prefix_PerRequestFee)); + if (storage is null) return 0; + return BitConverter.ToInt32(storage.Value); + } + } +} diff --git a/src/neo/Wallets/Wallet.cs b/src/neo/Wallets/Wallet.cs index 4831a82c96..f5fc46e4de 100644 --- a/src/neo/Wallets/Wallet.cs +++ b/src/neo/Wallets/Wallet.cs @@ -2,6 +2,8 @@ using Neo.IO; using Neo.Ledger; using Neo.Network.P2P.Payloads; +using Neo.Oracle; +using Neo.Oracle.Protocols.Https; using Neo.Persistence; using Neo.SmartContract; using Neo.SmartContract.Native; @@ -210,7 +212,7 @@ public virtual WalletAccount Import(string nep2, string passphrase, int N = 1638 return account; } - public Transaction MakeTransaction(TransferOutput[] outputs, UInt160 from = null) + public Transaction MakeTransaction(TransferOutput[] outputs, UInt160 from = null, OracleWalletBehaviour oracle = OracleWalletBehaviour.OracleWithAssert) { UInt160[] accounts; if (from is null) @@ -275,11 +277,11 @@ public Transaction MakeTransaction(TransferOutput[] outputs, UInt160 from = null Account = new UInt160(p.ToArray()) }).ToArray(); - return MakeTransaction(snapshot, script, new TransactionAttribute[0], cosigners, balances_gas); + return MakeTransaction(snapshot, script, new TransactionAttribute[0], cosigners, balances_gas, oracle); } } - public Transaction MakeTransaction(byte[] script, UInt160 sender = null, TransactionAttribute[] attributes = null, Cosigner[] cosigners = null) + public Transaction MakeTransaction(byte[] script, UInt160 sender = null, TransactionAttribute[] attributes = null, Cosigner[] cosigners = null, OracleWalletBehaviour oracle = OracleWalletBehaviour.OracleWithAssert) { UInt160[] accounts; if (sender is null) @@ -295,18 +297,35 @@ public Transaction MakeTransaction(byte[] script, UInt160 sender = null, Transac using (SnapshotView snapshot = Blockchain.Singleton.GetSnapshot()) { var balances_gas = accounts.Select(p => (Account: p, Value: NativeContract.GAS.BalanceOf(snapshot, p))).Where(p => p.Value.Sign > 0).ToList(); - return MakeTransaction(snapshot, script, attributes ?? new TransactionAttribute[0], cosigners ?? new Cosigner[0], balances_gas); + return MakeTransaction(snapshot, script, attributes ?? new TransactionAttribute[0], cosigners ?? new Cosigner[0], balances_gas, oracle); } } - private Transaction MakeTransaction(StoreView snapshot, byte[] script, TransactionAttribute[] attributes, Cosigner[] cosigners, List<(UInt160 Account, BigInteger Value)> balances_gas) + private Transaction MakeTransaction(StoreView snapshot, byte[] script, TransactionAttribute[] attributes, Cosigner[] cosigners, List<(UInt160 Account, BigInteger Value)> balances_gas, OracleWalletBehaviour oracle = OracleWalletBehaviour.OracleWithAssert) { + OracleExecutionCache oracleCache = null; + List oracleRequests = null; + + if (oracle != OracleWalletBehaviour.WithoutOracle) + { + // Wee need the full request in order to duplicate the call for asserts + + oracleRequests = new List(); + oracleCache = new OracleExecutionCache((request) => + { + oracleRequests.Add(request); + return OracleService.Process(request); + }); + } + + Start: + Random rand = new Random(); foreach (var (account, value) in balances_gas) { Transaction tx = new Transaction { - Version = 0, + Version = oracleCache?.Count > 0 ? TransactionVersion.OracleRequest : TransactionVersion.Transaction, Nonce = (uint)rand.Next(), Script = script, Sender = account, @@ -314,12 +333,66 @@ private Transaction MakeTransaction(StoreView snapshot, byte[] script, Transacti Attributes = attributes, Cosigners = cosigners }; + // will try to execute 'transfer' script to check if it works - using (ApplicationEngine engine = ApplicationEngine.Run(script, snapshot.Clone(), tx, testMode: true)) + using (ApplicationEngine engine = ApplicationEngine.Run(script, snapshot.Clone(), tx, testMode: true, oracle: oracleCache)) { if (engine.State.HasFlag(VMState.FAULT)) throw new InvalidOperationException($"Failed execution for '{script.ToHexString()}'"); tx.SystemFee = Math.Max(engine.GasConsumed - ApplicationEngine.GasFree, 0); + + // Change the Transaction type because it's an oracle request + + if (oracleRequests.Count > 0) + { + if (oracle == OracleWalletBehaviour.OracleWithAssert) + { + // If we want the same result for accept the response, we need to create asserts at the begining of the script + + var assertScript = new ScriptBuilder(); + + foreach (var oracleRequest in oracleRequests) + { + // Do the request in order to cache the result + + if (oracleRequest is OracleHttpsRequest https) + { + assertScript.EmitSysCall(InteropService.Oracle.Neo_Oracle_Get, https.URL.ToString(), https.Filter?.ContractHash, https.Filter?.FilterMethod); + } + else + { + throw new NotImplementedException(); + } + } + + // Clear the stack + + assertScript.Emit(OpCode.CLEAR); + + // Check that the hash of the whole responses are exactly the same + + assertScript.EmitSysCall(InteropService.Oracle.Neo_Oracle_Hash); + assertScript.EmitPush(oracleCache.Hash.ToArray()); + assertScript.Emit(OpCode.EQUAL); + assertScript.Emit(OpCode.ASSERT); + + // Concat two scripts [OracleAsserts+Script] + + script = assertScript.ToArray().Concat(script).ToArray(); + oracle = OracleWalletBehaviour.OracleWithoutAssert; + + // We need to remove new oracle calls (OracleService.Process) + + oracleCache = new OracleExecutionCache(oracleCache.Responses); + + // We need to compute the gas again with the right script + goto Start; + } + else + { + tx.Version = TransactionVersion.OracleRequest; + } + } } UInt160[] hashes = tx.GetScriptHashesForVerifying(snapshot); diff --git a/src/neo/neo.csproj b/src/neo/neo.csproj index df025e9ab3..8b184ba586 100644 --- a/src/neo/neo.csproj +++ b/src/neo/neo.csproj @@ -1,4 +1,4 @@ - + 2015-2019 The Neo Project @@ -30,4 +30,4 @@ - + \ No newline at end of file diff --git a/tests/neo.UnitTests/Extensions/Nep5NativeContractExtensions.cs b/tests/neo.UnitTests/Extensions/Nep5NativeContractExtensions.cs index 70098c9c04..78a1c8fd83 100644 --- a/tests/neo.UnitTests/Extensions/Nep5NativeContractExtensions.cs +++ b/tests/neo.UnitTests/Extensions/Nep5NativeContractExtensions.cs @@ -1,11 +1,9 @@ using FluentAssertions; -using Neo.Network.P2P.Payloads; +using Neo.Oracle; using Neo.Persistence; using Neo.SmartContract; using Neo.SmartContract.Native; using Neo.VM; -using System; -using System.IO; using System.Linq; using System.Numerics; @@ -13,34 +11,6 @@ namespace Neo.UnitTests.Extensions { public static class Nep5NativeContractExtensions { - internal class ManualWitness : IVerifiable - { - private readonly UInt160[] _hashForVerify; - - public Witness[] Witnesses - { - get => throw new NotImplementedException(); - set => throw new NotImplementedException(); - } - - public int Size => 0; - - public ManualWitness(params UInt160[] hashForVerify) - { - _hashForVerify = hashForVerify ?? new UInt160[0]; - } - - public void Deserialize(BinaryReader reader) { } - - public void DeserializeUnsigned(BinaryReader reader) { } - - public UInt160[] GetScriptHashesForVerifying(StoreView snapshot) => _hashForVerify; - - public void Serialize(BinaryWriter writer) { } - - public void SerializeUnsigned(BinaryWriter writer) { } - } - public static bool Transfer(this NativeContract contract, StoreView snapshot, byte[] from, byte[] to, BigInteger amount, bool signFrom) { var engine = new ApplicationEngine(TriggerType.Application, diff --git a/tests/neo.UnitTests/Ledger/UT_MemoryPool.cs b/tests/neo.UnitTests/Ledger/UT_MemoryPool.cs index 08d18ce58f..41c4e44ea7 100644 --- a/tests/neo.UnitTests/Ledger/UT_MemoryPool.cs +++ b/tests/neo.UnitTests/Ledger/UT_MemoryPool.cs @@ -170,12 +170,13 @@ public void BlockPersistMovesTxToUnverifiedAndReverification() { AddTransactions(70); + using var snapshot = Blockchain.Singleton.GetSnapshot(); _unit.SortedTxCount.Should().Be(70); var block = new Block { - Transactions = _unit.GetSortedVerifiedTransactions().Take(10) - .Concat(_unit.GetSortedVerifiedTransactions().Take(5)).ToArray() + Transactions = _unit.GetSortedVerifiedTransactions(snapshot).Take(10) + .Concat(_unit.GetSortedVerifiedTransactions(snapshot).Take(5)).ToArray() }; _unit.UpdatePoolForBlockPersisted(block, Blockchain.Singleton.GetSnapshot()); _unit.InvalidateVerifiedTransactions(); @@ -211,18 +212,18 @@ public void BlockPersistMovesTxToUnverifiedAndReverification() public void BlockPersistAndReverificationWillAbandonTxAsBalanceTransfered() { long txFee = 1; + using var snapshot = Blockchain.Singleton.GetSnapshot(); AddTransactionsWithBalanceVerify(70, txFee); _unit.SortedTxCount.Should().Be(70); var block = new Block { - Transactions = _unit.GetSortedVerifiedTransactions().Take(10).ToArray() + Transactions = _unit.GetSortedVerifiedTransactions(snapshot).Take(10).ToArray() }; // Simulate the transfer process in tx by burning the balance UInt160 sender = block.Transactions[0].Sender; - SnapshotView snapshot = Blockchain.Singleton.GetSnapshot(); BigInteger balance = NativeContract.GAS.BalanceOf(snapshot, sender); ApplicationEngine applicationEngine = new ApplicationEngine(TriggerType.All, block, snapshot, (long)balance); @@ -266,15 +267,16 @@ private void VerifyTransactionsSortedDescending(IEnumerable transac public void VerifySortOrderAndThatHighetFeeTransactionsAreReverifiedFirst() { AddTransactions(100); + using var snapshot = Blockchain.Singleton.GetSnapshot(); - var sortedVerifiedTxs = _unit.GetSortedVerifiedTransactions().ToList(); + var sortedVerifiedTxs = _unit.GetSortedVerifiedTransactions(snapshot).ToList(); // verify all 100 transactions are returned in sorted order sortedVerifiedTxs.Count.Should().Be(100); VerifyTransactionsSortedDescending(sortedVerifiedTxs); // move all to unverified var block = new Block { Transactions = new Transaction[0] }; - _unit.UpdatePoolForBlockPersisted(block, Blockchain.Singleton.GetSnapshot()); + _unit.UpdatePoolForBlockPersisted(block, snapshot); _unit.InvalidateVerifiedTransactions(); _unit.SortedTxCount.Should().Be(0); _unit.UnverifiedSortedTxCount.Should().Be(100); @@ -290,13 +292,13 @@ public void VerifySortOrderAndThatHighetFeeTransactionsAreReverifiedFirst() var minTransaction = sortedUnverifiedArray.Last(); // reverify 1 high priority and 1 low priority transaction - _unit.ReVerifyTopUnverifiedTransactionsIfNeeded(1, Blockchain.Singleton.GetSnapshot()); - var verifiedTxs = _unit.GetSortedVerifiedTransactions().ToArray(); + _unit.ReVerifyTopUnverifiedTransactionsIfNeeded(1, snapshot); + var verifiedTxs = _unit.GetSortedVerifiedTransactions(snapshot).ToArray(); verifiedTxs.Length.Should().Be(1); verifiedTxs[0].Should().BeEquivalentTo(maxTransaction); var blockWith2Tx = new Block { Transactions = new[] { maxTransaction, minTransaction } }; // verify and remove the 2 transactions from the verified pool - _unit.UpdatePoolForBlockPersisted(blockWith2Tx, Blockchain.Singleton.GetSnapshot()); + _unit.UpdatePoolForBlockPersisted(blockWith2Tx, snapshot); _unit.InvalidateVerifiedTransactions(); _unit.SortedTxCount.Should().Be(0); } @@ -305,7 +307,8 @@ public void VerifySortOrderAndThatHighetFeeTransactionsAreReverifiedFirst() void VerifyCapacityThresholdForAttemptingToAddATransaction() { - var sortedVerified = _unit.GetSortedVerifiedTransactions().ToArray(); + using var snapshot = Blockchain.Singleton.GetSnapshot(); + var sortedVerified = _unit.GetSortedVerifiedTransactions(snapshot).ToArray(); var txBarelyWontFit = CreateTransactionWithFee(sortedVerified.Last().NetworkFee - 1); _unit.CanTransactionFitInPool(txBarelyWontFit).Should().Be(false); @@ -396,12 +399,13 @@ public void TestIEnumerableGetEnumerator() [TestMethod] public void TestGetVerifiedTransactions() { + using var snapshot = Blockchain.Singleton.GetSnapshot(); var tx1 = CreateTransaction(); var tx2 = CreateTransaction(); _unit.TryAdd(tx1.Hash, tx1); _unit.InvalidateVerifiedTransactions(); _unit.TryAdd(tx2.Hash, tx2); - IEnumerable enumerable = _unit.GetVerifiedTransactions(); + IEnumerable enumerable = _unit.GetVerifiedTransactions(snapshot); enumerable.Count().Should().Be(1); var enumerator = enumerable.GetEnumerator(); enumerator.MoveNext(); diff --git a/tests/neo.UnitTests/Oracle/Protocols/Https/UT_OracleHTTPRequest.cs b/tests/neo.UnitTests/Oracle/Protocols/Https/UT_OracleHTTPRequest.cs new file mode 100644 index 0000000000..3f8e49725e --- /dev/null +++ b/tests/neo.UnitTests/Oracle/Protocols/Https/UT_OracleHTTPRequest.cs @@ -0,0 +1,44 @@ +using Microsoft.VisualStudio.TestTools.UnitTesting; +using Neo.Oracle.Protocols.Https; +using System; + +namespace Neo.UnitTests.Oracle.Protocols.Https +{ + [TestClass] + public class UT_OracleHTTPRequest + { + [TestMethod] + public void TestHash() + { + var requestA = CreateDefault(); + var requestB = CreateDefault(); + + Assert.AreEqual(requestA.Hash, requestB.Hash); + + requestB = CreateDefault(); + requestB.Filter = new Neo.Oracle.OracleFilter() + { + ContractHash = UInt160.Zero, + FilterMethod = "" + }; + Assert.AreNotEqual(requestA.Hash, requestB.Hash); + + requestB = CreateDefault(); + requestB.URL = new Uri("https://google.com/?dummy=1"); + Assert.AreNotEqual(requestA.Hash, requestB.Hash); + + requestB = CreateDefault(); + requestB.Method = (HttpMethod)0xFF; + Assert.AreNotEqual(requestA.Hash, requestB.Hash); + } + + private OracleHttpsRequest CreateDefault() + { + return new OracleHttpsRequest() + { + Method = HttpMethod.GET, + URL = new Uri("https://google.com") + }; + } + } +} diff --git a/tests/neo.UnitTests/Oracle/Protocols/Https/UT_OracleHttpsProtocol.cs b/tests/neo.UnitTests/Oracle/Protocols/Https/UT_OracleHttpsProtocol.cs new file mode 100644 index 0000000000..eb3990df45 --- /dev/null +++ b/tests/neo.UnitTests/Oracle/Protocols/Https/UT_OracleHttpsProtocol.cs @@ -0,0 +1,60 @@ +using Microsoft.VisualStudio.TestTools.UnitTesting; +using Neo.Oracle.Protocols.Https; +using System.Net; + +namespace Neo.UnitTests.Oracle.Protocols.Https +{ + [TestClass] + public class UT_OracleHttpsProtocol + { + static readonly string[] Internal = new string[] + { + "::", + "::1", + "0.0.0.0", + "255.255.255.255", + "127.0.0.1", + "192.168.1.1", + "172.22.32.11" + }; + + static readonly string[] External = new string[] + { + "88.22.32.11" + }; + + [TestMethod] + public void TestIsInternalAddress() + { + foreach (var i in Internal) + { + Assert.IsTrue(OracleHttpsProtocol.IsInternal(IPAddress.Parse(i)), $"{i} is not internal"); + } + + foreach (var i in External) + { + Assert.IsFalse(OracleHttpsProtocol.IsInternal(IPAddress.Parse(i)), $"{i} is internal"); + } + } + + [TestMethod] + public void TestIsPrivateHost() + { + IPHostEntry entry = new IPHostEntry(); + + foreach (var i in Internal) + { + entry.AddressList = new IPAddress[] { IPAddress.Parse(i) }; + + Assert.IsTrue(OracleHttpsProtocol.IsInternal(entry), $"{i} is not internal"); + } + + foreach (var i in External) + { + entry.AddressList = new IPAddress[] { IPAddress.Parse(i) }; + + Assert.IsFalse(OracleHttpsProtocol.IsInternal(entry), $"{i} is internal"); + } + } + } +} diff --git a/tests/neo.UnitTests/Oracle/UT_OracleExecutionCache.cs b/tests/neo.UnitTests/Oracle/UT_OracleExecutionCache.cs new file mode 100644 index 0000000000..edacb551cf --- /dev/null +++ b/tests/neo.UnitTests/Oracle/UT_OracleExecutionCache.cs @@ -0,0 +1,143 @@ +using Microsoft.VisualStudio.TestTools.UnitTesting; +using Neo.Oracle; +using Neo.Oracle.Protocols.Https; +using System; +using System.Linq; + +namespace Neo.UnitTests.Oracle +{ + [TestClass] + public class UT_OracleExecutionCache + { + class CounterRequest : OracleHttpsRequest + { + public int Counter = 0; + } + + private OracleResponse OracleLogic(OracleRequest arg) + { + var http = (CounterRequest)arg; + http.Counter++; + return OracleResponse.CreateResult(arg.Hash, BitConverter.GetBytes(http.Counter), 0); + } + + UInt256 _txHash; + + [TestInitialize] + public void Init() + { + var rand = new Random(); + var data = new byte[32]; + rand.NextBytes(data); + + _txHash = new UInt256(data); + } + + [TestMethod] + public void TestEnumerator() + { + var copy = UT_OracleResponse.CreateDefault(); + var entry = new OracleExecutionCache(UT_OracleResponse.CreateDefault()); + var entries = entry.ToArray(); + + Assert.AreEqual(entries[0].Value.Hash, copy.Hash); + } + + [TestMethod] + public void TestSerialization() + { + var entry = new OracleExecutionCache(UT_OracleResponse.CreateDefault()); + var data = Neo.IO.Helper.ToArray(entry); + + Assert.AreEqual(entry.Size, data.Length); + + var copy = Neo.IO.Helper.AsSerializable(data); + + Assert.AreEqual(entry.Count, copy.Count); + Assert.AreEqual(entry.FilterCost, copy.FilterCost); + Assert.AreEqual(entry.First().Value.Hash, copy.First().Value.Hash); + } + + [TestMethod] + public void TestWithOracle() + { + var cache = new OracleExecutionCache(OracleLogic); + + Assert.AreEqual(0, cache.Count); + Assert.IsFalse(cache.GetEnumerator().MoveNext()); + + // Test without cache + + var req = new CounterRequest() + { + Counter = 1, + URL = new Uri("https://google.es"), + Method = HttpMethod.GET + }; + Assert.IsTrue(cache.TryGet(req, out var ret)); + + Assert.AreEqual(2, req.Counter); + Assert.AreEqual(1, cache.Count); + Assert.IsFalse(ret.Error); + CollectionAssert.AreEqual(new byte[] { 0x02, 0x00, 0x00, 0x00 }, ret.Result); + + // Test cached + + Assert.IsTrue(cache.TryGet(req, out ret)); + + Assert.AreEqual(2, req.Counter); + Assert.AreEqual(1, cache.Count); + Assert.IsFalse(ret.Error); + CollectionAssert.AreEqual(new byte[] { 0x02, 0x00, 0x00, 0x00 }, ret.Result); + + // Check collection + + var array = cache.ToArray(); + Assert.AreEqual(1, array.Length); + Assert.AreEqual(req.Hash, array[0].Key); + Assert.IsFalse(array[0].Value.Error); + CollectionAssert.AreEqual(new byte[] { 0x02, 0x00, 0x00, 0x00 }, array[0].Value.Result); + } + + [TestMethod] + public void TestWithoutOracle() + { + var initReq = new OracleHttpsRequest() + { + URL = new Uri("https://google.es"), + Method = HttpMethod.GET + }; + + var initRes = OracleResponse.CreateError(initReq.Hash, OracleResultError.ServerError); + var cache = new OracleExecutionCache(initRes); + + Assert.AreEqual(1, cache.Count); + + // Check collection + + var array = cache.ToArray(); + Assert.AreEqual(1, array.Length); + Assert.AreEqual(initReq.Hash, array[0].Key); + Assert.IsTrue(array[0].Value.Error); + Assert.AreEqual(null, array[0].Value.Result); + + // Test without cache + + Assert.IsFalse(cache.TryGet(new OracleHttpsRequest() + { + URL = new Uri("https://google.es/?p=1"), + Method = HttpMethod.GET + } + , out var ret)); + + Assert.IsNull(ret); + + // Test cached + + Assert.IsTrue(cache.TryGet(initReq, out ret)); + Assert.IsNotNull(ret); + Assert.AreEqual(1, cache.Count); + Assert.IsTrue(ReferenceEquals(ret, initRes)); + } + } +} diff --git a/tests/neo.UnitTests/Oracle/UT_OracleResponse.cs b/tests/neo.UnitTests/Oracle/UT_OracleResponse.cs new file mode 100644 index 0000000000..c2bc07203e --- /dev/null +++ b/tests/neo.UnitTests/Oracle/UT_OracleResponse.cs @@ -0,0 +1,53 @@ +using Microsoft.VisualStudio.TestTools.UnitTesting; +using Neo.IO; +using Neo.Oracle; + +namespace Neo.UnitTests.Oracle +{ + [TestClass] + public class UT_OracleResponse + { + [TestMethod] + public void TestHash() + { + var requestA = CreateDefault(); + var requestB = CreateDefault(); + + requestB.Result = new byte[1]; + Assert.AreNotEqual(requestA.Hash, requestB.Hash); + + requestB = CreateDefault(); + requestB.RequestHash = UInt160.Parse("0xa400ff00ff00ff00ff00ff00ff00ff00ff00ff01"); + Assert.AreNotEqual(requestA.Hash, requestB.Hash); + + requestB = CreateDefault(); + requestB.Result = null; + Assert.AreNotEqual(requestA.Hash, requestB.Hash); + } + + [TestMethod] + public void TestSerialization() + { + var entry = CreateDefault(); + var data = entry.ToArray(); + + Assert.AreEqual(entry.Size, data.Length); + + var copy = data.AsSerializable(); + + Assert.AreEqual(entry.Hash, copy.Hash); + Assert.AreEqual(entry.Error, copy.Error); + Assert.AreEqual(entry.RequestHash, copy.RequestHash); + CollectionAssert.AreEqual(entry.Result, copy.Result); + } + + internal static OracleResponse CreateDefault() + { + return new OracleResponse() + { + RequestHash = UInt160.Parse("0xff00ff00ff00ff00ff00ff00ff00ff00ff00ff01"), + Result = new byte[] { 0x01, 0x02, 0x03 } + }; + } + } +} diff --git a/tests/neo.UnitTests/Oracle/UT_OracleService.cs b/tests/neo.UnitTests/Oracle/UT_OracleService.cs new file mode 100644 index 0000000000..15cd21cfb0 --- /dev/null +++ b/tests/neo.UnitTests/Oracle/UT_OracleService.cs @@ -0,0 +1,420 @@ +using Akka.TestKit; +using Akka.TestKit.Xunit2; +using Microsoft.AspNetCore.Builder; +using Microsoft.AspNetCore.Hosting; +using Microsoft.AspNetCore.Http; +using Microsoft.AspNetCore.ResponseCompression; +using Microsoft.Extensions.DependencyInjection; +using Microsoft.VisualStudio.TestTools.UnitTesting; +using Neo.IO; +using Neo.Ledger; +using Neo.Network.P2P; +using Neo.Network.P2P.Payloads; +using Neo.Oracle; +using Neo.Oracle.Protocols.Https; +using Neo.Persistence; +using Neo.SmartContract; +using Neo.SmartContract.Native; +using Neo.SmartContract.Native.Oracle; +using Neo.SmartContract.Native.Tokens; +using Neo.VM; +using Neo.Wallets; +using Neo.Wallets.NEP6; +using System; +using System.IO; +using System.IO.Compression; +using System.Linq; +using System.Net; +using System.Text; +using System.Threading; +using System.Threading.Tasks; + +namespace Neo.UnitTests.Oracle +{ + [TestClass] + public class UT_OracleService : TestKit + { + private WalletAccount _account; + private NEP6Wallet _wallet; + + [TestInitialize] + public void Init() + { + _wallet = TestUtils.GenerateTestWallet(); + using var unlockA = _wallet.Unlock("123"); + _account = _wallet.CreateAccount(); + + TestBlockchain.InitializeMockNeoSystem(); + } + + SnapshotView MockedSnapshotFactory() + { + // Mock blockchain is not possible + + var snapshot = Blockchain.Singleton.GetSnapshot(); + + foreach (var cn in Blockchain.StandbyValidators) + { + var key = NativeContract.Oracle.CreateStorageKey(OracleContract.Prefix_Validator, cn); + var value = snapshot.Storages.GetOrAdd(key, () => new StorageItem() { IsConstant = false }); + value.Value = _account.GetKey().PublicKey.ToArray(); + } + + return snapshot; + } + + public static IWebHost CreateServer(int port) + { + var server = new WebHostBuilder().UseKestrel(options => + { + options.Listen(IPAddress.Any, port, listenOptions => + { + if (File.Exists("UT-cert.pfx")) + { + listenOptions.UseHttps("UT-cert.pfx", "123", https => + { + https.CheckCertificateRevocation = false; + https.SslProtocols = System.Security.Authentication.SslProtocols.None; + }); + } + else if (File.Exists("../../../UT-cert.pfx")) + { + // Unix doesn't copy to the output dir + + listenOptions.UseHttps("../../../UT-cert.pfx", "123", https => + { + https.CheckCertificateRevocation = false; + https.SslProtocols = System.Security.Authentication.SslProtocols.None; + }); + } + }); + }) + .Configure(app => + { + app.UseResponseCompression(); + app.Run(ProcessAsync); + }) + .ConfigureServices(services => + { + services.AddResponseCompression(options => + { + // options.EnableForHttps = false; + options.Providers.Add(); + options.MimeTypes = ResponseCompressionDefaults.MimeTypes.Concat(new[] { "application/json-rpc" }); + }); + + services.Configure(options => + { + options.Level = CompressionLevel.Fastest; + }); + }) + .Build(); + + server.Start(); + return server; + } + + private static async Task ProcessAsync(HttpContext context) + { + var response = ""; + context.Response.ContentType = "text/plain"; + + switch (context.Request.Path.Value) + { + case "/json": + { + context.Response.ContentType = "application/json"; + response = +@"{ + 'Stores': [ + 'Lambton Quay', + 'Willis Street' + ], + 'Manufacturers': [ + { + 'Name': 'Acme Co', + 'Products': [ + { + 'Name': 'Anvil', + 'Price': 50 + } + ] + }, + { + 'Name': 'Contoso', + 'Products': [ + { + 'Name': 'Elbow Grease', + 'Price': 99.95 + }, + { + 'Name': 'Headlight Fluid', + 'Price': 4 + } + ] + } + ] +} +"; + break; + } + case "/ping": + { + response = "pong"; + break; + } + case "/timeout": + { + Thread.Sleep((int)(OracleService.HTTPSProtocol.TimeOut.TotalMilliseconds + 250)); + break; + } + case "/error": + { + context.Response.StatusCode = 503; + break; + } + default: + { + context.Response.StatusCode = 404; + break; + } + } + + await context.Response.WriteAsync(response, Encoding.UTF8); + } + + [TestMethod] + public void StartStop() + { + TestProbe subscriber = CreateTestProbe(); + + var service = new OracleService(TestBlockchain.TheNeoSystem, subscriber, _wallet, MockedSnapshotFactory, 10); + Assert.IsFalse(service.IsStarted); + service.Start(); + Assert.IsTrue(service.IsStarted); + service.Stop(); + } + + [TestMethod] + public void ProcessTx() + { + var port = 8443; + using var server = CreateServer(port); + + OracleService.HTTPSProtocol.AllowPrivateHost = true; + + TestProbe subscriber = CreateTestProbe(); + TestActorRef service = ActorOfAsTestActorRef(() => new OracleService(TestBlockchain.TheNeoSystem, subscriber, _wallet, MockedSnapshotFactory, 10)); + + service.UnderlyingActor.Start(); + + // Send start + + service.Tell(new OracleService.StartMessage() { NumberOfTasks = 1 }); + + // Send tx + + var tx = CreateTx($"https://127.0.0.1:{port}/ping", null); + service.Tell(tx); + + // Receive response + + var responseMsg = subscriber.ExpectMsg(TimeSpan.FromSeconds(10)); + Assert.IsInstanceOfType(responseMsg.Inventory, typeof(Transaction)); + + var response = responseMsg.Inventory as Transaction; + + Assert.AreEqual(TransactionVersion.OracleResponse, response.Version); + Assert.AreEqual(tx.Hash, response.OracleRequestTx); + + //var response = responseMsg.Inventory as OraclePayload; + //Assert.AreEqual(117, response.Data.Length); + //Assert.AreEqual(_account.GetKey().PublicKey, response.OraclePub); + + //Assert.IsNotNull(response.OracleSignature); + //// pong + //Assert.AreEqual("0x6f458eea71a0b63d3e9efc8cc54608e14d89a5cd", response.OracleSignature.OracleExecutionCacheHash.ToString()); + //Assert.AreEqual(64, response.OracleSignature.Signature.Length); + //Assert.AreEqual(tx.Hash, response.OracleSignature.TransactionRequestHash); + + service.UnderlyingActor.Stop(); + OracleService.HTTPSProtocol.AllowPrivateHost = false; + } + + private Transaction CreateTx(string url, OracleFilter filter) + { + using ScriptBuilder script = new ScriptBuilder(); + script.EmitSysCall(InteropService.Oracle.Neo_Oracle_Get, url, filter?.ContractHash, filter?.FilterMethod); + + return new Transaction() + { + Version = TransactionVersion.OracleRequest, + Attributes = new TransactionAttribute[0], + Cosigners = new Cosigner[0], + Script = script.ToArray(), + Sender = UInt160.Zero, + Witnesses = new Witness[0], + NetworkFee = 1_000_000, + SystemFee = 1_000_000, + }; + } + + [TestMethod] + public void TestOracleTx() + { + var port = 8443; + using var server = CreateServer(port); + + var wallet = TestUtils.GenerateTestWallet(); + var snapshot = Blockchain.Singleton.GetSnapshot(); + + // no password on this wallet + using (var unlock = wallet.Unlock("")) + { + var acc = wallet.CreateAccount(); + + // Fake balance + + var key = NativeContract.GAS.CreateStorageKey(20, acc.ScriptHash); + + var entry = snapshot.Storages.GetAndChange(key, () => new StorageItem + { + Value = new Nep5AccountState().ToByteArray() + }); + + entry.Value = new Nep5AccountState() + { + Balance = 10000 * NativeContract.GAS.Factor + } + .ToByteArray(); + + snapshot.Commit(); + + // Manually creating script + + byte[] script; + using (ScriptBuilder sb = new ScriptBuilder()) + { + sb.EmitSysCall(InteropService.Oracle.Neo_Oracle_Get, $"https://127.0.0.1:{port}/ping", null, null); + script = sb.ToArray(); + } + + // WithoutOracle + + Assert.ThrowsException(() => + { + _ = wallet.MakeTransaction(script, acc.ScriptHash, new TransactionAttribute[0], new Cosigner[0], oracle: OracleWalletBehaviour.WithoutOracle); + }); + + // OracleWithoutAssert + + var txWithout = wallet.MakeTransaction(script, acc.ScriptHash, new TransactionAttribute[0], new Cosigner[0], oracle: OracleWalletBehaviour.OracleWithoutAssert); + + Assert.IsNotNull(txWithout); + Assert.IsNull(txWithout.Witnesses); + Assert.AreEqual(TransactionVersion.OracleRequest, txWithout.Version); + + // OracleWithoutAssert + + var txWith = wallet.MakeTransaction(script, acc.ScriptHash, new TransactionAttribute[0], new Cosigner[0], oracle: OracleWalletBehaviour.OracleWithAssert); + + Assert.IsNotNull(txWith); + Assert.IsNull(txWith.Witnesses); + Assert.AreEqual(TransactionVersion.OracleRequest, txWith.Version); + + // Check that has more fee and the script is longer + + Assert.IsTrue(txWith.Script.Length > txWithout.Script.Length); + Assert.IsTrue(txWith.NetworkFee > txWithout.NetworkFee); + } + } + + [TestMethod] + public void TestOracleHttpsRequest() + { + var port = 8443; + using var server = CreateServer(port); + + // With local access (Only for UT) + + OracleService.HTTPSProtocol.AllowPrivateHost = true; + + // Timeout + + OracleService.HTTPSProtocol.TimeOut = TimeSpan.FromSeconds(2); + + var request = new OracleHttpsRequest() + { + Method = HttpMethod.GET, + URL = new Uri($"https://127.0.0.1:{port}/timeout") + }; + + var response = OracleService.Process(request); + + OracleService.HTTPSProtocol.TimeOut = TimeSpan.FromSeconds(5); + + Assert.IsTrue(response.Error); + Assert.IsTrue(response.Result == null); + Assert.AreEqual(request.Hash, response.RequestHash); + Assert.AreNotEqual(UInt160.Zero, response.Hash); + + // OK + + request = new OracleHttpsRequest() + { + Method = HttpMethod.GET, + URL = new Uri($"https://127.0.0.1:{port}/ping") + }; + + response = OracleService.Process(request); + + Assert.IsFalse(response.Error); + Assert.AreEqual("pong", Encoding.UTF8.GetString(response.Result)); + Assert.AreEqual(request.Hash, response.RequestHash); + Assert.AreNotEqual(UInt160.Zero, response.Hash); + + // Error + + request = new OracleHttpsRequest() + { + Method = HttpMethod.GET, + URL = new Uri($"https://127.0.0.1:{port}/error") + }; + + response = OracleService.Process(request); + + Assert.IsTrue(response.Error); + Assert.IsTrue(response.Result == null); + Assert.AreEqual(request.Hash, response.RequestHash); + Assert.AreNotEqual(UInt160.Zero, response.Hash); + + // Without local access + + OracleService.HTTPSProtocol.AllowPrivateHost = false; + response = OracleService.Process(request); + + Assert.IsTrue(response.Error); + Assert.IsTrue(response.Result == null); + Assert.AreEqual(request.Hash, response.RequestHash); + Assert.AreNotEqual(UInt160.Zero, response.Hash); + } + + class ErrorRequest : OracleRequest + { + public override OracleRequestType Type => OracleRequestType.HTTPS; + protected override byte[] GetHashData() => new byte[1]; + } + + [TestMethod] + public void TestOracleErrorRequest() + { + var request = new ErrorRequest(); + var response = OracleService.Process(request); + + Assert.IsTrue(response.Error); + Assert.AreEqual(null, response.Result); + Assert.AreEqual(request.Hash, response.RequestHash); + Assert.AreEqual("0x465885f2e323d8c1f33abde501f623a40533d5ec", response.Hash.ToString()); + } + } +} diff --git a/tests/neo.UnitTests/SmartContract/Native/Oracle/UT_OracleContract.cs b/tests/neo.UnitTests/SmartContract/Native/Oracle/UT_OracleContract.cs new file mode 100644 index 0000000000..b89cb879b8 --- /dev/null +++ b/tests/neo.UnitTests/SmartContract/Native/Oracle/UT_OracleContract.cs @@ -0,0 +1,355 @@ +using FluentAssertions; +using Microsoft.VisualStudio.TestTools.UnitTesting; +using Neo.Cryptography.ECC; +using Neo.IO; +using Neo.Ledger; +using Neo.Network.P2P.Payloads; +using Neo.Oracle; +using Neo.SmartContract; +using Neo.SmartContract.Native; +using Neo.SmartContract.Native.Tokens; +using Neo.UnitTests.Wallets; +using Neo.VM; +using Neo.Wallets; +using System; +using System.Linq; +using System.Numerics; +using static Neo.UnitTests.Extensions.Nep5NativeContractExtensions; + +namespace Neo.UnitTests.Oracle +{ + [TestClass] + public class UT_OracleContract + { + [TestInitialize] + public void TestSetup() + { + TestBlockchain.InitializeMockNeoSystem(); + } + + [TestMethod] + public void TestInitialize() + { + var snapshot = Blockchain.Singleton.GetSnapshot(); + var engine = new ApplicationEngine(TriggerType.System, null, snapshot, 0, true); + + snapshot.Storages.Delete(CreateStorageKey(11)); + snapshot.PersistingBlock = Blockchain.GenesisBlock; + engine = new ApplicationEngine(TriggerType.Application, null, snapshot, 0, true); + NativeContract.Oracle.Initialize(engine).Should().BeFalse(); // already registered + } + internal static StorageKey CreateStorageKey(byte prefix, byte[] key = null) + { + StorageKey storageKey = new StorageKey + { + Id = NativeContract.Oracle.Id, + Key = new byte[sizeof(byte) + (key?.Length ?? 0)] + }; + storageKey.Key[0] = prefix; + key?.CopyTo(storageKey.Key.AsSpan(1)); + return storageKey; + } + + [TestMethod] + public void Test_GetPerRequestFee() + { + var snapshot = Blockchain.Singleton.GetSnapshot(); + var script = new ScriptBuilder(); + script.EmitAppCall(NativeContract.Oracle.Hash, "getPerRequestFee"); + var engine = new ApplicationEngine(TriggerType.Application, null, snapshot, 0, true); + engine.LoadScript(script.ToArray()); + + engine.Execute().Should().Be(VMState.HALT); + var result = engine.ResultStack.Pop(); + result.Should().BeOfType(typeof(VM.Types.Integer)); + Assert.AreEqual(result, 1000); + } + + [TestMethod] + public void Test_SetPerRequestFee() + { + var snapshot = Blockchain.Singleton.GetSnapshot().Clone(); + + // Init + + var engine = new ApplicationEngine(TriggerType.Application, null, snapshot, 0, true); + var from = NativeContract.Oracle.GetOracleMultiSigAddress(snapshot); + var value = 12345; + + // Set + var script = new ScriptBuilder(); + script.EmitAppCall(NativeContract.Oracle.Hash, "setPerRequestFee", new ContractParameter(ContractParameterType.Integer) { Value = value }); + engine = new ApplicationEngine(TriggerType.Application, new ManualWitness(from), snapshot, 0, true); + engine.LoadScript(script.ToArray()); + + engine.Execute().Should().Be(VMState.HALT); + var result = engine.ResultStack.Pop(); + result.Should().BeOfType(typeof(VM.Types.Boolean)); + Assert.IsTrue((result as VM.Types.Boolean).ToBoolean()); + + // Set (wrong witness) + script = new ScriptBuilder(); + script.EmitAppCall(NativeContract.Oracle.Hash, "setPerRequestFee", new ContractParameter(ContractParameterType.Integer) { Value = value }); + engine = new ApplicationEngine(TriggerType.Application, new ManualWitness(null), snapshot, 0, true); + engine.LoadScript(script.ToArray()); + + engine.Execute().Should().Be(VMState.HALT); + result = engine.ResultStack.Pop(); + result.Should().BeOfType(typeof(VM.Types.Boolean)); + Assert.IsFalse((result as VM.Types.Boolean).ToBoolean()); + + // Set wrong (negative) + + script = new ScriptBuilder(); + script.EmitAppCall(NativeContract.Oracle.Hash, "setPerRequestFee", new ContractParameter(ContractParameterType.Integer) { Value = -1 }); + engine = new ApplicationEngine(TriggerType.Application, new ManualWitness(from), snapshot, 0, true); + engine.LoadScript(script.ToArray()); + + engine.Execute().Should().Be(VMState.HALT); + result = engine.ResultStack.Pop(); + result.Should().BeOfType(typeof(VM.Types.Boolean)); + Assert.IsFalse((result as VM.Types.Boolean).ToBoolean()); + + // Get + + script = new ScriptBuilder(); + script.EmitAppCall(NativeContract.Oracle.Hash, "getPerRequestFee"); + engine = new ApplicationEngine(TriggerType.Application, null, snapshot, 0, true); + engine.LoadScript(script.ToArray()); + + engine.Execute().Should().Be(VMState.HALT); + result = engine.ResultStack.Pop(); + result.Should().BeOfType(typeof(VM.Types.Integer)); + Assert.AreEqual(result, value); + } + + [TestMethod] + public void Test_GetHttpConfig() + { + var snapshot = Blockchain.Singleton.GetSnapshot(); + var script = new ScriptBuilder(); + script.EmitAppCall(NativeContract.Oracle.Hash, "getConfig", new ContractParameter(ContractParameterType.String) { Value = "HttpTimeout" }); + var engine = new ApplicationEngine(TriggerType.Application, null, snapshot, 0, true); + engine.LoadScript(script.ToArray()); + + engine.Execute().Should().Be(VMState.HALT); + var result = engine.ResultStack.Pop(); + result.Should().BeOfType(typeof(VM.Types.ByteString)); + Assert.AreEqual(result.GetBigInteger(), 5000); + } + + [TestMethod] + public void Test_SetConfig() + { + var snapshot = Blockchain.Singleton.GetSnapshot().Clone(); + + // Init + + var engine = new ApplicationEngine(TriggerType.Application, null, snapshot, 0, true); + var from = NativeContract.Oracle.GetOracleMultiSigAddress(snapshot); + var key = "HttpTimeout"; + var value = BitConverter.GetBytes(12345); + + // Set (wrong witness) + var script = new ScriptBuilder(); + script.EmitAppCall(NativeContract.Oracle.Hash, "setConfig", new ContractParameter(ContractParameterType.String) { Value = key }, new ContractParameter(ContractParameterType.ByteArray) { Value = value }); + engine = new ApplicationEngine(TriggerType.Application, new ManualWitness(null), snapshot, 0, true); + engine.LoadScript(script.ToArray()); + + engine.Execute().Should().Be(VMState.HALT); + var result = engine.ResultStack.Pop(); + result.Should().BeOfType(typeof(VM.Types.Boolean)); + Assert.IsFalse((result as VM.Types.Boolean).ToBoolean()); + + // Set good + + script = new ScriptBuilder(); + script.EmitAppCall(NativeContract.Oracle.Hash, "setConfig", new ContractParameter(ContractParameterType.String) { Value = key }, new ContractParameter(ContractParameterType.ByteArray) { Value = value }); + engine = new ApplicationEngine(TriggerType.Application, new ManualWitness(from), snapshot, 0, true); + engine.LoadScript(script.ToArray()); + + engine.Execute().Should().Be(VMState.HALT); + result = engine.ResultStack.Pop(); + result.Should().BeOfType(typeof(VM.Types.Boolean)); + Assert.IsTrue((result as VM.Types.Boolean).ToBoolean()); + + // Get + + script = new ScriptBuilder(); + script.EmitAppCall(NativeContract.Oracle.Hash, "getConfig", new ContractParameter(ContractParameterType.String) { Value = key }); + engine = new ApplicationEngine(TriggerType.Application, null, snapshot, 0, true); + engine.LoadScript(script.ToArray()); + + engine.Execute().Should().Be(VMState.HALT); + result = engine.ResultStack.Pop(); + result.Should().BeOfType(typeof(VM.Types.ByteString)); + Assert.AreEqual(result.GetBigInteger(), new BigInteger(value)); + } + + [TestMethod] + public void Test_GetOracleValidators() + { + var snapshot = Blockchain.Singleton.GetSnapshot(); + + // Fake a oracle validator has cosignee. + ECPoint[] oraclePubKeys = NativeContract.Oracle.GetOracleValidators(snapshot); + + ECPoint pubkey0 = oraclePubKeys[0]; // Validator0 is the cosignor + ECPoint cosignorPubKey = oraclePubKeys[1]; // Validator1 is the cosignee + var validator0Key = NativeContract.Oracle.CreateStorageKey(24, pubkey0); // 24 = Prefix_Validator + var validator0Value = new StorageItem() + { + Value = cosignorPubKey.ToArray() + }; + snapshot.Storages.Add(validator0Key, validator0Value); + + var script = new ScriptBuilder(); + script.EmitAppCall(NativeContract.Oracle.Hash, "getOracleValidators"); + var engine = new ApplicationEngine(TriggerType.Application, null, snapshot, 0, true); + engine.LoadScript(script.ToArray()); + + engine.Execute().Should().Be(VMState.HALT); + var result = engine.ResultStack.Pop(); + result.Should().BeOfType(typeof(VM.Types.Array)); + Assert.AreEqual(6, ((VM.Types.Array)result).Count); + + // The validator0's cosignee should be the validator1 + var validators = (VM.Types.Array)result; + var cosignee0Bytes = ((VM.Types.ByteString)validators[0]).GetSpan().ToHexString(); + var cosignee1Bytes = ((VM.Types.ByteString)validators[1]).GetSpan().ToHexString(); + Assert.AreNotEqual(cosignee0Bytes, cosignee1Bytes); + var validator1Bytes = cosignorPubKey.ToArray().ToHexString(); + Assert.AreNotEqual(cosignee1Bytes, validator1Bytes); + + // clear data + snapshot.Storages.Delete(validator0Key); + } + + [TestMethod] + public void Test_GetOracleValidatorsCount() + { + var snapshot = Blockchain.Singleton.GetSnapshot(); + var script = new ScriptBuilder(); + script.EmitAppCall(NativeContract.Oracle.Hash, "getOracleValidatorsCount"); + var engine = new ApplicationEngine(TriggerType.Application, null, snapshot, 0, true); + engine.LoadScript(script.ToArray()); + + engine.Execute().Should().Be(VMState.HALT); + var result = engine.ResultStack.Pop(); + result.Should().BeOfType(typeof(VM.Types.Integer)); + Assert.AreEqual(result, 7); + } + + [TestMethod] + public void Test_DelegateOracleValidator() + { + var snapshot = Blockchain.Singleton.GetSnapshot(); + + ECPoint[] oraclePubKeys = NativeContract.Oracle.GetOracleValidators(snapshot); + + ECPoint pubkey0 = oraclePubKeys[0]; + + byte[] privateKey1 = { 0x01,0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, + 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01}; + KeyPair keyPair1 = new KeyPair(privateKey1); + ECPoint pubkey1 = keyPair1.PublicKey; + + byte[] privateKey2 = { 0x02,0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, + 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01}; + KeyPair keyPair2 = new KeyPair(privateKey2); + ECPoint pubkey2 = keyPair2.PublicKey; + + using ScriptBuilder sb = new ScriptBuilder(); + sb.EmitAppCall(NativeContract.Oracle.Hash, "delegateOracleValidator", new ContractParameter + { + Type = ContractParameterType.ByteArray, + Value = pubkey0.ToArray() + }, new ContractParameter + { + Type = ContractParameterType.ByteArray, + Value = pubkey1.ToArray() + }); + + MyWallet wallet = new MyWallet(); + WalletAccount account = wallet.CreateAccount(privateKey1); + + // Fake balance + var key = NativeContract.GAS.CreateStorageKey(20, account.ScriptHash); + var entry = snapshot.Storages.GetAndChange(key, () => new StorageItem + { + Value = new Nep5AccountState().ToByteArray() + }); + entry.Value = new Nep5AccountState() + { + Balance = 1000000 * NativeContract.GAS.Factor + } + .ToByteArray(); + + snapshot.Commit(); + + // Fake an nonexist validator in delegatedOracleValidators + byte[] fakerPrivateKey = { 0x01,0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, + 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x02, 0x02, 0x02}; + KeyPair fakerKeyPair = new KeyPair(fakerPrivateKey); + ECPoint fakerPubkey = fakerKeyPair.PublicKey; + var invalidOracleValidatorKey = NativeContract.Oracle.CreateStorageKey(24, fakerPubkey); // 24 = Prefix_Validator + var invalidOracleValidatorValue = new StorageItem() + { + Value = fakerPubkey.ToArray() + }; + snapshot.Storages.Add(invalidOracleValidatorKey, invalidOracleValidatorValue); + + var tx = wallet.MakeTransaction(sb.ToArray(), account.ScriptHash, new TransactionAttribute[] { }); + ContractParametersContext context = new ContractParametersContext(tx); + wallet.Sign(context); + tx.Witnesses = context.GetWitnesses(); + + // wrong witness + var engine = new ApplicationEngine(TriggerType.Application, tx, snapshot, 0, true); + engine.LoadScript(tx.Script); + var state = engine.Execute(); + state.Should().Be(VMState.HALT); + var result = engine.ResultStack.Pop(); + result.Should().BeOfType(typeof(VM.Types.Boolean)); + Assert.IsFalse((result as VM.Types.Boolean).ToBoolean()); + + //wrong witness + using ScriptBuilder sb2 = new ScriptBuilder(); + sb2.EmitAppCall(NativeContract.Oracle.Hash, "delegateOracleValidator", new ContractParameter + { + Type = ContractParameterType.ByteArray, + Value = pubkey1.ToArray() + }, new ContractParameter + { + Type = ContractParameterType.ByteArray, + Value = pubkey2.ToArray() + }); + + var from = Contract.CreateSignatureContract(pubkey1).ScriptHash; + + engine = new ApplicationEngine(TriggerType.Application, new ManualWitness(from), snapshot, 0, true); + engine.LoadScript(sb2.ToArray()); + state = engine.Execute(); + state.Should().Be(VMState.HALT); + result = engine.ResultStack.Pop(); + result.Should().BeOfType(typeof(VM.Types.Boolean)); + Assert.IsFalse((result as VM.Types.Boolean).ToBoolean()); + + //correct + from = Contract.CreateSignatureContract(pubkey0).ScriptHash; + + engine = new ApplicationEngine(TriggerType.Application, new ManualWitness(from), snapshot, 0, true); + engine.LoadScript(tx.Script.ToArray()); + + engine.Execute().Should().Be(VMState.HALT); + result = engine.ResultStack.Pop(); + result.Should().BeOfType(typeof(VM.Types.Boolean)); + Assert.IsTrue((result as VM.Types.Boolean).ToBoolean()); + + // The invalid oracle validator should be removed + Assert.IsNull(snapshot.Storages.TryGet(invalidOracleValidatorKey)); + + Test_GetOracleValidators(); + } + } +} diff --git a/tests/neo.UnitTests/SmartContract/Native/Tokens/UT_NeoToken.cs b/tests/neo.UnitTests/SmartContract/Native/Tokens/UT_NeoToken.cs index b2696b4888..7f9beed2c9 100644 --- a/tests/neo.UnitTests/SmartContract/Native/Tokens/UT_NeoToken.cs +++ b/tests/neo.UnitTests/SmartContract/Native/Tokens/UT_NeoToken.cs @@ -6,6 +6,7 @@ using Neo.IO.Caching; using Neo.Ledger; using Neo.Network.P2P.Payloads; +using Neo.Oracle; using Neo.Persistence; using Neo.SmartContract; using Neo.SmartContract.Native; @@ -536,7 +537,7 @@ public void TestValidatorState_ToByteArray() internal static (bool State, bool Result) Check_Vote(StoreView snapshot, byte[] account, byte[] pubkey, bool signAccount) { var engine = new ApplicationEngine(TriggerType.Application, - new Nep5NativeContractExtensions.ManualWitness(signAccount ? new UInt160(account) : UInt160.Zero), snapshot, 0, true); + new ManualWitness(signAccount ? new UInt160(account) : UInt160.Zero), snapshot, 0, true); engine.LoadScript(NativeContract.NEO.Script); @@ -566,7 +567,7 @@ internal static (bool State, bool Result) Check_Vote(StoreView snapshot, byte[] internal static (bool State, bool Result) Check_RegisterValidator(StoreView snapshot, byte[] pubkey) { var engine = new ApplicationEngine(TriggerType.Application, - new Nep5NativeContractExtensions.ManualWitness(Contract.CreateSignatureRedeemScript(ECPoint.DecodePoint(pubkey, ECCurve.Secp256r1)).ToScriptHash()), snapshot, 0, true); + new ManualWitness(Contract.CreateSignatureRedeemScript(ECPoint.DecodePoint(pubkey, ECCurve.Secp256r1)).ToScriptHash()), snapshot, 0, true); engine.LoadScript(NativeContract.NEO.Script); diff --git a/tests/neo.UnitTests/SmartContract/Native/UT_PolicyContract.cs b/tests/neo.UnitTests/SmartContract/Native/UT_PolicyContract.cs index a173585182..241cd154b4 100644 --- a/tests/neo.UnitTests/SmartContract/Native/UT_PolicyContract.cs +++ b/tests/neo.UnitTests/SmartContract/Native/UT_PolicyContract.cs @@ -3,7 +3,7 @@ using Neo.IO; using Neo.Ledger; using Neo.Network.P2P.Payloads; -using Neo.Persistence; +using Neo.Oracle; using Neo.SmartContract; using Neo.SmartContract.Native; using Neo.UnitTests.Extensions; @@ -65,7 +65,7 @@ public void Check_SetMaxBlockSize() // Without signature - var ret = NativeContract.Policy.Call(snapshot, new Nep5NativeContractExtensions.ManualWitness(null), + var ret = NativeContract.Policy.Call(snapshot, new ManualWitness(null), "setMaxBlockSize", new ContractParameter(ContractParameterType.Integer) { Value = 1024 }); ret.Should().BeOfType(); ret.ToBoolean().Should().BeFalse(); @@ -76,7 +76,7 @@ public void Check_SetMaxBlockSize() // More than expected - ret = NativeContract.Policy.Call(snapshot, new Nep5NativeContractExtensions.ManualWitness(UInt160.Zero), + ret = NativeContract.Policy.Call(snapshot, new ManualWitness(UInt160.Zero), "setMaxBlockSize", new ContractParameter(ContractParameterType.Integer) { Value = Neo.Network.P2P.Message.PayloadMaxSize }); ret.Should().BeOfType(); ret.ToBoolean().Should().BeFalse(); @@ -87,7 +87,7 @@ public void Check_SetMaxBlockSize() // With signature - ret = NativeContract.Policy.Call(snapshot, new Nep5NativeContractExtensions.ManualWitness(UInt160.Zero), + ret = NativeContract.Policy.Call(snapshot, new ManualWitness(UInt160.Zero), "setMaxBlockSize", new ContractParameter(ContractParameterType.Integer) { Value = 1024 }); ret.Should().BeOfType(); ret.ToBoolean().Should().BeTrue(); @@ -111,7 +111,7 @@ public void Check_SetMaxTransactionsPerBlock() // Without signature - var ret = NativeContract.Policy.Call(snapshot, new Nep5NativeContractExtensions.ManualWitness(), + var ret = NativeContract.Policy.Call(snapshot, new ManualWitness(), "setMaxTransactionsPerBlock", new ContractParameter(ContractParameterType.Integer) { Value = 1 }); ret.Should().BeOfType(); ret.ToBoolean().Should().BeFalse(); @@ -122,7 +122,7 @@ public void Check_SetMaxTransactionsPerBlock() // With signature - ret = NativeContract.Policy.Call(snapshot, new Nep5NativeContractExtensions.ManualWitness(UInt160.Zero), + ret = NativeContract.Policy.Call(snapshot, new ManualWitness(UInt160.Zero), "setMaxTransactionsPerBlock", new ContractParameter(ContractParameterType.Integer) { Value = 1 }); ret.Should().BeOfType(); ret.ToBoolean().Should().BeTrue(); @@ -146,7 +146,7 @@ public void Check_SetFeePerByte() // Without signature - var ret = NativeContract.Policy.Call(snapshot, new Nep5NativeContractExtensions.ManualWitness(), + var ret = NativeContract.Policy.Call(snapshot, new ManualWitness(), "setFeePerByte", new ContractParameter(ContractParameterType.Integer) { Value = 1 }); ret.Should().BeOfType(); ret.ToBoolean().Should().BeFalse(); @@ -157,7 +157,7 @@ public void Check_SetFeePerByte() // With signature - ret = NativeContract.Policy.Call(snapshot, new Nep5NativeContractExtensions.ManualWitness(UInt160.Zero), + ret = NativeContract.Policy.Call(snapshot, new ManualWitness(UInt160.Zero), "setFeePerByte", new ContractParameter(ContractParameterType.Integer) { Value = 1 }); ret.Should().BeOfType(); ret.ToBoolean().Should().BeTrue(); @@ -181,7 +181,7 @@ public void Check_Block_UnblockAccount() // Block without signature - var ret = NativeContract.Policy.Call(snapshot, new Nep5NativeContractExtensions.ManualWitness(), + var ret = NativeContract.Policy.Call(snapshot, new ManualWitness(), "blockAccount", new ContractParameter(ContractParameterType.Hash160) { Value = UInt160.Zero }); ret.Should().BeOfType(); ret.ToBoolean().Should().BeFalse(); @@ -192,7 +192,7 @@ public void Check_Block_UnblockAccount() // Block with signature - ret = NativeContract.Policy.Call(snapshot, new Nep5NativeContractExtensions.ManualWitness(UInt160.Zero), + ret = NativeContract.Policy.Call(snapshot, new ManualWitness(UInt160.Zero), "blockAccount", new ContractParameter(ContractParameterType.Hash160) { Value = UInt160.Zero }); ret.Should().BeOfType(); ret.ToBoolean().Should().BeTrue(); @@ -204,7 +204,7 @@ public void Check_Block_UnblockAccount() // Unblock without signature - ret = NativeContract.Policy.Call(snapshot, new Nep5NativeContractExtensions.ManualWitness(), + ret = NativeContract.Policy.Call(snapshot, new ManualWitness(), "unblockAccount", new ContractParameter(ContractParameterType.Hash160) { Value = UInt160.Zero }); ret.Should().BeOfType(); ret.ToBoolean().Should().BeFalse(); @@ -216,7 +216,7 @@ public void Check_Block_UnblockAccount() // Unblock with signature - ret = NativeContract.Policy.Call(snapshot, new Nep5NativeContractExtensions.ManualWitness(UInt160.Zero), + ret = NativeContract.Policy.Call(snapshot, new ManualWitness(UInt160.Zero), "unblockAccount", new ContractParameter(ContractParameterType.Hash160) { Value = UInt160.Zero }); ret.Should().BeOfType(); ret.ToBoolean().Should().BeTrue(); diff --git a/tests/neo.UnitTests/SmartContract/UT_Syscalls.cs b/tests/neo.UnitTests/SmartContract/UT_Syscalls.cs index c516d1a871..52f5123606 100644 --- a/tests/neo.UnitTests/SmartContract/UT_Syscalls.cs +++ b/tests/neo.UnitTests/SmartContract/UT_Syscalls.cs @@ -32,7 +32,7 @@ public void System_Blockchain_GetBlock() SystemFee = 0x03, Nonce = 0x04, ValidUntilBlock = 0x05, - Version = 0x06, + Version = TransactionVersion.Transaction, Witnesses = new Witness[] { new Witness() { VerificationScript = new byte[] { 0x07 } } }, Sender = UInt160.Parse("FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF"), }; @@ -251,7 +251,7 @@ public void System_ExecutionEngine_GetScriptContainer() SystemFee = 0x03, Nonce = 0x04, ValidUntilBlock = 0x05, - Version = 0x06, + Version = TransactionVersion.Transaction, Witnesses = new Witness[] { new Witness() { VerificationScript = new byte[] { 0x07 } } }, Sender = UInt160.Parse("FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF"), }; @@ -263,7 +263,7 @@ public void System_ExecutionEngine_GetScriptContainer() Assert.AreEqual(1, engine.ResultStack.Count); Assert.IsInstanceOfType(engine.ResultStack.Peek(), typeof(ByteString)); Assert.AreEqual(engine.ResultStack.Pop().GetSpan().ToHexString(), - @"5b226770564846625133316969517a614f4c7a33523546394d6256715932596b7a5164324461785536677154303d222c362c342c222f2f2f2f2f2f2f2f2f2f2f2f2f2f2f2f2f2f2f2f2f2f2f2f2f2f383d222c332c322c352c2241513d3d225d"); + @"5b2262533635577a41363155363855585367426930466f4d767068356e4b4954566175507867782f6b515070303d222c302c342c222f2f2f2f2f2f2f2f2f2f2f2f2f2f2f2f2f2f2f2f2f2f2f2f2f2f383d222c332c322c352c2241513d3d225d"); Assert.AreEqual(0, engine.ResultStack.Count); } } diff --git a/tests/neo.UnitTests/UT-cert.pfx b/tests/neo.UnitTests/UT-cert.pfx new file mode 100644 index 0000000000000000000000000000000000000000..09bd730b35f912d2f2a5645a26fbb0c7b90ebb12 GIT binary patch literal 2469 zcmV;W30n3rf(fAl0Ru3C31CotfRbmr zsOs;%5D9Hb{H~tZm-2ZUlVHre$D2`olAV?3y_r~+8Yxe9FLV`5n-=moX$xybY&rZE z+~OC6IGt`_piebaM)mf>5H;YMQ1!pUe*iT&ic^YlmeUEPiE@L<>b0!Jl7YL0xa1}--|TV3^o4eGzc7T8xJj|v z5P<2^iF(K;f9*!xdr5XD22El+trtYTbkNZ^&US+Su7Yf2Gx#OcvA*irHwl*`)A!tO zenaYQzPz~M=vE}x<&6+oYxz>XV-PM-uypG<92(Fg+xNVx5KSRQ?yc#lRsoz*7LF`9 z7W|@{%-8}%zICup=0B${thj`oD$tlP<<5ElQqVZmoe0~RsUkabjUcHro#3nwyYf|e zuogBnOG(y=dg_OK7@{^j1}(MlfR5gIaf8Sa$>_0oU-sJfOKGlNH){2iYpkI6v)bs? zu7Ea=ptmI-cC^@tg1g{Bk;64x5snI!Srb5{duTH{C>u=JIfMV2#BL&GOE)(rjXh@k znebV69UlLFCL<2{Oi7p!Fz=osQlC!A@=2Rx-qQ0QTZRIl#Q&Cx7hlQ@je&=ax8uYY z{$2Xr211amfK32=qc(O=nUqm=ja+%5!+t^>L7ShnxhjKDf&^+yuHv{2?a!@44C0ww z6Dpx+2Ctz2Ol^>OoL>@zR=f?Xip_{*WiRF=VkTA?xmR6mM4RkuWJEko2ck|7J=!q9 z2hn(Vj;m)_BVZyE9CkJTU%$@3T4ZUNmWWC;HXpB1=W9=>an)#sr9|;U zBs%wvF6K9gjk-?dw;UwG({NLmw!ei}2Mb6$Vy@zMk}h6p%RLJoyu&Rfnql+s$u{aF zQ8O*@Xw+eZ)_VATT%(o2_5Z2qmbwzT%Ccfkh=o`>iP#_(B?Ei;0yLsm71A<+eOXg%5&vCEBjJy$nd#i2ba6)AIBoo3^$=SWX2Qu8y<5fvv`ey?FoFd^1_>&LNQUwvj$N}B1&||`Xh_~E5uUw=Sqri;{lccvBc5s) z=%f6)t*fq@S4SOg`&BYMJ7c-S`W+(s2|okuAESE5O!@%OwPO!F@R?XpT{!EOZ$tPI zK>)g-I9^Zl1m;y|BN~euotjj4JeH_SY~~s$5HC)Ov2m>uejm-2%h@cA44rvIC)62& zM)YOi{1)k6y#>~^iT1Q*$pJ7RkA=X9JiBDV(w2>vtl{R-{Pi!ml82{c+!&^;=R{|v zYCgnLT35Wy);LgsI2x<;{wWL?U)`=)R~>YB89o)f1K62^6#fN&j_A?6sGN)|9IhR- zp{D;Dzvs9kM3*kQzQ8l{*>#36`g%hEM2IGA>1QWnS@r8O*?oxa`yG7j?@tBKCbUuz z{akIVgEAQke;GD(XGd?W71Fz8Jq!Uw)lXhI))NZzWloQN`zz=}1xDRW@ac`D97qku ziMbO8ypAYC*h%8q27;A>sFL-NGn8X<~5%DHIlIi#Zd znvf6-1_XjEl!@WIE_ej=E=8-|<2`EYYt9bFCBsgzyN>?20h=0%CV+@r>=_8VHOfs5 znMWuK?`iT!yKp&gPUn)dtz;Swxsmbc?;C!E!rwz))f$?`LH9q>ackexw4IHdcg*+S zK#5VcRRS&lZX(RxH)}i>Qh_6V)7iS*V@I|wz9p<9GD5OtF)PwaPJDaC{5zRWX!opr z6wLgwFh8a{{q&hM6Am^pd78SIT zjcQLi)X~=4WAg6o?VwzORem~$BMDF5p*ld0XXZ*GeSa16LgC10BSgkzvBaCK9mxp* zn?vE3MB4~RWhT>&J#C7Hg1fPF&cL*SM7WYy(EPZ~2iZKphIxAJw{k;&ex4mLt*gX! z;+drSG+Zm3m+?l9F0*68L3Q(?^1z9SBc&#{g#X~x_zd-k>2B-$-g4m`;?|JY7IsiQ zjENCE9n(f~(MlV}`I&y`71O9jonv{tx)bK&68LHZDk!pB104z7a5ui;1kc4-JIKfN zTIKHSefkv1H+?I-BQvd}SQLKk*k4+URQ*Y{QPEIs@4fEIlvXR@JnJ+eRxwW+4D1bj zCmD(p0_!);MpnZ$y>8sJ@;FY8jQQ}YPb7}Q?nFeZ9cgM>>S8dI)4ZC;wGA;PFe3&D zDuzgg_YDCF6)_eB6l-$;9PAI>jl`Mmfj2jB#y+Oz)G#qHAutIB1uG5%0vZJX1QeoS j|7%M-Tnm!N9V2)?DX9RjFfIfLF;{W=YD&>~0s;sCJQbF4 literal 0 HcmV?d00001 diff --git a/tests/neo.UnitTests/neo.UnitTests.csproj b/tests/neo.UnitTests/neo.UnitTests.csproj index 25aa788129..d1352da843 100644 --- a/tests/neo.UnitTests/neo.UnitTests.csproj +++ b/tests/neo.UnitTests/neo.UnitTests.csproj @@ -1,4 +1,4 @@ - + Exe @@ -12,10 +12,14 @@ PreserveNewest + + PreserveNewest + + @@ -31,4 +35,5 @@ +