Add DATAHASH opcode (#4894)

* Reformat whitespaces * add withdrawalApplier to blockchainProcessor * add withdrawals to ExecutionPayload * test fixes * fix EVM.test * fix EthereumTests * fix Benchmarks.sln * Revise withdrawals representation according to the spec and refactor * Rename `IWithdrawalApplier` to `IWithdrawalProcessor` * Add tests for withdrawal encoding/decoding * Rename `Recipient` to `Address` * Reformat whitespaces * Revise withdrawals length calculation in block encoding * HasBody? * Remove redundant withdrawals hash check * fix withdrawals_test chainspec && added IReleaseSpec.WithdrawalsEnabled * Fixes * fix GenesisLoader * fix extra-data * Refactor withdrawal validation by implementing `IWithdrawalValidator` * Fix failing tests * Reformat whitespaces * fix withdrawalsTimestamp * temporary change validation * null handling? * Applying Marek's suggestion. Not sure about this one * Fix Ethereum tests * Fix test cases * Fix benchmark build * Fix payload attributes validation * Remove "V1" from `IForkchoiceUpdatedHandler` name * Update tests * hack AuRa tests for now * fix more tests * fix more tests * fix build * + fix one more tests * fix Synchronization tests * Update tests to 0aa59689101f64cab8fa1526d9cf6e647ddba946 * fixed withdrawal chainspec * withdrawals block validator tests * Implement Engine API tests * Reformat code * fix chainspec? * Fix withdrawals decoding * fix chainspec timestamp * Add and fix Engine API tests * Revise block decoder tests * Revise file headers * Revise file headers * Add missing file headers * fix AuRa test * adjusting comments * load genesis tests * Lukasz suggestions * formating * Fix build * withdrawals_hivetests.json + cosmetic * Add more unit tests * spacing * Try on fixing timestamp activation with same value as genesis timestamp * Fix gnosis and chiado ForkId Calculations * fix tests * Cleanup and more tests * Janky but working solution to very rare edge case? * Fix flakiness of caused by Parallelizable * Correct usage of chain ID and network ID * adding engine tests * working on more tests * add loop in test * Can_apply_withdrawals_correctly test * more test cases * Update withdrawals hive tests configuration * Refactor and fix some null reference warnings * Expose withdrawals to JSON-RPC modules * Fix broken tests * Revise `ForkchoiceUpdatedHandler` string output * fix Can_apply_withdrawals_correctly * fix whitespaces * fix whitespaces * more whitespaces fixes * add more tests * Fix chainId loading from spec * Rename for clarity * work on Withdrawals_transition test cases * adjust CustomSpecProvider * Introduce IEip1995Spec * Make test NetworkId vs ChainId different Changing ChainId to another number breaks a lot of test, that are based to hashes and RLP. Let at least test if the values differ * small changes in tests * Fix license * add comments * Apply Marek Suggestion * Adjust Lic to Rubo * Marek Suggestions * Fix * fix withdrawals in ChainLevelHelper * Fix chainid for account abstraction; add and improve tests; fix a tutorial link * Fix naming in other projects, whitespaces * Fix more renaming * fix hive sync tests * fix? * Revise withdrawals root hash encoding/decoding and its tests * fix PayloadAttributes ToString * add more temp logs to investigate hive tests * more logs * revert not needed logs * fix test * ignore incorrect tests * Fix tests broken by withdrawals decoding revision * Revise file headers * Final appraoch, removed GetSpec complexity * Forgotten changes * SecondsPerSlot to BlocksConfig * Minor + config changes * Typo fix * remove empty line * Move BlocksConfig to Nethermind.Config project * Benchmark build fix * Rename `ExecutionPayloadV1` to `ExecutionPayloadV2` * Revise withdrawals check * Refactor tests * Optimize withdrawals root hash decoding in block header * Add fields, spec, header encoding * Implements ForkId tests that are ub EIP-6122 * Spacing * Fix build * Fix ForkId Test case * Removing 3675 and using MergeForkId Transition * Final test fix * Fix tests * Rename `ExecutionPayloadV2` to `ExecutionPayload` * add one more line in Nlog (temp) * Revert NLog for jsonRpc * fix missing body? * adjust TxPool logs * Add more test cases for `BlockBodiesMessageSerializer` * Add tests for `BlockHeader.HasBody` * Revise `null` handling for `BlockHeader.HasBody` * fix BlockBody empty * cosmetic * fix CI * Revise tests * Update null checks with pattern matching * Refactor block body initialization and add tests * Revise whitespaces * cosmetic * ForkId calculation polish (#5068) * Refactor - mainly ChainSpecBasedSpecProvider * fixes * Better warning message * Simplify sanity check * one more simplification * Revise Clique block production according to withdrawals rules * Add withdrawals to `eth_getBlockByNumber` tests * SecondsPerSlot move to BlocksConfig (#4944) Co-authored-by: MarekM25 <marekm2504@gmail.com> Co-authored-by: lukasz.rozmej <lukasz.rozmej@gmail.com> Resolves #4871 * fix build * removed duplicated NSubstitute * Fix network id in node info * Rollback spec change * Rename NetworkId class to BlockchainIds; more renaming and usage of TestBlockChainIds * Improve a test * Fix chainid/networkid usage * Refactors * Improve CustomSpecProvider ctor * Use test provider * ethstats fix * whitespace * Fix * Fix header encoding: loading from genesis * Remove fork, simplify codfe, fix merge conflicts * Improve tests, fix spaces * Fix whitespaces * Fix tests * Fix space * Fix Rlp for headers * Add DATAHASH opcode * LogFinder should be initialized befre UserOperationPools * Fix postmerge * Add no blobs for benchmarks, let's add it with DATAHASH benchmarks Co-authored-by: Ruben Buniatyan <rubo@users.noreply.github.com> Co-authored-by: MarekM25 <marekm2504@gmail.com> Co-authored-by: smartprogrammer <smartprogrammer@windowslive.com> Co-authored-by: Lukasz Rozmej <lukasz.rozmej@gmail.com> Co-authored-by: Ahmad Bitar <33181301+smartprogrammer93@users.noreply.github.com>
NethermindEth · Jan 21, 2023 · 6fe2983 · 6fe2983
1 parent bf29cbf
commit 6fe2983
Show file tree

Hide file tree

Showing 15 changed files with 109 additions and 21 deletions.
diff --git a/src/Nethermind/Nethermind.Core.Test/Blockchain/TestBlockchain.cs b/src/Nethermind/Nethermind.Core.Test/Blockchain/TestBlockchain.cs
@@ -66,6 +66,7 @@ public IBlockFinder BlockFinder
             set => _blockFinder = value;
         }
 
+        public ILogFinder LogFinder { get; private set; } = null!;
         public IJsonSerializer JsonSerializer { get; set; } = null!;
         public IStateProvider State { get; set; } = null!;
         public IReadOnlyStateProvider ReadOnlyState { get; private set; } = null!;
@@ -165,6 +166,10 @@ protected virtual async Task<TestBlockchain> Build(ISpecProvider? specProvider =
             PoSSwitcher = NoPoS.Instance;
             ISealer sealer = new NethDevSealEngine(TestItem.AddressD);
             SealEngine = new SealEngine(sealer, Always.Valid);
+
+            BloomStorage bloomStorage = new(new BloomConfig(), new MemDb(), new InMemoryDictionaryFileStoreFactory());
+            ReceiptsRecovery receiptsRecovery = new(new EthereumEcdsa(SpecProvider.ChainId, LimboLogs.Instance), SpecProvider);
+            LogFinder = new LogFinder(BlockTree, ReceiptStorage, ReceiptStorage, bloomStorage, LimboLogs.Instance, receiptsRecovery);
             BlockProcessor = CreateBlockProcessor();
 
             BlockchainProcessor chainProcessor = new(BlockTree, BlockProcessor, BlockPreprocessorStep, StateReader, LogManager, Consensus.Processing.BlockchainProcessor.Options.Default);

diff --git a/src/Nethermind/Nethermind.Core.Test/Nethermind.Core.Test.csproj b/src/Nethermind/Nethermind.Core.Test/Nethermind.Core.Test.csproj
@@ -20,6 +20,7 @@
   <ItemGroup>
     <ProjectReference Include="..\Nethermind.Blockchain\Nethermind.Blockchain.csproj" />
     <ProjectReference Include="..\Nethermind.Core\Nethermind.Core.csproj" />
+    <ProjectReference Include="..\Nethermind.Facade\Nethermind.Facade.csproj" />
     <ProjectReference Include="..\Nethermind.Network\Nethermind.Network.csproj" />
     <ProjectReference Include="..\Nethermind.Specs.Test\Nethermind.Specs.Test.csproj" />
   </ItemGroup>

diff --git a/src/Nethermind/Nethermind.Evm.Benchmark/EvmBenchmarks.cs b/src/Nethermind/Nethermind.Evm.Benchmark/EvmBenchmarks.cs
@@ -61,7 +61,7 @@ public void GlobalSetup()
                 CodeInfo = new CodeInfo(ByteCode),
                 Value = 0,
                 TransferValue = 0,
-                TxExecutionContext = new TxExecutionContext(_header, Address.Zero, 0)
+                TxExecutionContext = new TxExecutionContext(_header, Address.Zero, 0, null)
             };
 
             _evmState = new EvmState(long.MaxValue, _environment, ExecutionType.Transaction, true, _worldState.TakeSnapshot(), false);

diff --git a/src/Nethermind/Nethermind.Evm.Benchmark/MultipleUnsignedOperations.cs b/src/Nethermind/Nethermind.Evm.Benchmark/MultipleUnsignedOperations.cs
@@ -91,7 +91,7 @@ public void GlobalSetup()
             CodeInfo = new CodeInfo(_bytecode.Concat(_bytecode).Concat(_bytecode).Concat(_bytecode).ToArray()),
             Value = 0,
             TransferValue = 0,
-            TxExecutionContext = new TxExecutionContext(_header, Address.Zero, 0)
+            TxExecutionContext = new TxExecutionContext(_header, Address.Zero, 0, null)
         };
 
         _evmState = new EvmState(100_000_000L, _environment, ExecutionType.Transaction, true, _worldState.TakeSnapshot(), false);

diff --git a/src/Nethermind/Nethermind.Evm.Benchmark/StaticCallBenchmarks.cs b/src/Nethermind/Nethermind.Evm.Benchmark/StaticCallBenchmarks.cs
@@ -102,7 +102,7 @@ public void GlobalSetup()
                 CodeInfo = new CodeInfo(Bytecode),
                 Value = 0,
                 TransferValue = 0,
-                TxExecutionContext = new TxExecutionContext(_header, Address.Zero, 0)
+                TxExecutionContext = new TxExecutionContext(_header, Address.Zero, 0, null)
             };
 
             _evmState = new EvmState(100_000_000L, _environment, ExecutionType.Transaction, true, _worldState.TakeSnapshot(), false);

diff --git a/src/Nethermind/Nethermind.Evm.Test/Eip4844Tests.cs b/src/Nethermind/Nethermind.Evm.Test/Eip4844Tests.cs
@@ -0,0 +1,60 @@
+// SPDX-FileCopyrightText: 2022 Demerzel Solutions Limited
+// SPDX-License-Identifier: LGPL-3.0-only
+
+using FluentAssertions;
+using Nethermind.Specs;
+using NUnit.Framework;
+using System;
+using Nethermind.Int256;
+using System.Linq;
+
+namespace Nethermind.Evm.Test;
+
+[TestFixture]
+public class Eip4844Tests : VirtualMachineTestsBase
+{
+    protected override long BlockNumber => MainnetSpecProvider.GrayGlacierBlockNumber;
+    protected override ulong Timestamp => MainnetSpecProvider.CancunBlockTimestamp;
+
+    [TestCase(0, 0, Description = "Should return 0 when no hashes")]
+    [TestCase(1, 1, Description = "Should return 0 when out of range")]
+    [TestCase(2, 1, Description = "Should return 0 when way out of range")]
+    [TestCase(0, 1, Description = "Should return hash, when exists")]
+    [TestCase(1, 3, Description = "Should return hash, when exists")]
+    public void Test_datahash_index_in_range(int index, int datahashesCount)
+    {
+        byte[][] hashes = new byte[datahashesCount][];
+        for (int i = 0; i < datahashesCount; i++)
+        {
+            hashes[i] = new byte[32];
+            for (int n = 0; n < datahashesCount; n++)
+            {
+                hashes[i][n] = (byte)((i * 3 + 10 * 7) % 256);
+            }
+        }
+        byte[] expectedOutput = datahashesCount > index ? hashes[index] : new byte[32];
+
+        // Cost of transaction call + PUSH1 x4 + MSTORE (entry cost + 1 memory cell used)
+        const long GasCostOfCallingWrapper = GasCostOf.Transaction + GasCostOf.VeryLow * 5 + GasCostOf.Memory;
+
+        byte[] code = Prepare.EvmCode
+            .PushData(new UInt256((ulong)index))
+            .DATAHASH()
+            .MSTORE(0)
+            .Return(32, 0)
+            .Done;
+
+        TestAllTracerWithOutput result = Execute(BlockNumber, 50000, code, blobVersionedHashes: hashes, timestamp: Timestamp);
+
+        result.StatusCode.Should().Be(StatusCode.Success);
+        result.ReturnValue.SequenceEqual(expectedOutput);
+        AssertGas(result, GasCostOfCallingWrapper + GasCostOf.DataHash);
+    }
+
+    protected override TestAllTracerWithOutput CreateTracer()
+    {
+        TestAllTracerWithOutput tracer = base.CreateTracer();
+        tracer.IsTracingAccess = false;
+        return tracer;
+    }
+}
diff --git a/src/Nethermind/Nethermind.Evm.Test/InvalidOpcodeTests.cs b/src/Nethermind/Nethermind.Evm.Test/InvalidOpcodeTests.cs
@@ -106,7 +106,7 @@ public class InvalidOpcodeTests : VirtualMachineTestsBase
                 {
                     Instruction.TSTORE,
                     Instruction.TLOAD,
-                    // TODO: Add DATAHASH
+                    Instruction.DATAHASH,
                 }
             ).ToArray();
 

diff --git a/src/Nethermind/Nethermind.Evm.Test/VirtualMachineTestsBase.cs b/src/Nethermind/Nethermind.Evm.Test/VirtualMachineTestsBase.cs
@@ -110,9 +110,9 @@ protected TestAllTracerWithOutput Execute(params byte[] code)
             return tracer;
         }
 
-        protected TestAllTracerWithOutput Execute(long blockNumber, long gasLimit, byte[] code, long blockGasLimit = DefaultBlockGasLimit, ulong timestamp = 0)
+        protected TestAllTracerWithOutput Execute(long blockNumber, long gasLimit, byte[] code, long blockGasLimit = DefaultBlockGasLimit, ulong timestamp = 0, byte[][] blobVersionedHashes = null)
         {
-            (Block block, Transaction transaction) = PrepareTx(blockNumber, gasLimit, code, blockGasLimit: blockGasLimit, timestamp: timestamp);
+            (Block block, Transaction transaction) = PrepareTx(blockNumber, gasLimit, code, blockGasLimit: blockGasLimit, timestamp: timestamp, blobVersionedHashes: blobVersionedHashes);
             TestAllTracerWithOutput tracer = CreateTracer();
             _processor.Execute(transaction, block.Header, tracer);
             return tracer;
@@ -125,7 +125,8 @@ protected TestAllTracerWithOutput Execute(long blockNumber, long gasLimit, byte[
             SenderRecipientAndMiner senderRecipientAndMiner = null,
             int value = 1,
             long blockGasLimit = DefaultBlockGasLimit,
-            ulong timestamp = 0)
+            ulong timestamp = 0,
+            byte[][] blobVersionedHashes = null)
         {
             senderRecipientAndMiner ??= SenderRecipientAndMiner.Default;
             TestState.CreateAccount(senderRecipientAndMiner.Sender, 100.Ether());
@@ -144,6 +145,7 @@ protected TestAllTracerWithOutput Execute(long blockNumber, long gasLimit, byte[
                 .WithGasLimit(gasLimit)
                 .WithGasPrice(1)
                 .WithValue(value)
+                .WithBlobVersionedHashes(blobVersionedHashes)
                 .To(senderRecipientAndMiner.Recipient)
                 .SignedAndResolved(_ethereumEcdsa, senderRecipientAndMiner.SenderKey)
                 .TestObject;

diff --git a/src/Nethermind/Nethermind.Evm/ByteCodeBuilderExtensions.cs b/src/Nethermind/Nethermind.Evm/ByteCodeBuilderExtensions.cs
@@ -1,15 +1,8 @@
 // SPDX-FileCopyrightText: 2022 Demerzel Solutions Limited
 // SPDX-License-Identifier: LGPL-3.0-only
 
-using System;
-using System.Collections;
-using System.Collections.Generic;
 using Nethermind.Core;
-using Nethermind.Core.Collections;
-using Nethermind.Core.Extensions;
 using Nethermind.Int256;
-using Nethermind.Serialization.Json;
-using Org.BouncyCastle.Asn1.Mozilla;
 
 namespace Nethermind.Evm
 {
@@ -92,6 +85,8 @@ public static Prepare SELFBALANCE(this Prepare @this)
             => @this.Op(Instruction.SELFBALANCE);
         public static Prepare BASEFEE(this Prepare @this)
             => @this.Op(Instruction.BASEFEE);
+        public static Prepare DATAHASH(this Prepare @this)
+            => @this.Op(Instruction.DATAHASH);
         public static Prepare POP(this Prepare @this)
             => @this.Op(Instruction.POP);
         public static Prepare PC(this Prepare @this)

diff --git a/src/Nethermind/Nethermind.Evm/GasCostOf.cs b/src/Nethermind/Nethermind.Evm/GasCostOf.cs
@@ -39,6 +39,7 @@ public static class GasCostOf
         public const long TxDataNonZero = 68;
         public const long TxDataNonZeroEip2028 = 16;
         public const long Transaction = 21000;
+        public const long DataHash = 3;
         public const long Log = 375;
         public const long LogTopic = 375;
         public const long LogData = 8;

diff --git a/src/Nethermind/Nethermind.Evm/Instruction.cs b/src/Nethermind/Nethermind.Evm/Instruction.cs
@@ -67,6 +67,7 @@ public enum Instruction : byte
         CHAINID = 0x46,
         SELFBALANCE = 0x47,
         BASEFEE = 0x48,
+        DATAHASH = 0x49,
 
         POP = 0x50,
         MLOAD = 0x51,

diff --git a/src/Nethermind/Nethermind.Evm/TransactionProcessing/TransactionProcessor.cs b/src/Nethermind/Nethermind.Evm/TransactionProcessing/TransactionProcessor.cs
@@ -321,7 +321,7 @@ senderBalance < (UInt256)transaction.GasLimit * transaction.MaxFeePerGas + value
                 recipientOrNull = recipient;
 
                 ExecutionEnvironment env = new();
-                env.TxExecutionContext = new TxExecutionContext(block, caller, effectiveGasPrice);
+                env.TxExecutionContext = new TxExecutionContext(block, caller, effectiveGasPrice, transaction.BlobVersionedHashes);
                 env.Value = value;
                 env.TransferValue = value;
                 env.Caller = caller;

diff --git a/src/Nethermind/Nethermind.Evm/TxExecutionContext.cs b/src/Nethermind/Nethermind.Evm/TxExecutionContext.cs
@@ -11,12 +11,14 @@ namespace Nethermind.Evm
         public BlockHeader Header { get; }
         public Address Origin { get; }
         public UInt256 GasPrice { get; }
+        public byte[][]? BlobVersionedHashes { get; }
 
-        public TxExecutionContext(BlockHeader blockHeader, Address origin, in UInt256 gasPrice)
+        public TxExecutionContext(BlockHeader blockHeader, Address origin, in UInt256 gasPrice, byte[][] blobVersionedHashes)
         {
             Header = blockHeader;
             Origin = origin;
             GasPrice = gasPrice;
+            BlobVersionedHashes = blobVersionedHashes;
         }
     }
 }
diff --git a/src/Nethermind/Nethermind.Evm/VirtualMachine.cs b/src/Nethermind/Nethermind.Evm/VirtualMachine.cs
@@ -1720,6 +1720,32 @@ void UpdateMemoryCost(in UInt256 position, in UInt256 length)
                             stack.PushUInt256(in baseFee);
                             break;
                         }
+                    case Instruction.DATAHASH:
+                        {
+                            if (!spec.IsEip4844Enabled)
+                            {
+                                EndInstructionTraceError(EvmExceptionType.BadInstruction);
+                                return CallResult.InvalidInstructionException;
+                            }
+
+                            if (!UpdateGas(GasCostOf.DataHash, ref gasAvailable))
+                            {
+                                EndInstructionTraceError(EvmExceptionType.OutOfGas);
+                                return CallResult.OutOfGasException;
+                            }
+
+                            stack.PopUInt256(out UInt256 blobIndex);
+
+                            if (txCtx.BlobVersionedHashes is not null && blobIndex < txCtx.BlobVersionedHashes.Length)
+                            {
+                                stack.PushBytes(txCtx.BlobVersionedHashes[blobIndex.u0]);
+                            }
+                            else
+                            {
+                                stack.PushZero();
+                            }
+                            break;
+                        }
                     case Instruction.POP:
                         {
                             if (!UpdateGas(GasCostOf.Base, ref gasAvailable))

diff --git a/src/Nethermind/Nethermind.JsonRpc.Test/Modules/TestRpcBlockchain.cs b/src/Nethermind/Nethermind.JsonRpc.Test/Modules/TestRpcBlockchain.cs
@@ -40,8 +40,6 @@ public class TestRpcBlockchain : TestBlockchain
         public IBlockchainBridge Bridge { get; private set; } = null!;
         public ITxSealer TxSealer { get; private set; } = null!;
         public ITxSender TxSender { get; private set; } = null!;
-        public ILogFinder LogFinder { get; private set; } = null!;
-
         public IReceiptFinder ReceiptFinder { get; private set; } = null!;
         public IGasPriceOracle GasPriceOracle { get; private set; } = null!;
 
@@ -108,14 +106,11 @@ public async Task<T> Build(ISpecProvider? specProvider = null, UInt256? initialV
 
         protected override async Task<TestBlockchain> Build(ISpecProvider? specProvider = null, UInt256? initialValues = null)
         {
-            BloomStorage bloomStorage = new(new BloomConfig(), new MemDb(), new InMemoryDictionaryFileStoreFactory());
             specProvider ??= new TestSpecProvider(Berlin.Instance);
             await base.Build(specProvider, initialValues);
             IFilterStore filterStore = new FilterStore();
             IFilterManager filterManager = new FilterManager(filterStore, BlockProcessor, TxPool, LimboLogs.Instance);
 
-            ReceiptsRecovery receiptsRecovery = new(new EthereumEcdsa(specProvider.ChainId, LimboLogs.Instance), specProvider);
-            LogFinder = new LogFinder(BlockTree, ReceiptStorage, ReceiptStorage, bloomStorage, LimboLogs.Instance, receiptsRecovery);
 
             ReadOnlyTxProcessingEnv processingEnv = new(
                 new ReadOnlyDbProvider(DbProvider, false),