Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add VM Version Paris and new built in function prevrandao #13531

Closed
wants to merge 4 commits into from
Closed
Show file tree
Hide file tree
Changes from 3 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
2 changes: 1 addition & 1 deletion .circleci/soltest_all.sh
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ REPODIR="$(realpath "$(dirname "$0")"/..)"
# shellcheck source=scripts/common.sh
source "${REPODIR}/scripts/common.sh"

EVM_VALUES=(homestead byzantium constantinople petersburg istanbul berlin london)
EVM_VALUES=(homestead byzantium constantinople petersburg istanbul berlin london paris)
axic marked this conversation as resolved.
Show resolved Hide resolved
DEFAULT_EVM=london
[[ " ${EVM_VALUES[*]} " =~ $DEFAULT_EVM ]]
OPTIMIZE_VALUES=(0 1)
Expand Down
7 changes: 4 additions & 3 deletions Changelog.md
Original file line number Diff line number Diff line change
@@ -1,12 +1,13 @@
### 0.8.18 (unreleased)

Language Features:
* Add support for ``prevrandao()`` which was introduced in EVM version to "Paris", supplanting ``difficulty`` in the process. Both can be used interchangeably.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As mentioned on the issue I think this is a bad idea, because the two outputs are expected to be vastly different and hence expect different use cases.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well, this is exactly implemented according to the summary after the discussion, so I'm not sure what to tell you here...
#13512 (comment)

Copy link
Member

@ekpyron ekpyron Oct 5, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The main point is that we should not suggest to use them interchangeably. Supporting both (with a warning on mismatch) is a compromise for not being breaking for hypothetical libraries that want to use it as a source of randomness in a way that can rely on either version - but in general you should still very much use the correct one and not use them interchangeably.



Compiler Features:
* Commandline Interface: Add `--no-cbor-metadata` that skips CBOR metadata from getting appended at the end of the bytecode.
* Standard JSON: Add a boolean field `settings.metadata.appendCBOR` that skips CBOR metadata from getting appended at the end of the bytecode.
* Yul Optimizer: Allow replacing the previously hard-coded cleanup sequence by specifying custom steps after a colon delimiter (``:``) in the sequence string.
* Commandline Interface: Add `--no-cbor-metadata` that skips CBOR metadata from getting appended at the end of the bytecode.
* Standard JSON: Add a boolean field `settings.metadata.appendCBOR` that skips CBOR metadata from getting appended at the end of the bytecode.
* Yul Optimizer: Allow replacing the previously hard-coded cleanup sequence by specifying custom steps after a colon delimiter (``:``) in the sequence string.


Bugfixes:
Expand Down
3 changes: 2 additions & 1 deletion docs/cheatsheet.rst
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ Order of Precedence of Operators
================================
.. include:: types/operator-precedence-table.rst

.. index:: assert, block, coinbase, difficulty, number, block;number, timestamp, block;timestamp, msg, data, gas, sender, value, gas price, origin, revert, require, keccak256, ripemd160, sha256, ecrecover, addmod, mulmod, cryptography, this, super, selfdestruct, balance, codehash, send
.. index:: assert, block, coinbase, difficulty, prevrandao, number, block;number, timestamp, block;timestamp, msg, data, gas, sender, value, gas price, origin, revert, require, keccak256, ripemd160, sha256, ecrecover, addmod, mulmod, cryptography, this, super, selfdestruct, balance, codehash, send

Global Variables
================
Expand All @@ -33,6 +33,7 @@ Global Variables
- ``block.chainid`` (``uint``): current chain id
- ``block.coinbase`` (``address payable``): current block miner's address
- ``block.difficulty`` (``uint``): current block difficulty
- ``block.prevrandao`` (``uint``): randomness provided by the beacon chain
- ``block.gaslimit`` (``uint``): current block gaslimit
- ``block.number`` (``uint``): current block number
- ``block.timestamp`` (``uint``): current block timestamp in seconds since Unix epoch
Expand Down
4 changes: 2 additions & 2 deletions docs/grammar/SolidityLexer.g4
Original file line number Diff line number Diff line change
Expand Up @@ -291,8 +291,8 @@ YulEVMBuiltin:
| 'returndatacopy' | 'extcodehash' | 'create' | 'create2' | 'call' | 'callcode'
| 'delegatecall' | 'staticcall' | 'return' | 'revert' | 'selfdestruct' | 'invalid'
| 'log0' | 'log1' | 'log2' | 'log3' | 'log4' | 'chainid' | 'origin' | 'gasprice'
| 'blockhash' | 'coinbase' | 'timestamp' | 'number' | 'difficulty' | 'gaslimit'
| 'basefee';
| 'blockhash' | 'coinbase' | 'timestamp' | 'number' | 'difficulty' | 'prevrandao'
| 'gaslimit' | 'basefee';

YulLBrace: '{' -> pushMode(YulMode);
YulRBrace: '}' -> popMode;
Expand Down
3 changes: 2 additions & 1 deletion docs/units-and-global-variables.rst
Original file line number Diff line number Diff line change
Expand Up @@ -65,7 +65,7 @@ There are special variables and functions which always exist in the global
namespace and are mainly used to provide information about the blockchain
or are general-use utility functions.

.. index:: abi, block, coinbase, difficulty, encode, number, block;number, timestamp, block;timestamp, msg, data, gas, sender, value, gas price, origin
.. index:: abi, block, coinbase, difficulty, prevrandao, encode, number, block;number, timestamp, block;timestamp, msg, data, gas, sender, value, gas price, origin


Block and Transaction Properties
Expand All @@ -76,6 +76,7 @@ Block and Transaction Properties
- ``block.chainid`` (``uint``): current chain id
- ``block.coinbase`` (``address payable``): current block miner's address
- ``block.difficulty`` (``uint``): current block difficulty
- ``block.prevrandao`` (``uint``): randomness provided by the beacon chain
- ``block.gaslimit`` (``uint``): current block gaslimit
- ``block.number`` (``uint``): current block number
- ``block.timestamp`` (``uint``): current block timestamp as seconds since unix epoch
Expand Down
3 changes: 3 additions & 0 deletions docs/using-the-compiler.rst
Original file line number Diff line number Diff line change
Expand Up @@ -172,6 +172,9 @@ at each version. Backward compatibility is not guaranteed between each version.
the optimizer.
- ``london`` (**default**)
- The block's base fee (`EIP-3198 <https://eips.ethereum.org/EIPS/eip-3198>`_ and `EIP-1559 <https://eips.ethereum.org/EIPS/eip-1559>`_) can be accessed via the global ``block.basefee`` or ``basefee()`` in inline assembly.
- ``paris``
- The ``difficulty`` opcode was supplanted by the new opcode ``prevrandao`` (`EIP-4399 <https://eips.ethereum.org/EIPS/eip-4399>`_).
Both can be used interchangeably. ``prevrandao`` is a reserved keyword in yul code.


.. index:: ! standard JSON, ! --standard-json
Expand Down
10 changes: 7 additions & 3 deletions docs/yul.rst
Original file line number Diff line number Diff line change
Expand Up @@ -755,8 +755,8 @@ This document does not want to be a full description of the Ethereum virtual mac
Please refer to a different document if you are interested in the precise semantics.

Opcodes marked with ``-`` do not return a result and all others return exactly one value.
Opcodes marked with ``F``, ``H``, ``B``, ``C``, ``I`` and ``L`` are present since Frontier, Homestead,
Byzantium, Constantinople, Istanbul or London respectively.
Opcodes marked with ``F``, ``H``, ``B``, ``C``, ``I``, ``L`` and ``P`` are present since Frontier, Homestead,
Byzantium, Constantinople, Istanbul, London or Paris respectively.

In the following, ``mem[a...b)`` signifies the bytes of memory starting at position ``a`` up to
but not including position ``b`` and ``storage[p]`` signifies the storage contents at slot ``p``.
Expand Down Expand Up @@ -933,7 +933,11 @@ the ``dup`` and ``swap`` instructions as well as ``jump`` instructions, labels a
+-------------------------+-----+---+-----------------------------------------------------------------+
| number() | | F | current block number |
+-------------------------+-----+---+-----------------------------------------------------------------+
| difficulty() | | F | difficulty of the current block |
| difficulty() | | F | difficulty of the current block, supplanted by ``prevrandao()`` |
| | | | in evm version paris |
+-------------------------+-----+---+-----------------------------------------------------------------+
| prevrandao() | | P | randomness provided by the beacon chain, supplants |
| | | | ``difficulty()`` starting with evm version paris |
+-------------------------+-----+---+-----------------------------------------------------------------+
| gaslimit() | | F | block gas limit of the current block |
+-------------------------+-----+---+-----------------------------------------------------------------+
Expand Down
50 changes: 28 additions & 22 deletions libevmasm/Assembly.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -426,7 +426,7 @@ map<u256, u256> const& Assembly::optimiseInternal(
AssemblyItems optimisedItems;

bool usesMSize = ranges::any_of(m_items, [](AssemblyItem const& _i) {
return _i == AssemblyItem{Instruction::MSIZE} || _i.type() == VerbatimBytecode;
return _i == AssemblyItem{InternalInstruction::MSIZE} || _i.type() == VerbatimBytecode;
});

auto iter = m_items.begin();
Expand Down Expand Up @@ -537,16 +537,22 @@ LinkerObject const& Assembly::assemble() const
multimap<size_t, size_t> subRef;
vector<unsigned> sizeRef; ///< Pointers to code locations where the size of the program is inserted
unsigned bytesPerTag = numberEncodingSize(bytesRequiredForCode);
uint8_t tagPush = static_cast<uint8_t>(pushInstruction(bytesPerTag));
InternalInstruction tagPush = pushInstruction(bytesPerTag);

unsigned bytesRequiredIncludingData = bytesRequiredForCode + 1 + static_cast<unsigned>(m_auxiliaryData.size());
for (auto const& sub: m_subs)
bytesRequiredIncludingData += static_cast<unsigned>(sub->assemble().bytecode.size());

unsigned bytesPerDataRef = numberEncodingSize(bytesRequiredIncludingData);
uint8_t dataRefPush = static_cast<uint8_t>(pushInstruction(bytesPerDataRef));
InternalInstruction dataRefPush = pushInstruction(bytesPerDataRef);

ret.bytecode.reserve(bytesRequiredIncludingData);

auto const appendInstruction = [&](InternalInstruction _instruction)
{
ret.bytecode.push_back(static_cast<uint8_t>((evmasm::instructionOpcode(_instruction))));
};

for (AssemblyItem const& i: m_items)
{
// store position of the invalid jump destination
Expand All @@ -556,32 +562,32 @@ LinkerObject const& Assembly::assemble() const
switch (i.type())
{
case Operation:
ret.bytecode.push_back(static_cast<uint8_t>(i.instruction()));
appendInstruction(i.instruction());
break;
case Push:
{
unsigned b = max<unsigned>(1, numberEncodingSize(i.data()));
ret.bytecode.push_back(static_cast<uint8_t>(pushInstruction(b)));
appendInstruction(pushInstruction(b));
ret.bytecode.resize(ret.bytecode.size() + b);
bytesRef byr(&ret.bytecode.back() + 1 - b, b);
toBigEndian(i.data(), byr);
break;
}
case PushTag:
{
ret.bytecode.push_back(tagPush);
appendInstruction(tagPush);
tagRef[ret.bytecode.size()] = i.splitForeignPushTag();
ret.bytecode.resize(ret.bytecode.size() + bytesPerTag);
break;
}
case PushData:
ret.bytecode.push_back(dataRefPush);
appendInstruction(dataRefPush);
dataRef.insert(make_pair(h256(i.data()), ret.bytecode.size()));
ret.bytecode.resize(ret.bytecode.size() + bytesPerDataRef);
break;
case PushSub:
assertThrow(i.data() <= numeric_limits<size_t>::max(), AssemblyException, "");
ret.bytecode.push_back(dataRefPush);
appendInstruction(dataRefPush);
subRef.insert(make_pair(static_cast<size_t>(i.data()), ret.bytecode.size()));
ret.bytecode.resize(ret.bytecode.size() + bytesPerDataRef);
break;
Expand All @@ -591,26 +597,26 @@ LinkerObject const& Assembly::assemble() const
auto s = subAssemblyById(static_cast<size_t>(i.data()))->assemble().bytecode.size();
i.setPushedValue(u256(s));
unsigned b = max<unsigned>(1, numberEncodingSize(s));
ret.bytecode.push_back(static_cast<uint8_t>(pushInstruction(b)));
appendInstruction(pushInstruction(b));
ret.bytecode.resize(ret.bytecode.size() + b);
bytesRef byr(&ret.bytecode.back() + 1 - b, b);
toBigEndian(s, byr);
break;
}
case PushProgramSize:
{
ret.bytecode.push_back(dataRefPush);
appendInstruction(dataRefPush);
sizeRef.push_back(static_cast<unsigned>(ret.bytecode.size()));
ret.bytecode.resize(ret.bytecode.size() + bytesPerDataRef);
break;
}
case PushLibraryAddress:
ret.bytecode.push_back(static_cast<uint8_t>(Instruction::PUSH20));
appendInstruction(InternalInstruction::PUSH20);
ret.linkReferences[ret.bytecode.size()] = m_libraries.at(i.data());
ret.bytecode.resize(ret.bytecode.size() + 20);
break;
case PushImmutable:
ret.bytecode.push_back(static_cast<uint8_t>(Instruction::PUSH32));
appendInstruction(InternalInstruction::PUSH32);
// Maps keccak back to the "identifier" string of that immutable.
ret.immutableReferences[i.data()].first = m_immutables.at(i.data());
// Record the bytecode offset of the PUSH32 argument.
Expand All @@ -629,26 +635,26 @@ LinkerObject const& Assembly::assemble() const
{
if (i != offsets.size() - 1)
{
ret.bytecode.push_back(uint8_t(Instruction::DUP2));
ret.bytecode.push_back(uint8_t(Instruction::DUP2));
appendInstruction(InternalInstruction::DUP2);
appendInstruction(InternalInstruction::DUP2);
}
// TODO: should we make use of the constant optimizer methods for pushing the offsets?
bytes offsetBytes = toCompactBigEndian(u256(offsets[i]));
ret.bytecode.push_back(static_cast<uint8_t>(pushInstruction(static_cast<unsigned>(offsetBytes.size()))));
appendInstruction(pushInstruction(static_cast<unsigned>(offsetBytes.size())));
ret.bytecode += offsetBytes;
ret.bytecode.push_back(uint8_t(Instruction::ADD));
ret.bytecode.push_back(uint8_t(Instruction::MSTORE));
appendInstruction(InternalInstruction::ADD);
appendInstruction(InternalInstruction::MSTORE);
}
if (offsets.empty())
{
ret.bytecode.push_back(uint8_t(Instruction::POP));
ret.bytecode.push_back(uint8_t(Instruction::POP));
appendInstruction(InternalInstruction::POP);
appendInstruction(InternalInstruction::POP);
}
immutableReferencesBySub.erase(i.data());
break;
}
case PushDeployTimeAddress:
ret.bytecode.push_back(static_cast<uint8_t>(Instruction::PUSH20));
appendInstruction(InternalInstruction::PUSH20);
ret.bytecode.resize(ret.bytecode.size() + 20);
break;
case Tag:
Expand All @@ -659,7 +665,7 @@ LinkerObject const& Assembly::assemble() const
assertThrow(ret.bytecode.size() < 0xffffffffL, AssemblyException, "Tag too large.");
assertThrow(m_tagPositionsInBytecode[tagId] == numeric_limits<size_t>::max(), AssemblyException, "Duplicate tag position.");
m_tagPositionsInBytecode[tagId] = ret.bytecode.size();
ret.bytecode.push_back(static_cast<uint8_t>(Instruction::JUMPDEST));
appendInstruction(InternalInstruction::JUMPDEST);
break;
}
default:
Expand All @@ -677,7 +683,7 @@ LinkerObject const& Assembly::assemble() const

if (!m_subs.empty() || !m_data.empty() || !m_auxiliaryData.empty())
// Append an INVALID here to help tests find miscompilation.
ret.bytecode.push_back(static_cast<uint8_t>(Instruction::INVALID));
appendInstruction(InternalInstruction::INVALID);

for (auto const& [subIdPath, bytecodeOffset]: subRef)
{
Expand Down
12 changes: 7 additions & 5 deletions libevmasm/Disassemble.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,8 @@
*/
// SPDX-License-Identifier: GPL-3.0

#include "libevmasm/Instruction.h"

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Coding style error

#include "liblangutil/EVMVersion.h"

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Coding style error

#include <libevmasm/Disassemble.h>

#include <libsolutil/Common.h>
Expand All @@ -27,15 +29,15 @@ using namespace solidity;
using namespace solidity::util;
using namespace solidity::evmasm;


void solidity::evmasm::eachInstruction(
bytes const& _mem,
function<void(Instruction,u256 const&)> const& _onInstruction
langutil::EVMVersion _evmVersion,
function<void(InternalInstruction,u256 const&)> const& _onInstruction
)
{
for (auto it = _mem.begin(); it < _mem.end(); ++it)
{
Instruction const instr{*it};
InternalInstruction instr = internalInstruction(InstructionOpCode(*it), _evmVersion);
int additional = 0;
if (isValidInstruction(instr))
additional = instructionInfo(instr).additional;
Expand All @@ -57,10 +59,10 @@ void solidity::evmasm::eachInstruction(
}
}

string solidity::evmasm::disassemble(bytes const& _mem, string const& _delimiter)
string solidity::evmasm::disassemble(bytes const& _mem, langutil::EVMVersion _evmVersion, string const& _delimiter)
{
stringstream ret;
eachInstruction(_mem, [&](Instruction _instr, u256 const& _data) {
eachInstruction(_mem, _evmVersion, [&](InternalInstruction _instr, u256 const& _data) {
if (!isValidInstruction(_instr))
ret << "0x" << std::uppercase << std::hex << static_cast<int>(_instr) << _delimiter;
Comment on lines 66 to 67
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This doesn't look right - in case the byte wasn't a valid instruction, it should output the byte directly, not the internal enum value... so both this and eachInstruction will probably need to change slightly.

else
Expand Down
5 changes: 3 additions & 2 deletions libevmasm/Disassemble.h
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@

#pragma once

#include "liblangutil/EVMVersion.h"
#include <libsolutil/Common.h>
#include <libsolutil/Numeric.h>

Expand All @@ -30,9 +31,9 @@ namespace solidity::evmasm
{

/// Iterate through EVM code and call a function on each instruction.
void eachInstruction(bytes const& _mem, std::function<void(Instruction, u256 const&)> const& _onInstruction);
void eachInstruction(bytes const& _mem, langutil::EVMVersion _evmVersion, std::function<void(InternalInstruction, u256 const&)> const& _onInstruction);

/// Convert from EVM code to simple EVM assembly language.
std::string disassemble(bytes const& _mem, std::string const& _delimiter = " ");
std::string disassemble(bytes const& _mem, langutil::EVMVersion _evmVersion, std::string const& _delimiter = " ");

}