HIP-260 - Smart Contract Traceability

[shemnon]

Discuss HIP-206 - Smart Contract Traceability

#260

---

Abstract

Proposes additional data placed in Transaction Records to enable full replay of
transactions by mirror nodes and downstream users of Hedera.

Motivation

Enabling full replay of smart contract service transactions will improve the
usability, auditability, and debuggability of smart contract services
transactions. Current facilities are insufficient to inspect "internal"
transactions executed across contracts, and provide insufficient error messages
when correcting errors during smart contract development.

Rationale

Creating a traceable output is always the classic balancing act of time vs.
memory. The service node could easily put an entire operation-by-operation trace
in the record stream but the storage and data transmission costs would quickly
become unmanageable.

Instead, we add the minimum needed data that allows a user to fully replay the
transaction and derive the same set of results. We also need to presume that the
user does not have access to the full historical state of the chain.

This will allow for the same level of transaction inspection that popular
ethereum chains have on their block explorer sites without requiring speculative
calculation of traces from transient execution data.

User stories

As a user of a mirror node I want to be able to see a list of state changes, a
full representation of the calls between contracts, and a step by step
evaluation of all EVM operations. I want to be able to create these records
disconnected from the Hedera Network with only the record streams of the
transactions.

Specification

Add ContractStorageChanges to ContractFunctionResult

Add and populate a new field stateChanges of type ContractStorageChagnes to
the ContractFunctionResult message. This will be populated whenever a
transaction causes contract storage states to change.

message ContractStateChange {
    ContractID contractID = 1;
    repeated StorageChange storageChanges = 2;
}

message StorageChange {
    bytes slot = 1;
    bytes valueRead = 2;

    oneof writeMode {
        bytes valueWritten = 3;
        bool readOnly = 4;
    }
}

For each slot read or written by the smart contract transaction
a ContactStateChange record should be added to the transaction record. As
there can be multiple contracts called in one transaction the storage is grouped
by contract.

The slot, valueRead, and valueWritten values are byte strings of up to 32
bytes. Semantically they are uint256 stored in big endian notation with leading
zeros stripped. Hence, any value less than 32 bytes is semantically identical to
a value left padded with zeros.

If a value is only read and is not updated the readOnly flag MUST be set to
zero. Otherwise, it is impossible to distinguish writing a zero value from a
read of that value without an update.

Values that are updated without first being read must have their valueRead
field set regardless. This is required as gas calculation is dependent on the
original value fo the field updated. A message without a valueRead field will
be treated as though a value of zero had been read, as that is what will be
encoded.

Canonically the fields are sorted by the contractID then by slot. Canonically
the slot, valueRead, and valueWritten are stripped of preceding zero
bytes. Because of this zero values will not be written by the protobuf encoder,
and a message missing both the valueWritten and readOnly fields will be
treated as an update to all zeros. Hence, a contract that reads and writes zeros
to slot zero will have an empty message body.

The value in valueRead reflects the storage value prior to the execution of
the smart contract. The value in valueWritten, if present, represents the
final updated value of the storage slot after the completion of the smart
contract call. Transient states between the start and finish of the contract are
not stored in the record stream.

Populating contractCallResult field in Precompiled Contract Records

As an update to HIP-206 smart contracts will need to populate
the contractCallResult for all transaction records generated by precompiled
contracts recording key facts about the smart contract call. The semantics of
each field will be as follows:

• contractID will be set to the address of the precompile that generated the
  record.
• gasUsed will be set to the gas that the precompiled contract charged for its
  execution.
• contractCallResult will be populated with the return of the call, if any.
• bloom will be populated if the precompile produces EVM Logs.
• logInfo will be populated if the precompiled contract produces logs.
• stateChanges will be populated if the precompiled contract changes state in
  any contract.
• createdContractIDs will not be populated.

Replaying the Transaction

When replaying the transaction it is presumed the program has access to the
smart contracts deployed on the chain or can quickly address them on demand.
From this byte code along with precompiled system contract results a replay can
be performed.

First, all the read values of the storage changes are loaded into the local
world state. The entire state of the contracts involved are not needed and for
large contracts is not desirable for efficient execution.

Next the EVM operations are executed against this world state. For calls that go
to other contracts the bytecode is provided by the mirror node. For calls that
go to precompiled system contracts the results are mocked out by the related
record stream.

Finally, the execution can compare the record stream results with its own
results. This includes gas used and storage value changes.

A tracer can be attached to the EVM to capture intermediate data for
presentation by block explorers. Three kinds of data are most relevant. State
changes can be extracted from the record stream directly without replay.
Internal Transactions or "flat traces" can be

Backwards Compatibility

The added fields can be ignored by clients not wishing to trace or replay smart
contract transactions.

Prior transactions lacking these fields cannot be replayed efficiently, and will
require historical state reconstruction. This reflects current practice.

Security Implications

All data stored can be recreated by anyone who already has full access to the
historical records. This EIP makes the data more efficient and economical to
access.

In adversarial conditions it is estimated the record stream can have 30 KB of
data added per million gas spent. This does not make smart contracts the most
efficient attack surface for record stream bloating.

Because we are not matching external gas pricing for our precompiled system
contracts we will be able to adjust the gas price of precompiled contracts if
they become an economical way to bloat the record stream size.

How to Teach This

For creating a transaction replay clients should be pointed to the reference
implementation.

For end users looking for the benefits of state, flat, and operation traces user
help documentation and possibly a technical blog post should be sufficient.

Reference Implementation

None yet. Expected no sooner than 0.22.0

Rejected Ideas

Trace Cross-contract calls in the record stream

The initial proposal included adding the full "flat trace" as a part of the
record. Early development showed that in adversarial interactions the record
stream would produce about one byte of data per unit of gas spent on the
transaction. When targeting millions of gas per second this would result in
megabytes of record data generated per second. This load in the record stream is
not sustainable for further scaling efforts.