Skip to content

Merge upstream for Pectra#3

Merged
blockchaindevsh merged 367 commits intoop-esfrom
op-merge
Feb 17, 2025
Merged

Merge upstream for Pectra#3
blockchaindevsh merged 367 commits intoop-esfrom
op-merge

Conversation

@blockchaindevsh
Copy link
Copy Markdown
Collaborator

@blockchaindevsh blockchaindevsh commented Feb 17, 2025

This PR merges latest op-es(7bb32fe) with latest upstream release, which is also the version referenced by latest upstream monorepo .

Conflicts:

core/state_transition.go
core/types/transaction_signing.go

holiman and others added 30 commits November 19, 2024 14:50
This change makes the trie commit operation concurrent, if the number of changes exceed 100. 

Co-authored-by: stevemilk <wangpeculiar@gmail.com>
Co-authored-by: Gary Rong <garyrong0905@gmail.com>
Changelog: https://golangci-lint.run/product/changelog/#1610 

Removes `exportloopref` (no longer needed), replaces it with
`copyloopvar` which is basically the opposite.

Also adds: 
- `durationcheck`
- `gocheckcompilerdirectives`
- `reassign`
- `mirror`
- `tenv`

---------

Co-authored-by: Marius van der Wijden <m.vanderwijden@live.de>
This change brings geth into compliance with the current engine API
specification for the Prague fork. I have moved the assignment of
ExecutionPayloadEnvelope.Requests into BlockToExecutableData to ensure
there is a single place where the type is removed.

While doing so, I noticed that handling of requests in the miner was not
quite correct for the empty payload. It would return `nil` requests for
the empty payload even for blocks after the Prague fork. To fix this, I
have added the emptyRequests field in miner.Payload.
calculating a reasonable tx blob fee cap (`max_blob_fee_per_gas *
total_blob_gas`) only depends on the excess blob gas of the parent
header. The parent header is assumed to be correct, so the method should
not be able to fail and return an error.
Use `github.com/decred/dcrd/dcrec/secp256k1/v4` directly rather than
`github.com/btcsuite/btcd/btcec/v2` which is just a wrapper around the
underlying decred library. Inspired by
cosmos/cosmos-sdk#15018

`github.com/btcsuite/btcd/btcec/v2` has a very annoying breaking change
when upgrading from `v2.3.3` to `v2.3.4`. The easiest way to workaround
this is to just remove the wrapper.

Would be very nice if you could backport this to the release branches.

References:
- btcsuite/btcd#2221
- cometbft/cometbft#4294
- cometbft/cometbft#3728
- zeta-chain/node#2934
## Description

Omit null `witness` field from payload envelope.

## Motivation

Currently, JSON encoded payload types always include `"witness": null`,
which, I believe, is not intentional.
~~Opening this as a draft to have a discussion.~~ Pressed the wrong
button
I had [a previous PR
](ethereum/go-ethereum#24616 long time ago
which reduced the peak memory used during reorgs by not accumulating all
transactions and logs.
This PR reduces the peak memory further by not storing the blocks in
memory.
However this means we need to pull the blocks back up from storage
multiple times during the reorg.
I collected the following numbers on peak memory usage: 

// Master: BenchmarkReorg-8 10000 899591 ns/op 820154 B/op 1440
allocs/op 1549443072 bytes of heap used
// WithoutOldChain: BenchmarkReorg-8 10000 1147281 ns/op 943163 B/op
1564 allocs/op 1163870208 bytes of heap used
// WithoutNewChain: BenchmarkReorg-8 10000 1018922 ns/op 943580 B/op
1564 allocs/op 1171890176 bytes of heap used

Each block contains a transaction with ~50k bytes and we're doing a 10k
block reorg, so the chain should be ~500MB in size

---------

Co-authored-by: Péter Szilágyi <peterke@gmail.com>
Breaking changes:

- The ChainConfig was exposed to tracers via VMContext passed in
`OnTxStart`. This is unnecessary specially looking through the lens of
live tracers as chain config remains the same throughout the lifetime of
the program. It was there so that native API-invoked tracers could
access it. So instead we moved it to the constructor of API tracers.

Non-breaking:

- Change the default config of the tracers to be `{}` instead of nil.
This way an extra nil check can be avoided.

Refactoring:

- Rename `supply` struct to `supplyTracer`.
- Un-export some hook definitions.
Fixes an issue missed in #30576 where we send empty requests for a full
payload being resolved, causing hash mismatch later on when we get the
payload back via `NewPayload`.
Swarm moved out more than 5 years ago, time to let it go.
Clean out some ancient stuff from git ignore.
This change ports some non-important changes from ethereum/go-ethereum#30159, including interface renaming and some trivial refactorings.
Co-authored-by: lightclient <lightclient@protonmail.com>
…638)

This PR tries to break the ci.go to common dependency by moving the
version number out of params.
While looking at some mem profiles from `evm` runs, I noticed that
`goja` compilation of the bigint library was present. The bigint library
compilation happens in a package `init`, whenever the package
`eth/tracers/js` is loaded. This PR changes it to load lazily when
needed.

It becomes slightly faster with this change, and slightly less alloc:y. 

Non-scientific benchmark with 100 executions: 
```
time for i in {1..100}; do ./evm --code 6040 run; done;
 ```

current `master`:

```
real    0m6.634s
user    0m5.213s
sys     0m2.277s
```
Without compiling bigint
```
real    0m5.802s
user    0m4.191s
sys     0m1.965s
```
Clique currently depends on the `accounts` package. This was a bit of a
big cannon even in the past, just to pass a signer "account" to the
Clique block producer. Either way, nowadays Geth does not support clique
mining any more, so by removing that bit of functionality from our code,
we can also break this dependency.

Clique should ideally be further torn out, but this at least gets us one
step closer to cleanups.
Way back we've added `common.math.BigMin` and `common.math.BigMax`.
These were kind of cute helpers, but unfortunate ones, because package
all over out codebase added dependencies to this package just to avoid
having to write out 3 lines of code.

Because of this, we've also started having package name clashes with the
stdlib `math`, which got solves even more badly by moving some helpers
over ***from*** the stdlib into our custom lib (e.g. MaxUint64). The
latter ones were nuked out in a previous PR and this PR nukes out BigMin
and BigMax, inlining them at all call sites.

As we're transitioning to uint256, if need be, we can add a min and max
to that.
This PR moves the logging/tracing-facilities out of `*state.StateDB`,
in to a wrapping struct which implements `vm.StateDB` instead.

In most places, it is a pretty straight-forward change: 
- First, hoisting the invocations from state objects up to the statedb. 
- Then making the mutation-methods simply return the previous value, so
that the external logging layer could log everything.

Some internal code uses the direct object-accessors to mutate the state,
particularly in testing and in setting up state overrides, which means
that these changes are unobservable for the hooked layer. Thus, configuring
the overrides are not necessarily part of the API we want to publish.

The trickiest part about the layering is that when the selfdestructs are
finally deleted during `Finalise`, there's the possibility that someone
sent some ether to it, which is burnt at that point, and thus needs to
be logged. The hooked layer reaches into the inner layer to figure out
these events.

In package `vm`, the conversion from `state.StateDB + hooks` into a
hooked `vm.StateDB` is performed where needed.

---------

Co-authored-by: Gary Rong <garyrong0905@gmail.com>
… (#30585)

Makes the gnark precompile methods more amenable to fuzzing
rebased ethereum/go-ethereum#29766 . The
downstream branch appears to have been deleted and I don't have perms to
push to that fork.

`TerminalTotalDifficultyPassed` is removed. `TerminalTotalDifficulty`
must now be non-nil, and it is expected that networks are already
merged: we can only import PoW/Clique chains, not produce blocks on
them.

---------

Co-authored-by: stevemilk <wangpeculiar@gmail.com>
sebastianst and others added 14 commits January 31, 2025 11:14
…_review_prerelease

all: Merge upstream pre-v1.15.0, based on ethereum-optimism#474, plus review fixes
We know about upstream releases now, but we aren't actually aiming to always pull them in immediately so reduce the noise.
* feat: ensure requests array empty post-Isthmus

* Add IsIsthmus to block config

* Remove unnecessary isthmus check

* Reword fix

---------

Co-authored-by: refcell <abigger87@gmail.com>
* feat: add Isthmus precompiles

Make sure g1 add implemented

* feat: add BLS precompile size limits for Isthmus

* Fix

* Fix tests

* Update definition order
…sm#491)

* Enable miner rpc in authrpc if enabled in http

* Use slices.Contains fn
* pathdb: limit diff layers in write-buffer

* fork.yaml: document pathdb fix
Tag created with op-workbench.
@blockchaindevsh blockchaindevsh changed the base branch from optimism to op-es February 17, 2025 05:35
@syntrust
Copy link
Copy Markdown
Collaborator

Does mise run dt still apply?

@blockchaindevsh
Copy link
Copy Markdown
Collaborator Author

Does mise run dt still apply?

There's no such task defined in op-geth.

@blockchaindevsh blockchaindevsh merged commit 97596a4 into op-es Feb 17, 2025
blockchaindevsh pushed a commit that referenced this pull request Jul 7, 2025
This PR is #3 of a 3-part series that implements the new log index
intended to replace core/bloombits.
Based on ethereum/go-ethereum#31079 and
ethereum/go-ethereum#31080
Replaces ethereum/go-ethereum#30370

This part removes the old bloombits package and the chain indexer that
was only used by bloombits. Deletes the old bloombits database.

FilterMaps data structure explanation:
https://gist.github.com/zsfelfoldi/a60795f9da7ae6422f28c7a34e02a07e

Log index generator code overview:
https://gist.github.com/zsfelfoldi/97105dff0b1a4f5ed557924a24b9b9e7

Search pattern matcher code overview:
https://gist.github.com/zsfelfoldi/5981735641c956afb18065e84f8aff34

Note that the possibility of a tree hashing scheme and remote proof
protocol are mentioned in the documents above but they are not exactly
specified yet. These specs are WIP and will be finalized after the local
log indexer/filter code is finalized and merged.

---------

Co-authored-by: Felix Lange <fjl@twurst.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.