Merge/foundation master 20201111 #230
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
* core/vm: marshall returnData as hexstring in trace logs * core/vm: marshall returnData as hexstring in trace logs
* add interrupt counter * remove interrupt counter, allow ctrl-C to clear ONLY, ctrl-D will terminate console, stop node * format * add instructions to exit * fix tests
* core: add background indexer to waitgroup * core: make indexer stopable * core/rawdb: add unit tests * core/rawdb: fix lint * core/rawdb: fix tests * core/rawdb: fix linter
* Fix potential nil pointer error when neither block number nor hash is specified to accountRange * Update error description
* les: move NodeStateMachine from clientPool to LesServer * les: new header broadcaster * les: peerCommons.headInfo always contains last announced head * les: remove clientPeerSet and serverSet * les: fixed panic * les: fixed --nodiscover option * les: disconnect all peers at ns.Stop() * les: added comments and fixed signed broadcasts * les: removed unused parameter, fixed tests
The version check logic did not take into account the second digit (i.e. the '4' in v1.4.0) - this one line patch corrects this.
… + yolo-v2 (#21509) * core/vm, core/state: implement EIP-2929 + YOLOv2 * core/state, core/vm: fix some review concerns * core/state, core/vm: address review concerns * core/vm: address review concerns * core/vm: better documentation * core/vm: unify sload cost as fully dynamic * core/vm: fix typo * core/vm/runtime: fix compilation flaw * core/vm/runtime: fix renaming-err leftovers * core/vm: renaming * params/config: use correct yolov2 chainid for config * core, params: use a proper new genesis for yolov2 * core/state/tests: golinter nitpicks
The old one was wrong in two ways: the first block in chain.rlp was the genesis block, and the genesis difficulty was below minimum difficulty. This also contains some other fixes to the test.
utils, params: add yolov2 bootnode
params: update yolov2 bootnode with elastic ip
* accounts/abi/bind: restore error functionality * Update accounts/abi/bind/base.go Co-authored-by: Guillaume Ballet <gballet@gmail.com> Co-authored-by: Guillaume Ballet <gballet@gmail.com>
* core/state: maintain one more diff layer * core/state: address comment
…(#21682) * core/state/snapshot: add diskRoot function * core/state/snapshot: disable iteration if the snapshot is generating * core/state/snapshot: simplify the function * core/state: panic for undefined layer
* core/state/snapshot: introduce snapshot journal version * core: update the disk layer in an atomic way * core: persist the disk layer generator periodically * core/state/snapshot: improve logging * core/state/snapshot: forcibly ensure the legacy snapshot is matched * core/state/snapshot: add debug logs * core, tests: fix tests and special recovery case * core: polish * core: add more blockchain tests for snapshot recovery * core/state: fix comment * core: add recovery flag for snapshot * core: add restart after start-after-crash tests * core/rawdb: fix imports * core: fix tests * core: remove log * core/state/snapshot: fix snapshot * core: avoid callbacks in SetHead * core: fix setHead cornercase where the threshold root has state * core: small docs for the test cases Co-authored-by: Péter Szilágyi <peterke@gmail.com>
…(#21761) This adds a few tiny fixes for les and the p2p simulation framework: LES Parts - Keep the LES-SERVER connection even it's non-synced We had this idea to reject the connections in LES protocol if the les-server itself is not synced. However, in LES protocol we will also receive the connection from another les-server. In this case even the local node is not synced yet, we should keep the tcp connection for other protocols(e.g. eth protocol). - Don't count "invalid message" for non-existing GetBlockHeadersMsg request In the eth syncing mechanism (full sync, fast sync, light sync), it will try to fetch some non-existent blocks or headers(to ensure we indeed download all the missing chain). In this case, it's possible that the les-server will receive the request for non-existent headers. So don't count it as the "invalid message" for scheduling dropping. - Copy the announce object in the closure Before the les-server pushes the latest headers to all connected clients, it will create a closure and queue it in the underlying request scheduler. In some scenarios it's problematic. E.g, in private networks, the block can be mined very fast. So before the first closure is executed, we may already update the latest_announce object. So actually the "announce" object we want to send is replaced. The downsize is the client will receive two announces with the same td and then drop the server. P2P Simulation Framework - Don't double register the protocol services in p2p-simulation "Start". The protocols upon the devp2p are registered in the "New node stage". So don't reigster them again when starting a node in the p2p simulation framework - Add one more new config field "ExternalSigner", in order to use clef service in the framework.
ToHex was deprecated a couple years ago. The last remaining use was in ToHexArray, which itself only had a single call site. This just moves ToHexArray near its only remaining call site and implements it using hexutil.Encode. This changes the default behaviour of ToHexArray and with it the behaviour of eth_getProof. Previously we encoded an empty slice as 0, now the empty slice is encoded as 0x.
… (#21775) * core/state/snapshot: print warning if failed to resolve journal * core/state/snapshot: fix snapshot recovery When we meet the snapshot journal consisted with: - disk layer generator with new-format - diff layer journal with old-format The base layer should be returned without error. The broken diff layer can be reconstructed later but we definitely don't want to reconstruct the huge diff layer. * core: add tests
TAP is a text format for test results. Parsers for it are available in many languages, making it easy to consume. I want TAP output from our protocol tests because the Hive wrapper around them needs to know about the test names and their individual results and logs. It would also be possible to just write this info as JSON, but I don't want to invent a new format. This also improves the normal console output for tests (when running without --tap). It now prints -- RUN lines before any output from the test, and indents the log output by one space.
… output (#21782) This PR replaces the old test genesis.json and chain.rlp files in the testdata directory for the eth protocol test suite, and also adds documentation for running the eth test suite locally. It also improves the test output text and adds more timeouts. Co-authored-by: Felix Lange <fjl@twurst.com>
This PR contains a minor optimization in derivesha, by exposing the RLP int-encoding and making use of it to write integers directly to a buffer (an RLP integer is known to never require more than 9 bytes total). rlp.AppendUint64 might be useful in other places too. The code assumes, just as before, that the hasher (a trie) will copy the key internally, which it does when doing keybytesToHex(key). Co-authored-by: Felix Lange <fjl@twurst.com>
build: stop verbose output to keep travis from overflowing
…dation-master-20201111
Conflicts:
.travis.yml
appveyor.yml
params/version.go
…re-geth into merge/foundation-master-20201111
|
Rel #230 (comment), I just merged |
|
Double check:
|
meowsbits
commented
Dec 10, 2020
| devp2p rlpx eth-test <enode ID> cmd/devp2p/internal/ethtest/testdata/fullchain.rlp cmd/devp2p/internal/ethtest/testdata/genesis.json | ||
| ``` | ||
|
|
||
| [eth]: https://github.com/ethereum/devp2p/blob/master/caps/eth.md |
There was a problem hiding this comment.
Are the tests described here run programmatically anywhere? (Will they be run in the CI?)
meowsbits
commented
Dec 10, 2020
| @@ -287,9 +288,29 @@ func NewBlockChain(db ethdb.Database, cacheConfig *CacheConfig, chainConfig ctyp | |||
| // Make sure the state associated with the block is available | |||
| head := bc.CurrentBlock() | |||
| if _, err := state.New(head.Root(), bc.stateCache, bc.snaps); err != nil { | |||
There was a problem hiding this comment.
bc.snaps:
snaps *snapshot.Tree // Snapshot tree for fast trie leaf access// SnapshotTree is an Ethereum state snapshot tree. It consists of one persistent
// base layer backed by a key-value store, on top of which arbitrarily many in-
// memory diff layers are topped. The memory diffs can form a tree with branching,
// but the disk layer is singleton and common to all. If a reorg goes deeper than
// the disk layer, everything needs to be deleted.
//
// The goal of a state snapshot is twofold: to allow direct access to account and
// storage data to avoid expensive multi-level trie lookups; and to allow sorted,
// cheap iteration of the account/storage tries for sync aid.
type Tree struct {
diskdb ethdb.KeyValueStore // Persistent database to store the snapshot
triedb *trie.Database // In-memory cache to access the trie through
cache int // Megabytes permitted to use for read caches
layers map[common.Hash]snapshot // Collection of all known layers
lock sync.RWMutex
}
meowsbits
commented
Dec 10, 2020
| // disk layer point of snapshot(if it's enabled). Make sure the | ||
| // rewound point is lower than disk layer. | ||
| var diskRoot common.Hash | ||
| if bc.cacheConfig.SnapshotLimit > 0 { |
There was a problem hiding this comment.
SnapshotLimit int // Memory allowance (MB) to use for caching snapshot entries in memory
meowsbits
commented
Dec 10, 2020
| // retaining chain consistency. | ||
| // | ||
| // The method returns the block number where the requested root cap was found. | ||
| func (bc *BlockChain) SetHeadBeyondRoot(head uint64, root common.Hash) (uint64, error) { |
There was a problem hiding this comment.
It seems an extra condition is added to the original SetHead method, but the original method's logic is not preserved (since SetHead jus wraps SetHeadBeyondRoot).
meowsbits
commented
Dec 10, 2020
| // - head layer is paired with HEAD state | ||
| // - head-1 layer is paired with HEAD-1 state | ||
| // - head-127 layer(bottom-most diff layer) is paired with HEAD-127 state | ||
| if err := s.snaps.Cap(root, 128); err != nil { |
There was a problem hiding this comment.
Note 128 is hardcoded here, and got bumped from 127.
meowsbits
commented
Dec 10, 2020
This fixes value-copying logic for the chain config interface value in the tracer in case of tracing-specific overrides. Tests, though embarassingly amateurish, shamelessly show the logic and legitimacy of the change. Date: 2020-12-10 15:23:29-06:00 Signed-off-by: meows <b5c6@protonmail.com>
Date: 2020-12-10 15:25:47-06:00 Signed-off-by: meows <b5c6@protonmail.com>
meowsbits
commented
Dec 10, 2020
|
@meowsbits the |
Conflicts: cmd/puppeth/wizard_genesis.go cmd/utils/flags.go consensus/ethash/ethash.go
|
@meowsbits upstream master bring more changes with regards snapshot. I am not saying to merge them in. Just that we might want to have an overview of what it brings in, in order to better evaluate the snapshot feature and what we merge in/how we resolve conflicts. |
|
Sorry, I think there is a naming confusion here from "master-upstream". The This latest commit (ce45de9) only did the following... # Checkout the master-upstream branch which is a hybrid between
# the current master and current pending upstream changes (ie this PR).
git checkout master-upstream
# Resolve conflicts via merging CURRENT master.
git merge master
git push etclabscore master-upstream
# Checkout this branch.
git checkout merge/foundation-master-20201111
# Update to master-upstream and resolve conflicts.
git merge master-upstream |
|
The intention of the poorly-named This allows us to handle our own features and bugs relative to |
|
My bad @meowsbits as I was meaning foundation master (ethereum/go-ethereum#master). Sorry for the naming confusion I brought in. |
|
iquidus
approved these changes
Dec 23, 2020
ziogaschr
approved these changes
Dec 23, 2020
|
This was merged to |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There are a lot of changes related to YoloV2 including some additional EIPs, see