Move more code into state-account #10840

dvdplm · 2019-07-04T14:05:05Z

Moves ethcore/state into a new account-state crate.

State management has a piece that is tightly coupled to other ethcore modules, mainly Machine and Excutive (and related types and support code).
In order to extract the state management code from ethcore I have left one module – executive_state – in ethcore that isolates the code where the state code is too entangled with the rest of ethcore to be tackled at this time. The tests for state management are also still in ethcore because of the tight coupling with other ethcore modules. While this is rather ungainly and unsatisfactory I think it's an improvement to have this code in an isolated unit that makes it obvious where the coupling takes place.

The end result is that we now have 6k loc less in ethcore and the following new crates:

trace
factories
account-state
pod
account-db from Extract AccountDB to account-db #10839

We also have a somewhat smaller API surface on ethcore.

The diff is very noisy and for that I apologise. The only really new code is in executive_state.rs and the rest is mostly cleanup and munging. None of the TODOs that show up as "green" are mine (but still worthy of consideration ofc). I have used this branch to sync Goerli.

Builds on #10839

Remove botched ethcore-error crate

* master: Extricate PodAccount and state Account to own crates (#10838) logs (#10817) refactor: whisper: Add type aliases and update rustdocs in message.rs (#10812) Break circular dependency between Client and Engine (part 1) (#10833) tests: Relates to #10655: Test instructions for Readme (#10835) refactor: Related #9459 - evmbin: replace untyped json! macro with fully typed serde serialization using Rust structs (#10657)

…hore/extricate-state-backend * dp/chore/extricate-account-db-into-own-crate: third time's the charm test failure 2 test failure Extricate PodAccount and state Account to own crates (#10838) logs (#10817) refactor: whisper: Add type aliases and update rustdocs in message.rs (#10812) Break circular dependency between Client and Engine (part 1) (#10833) tests: Relates to #10655: Test instructions for Readme (#10835) refactor: Related #9459 - evmbin: replace untyped json! macro with fully typed serde serialization using Rust structs (#10657)

* master: Extract AccountDB to account-db (#10839) test: Update Whisper test for invalid pool size (#10811)

* master: Improve logging and cleanup in miner around block sealing (#10745)

debris

Great PR! I have only a few questions about the code. I believe that those changes are very needed and are a huge step towards getting ethcore more modular

ethcore/factories/src/lib.rs

ethcore/src/client/evm_test_client.rs

ethcore/src/error.rs

ethcore/trace/Cargo.toml

ethcore/trace/src/db.rs

ethcore/trace/src/types/flat.rs

rpc/src/lib.rs

This reverts commit 2f839f8.

* master: ethcore-builtin (#10850)

* master: EIP-1702: Generalized Account Versioning Scheme (#10771)

tomusdrw

Looks good, definitely a good step forward.

ethcore/account-state/Cargo.toml

ethcore/account-state/src/substate.rs

ethcore/blockchain/src/database_extras.rs

ethcore/factories/Cargo.toml

ethcore/trace/src/db.rs

ethcore/trace/Cargo.toml

tomusdrw · 2019-07-08T13:19:13Z

ethcore/trace/Cargo.toml

+ethcore-blockchain = { path = "../blockchain" }
+ethcore-db = { path = "../db" }
+ethereum-types = "0.6"
+evm = { path = "../evm" }


Again requiring entire EVM seems odd, would be worth to remove it in the future.

We use CallType from the evm, that's all. Seems like a good opportunity for a refactor indeed.

tomusdrw · 2019-07-08T13:19:25Z

ethcore/trace/Cargo.toml

+
+[dependencies]
+common-types = { path = "../types"}
+ethcore-blockchain = { path = "../blockchain" }


Do we really need the entire blockchain or just some trait definitions?

Currently we use BlockChainDB and the contentious DatabaseExtras, e.g.:

pub struct TraceDB<T> where T: DatabaseExtras { /// cache traces: RwLock<HashMap<H256, FlatBlockTraces>>, /// hashes of cached traces cache_manager: RwLock<CacheManager<H256>>, /// db db: Arc<dyn BlockChainDB>, /// tracing enabled enabled: bool, /// extras extras: Arc<T>, }

What is your thinking here? Are you thinking we should move traits to a common-traits crate?

ethcore/trace/src/types/flat.rs

debris

looks good. I think we should get it in asap and we can fix trace <-> blockchain relation later on

debris · 2019-07-08T15:42:32Z

ethcore/src/client/evm_test_client.rs

@@ -100,7 +108,7 @@ impl<'a> EvmTestClient<'a> {
 	}

 	/// Change default function for dump state (default does not dump)
-	pub fn set_dump_state_fn(&mut self, dump_state: fn(&state::State<state_db::StateDB>) -> Option<pod_state::PodState>) {
+	pub fn set_dump_state(&mut self) {
 		self.dump_state = dump_state;


this function looks a bit weird

dvdplm · 2019-07-08T16:17:42Z

@tomusdrw @debris Merging this but I'll make sure the following gets addressed in follow up PRs:

sort out the trace <–> blockchain interdependencies
cf. Move more code into state-account #10840 (comment)
attempt to avoid pulling in EVM in trace and account-state
make the conditional "dump state" suck a bit less

* master: Move more code into state-account (#10840) Remove compiler warning (#10865) [ethash] use static_assertions crate (#10860)

* master: Run cargo fix on a few of the worst offenders (#10854) removed redundant fork choice abstraction (#10849) Extract state-db from ethcore (#10858) Fix fork choice (#10837) Move more code into state-account (#10840) Remove compiler warning (#10865) [ethash] use static_assertions crate (#10860)

dvdplm added 9 commits July 3, 2019 15:57

WIP move errors, pod_account and state account to own crates

f144f88

Sort out dependencies, fix broken code and tests

c8bf87f

Remove botched ethcore-error crate

remove template line

47bc6fb

fix review feedback

f8756bb

Remove test-only AccountDBMut::new

d54ebab

Extract AccountDB to account-db

7792810

Move Substate to state-account – wip

a55ca1e

Add lib.rs

201dac7

cleanup

e23acad

dvdplm self-assigned this Jul 4, 2019

dvdplm added 20 commits July 4, 2019 16:09

test failure

e0701e4

test failure 2

b42d16a

third time's the charm

e45d13f

Merge branch 'master' into dp/chore/extricate-state-backend

9b60a57

* master: Extract AccountDB to account-db (#10839) test: Update Whisper test for invalid pool size (#10811)

Merge branch 'master' into dp/chore/extricate-state-backend

c55e6b1

* master: Improve logging and cleanup in miner around block sealing (#10745)

Add factories crate

1c0722b

Use new factories crate

2114c12

Use factories crate

80ededa

Extract trace

d9ceaa6

Fix tests

9f3aa73

Sort out parity-util-mem and parking_lot

a6a25ba

cleanup

44a0904

WIP port over the rest of state from ethcore

ce3a211

Collect all impls for Machine

9ed1d47

some notes

67332d7

Rename pod-account to pod

0a42ea8

Move PodState to pod crate

d9ce43d

Use PodState from pod crate

aebee1d

dvdplm added 2 commits July 6, 2019 23:18

Let private-tx use StateDB. :(

081e678

Remove ethcore/src/pod_state.rs

659440a

debris reviewed Jul 7, 2019

View reviewed changes

dvdplm added 7 commits July 7, 2019 16:18

Inner type does not need to be pub/pub(crate)

2f839f8

optimise imports

df23dcc

Revert "Inner type does not need to be pub/pub(crate)"

e23cb6a

This reverts commit 2f839f8.

Move DatabaseExtras to ethcore-blockchain

de2aa80

Add database_extra module to ethcore-blockchain

b4ea86a

Merge branch 'master' into dp/chore/extricate-state-backend

1cf8254

* master: ethcore-builtin (#10850)

Remove to-pod-full feature

a57517c

dvdplm mentioned this pull request Jul 8, 2019

Extract state-db from ethcore #10858

Merged

dvdplm requested a review from debris July 8, 2019 10:00

dvdplm added 2 commits July 8, 2019 13:35

Merge branch 'master' into dp/chore/extricate-state-backend

75efc47

* master: EIP-1702: Generalized Account Versioning Scheme (#10771)

Sort out the merge

994424e

tomusdrw approved these changes Jul 8, 2019

View reviewed changes

dvdplm added 4 commits July 8, 2019 16:38

sort imports

dda55bd

address grumbles

268940b

rename crate

20cb69c

address more grumbles

445b0a1

debris approved these changes Jul 8, 2019

View reviewed changes

dvdplm merged commit 44cc442 into master Jul 8, 2019

dvdplm deleted the dp/chore/extricate-state-backend branch July 8, 2019 16:18

dvdplm added a commit that referenced this pull request Jul 8, 2019

Merge branch 'master' into dp/chore/extract-state-db

d51cede

* master: Move more code into state-account (#10840) Remove compiler warning (#10865) [ethash] use static_assertions crate (#10860)

dvdplm mentioned this pull request Jul 8, 2019

Move DatabaseExtras back to trace #10868

Merged

This was referenced Nov 5, 2019

Backport code structure changes #11241

Closed

Backport code structure changes #11242

Closed

Backport code structure changes #11243

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Move more code into state-account #10840

Move more code into state-account #10840

dvdplm commented Jul 4, 2019 •

edited

debris left a comment

tomusdrw left a comment

tomusdrw Jul 8, 2019

dvdplm Jul 8, 2019

tomusdrw Jul 8, 2019

dvdplm Jul 8, 2019

dvdplm Jul 8, 2019 •

edited

debris left a comment

debris Jul 8, 2019

dvdplm commented Jul 8, 2019

Move more code into state-account #10840

Move more code into state-account #10840

Conversation

dvdplm commented Jul 4, 2019 • edited

debris left a comment

Choose a reason for hiding this comment

tomusdrw left a comment

Choose a reason for hiding this comment

tomusdrw Jul 8, 2019

Choose a reason for hiding this comment

dvdplm Jul 8, 2019

Choose a reason for hiding this comment

tomusdrw Jul 8, 2019

Choose a reason for hiding this comment

dvdplm Jul 8, 2019

Choose a reason for hiding this comment

dvdplm Jul 8, 2019 • edited

Choose a reason for hiding this comment

debris left a comment

Choose a reason for hiding this comment

debris Jul 8, 2019

Choose a reason for hiding this comment

dvdplm commented Jul 8, 2019

dvdplm commented Jul 4, 2019 •

edited

dvdplm Jul 8, 2019 •

edited