Skip to content
This repository has been archived by the owner on Feb 26, 2024. It is now read-only.

perf: add a disk cache for forking requests #1339

Merged
merged 96 commits into from
Nov 11, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
96 commits
Select commit Hold shift + click to select a range
8666e8f
perf: add an disk cache for forking requests
davidmurdoch Sep 25, 2021
866a9dc
some fixes
davidmurdoch Oct 8, 2021
b6dadaa
test fixes
davidmurdoch Oct 9, 2021
7121a6a
test fix
davidmurdoch Oct 9, 2021
8ccb894
remove .only
davidmurdoch Oct 9, 2021
046bd01
fix test
davidmurdoch Oct 9, 2021
6eedfb3
fix: make sure we update the parent after moving descendant
davidmurdoch Oct 11, 2021
06c29fa
don't filter found related nodes
davidmurdoch Oct 12, 2021
3e065f8
remove irrelevant comment
davidmurdoch Oct 12, 2021
41d130b
fix comment typo
davidmurdoch Oct 12, 2021
918add7
clean up
davidmurdoch Oct 12, 2021
d99647d
more cache-test cleanup
davidmurdoch Oct 12, 2021
dc602c3
fix typeo
davidmurdoch Oct 12, 2021
482eea3
remove unused and fix typo
davidmurdoch Oct 12, 2021
f30e83d
remove old comment
davidmurdoch Oct 12, 2021
33117eb
remove timeout limit in ethereum tests
davidmurdoch Oct 12, 2021
ea06d78
fix cache issue and increase test timeouts
davidmurdoch Oct 12, 2021
5839621
fix tests
davidmurdoch Oct 12, 2021
968d648
extend timeout
davidmurdoch Oct 12, 2021
7025051
don't return the wrong data
davidmurdoch Oct 13, 2021
26b42f9
extend timeout even more
davidmurdoch Oct 13, 2021
c4e3d8b
extend timeout
davidmurdoch Oct 13, 2021
ccecf6d
extend all tests in forking->transactions
davidmurdoch Oct 13, 2021
5d11997
add more info to a failling test
davidmurdoch Oct 13, 2021
9e17e24
fixes
davidmurdoch Oct 26, 2021
8c47853
fix fork deletions
davidmurdoch Oct 26, 2021
8cdeeaa
refactor bufferify and add tests
davidmurdoch Oct 27, 2021
dbbc500
fix checkpoint issue
davidmurdoch Oct 28, 2021
358ee62
stream results from bufferify as they are parsed
davidmurdoch Oct 28, 2021
66a885f
fix batch
davidmurdoch Oct 29, 2021
53b011f
remove unused variable
davidmurdoch Oct 29, 2021
46b2139
delete test.js
davidmurdoch Oct 29, 2021
87f5d42
remove chunk streaming from ws-server
davidmurdoch Oct 30, 2021
7993cde
Merge branch 'perf/persistent-fork-cache' of github.com:trufflesuite/…
davidmurdoch Oct 30, 2021
d300bbc
track metadata dbs, not checkpoints!
davidmurdoch Nov 1, 2021
bcb3e29
remove this test as the transaction traced for `debug_storageRangeAt`…
davidmurdoch Nov 1, 2021
764682d
fix traceTransaction and debug_traceTransaction events
davidmurdoch Nov 1, 2021
52973dc
delete old file import
davidmurdoch Nov 1, 2021
9da11d3
fix typo and omission
davidmurdoch Nov 1, 2021
3d1caca
.
davidmurdoch Nov 1, 2021
bbc0ec3
add info to flaky test
davidmurdoch Nov 1, 2021
3c9e0cf
add streaming support to websockets
davidmurdoch Nov 2, 2021
55e3ad3
update uws
davidmurdoch Nov 3, 2021
27332ab
remove .only
davidmurdoch Nov 3, 2021
77b104d
disable a test
davidmurdoch Nov 3, 2021
b43df57
oops
davidmurdoch Nov 3, 2021
458f1c5
fix
davidmurdoch Nov 4, 2021
18afb3e
add clarifying comment
davidmurdoch Nov 4, 2021
b984af7
fix options
davidmurdoch Nov 4, 2021
7099e9c
clarifying comment
davidmurdoch Nov 4, 2021
f9ad181
make sure bufferify creates valid JSON
davidmurdoch Nov 4, 2021
7834d71
only
davidmurdoch Nov 4, 2021
c34672d
make build work
davidmurdoch Nov 4, 2021
348ec57
add pre-fork block trie optimization
davidmurdoch Nov 4, 2021
ffb1696
revert a change
davidmurdoch Nov 5, 2021
2b4041d
update comment
davidmurdoch Nov 5, 2021
560abc4
clarify comment
davidmurdoch Nov 5, 2021
acd10cd
remove unncessary check
davidmurdoch Nov 5, 2021
a6fcf37
simplify
davidmurdoch Nov 5, 2021
5441044
simplify more
davidmurdoch Nov 5, 2021
9c060ec
refactor bufferify 's objectToBuffer
davidmurdoch Nov 5, 2021
d1cf676
add more bufferification test props
davidmurdoch Nov 5, 2021
cadbd5d
fix comment typo
davidmurdoch Nov 5, 2021
3903441
code review changes
davidmurdoch Nov 5, 2021
423945b
remove Buffer import from tests
davidmurdoch Nov 5, 2021
57e63cb
add comments about ws-server
davidmurdoch Nov 5, 2021
e56e9af
clarify variable name
davidmurdoch Nov 5, 2021
567e24f
move this weird line up
davidmurdoch Nov 5, 2021
1f36cf9
clarifying comments again
davidmurdoch Nov 5, 2021
7afa74f
move it again
davidmurdoch Nov 5, 2021
7ad162d
better with brackets
davidmurdoch Nov 5, 2021
30747b2
_
davidmurdoch Nov 5, 2021
a3d8be4
add clarifying comment
davidmurdoch Nov 5, 2021
bb6c7eb
fix typo
davidmurdoch Nov 6, 2021
2c06a70
partially switch to block confirmations for "latest"
davidmurdoch Nov 8, 2021
c202b2e
Update src/chains/ethereum/ethereum/src/forking/trie.ts
davidmurdoch Nov 9, 2021
1566e28
Update src/chains/ethereum/ethereum/src/forking/trie.ts
davidmurdoch Nov 9, 2021
381ae91
rename confusing variable
davidmurdoch Nov 9, 2021
5ba148f
Update src/chains/ethereum/ethereum/src/forking/trie.ts
davidmurdoch Nov 9, 2021
d7d65ce
Rename `blockAge` to `preLatestConfirmations`
davidmurdoch Nov 9, 2021
74eb2ed
Update src/chains/ethereum/options/src/fork-options.ts
davidmurdoch Nov 9, 2021
67ab31d
Merge branch 'perf/persistent-fork-cache' of github.com:trufflesuite/…
davidmurdoch Nov 9, 2021
6302d16
fix setDbVersion function
davidmurdoch Nov 9, 2021
cf0947d
Update src/chains/ethereum/ethereum/src/forking/trie.ts
davidmurdoch Nov 9, 2021
95b779e
Update src/chains/ethereum/options/src/fork-options.ts
davidmurdoch Nov 9, 2021
fdb7c77
rename noCache to disableCache
davidmurdoch Nov 9, 2021
c4b36de
Update src/chains/ethereum/options/src/fork-options.ts
davidmurdoch Nov 9, 2021
a2ac1e9
Update src/chains/ethereum/options/src/fork-options.ts
davidmurdoch Nov 9, 2021
6faae0e
fix commit suggestion typo
davidmurdoch Nov 9, 2021
25c38b5
fix error in slowCache
davidmurdoch Nov 9, 2021
7cb9cf6
remove mock-provider.ts
MicaiahReid Nov 10, 2021
ed038fe
code review changes
davidmurdoch Nov 10, 2021
259b3a2
fix up
davidmurdoch Nov 10, 2021
5d60810
better var name
davidmurdoch Nov 10, 2021
09dd1f6
nice cache!
davidmurdoch Nov 10, 2021
a6aa654
Merge branch 'perf/persistent-fork-cache' of github.com:trufflesuite/…
davidmurdoch Nov 10, 2021
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 3 additions & 2 deletions src/chains/ethereum/block/src/runtime-block.ts
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,8 @@ import {
Quantity,
BUFFER_EMPTY,
BUFFER_32_ZERO,
BUFFER_8_ZERO
BUFFER_8_ZERO,
BUFFER_ZERO
} from "@ganache/utils";
import { BN, KECCAK256_RLP_ARRAY } from "ethereumjs-util";
import { EthereumRawBlockHeader, serialize } from "./serialize";
Expand Down Expand Up @@ -128,7 +129,7 @@ export class RuntimeBlock {
timestamp: new BnExtra(ts),
baseFeePerGas:
baseFeePerGas === undefined
? undefined
? new BnExtra(BUFFER_ZERO)
MicaiahReid marked this conversation as resolved.
Show resolved Hide resolved
: new BnExtra(Quantity.from(baseFeePerGas).toBuffer())
};
}
Expand Down
69 changes: 50 additions & 19 deletions src/chains/ethereum/ethereum/package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

8 changes: 5 additions & 3 deletions src/chains/ethereum/ethereum/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@
"docs.preview": "ws --open --port 3010 --directory ../../../../docs",
"tsc": "ttsc --build",
"test": "nyc --reporter lcov npm run mocha",
"mocha": "cross-env TS_NODE_COMPILER=ttypescript TS_NODE_FILES=true mocha -s 0 -t 0 --exit --check-leaks --throw-deprecation --trace-warnings --require ts-node/register 'tests/**/*.test.ts'"
"mocha": "cross-env TS_NODE_COMPILER=ttypescript TS_NODE_FILES=true mocha -s 0 --exit --check-leaks --throw-deprecation --trace-warnings --require ts-node/register 'tests/**/*.test.ts'"
MicaiahReid marked this conversation as resolved.
Show resolved Hide resolved
},
"bugs": {
"url": "https://github.com/trufflesuite/ganache/issues"
Expand Down Expand Up @@ -67,6 +67,7 @@
"abort-controller": "3.0.0",
"bip39": "3.0.4",
"emittery": "0.7.2",
"env-paths": "2.2.1",
"eth-sig-util": "2.5.3",
"ethereumjs-abi": "0.6.8",
"ethereumjs-util": "7.1.0",
Expand All @@ -83,11 +84,11 @@
"semaphore": "1.1.0",
"subleveldown": "5.0.1",
"tmp-promise": "3.0.2",
"ws": "7.5.3"
"ws": "8.2.3"
},
"devDependencies": {
"@trufflesuite/typedoc-default-themes": "0.6.1",
"@trufflesuite/uws-js-unofficial": "18.14.0-unofficial.12",
"@trufflesuite/uws-js-unofficial": "20.4.0-unofficial.2",
"@types/encoding-down": "5.0.0",
"@types/fs-extra": "9.0.2",
"@types/keccak": "3.0.1",
Expand All @@ -102,6 +103,7 @@
"abstract-leveldown": "6.3.0",
"cheerio": "1.0.0-rc.3",
"cross-env": "7.0.3",
"fast-check": "2.17.0",
"fs-extra": "9.0.1",
"local-web-server": "4.2.1",
"mocha": "8.4.0",
Expand Down
Loading