Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(lib/trie): record deleted Merkle values fixed #2873

Merged
merged 14 commits into from
Jan 10, 2023

Conversation

qdm12
Copy link
Contributor

@qdm12 qdm12 commented Oct 5, 2022

Changes

  • Record Merkle values deleted only if they are NOT 'dirty' (the same as in the last trie snapshot)
  • Record Merkle values deleted when one or more nodes are deleted from the trie
  • Do NOT record Merkle values of inlined nodes in the deleted set of Merkle values
  • Lazy calculate Merkle values if they are missing (especially for delete operations)
  • Update tests to have full trie.go coverage again (except hashing error due to the buffer sync pools at global scope)
  • Fix preparing branch for mutation only once before for loop (instead of N times)
  • Check errors from merkle value computation (useful for lazy loading to handle database errors)

Eventually add more test cases to test for correct registration of deleted Merkle values, but meh might not be that useful with incoming lazy loading changes

Tests

go test -tags integration github.com/ChainSafe/gossamer/lib/trie/...

Issues

Blocking #2854 and #2838

Primary Reviewer

@timwu20

@qdm12 qdm12 changed the base branch from development to qdm12/trie/fix-populate-merkle October 6, 2022 06:28
@qdm12 qdm12 force-pushed the qdm12/trie/fix-deleted-keys branch from f848961 to 16a315b Compare October 6, 2022 14:19
@qdm12 qdm12 force-pushed the qdm12/trie/fix-populate-merkle branch from 7aa5ea7 to 68b6b3d Compare October 6, 2022 14:21
@qdm12 qdm12 force-pushed the qdm12/trie/fix-deleted-keys branch from 16a315b to 7eab810 Compare October 6, 2022 14:21
@qdm12 qdm12 marked this pull request as ready for review October 6, 2022 15:31
@qdm12 qdm12 force-pushed the qdm12/trie/fix-populate-merkle branch from 68b6b3d to d503984 Compare October 11, 2022 11:22
@qdm12 qdm12 force-pushed the qdm12/trie/fix-deleted-keys branch from ad9202e to d4808ec Compare October 11, 2022 11:37
@qdm12 qdm12 force-pushed the qdm12/trie/fix-populate-merkle branch from d503984 to 94c3530 Compare October 12, 2022 09:50
@qdm12 qdm12 force-pushed the qdm12/trie/fix-deleted-keys branch from d4808ec to 67676eb Compare October 12, 2022 09:52
@qdm12 qdm12 force-pushed the qdm12/trie/fix-populate-merkle branch from 94c3530 to 9fca62f Compare October 12, 2022 10:26
@qdm12 qdm12 force-pushed the qdm12/trie/fix-deleted-keys branch from 67676eb to 146fbbb Compare October 12, 2022 12:05
lib/runtime/storage/trie.go Outdated Show resolved Hide resolved
lib/runtime/wasmer/imports.go Show resolved Hide resolved
lib/trie/trie.go Outdated Show resolved Hide resolved
lib/trie/trie.go Outdated Show resolved Hide resolved
lib/trie/trie.go Show resolved Hide resolved
lib/trie/trie.go Show resolved Hide resolved
@qdm12 qdm12 force-pushed the qdm12/trie/fix-deleted-keys branch 2 times, most recently from 1ee69d6 to e0c6446 Compare October 18, 2022 07:30
@qdm12 qdm12 force-pushed the qdm12/trie/fix-populate-merkle branch from 7ecd513 to dbf36e7 Compare October 31, 2022 13:14
@qdm12 qdm12 force-pushed the qdm12/trie/fix-deleted-keys branch from e0c6446 to b72a7c6 Compare October 31, 2022 13:14
@qdm12 qdm12 force-pushed the qdm12/trie/fix-populate-merkle branch from dbf36e7 to 21032bc Compare November 2, 2022 15:36
@qdm12 qdm12 force-pushed the qdm12/trie/fix-deleted-keys branch from b72a7c6 to 400a69e Compare November 2, 2022 16:05
@qdm12 qdm12 force-pushed the qdm12/trie/fix-populate-merkle branch from c8f6cc4 to fd8b0c2 Compare November 4, 2022 11:47
@qdm12 qdm12 force-pushed the qdm12/trie/fix-deleted-keys branch from 400a69e to 2d2cf27 Compare November 4, 2022 11:47
@qdm12 qdm12 force-pushed the qdm12/trie/fix-deleted-keys branch 2 times, most recently from 9a278fe to 4d8dc3f Compare January 9, 2023 17:09
lib/trie/trie.go Outdated Show resolved Hide resolved
Copy link
Contributor

@jimjbrettj jimjbrettj left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

great work, just a few comments!

@qdm12 qdm12 force-pushed the qdm12/trie/fix-deleted-keys branch from 233b69b to 0d91179 Compare January 10, 2023 14:33
- Record Merkle values deleted only if they were not changed since the last trie snapshot
- Record Merkle values deleted when one or more nodes are deleted from the trie
- Do NOT record Merkle values of inlined nodes in the deleted set of Merkle values (since they are not stored to database)
- Lazy calculate Merkle values if they are missing (especially for delete operations)
- Update tests to have full trie.go coverage again
- Fix preparing branch for mutation only once
- Fix merkle value calculation for future-to-be root node
- Remove `prepLeafForMutation`
- Remove `prepBranchForMutation`
- Remove `updateGeneration`
- Replace above with `prepForMutation`
@qdm12 qdm12 force-pushed the qdm12/trie/fix-deleted-keys branch from 0d91179 to c61ae90 Compare January 10, 2023 14:50
@qdm12 qdm12 merged commit 61f0216 into development Jan 10, 2023
@qdm12 qdm12 deleted the qdm12/trie/fix-deleted-keys branch January 10, 2023 16:04
github-actions bot pushed a commit that referenced this pull request Dec 11, 2023
# [0.8.0](v0.7.0...v0.8.0) (2023-12-11)

### Bug Fixes

* **.github/workflows:** update node.js version ([#3637](#3637)) ([619be1b](619be1b))
* **babe:** Add support for versioned NextConfigData decoding ([#3239](#3239)) ([5ee3a64](5ee3a64))
* **blockstate:** if blocktree fails to search a hash in memory, load it from disk ([#3059](#3059)) ([6442544](6442544))
* cache slot to header data while checking BABE equivocation ([#3364](#3364)) ([dcfa4a4](dcfa4a4))
* **chain:** Fix `chain=westend` option  ([#3123](#3123)) ([64dbba6](64dbba6))
* **ci:** cancel previous workflow runs ([#3140](#3140)) ([a322a19](a322a19))
* **ci:** fix all Deepsource issues ([#3046](#3046)) ([4ea0a70](4ea0a70))
* **ci:** fix broken docker build ([#3231](#3231)) ([f796430](f796430))
* **ci:** Fix staging metrics collection ([#3138](#3138)) ([05a5c4c](05a5c4c))
* **cli:** parse module log-levels ([#3285](#3285)) ([86c7577](86c7577))
* **cmd/gossamer:** embed default toml config files ([#3091](#3091)) ([af38364](af38364))
* **cmd/gossamer:** update error message ([#3301](#3301)) ([960a9d4](960a9d4))
* **dot/babe:** use `bs.latestFinalised` instead of using `round/set id` ([#3167](#3167)) ([46c0ef7](46c0ef7))
* **dot/digest:** create `BlockImportHandler` and remove channel ([#3312](#3312)) ([a179855](a179855))
* **dot/network:** `findPeers` returns on timeout if a peer is found ([#3001](#3001)) ([2a05ce7](2a05ce7))
* **dot/network:** remove `maxReads` limitation to read stream ([#3287](#3287)) ([483b23f](483b23f))
* **dot/state:** clean up scheduled changes once a forced change is applied ([#3219](#3219)) ([5ebec46](5ebec46))
* **dot/state:** fix a bug in IsDescendantOf ([#3125](#3125)) ([4fd4a89](4fd4a89))
* **dot/state:** store raw authority keys and decode when verifying block signature ([#3627](#3627)) ([58f741d](58f741d))
* **dot/sync:** fix `Timestamp slot must match 'CurrentSlot'` while using `westend` spec file ([#3040](#3040)) ([e6da01b](e6da01b))
* **dot/sync:** Revert verify justification before importing blocks ([#3615](#3615)) ([11b96dc](11b96dc))
* **dot/sync:** rework on bootstrap/tip sync ([#3227](#3227)) ([ab6650a](ab6650a))
* **dot/sync:** use `Range` instead of `SubChain` at `handleDescedingRequest` ([#3006](#3006)) ([a83c1a3](a83c1a3))
* **dot/sync:** verify justification before importing blocks ([#3576](#3576)) ([2954fc0](2954fc0))
* **dot/sync:** wrong error message at `checkOrGetDescendantHash` ([#2971](#2971)) ([b1c6bf1](b1c6bf1))
* **dot:** use tempDir in tests as base path to avoid creating `dot/~` ([#3363](#3363)) ([04514d5](04514d5))
* **go.mod:** Replace `centrifuge/go-substrate-rpc-client` dependency to `timwu20/go-substrate-rpc-client` temporarily to fix build ([#3572](#3572)) ([ea49251](ea49251))
* **lib/babe:** Add context and additional assertion to TestBuildBlock_ok ([#3101](#3101)) ([a9a89ed](a9a89ed))
* **lib/babe:** rewrite TestBuildBlock_failing ([#3089](#3089)) ([28a3d0b](28a3d0b))
* **lib/babe:** use current system time to yield a new slot ([#3133](#3133)) ([9cd6f25](9cd6f25))
* **lib/grandpa:**  on verifying block justification, compare given block hash with finalised hash ([#3081](#3081)) ([fc91843](fc91843))
* **lib/grandpa:** ensure `finalisationEngine` exits when stop channel is triggered ([#3141](#3141)) ([d7f7c06](d7f7c06))
* **lib/runtime:** `ext_default_child_storage_next_key_version_1` return `None` correctly ([#3473](#3473)) ([c7d574b](c7d574b))
* **lib/runtime:** Fix `wasm error: out of bounds memory access` at `[#9412261](https://github.com/ChainSafe/gossamer/issues/9412261)` ([#3588](#3588)) ([ecb1ad9](ecb1ad9))
* **lib/runtime:** prevents polkadot zero-address bug using `sr25519_verify` version 1 ([#3494](#3494)) ([8b93d5e](8b93d5e))
* **lib/runtime:** return correct encoded value for `None` ([#3451](#3451)) ([3e11bc2](3e11bc2))
* **lib/runtime:** update `MaxPossibleAllocation` to `2^25` ([#3393](#3393)) ([91eabdc](91eabdc))
* **lib/runtime:** use `westend-dev` spec file in `TestNodeRuntime_ValidateTransaction` ([#3047](#3047)) ([043f5eb](043f5eb))
* **lib/trie:** `ClearFromChild` should update parent trie ([#3482](#3482)) ([70e2d2b](70e2d2b))
* **lib/trie:** create an empty child trie if not found ([#3459](#3459)) ([5d68447](5d68447))
* **lib/trie:** record deleted Merkle values fixed ([#2873](#2873)) ([61f0216](61f0216))
* **peerset:** check for incoming slot error ([#2952](#2952)) ([a1602bc](a1602bc))
* **rpc-tests:** Fix node port to execute tests on macOS ([#3223](#3223)) ([f758575](f758575))
* **rpc/modules:** use `westend-local` in `TestAuthorModule_SubmitExtrinsic_invalid` test ([#3051](#3051)) ([b6429b7](b6429b7))
* **runtime:** initialize TransactionState when creating runtime instance ([#3188](#3188)) ([29fe7a0](29fe7a0))
* **scale:** Use *int for scale index ([#3274](#3274)) ([9b04d30](9b04d30))
* **staging:** fixes the staging deployment issues caused by the new cli ([#3266](#3266)) ([1f4e786](1f4e786))
* **state:** clarify node hashes vs merkle values ([#2915](#2915)) ([e4033e8](e4033e8))
* **test/rpc:** use `westend-local` in `TestStateRPCAPI` ([#3049](#3049)) ([c57ade6](c57ade6))
* **tests/polkadot_js:** Use `westend-local` to run polkadot js test suite ([#3052](#3052)) ([2d5ead1](2d5ead1))
* **tests/rpc:** ensure new blocks are created before assert ([#3042](#3042)) ([a116d58](a116d58))
* **tests/rpc:** flaky test `TestChainSubscriptionRPC/chain_subscribeNewHeads` ([#3092](#3092)) ([5b56238](5b56238))
* **tests/rpc:** place `GreaterOrEqual` arguments in the correct order at `chain_subscribeNewHeads` test ([#3137](#3137)) ([33bdf28](33bdf28))
* **tests:** Export unimplemented runtime fuctions ([#3461](#3461)) ([3e4546c](3e4546c))
* **trie:** do not create buffer for nil child ([#2928](#2928)) ([d70af4f](d70af4f))
* **wasmer:** `ext_storage_exists_version_1` for empty values ([#2973](#2973)) ([059268e](059268e))

### Features

* **chain:** Add Westend network as command line `chain` option ([#3103](#3103)) ([d9cdd45](d9cdd45))
* **chain:** remove unneeded spec files ([#3086](#3086)) ([c76387d](c76387d))
* **cli:** use a single flag for log level in the CLI ([#3303](#3303)) ([caf3ea4](caf3ea4))
* **dot/network:** introduce libp2p resource manager + prometheus metrics ([#3333](#3333)) ([f166746](f166746))
* **dot/rpc:** export block trie state entries for a block hash ([#3607](#3607)) ([43828fe](43828fe))
* **dot/rpc:** implement RPC method state_queryStorageAt ([#3191](#3191)) ([3bbdfe0](3bbdfe0))
* **dot/state:** create `Range` to traverse the blocktree and the blocks in the disk ([#2990](#2990)) ([4442eee](4442eee))
* **dot/state:** keep latest state trie in memory ([#3386](#3386)) ([421d087](421d087))
* **dot/sync:** include block origin and skip extra validation on `initialSync` ([#3392](#3392)) ([8e1650e](8e1650e))
* **dot/sync:** Remove the `EndBlockHash` from `BlockRequestMessage` ([#2977](#2977)) ([b25e0b4](b25e0b4))
* **internal/database:** replace `chaindb/badgerdb` with `pebbledb` ([#3434](#3434)) ([344461d](344461d))
* introduces `SaturatingAdd` and `SaturatingSub` ([#3519](#3519)) ([daa9e25](daa9e25))
* **lib/allocator:** Refactor `FreeingBumpHeapAllocator` ([#3570](#3570)) ([39ca47f](39ca47f))
* **lib/babe:** Submit BABE equivocation report  ([#2947](#2947)) ([55de62e](55de62e)), closes [#2853](#2853)
* **lib/blocktree:** reduce the entries in the runtimes mapping ([#3151](#3151)) ([1a34972](1a34972))
* **lib/grandpa:** include `t.Parallel()` to all `lib/grandpa` tests ([#2840](#2840)) ([5c93488](5c93488))
* **lib/runtime/wasmer:** report grandpa equivocations ([#3007](#3007)) ([e63aeea](e63aeea))
* **lib/runtime:** `wazero` implementation of `runtime.Instance` ([#3279](#3279)) ([115d6f5](115d6f5))
* **lib/runtime:** add extra required runtime imports for parachain validation ([#3254](#3254)) ([dc1a521](dc1a521))
* **lib/runtime:** Update default `runtime.Instance` to `wazero` ([#3352](#3352)) ([308b10a](308b10a))
* **pkg/scale:** `VaryingDataType` String method ([#2970](#2970)) ([841636e](841636e))
* **pkg/scale:** Add `Marshaler` and `Unmarshaler` interfaces and functionality ([#3617](#3617)) ([4888ce4](4888ce4))
* **pkg/scale:** Use `New()` receiver function for construction of custom `VaryingDataType`  ([#3315](#3315)) ([9688f6c](9688f6c))
* **runtime/wasmer:** create wrapper around wasmer.Memory  ([#3160](#3160)) ([fc1055d](fc1055d))
* **runtime/wasmer:** write wasmer using latest wasmer version ([#3168](#3168)) ([32f1aa8](32f1aa8))
* **scale:** add `MustMarshal` function ([#2991](#2991)) ([32a80aa](32a80aa))
* **sync:** Validate bad blocks ([#3220](#3220)) ([0d0354b](0d0354b))
* **telemetry:** Add scheduled and force changes telemetry metrics ([#3226](#3226)) ([c53b1cd](c53b1cd))
* **trie:** Add trie v1 new headers support ([#3295](#3295)) ([c30f463](c30f463))
* **zombienet:** Add javascript tests to zombienet testing ([#3200](#3200)) ([aca9a5b](aca9a5b))
* **zombienet:** add zombienet testing to github workflow ([#3192](#3192)) ([d788bd6](d788bd6))

### Reverts

* **lib/runtime:** rollback wasmer update `PR[#3168](#3168 ([#3264](#3264)) ([e7ff0cf](e7ff0cf))
Copy link

🎉 This PR is included in version 0.8.0 🎉

The release is available on GitHub release

Your semantic-release bot 📦🚀

timwu20 pushed a commit that referenced this pull request Apr 18, 2024
# [0.8.0](v0.7.0...v0.8.0) (2023-12-11)

### Bug Fixes

* **.github/workflows:** update node.js version ([#3637](#3637)) ([619be1b](619be1b))
* **babe:** Add support for versioned NextConfigData decoding ([#3239](#3239)) ([5ee3a64](5ee3a64))
* **blockstate:** if blocktree fails to search a hash in memory, load it from disk ([#3059](#3059)) ([6442544](6442544))
* cache slot to header data while checking BABE equivocation ([#3364](#3364)) ([dcfa4a4](dcfa4a4))
* **chain:** Fix `chain=westend` option  ([#3123](#3123)) ([64dbba6](64dbba6))
* **ci:** cancel previous workflow runs ([#3140](#3140)) ([a322a19](a322a19))
* **ci:** fix all Deepsource issues ([#3046](#3046)) ([4ea0a70](4ea0a70))
* **ci:** fix broken docker build ([#3231](#3231)) ([f796430](f796430))
* **ci:** Fix staging metrics collection ([#3138](#3138)) ([05a5c4c](05a5c4c))
* **cli:** parse module log-levels ([#3285](#3285)) ([86c7577](86c7577))
* **cmd/gossamer:** embed default toml config files ([#3091](#3091)) ([af38364](af38364))
* **cmd/gossamer:** update error message ([#3301](#3301)) ([960a9d4](960a9d4))
* **dot/babe:** use `bs.latestFinalised` instead of using `round/set id` ([#3167](#3167)) ([46c0ef7](46c0ef7))
* **dot/digest:** create `BlockImportHandler` and remove channel ([#3312](#3312)) ([a179855](a179855))
* **dot/network:** `findPeers` returns on timeout if a peer is found ([#3001](#3001)) ([2a05ce7](2a05ce7))
* **dot/network:** remove `maxReads` limitation to read stream ([#3287](#3287)) ([483b23f](483b23f))
* **dot/state:** clean up scheduled changes once a forced change is applied ([#3219](#3219)) ([5ebec46](5ebec46))
* **dot/state:** fix a bug in IsDescendantOf ([#3125](#3125)) ([4fd4a89](4fd4a89))
* **dot/state:** store raw authority keys and decode when verifying block signature ([#3627](#3627)) ([58f741d](58f741d))
* **dot/sync:** fix `Timestamp slot must match 'CurrentSlot'` while using `westend` spec file ([#3040](#3040)) ([e6da01b](e6da01b))
* **dot/sync:** Revert verify justification before importing blocks ([#3615](#3615)) ([11b96dc](11b96dc))
* **dot/sync:** rework on bootstrap/tip sync ([#3227](#3227)) ([ab6650a](ab6650a))
* **dot/sync:** use `Range` instead of `SubChain` at `handleDescedingRequest` ([#3006](#3006)) ([a83c1a3](a83c1a3))
* **dot/sync:** verify justification before importing blocks ([#3576](#3576)) ([2954fc0](2954fc0))
* **dot/sync:** wrong error message at `checkOrGetDescendantHash` ([#2971](#2971)) ([b1c6bf1](b1c6bf1))
* **dot:** use tempDir in tests as base path to avoid creating `dot/~` ([#3363](#3363)) ([04514d5](04514d5))
* **go.mod:** Replace `centrifuge/go-substrate-rpc-client` dependency to `timwu20/go-substrate-rpc-client` temporarily to fix build ([#3572](#3572)) ([ea49251](ea49251))
* **lib/babe:** Add context and additional assertion to TestBuildBlock_ok ([#3101](#3101)) ([a9a89ed](a9a89ed))
* **lib/babe:** rewrite TestBuildBlock_failing ([#3089](#3089)) ([28a3d0b](28a3d0b))
* **lib/babe:** use current system time to yield a new slot ([#3133](#3133)) ([9cd6f25](9cd6f25))
* **lib/grandpa:**  on verifying block justification, compare given block hash with finalised hash ([#3081](#3081)) ([fc91843](fc91843))
* **lib/grandpa:** ensure `finalisationEngine` exits when stop channel is triggered ([#3141](#3141)) ([d7f7c06](d7f7c06))
* **lib/runtime:** `ext_default_child_storage_next_key_version_1` return `None` correctly ([#3473](#3473)) ([c7d574b](c7d574b))
* **lib/runtime:** Fix `wasm error: out of bounds memory access` at `[#9412261](https://github.com/ChainSafe/gossamer/issues/9412261)` ([#3588](#3588)) ([ecb1ad9](ecb1ad9))
* **lib/runtime:** prevents polkadot zero-address bug using `sr25519_verify` version 1 ([#3494](#3494)) ([8b93d5e](8b93d5e))
* **lib/runtime:** return correct encoded value for `None` ([#3451](#3451)) ([3e11bc2](3e11bc2))
* **lib/runtime:** update `MaxPossibleAllocation` to `2^25` ([#3393](#3393)) ([91eabdc](91eabdc))
* **lib/runtime:** use `westend-dev` spec file in `TestNodeRuntime_ValidateTransaction` ([#3047](#3047)) ([043f5eb](043f5eb))
* **lib/trie:** `ClearFromChild` should update parent trie ([#3482](#3482)) ([70e2d2b](70e2d2b))
* **lib/trie:** create an empty child trie if not found ([#3459](#3459)) ([5d68447](5d68447))
* **lib/trie:** record deleted Merkle values fixed ([#2873](#2873)) ([61f0216](61f0216))
* **peerset:** check for incoming slot error ([#2952](#2952)) ([a1602bc](a1602bc))
* **rpc-tests:** Fix node port to execute tests on macOS ([#3223](#3223)) ([f758575](f758575))
* **rpc/modules:** use `westend-local` in `TestAuthorModule_SubmitExtrinsic_invalid` test ([#3051](#3051)) ([b6429b7](b6429b7))
* **runtime:** initialize TransactionState when creating runtime instance ([#3188](#3188)) ([29fe7a0](29fe7a0))
* **scale:** Use *int for scale index ([#3274](#3274)) ([9b04d30](9b04d30))
* **staging:** fixes the staging deployment issues caused by the new cli ([#3266](#3266)) ([1f4e786](1f4e786))
* **state:** clarify node hashes vs merkle values ([#2915](#2915)) ([e4033e8](e4033e8))
* **test/rpc:** use `westend-local` in `TestStateRPCAPI` ([#3049](#3049)) ([c57ade6](c57ade6))
* **tests/polkadot_js:** Use `westend-local` to run polkadot js test suite ([#3052](#3052)) ([2d5ead1](2d5ead1))
* **tests/rpc:** ensure new blocks are created before assert ([#3042](#3042)) ([a116d58](a116d58))
* **tests/rpc:** flaky test `TestChainSubscriptionRPC/chain_subscribeNewHeads` ([#3092](#3092)) ([5b56238](5b56238))
* **tests/rpc:** place `GreaterOrEqual` arguments in the correct order at `chain_subscribeNewHeads` test ([#3137](#3137)) ([33bdf28](33bdf28))
* **tests:** Export unimplemented runtime fuctions ([#3461](#3461)) ([3e4546c](3e4546c))
* **trie:** do not create buffer for nil child ([#2928](#2928)) ([d70af4f](d70af4f))
* **wasmer:** `ext_storage_exists_version_1` for empty values ([#2973](#2973)) ([059268e](059268e))

### Features

* **chain:** Add Westend network as command line `chain` option ([#3103](#3103)) ([d9cdd45](d9cdd45))
* **chain:** remove unneeded spec files ([#3086](#3086)) ([c76387d](c76387d))
* **cli:** use a single flag for log level in the CLI ([#3303](#3303)) ([caf3ea4](caf3ea4))
* **dot/network:** introduce libp2p resource manager + prometheus metrics ([#3333](#3333)) ([f166746](f166746))
* **dot/rpc:** export block trie state entries for a block hash ([#3607](#3607)) ([43828fe](43828fe))
* **dot/rpc:** implement RPC method state_queryStorageAt ([#3191](#3191)) ([3bbdfe0](3bbdfe0))
* **dot/state:** create `Range` to traverse the blocktree and the blocks in the disk ([#2990](#2990)) ([4442eee](4442eee))
* **dot/state:** keep latest state trie in memory ([#3386](#3386)) ([421d087](421d087))
* **dot/sync:** include block origin and skip extra validation on `initialSync` ([#3392](#3392)) ([8e1650e](8e1650e))
* **dot/sync:** Remove the `EndBlockHash` from `BlockRequestMessage` ([#2977](#2977)) ([b25e0b4](b25e0b4))
* **internal/database:** replace `chaindb/badgerdb` with `pebbledb` ([#3434](#3434)) ([344461d](344461d))
* introduces `SaturatingAdd` and `SaturatingSub` ([#3519](#3519)) ([daa9e25](daa9e25))
* **lib/allocator:** Refactor `FreeingBumpHeapAllocator` ([#3570](#3570)) ([39ca47f](39ca47f))
* **lib/babe:** Submit BABE equivocation report  ([#2947](#2947)) ([55de62e](55de62e)), closes [#2853](#2853)
* **lib/blocktree:** reduce the entries in the runtimes mapping ([#3151](#3151)) ([1a34972](1a34972))
* **lib/grandpa:** include `t.Parallel()` to all `lib/grandpa` tests ([#2840](#2840)) ([5c93488](5c93488))
* **lib/runtime/wasmer:** report grandpa equivocations ([#3007](#3007)) ([e63aeea](e63aeea))
* **lib/runtime:** `wazero` implementation of `runtime.Instance` ([#3279](#3279)) ([115d6f5](115d6f5))
* **lib/runtime:** add extra required runtime imports for parachain validation ([#3254](#3254)) ([dc1a521](dc1a521))
* **lib/runtime:** Update default `runtime.Instance` to `wazero` ([#3352](#3352)) ([308b10a](308b10a))
* **pkg/scale:** `VaryingDataType` String method ([#2970](#2970)) ([841636e](841636e))
* **pkg/scale:** Add `Marshaler` and `Unmarshaler` interfaces and functionality ([#3617](#3617)) ([4888ce4](4888ce4))
* **pkg/scale:** Use `New()` receiver function for construction of custom `VaryingDataType`  ([#3315](#3315)) ([9688f6c](9688f6c))
* **runtime/wasmer:** create wrapper around wasmer.Memory  ([#3160](#3160)) ([fc1055d](fc1055d))
* **runtime/wasmer:** write wasmer using latest wasmer version ([#3168](#3168)) ([32f1aa8](32f1aa8))
* **scale:** add `MustMarshal` function ([#2991](#2991)) ([32a80aa](32a80aa))
* **sync:** Validate bad blocks ([#3220](#3220)) ([0d0354b](0d0354b))
* **telemetry:** Add scheduled and force changes telemetry metrics ([#3226](#3226)) ([c53b1cd](c53b1cd))
* **trie:** Add trie v1 new headers support ([#3295](#3295)) ([c30f463](c30f463))
* **zombienet:** Add javascript tests to zombienet testing ([#3200](#3200)) ([aca9a5b](aca9a5b))
* **zombienet:** add zombienet testing to github workflow ([#3192](#3192)) ([d788bd6](d788bd6))

### Reverts

* **lib/runtime:** rollback wasmer update `PR[#3168](#3168 ([#3264](#3264)) ([e7ff0cf](e7ff0cf))
timwu20 pushed a commit that referenced this pull request Apr 19, 2024
# [0.8.0](v0.7.0...v0.8.0) (2023-12-11)

### Bug Fixes

* **.github/workflows:** update node.js version ([#3637](#3637)) ([619be1b](619be1b))
* **babe:** Add support for versioned NextConfigData decoding ([#3239](#3239)) ([5ee3a64](5ee3a64))
* **blockstate:** if blocktree fails to search a hash in memory, load it from disk ([#3059](#3059)) ([6442544](6442544))
* cache slot to header data while checking BABE equivocation ([#3364](#3364)) ([dcfa4a4](dcfa4a4))
* **chain:** Fix `chain=westend` option  ([#3123](#3123)) ([64dbba6](64dbba6))
* **ci:** cancel previous workflow runs ([#3140](#3140)) ([a322a19](a322a19))
* **ci:** fix all Deepsource issues ([#3046](#3046)) ([4ea0a70](4ea0a70))
* **ci:** fix broken docker build ([#3231](#3231)) ([f796430](f796430))
* **ci:** Fix staging metrics collection ([#3138](#3138)) ([05a5c4c](05a5c4c))
* **cli:** parse module log-levels ([#3285](#3285)) ([86c7577](86c7577))
* **cmd/gossamer:** embed default toml config files ([#3091](#3091)) ([af38364](af38364))
* **cmd/gossamer:** update error message ([#3301](#3301)) ([960a9d4](960a9d4))
* **dot/babe:** use `bs.latestFinalised` instead of using `round/set id` ([#3167](#3167)) ([46c0ef7](46c0ef7))
* **dot/digest:** create `BlockImportHandler` and remove channel ([#3312](#3312)) ([a179855](a179855))
* **dot/network:** `findPeers` returns on timeout if a peer is found ([#3001](#3001)) ([2a05ce7](2a05ce7))
* **dot/network:** remove `maxReads` limitation to read stream ([#3287](#3287)) ([483b23f](483b23f))
* **dot/state:** clean up scheduled changes once a forced change is applied ([#3219](#3219)) ([5ebec46](5ebec46))
* **dot/state:** fix a bug in IsDescendantOf ([#3125](#3125)) ([4fd4a89](4fd4a89))
* **dot/state:** store raw authority keys and decode when verifying block signature ([#3627](#3627)) ([58f741d](58f741d))
* **dot/sync:** fix `Timestamp slot must match 'CurrentSlot'` while using `westend` spec file ([#3040](#3040)) ([e6da01b](e6da01b))
* **dot/sync:** Revert verify justification before importing blocks ([#3615](#3615)) ([11b96dc](11b96dc))
* **dot/sync:** rework on bootstrap/tip sync ([#3227](#3227)) ([ab6650a](ab6650a))
* **dot/sync:** use `Range` instead of `SubChain` at `handleDescedingRequest` ([#3006](#3006)) ([a83c1a3](a83c1a3))
* **dot/sync:** verify justification before importing blocks ([#3576](#3576)) ([2954fc0](2954fc0))
* **dot/sync:** wrong error message at `checkOrGetDescendantHash` ([#2971](#2971)) ([b1c6bf1](b1c6bf1))
* **dot:** use tempDir in tests as base path to avoid creating `dot/~` ([#3363](#3363)) ([04514d5](04514d5))
* **go.mod:** Replace `centrifuge/go-substrate-rpc-client` dependency to `timwu20/go-substrate-rpc-client` temporarily to fix build ([#3572](#3572)) ([ea49251](ea49251))
* **lib/babe:** Add context and additional assertion to TestBuildBlock_ok ([#3101](#3101)) ([a9a89ed](a9a89ed))
* **lib/babe:** rewrite TestBuildBlock_failing ([#3089](#3089)) ([28a3d0b](28a3d0b))
* **lib/babe:** use current system time to yield a new slot ([#3133](#3133)) ([9cd6f25](9cd6f25))
* **lib/grandpa:**  on verifying block justification, compare given block hash with finalised hash ([#3081](#3081)) ([fc91843](fc91843))
* **lib/grandpa:** ensure `finalisationEngine` exits when stop channel is triggered ([#3141](#3141)) ([d7f7c06](d7f7c06))
* **lib/runtime:** `ext_default_child_storage_next_key_version_1` return `None` correctly ([#3473](#3473)) ([c7d574b](c7d574b))
* **lib/runtime:** Fix `wasm error: out of bounds memory access` at `[#9412261](https://github.com/ChainSafe/gossamer/issues/9412261)` ([#3588](#3588)) ([ecb1ad9](ecb1ad9))
* **lib/runtime:** prevents polkadot zero-address bug using `sr25519_verify` version 1 ([#3494](#3494)) ([8b93d5e](8b93d5e))
* **lib/runtime:** return correct encoded value for `None` ([#3451](#3451)) ([3e11bc2](3e11bc2))
* **lib/runtime:** update `MaxPossibleAllocation` to `2^25` ([#3393](#3393)) ([91eabdc](91eabdc))
* **lib/runtime:** use `westend-dev` spec file in `TestNodeRuntime_ValidateTransaction` ([#3047](#3047)) ([043f5eb](043f5eb))
* **lib/trie:** `ClearFromChild` should update parent trie ([#3482](#3482)) ([70e2d2b](70e2d2b))
* **lib/trie:** create an empty child trie if not found ([#3459](#3459)) ([5d68447](5d68447))
* **lib/trie:** record deleted Merkle values fixed ([#2873](#2873)) ([61f0216](61f0216))
* **peerset:** check for incoming slot error ([#2952](#2952)) ([a1602bc](a1602bc))
* **rpc-tests:** Fix node port to execute tests on macOS ([#3223](#3223)) ([f758575](f758575))
* **rpc/modules:** use `westend-local` in `TestAuthorModule_SubmitExtrinsic_invalid` test ([#3051](#3051)) ([b6429b7](b6429b7))
* **runtime:** initialize TransactionState when creating runtime instance ([#3188](#3188)) ([29fe7a0](29fe7a0))
* **scale:** Use *int for scale index ([#3274](#3274)) ([9b04d30](9b04d30))
* **staging:** fixes the staging deployment issues caused by the new cli ([#3266](#3266)) ([1f4e786](1f4e786))
* **state:** clarify node hashes vs merkle values ([#2915](#2915)) ([e4033e8](e4033e8))
* **test/rpc:** use `westend-local` in `TestStateRPCAPI` ([#3049](#3049)) ([c57ade6](c57ade6))
* **tests/polkadot_js:** Use `westend-local` to run polkadot js test suite ([#3052](#3052)) ([2d5ead1](2d5ead1))
* **tests/rpc:** ensure new blocks are created before assert ([#3042](#3042)) ([a116d58](a116d58))
* **tests/rpc:** flaky test `TestChainSubscriptionRPC/chain_subscribeNewHeads` ([#3092](#3092)) ([5b56238](5b56238))
* **tests/rpc:** place `GreaterOrEqual` arguments in the correct order at `chain_subscribeNewHeads` test ([#3137](#3137)) ([33bdf28](33bdf28))
* **tests:** Export unimplemented runtime fuctions ([#3461](#3461)) ([3e4546c](3e4546c))
* **trie:** do not create buffer for nil child ([#2928](#2928)) ([d70af4f](d70af4f))
* **wasmer:** `ext_storage_exists_version_1` for empty values ([#2973](#2973)) ([059268e](059268e))

### Features

* **chain:** Add Westend network as command line `chain` option ([#3103](#3103)) ([d9cdd45](d9cdd45))
* **chain:** remove unneeded spec files ([#3086](#3086)) ([c76387d](c76387d))
* **cli:** use a single flag for log level in the CLI ([#3303](#3303)) ([caf3ea4](caf3ea4))
* **dot/network:** introduce libp2p resource manager + prometheus metrics ([#3333](#3333)) ([f166746](f166746))
* **dot/rpc:** export block trie state entries for a block hash ([#3607](#3607)) ([43828fe](43828fe))
* **dot/rpc:** implement RPC method state_queryStorageAt ([#3191](#3191)) ([3bbdfe0](3bbdfe0))
* **dot/state:** create `Range` to traverse the blocktree and the blocks in the disk ([#2990](#2990)) ([4442eee](4442eee))
* **dot/state:** keep latest state trie in memory ([#3386](#3386)) ([421d087](421d087))
* **dot/sync:** include block origin and skip extra validation on `initialSync` ([#3392](#3392)) ([8e1650e](8e1650e))
* **dot/sync:** Remove the `EndBlockHash` from `BlockRequestMessage` ([#2977](#2977)) ([b25e0b4](b25e0b4))
* **internal/database:** replace `chaindb/badgerdb` with `pebbledb` ([#3434](#3434)) ([344461d](344461d))
* introduces `SaturatingAdd` and `SaturatingSub` ([#3519](#3519)) ([daa9e25](daa9e25))
* **lib/allocator:** Refactor `FreeingBumpHeapAllocator` ([#3570](#3570)) ([39ca47f](39ca47f))
* **lib/babe:** Submit BABE equivocation report  ([#2947](#2947)) ([55de62e](55de62e)), closes [#2853](#2853)
* **lib/blocktree:** reduce the entries in the runtimes mapping ([#3151](#3151)) ([1a34972](1a34972))
* **lib/grandpa:** include `t.Parallel()` to all `lib/grandpa` tests ([#2840](#2840)) ([5c93488](5c93488))
* **lib/runtime/wasmer:** report grandpa equivocations ([#3007](#3007)) ([e63aeea](e63aeea))
* **lib/runtime:** `wazero` implementation of `runtime.Instance` ([#3279](#3279)) ([115d6f5](115d6f5))
* **lib/runtime:** add extra required runtime imports for parachain validation ([#3254](#3254)) ([dc1a521](dc1a521))
* **lib/runtime:** Update default `runtime.Instance` to `wazero` ([#3352](#3352)) ([308b10a](308b10a))
* **pkg/scale:** `VaryingDataType` String method ([#2970](#2970)) ([841636e](841636e))
* **pkg/scale:** Add `Marshaler` and `Unmarshaler` interfaces and functionality ([#3617](#3617)) ([4888ce4](4888ce4))
* **pkg/scale:** Use `New()` receiver function for construction of custom `VaryingDataType`  ([#3315](#3315)) ([9688f6c](9688f6c))
* **runtime/wasmer:** create wrapper around wasmer.Memory  ([#3160](#3160)) ([fc1055d](fc1055d))
* **runtime/wasmer:** write wasmer using latest wasmer version ([#3168](#3168)) ([32f1aa8](32f1aa8))
* **scale:** add `MustMarshal` function ([#2991](#2991)) ([32a80aa](32a80aa))
* **sync:** Validate bad blocks ([#3220](#3220)) ([0d0354b](0d0354b))
* **telemetry:** Add scheduled and force changes telemetry metrics ([#3226](#3226)) ([c53b1cd](c53b1cd))
* **trie:** Add trie v1 new headers support ([#3295](#3295)) ([c30f463](c30f463))
* **zombienet:** Add javascript tests to zombienet testing ([#3200](#3200)) ([aca9a5b](aca9a5b))
* **zombienet:** add zombienet testing to github workflow ([#3192](#3192)) ([d788bd6](d788bd6))

### Reverts

* **lib/runtime:** rollback wasmer update `PR[#3168](#3168 ([#3264](#3264)) ([e7ff0cf](e7ff0cf))
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants