feat: add config `iavl-lazy-loading` to enable lazy loading of iavl store #14189

yihuang · 2022-12-07T09:32:43Z

Description

It improve start time of archive nodes (from dozens of minutes to seconds).

It seems to be running alright on production node, but not sure of unknown side effects.

Author Checklist

All items are required. Please add a note to the item if the item is not applicable and
please add links to any relevant follow up issues.

I have...

Reviewers Checklist

All items are required. Please add a note if the item is not applicable and please add
your handle next to the items reviewed if you only reviewed selected items.

I have...

confirmed the correct type prefix in the PR title
confirmed ! in the type prefix if API or client breaking change
confirmed all author checklist items have been addressed
reviewed state machine logic
reviewed API design and naming
reviewed documentation is accurate
reviewed tests and test coverage
manually tested (if applicable)

CHANGELOG.md

…ackport cosmos#14189) improve start time of archive nodes. Update CHANGELOG.md

github-actions · 2023-01-08T00:05:01Z

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

yihuang · 2023-01-08T02:42:30Z

unstall

improve start time of archive nodes. Update CHANGELOG.md fix test

tac0turtle · 2023-01-30T16:28:02Z

@yihuang want to update this to alpha.2?

yihuang · 2023-01-31T00:56:35Z

@yihuang want to update this to alpha.2?

done, added a bunch of replacements to make it build though, not sure what's the best way to do it, do I need to make a separate PR for store package?

yihuang · 2023-02-01T02:00:11Z

can we backport to 0.47 and 0.46, it's important UX improvement for existing chains. @tac0turtle

tac0turtle · 2023-02-01T09:34:32Z

@Mergifyio backport release/v0.46.x

tac0turtle · 2023-02-01T09:34:47Z

@Mergifyio backport release/v0.47.x

tac0turtle · 2023-02-01T09:35:10Z

heads up these prs will need to be reduced to 0.19.5

…tore (#14189) Co-authored-by: Marko <marbar3778@yahoo.com> (cherry picked from commit b266522) # Conflicts: # CHANGELOG.md # cosmovisor/go.sum # go.mod # go.sum # server/util.go # simapp/go.mod # simapp/go.sum # store/go.mod # store/go.sum # tests/go.mod # tests/go.sum # x/nft/go.mod # x/upgrade/go.mod # x/upgrade/go.sum

mergify · 2023-02-01T09:36:09Z

backport release/v0.46.x

✅ Backports have been created

#14862 feat: add config iavl-lazy-loading to enable lazy loading of iavl store (backport #14189) has been created for branch release/v0.46.x

…tore (#14189) Co-authored-by: Marko <marbar3778@yahoo.com> (cherry picked from commit b266522) # Conflicts: # CHANGELOG.md # go.mod # go.sum # simapp/go.mod # simapp/go.sum # store/go.mod # store/go.sum # tests/go.mod # tests/go.sum # x/nft/go.mod # x/nft/go.sum # x/upgrade/go.mod # x/upgrade/go.sum

mergify · 2023-02-01T09:36:28Z

backport release/v0.47.x

✅ Backports have been created

#14863 feat: add config iavl-lazy-loading to enable lazy loading of iavl store (backport #14189) has been created for branch release/v0.47.x

…tore (cosmos#14189) Co-authored-by: Marko <marbar3778@yahoo.com> (cherry picked from commit b266522) # Conflicts: # CHANGELOG.md # cosmovisor/go.sum # go.mod # go.sum # server/util.go # simapp/go.mod # simapp/go.sum # store/go.mod # store/go.sum # tests/go.mod # tests/go.sum # x/nft/go.mod # x/upgrade/go.mod # x/upgrade/go.sum

…tore (backport #14189) (#14863) Co-authored-by: yihuang <huang@crypto.com> Co-authored-by: marbar3778 <marbar3778@yahoo.com>

…tore (backport #14189) (#14862) Co-authored-by: yihuang <huang@crypto.com> Co-authored-by: marbar3778 <marbar3778@yahoo.com>

…tore (backport cosmos#14189) (cosmos#14862) Co-authored-by: yihuang <huang@crypto.com> Co-authored-by: marbar3778 <marbar3778@yahoo.com>

…518) * fix: Fix Evidence CLI query flag parsing (backport cosmos#13458) (cosmos#14799) Co-authored-by: Likhita Polavarapu <78951027+likhita-809@users.noreply.github.com> Co-authored-by: HuangYi <huang@crypto.com> Co-authored-by: Julien Robert <julien@rbrt.fr> * fix: nested iterator on cache store (cosmos#14798) * chore: update IAVL (backport cosmos#14830) (cosmos#14835) Co-authored-by: Marko <marbar3778@yahoo.com> * Mark v0.46.8-pio-2 in the changelog and release notes. * feat: add config `iavl-lazy-loading` to enable lazy loading of iavl store (backport cosmos#14189) (cosmos#14862) Co-authored-by: yihuang <huang@crypto.com> Co-authored-by: marbar3778 <marbar3778@yahoo.com> * chore: bump IAVL to v0.19.5 (backport cosmos#14889) (cosmos#14892) Co-authored-by: Julien Robert <julien@rbrt.fr> * build(deps): bump tendermint to v0.34.25 (cosmos#14903) * build(deps): bump tendermint to v0.34.26 (cosmos#14911) * fix: return named error for write validators (backport cosmos#14919) (cosmos#14921) Co-authored-by: mmsqe <mavis@crypto.com> Co-authored-by: Julien Robert <julien@rbrt.fr> * Bump Tendermint to v0.34.25 (from v0.34.24). (#498) * Fix a few listener proto comments. (#499) * build(deps): bump btcd in v0.46 (cosmos#14846) Co-authored-by: yihuang <huang@crypto.com> Co-authored-by: Julien Robert <julien@rbrt.fr> Co-authored-by: Elias Naur <103319121+elias-orijtech@users.noreply.github.com> Co-authored-by: Marko <marbar3778@yahoo.com> * ci: update release slack channel and algolia api key (backport cosmos#14942) (cosmos#14945) Co-authored-by: Julien Robert <julien@rbrt.fr> * feat: support profiling block replay during abci handshake (backport cosmos#14953) (cosmos#14964) Co-authored-by: yihuang <huang@crypto.com> Co-authored-by: Julien Robert <julien@rbrt.fr> * Revert "Revert cosmos#14349 - cache iterator optimizations (#444)" This reverts commit cd9d192. * Mark v0.46.8-pio-3 in the changelog and release notes. * fix: avoid broadcasting tx through query (backport cosmos#15044) (cosmos#15047) Co-authored-by: Facundo Medica <14063057+facundomedica@users.noreply.github.com> Co-authored-by: Julien Robert <julien@rbrt.fr> * Add CLI commands for sanction Tx gov props (#510) * Fix the changelog. --------- Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> Co-authored-by: Likhita Polavarapu <78951027+likhita-809@users.noreply.github.com> Co-authored-by: HuangYi <huang@crypto.com> Co-authored-by: Julien Robert <julien@rbrt.fr> Co-authored-by: Marko <marbar3778@yahoo.com> Co-authored-by: mmsqe <mavis@crypto.com> Co-authored-by: Robert Zaremba <robert@zaremba.ch> Co-authored-by: Elias Naur <103319121+elias-orijtech@users.noreply.github.com> Co-authored-by: Facundo Medica <14063057+facundomedica@users.noreply.github.com>

…tore (backport cosmos#14189) (cosmos#14862) Co-authored-by: yihuang <huang@crypto.com> Co-authored-by: marbar3778 <marbar3778@yahoo.com>

…tore (cosmos#14189) Co-authored-by: Marko <marbar3778@yahoo.com>

yihuang requested a review from a team as a code owner December 7, 2022 09:32

github-actions bot added the C:Store label Dec 7, 2022

yihuang marked this pull request as draft December 7, 2022 09:33

yihuang commented Dec 7, 2022

View reviewed changes

CHANGELOG.md Outdated Show resolved Hide resolved

yihuang force-pushed the iavl-lazy-loading branch 2 times, most recently from d14d643 to cdcfc1d Compare December 7, 2022 09:46

yihuang added a commit to yihuang/cosmos-sdk that referenced this pull request Dec 7, 2022

feat: add iavl-lazy-loading to enable lazy loading of iavl store (b…

31cdff3

…ackport cosmos#14189) improve start time of archive nodes. Update CHANGELOG.md

yihuang mentioned this pull request Dec 7, 2022

feat: make LazyLoadVersion validate InitialVersion the same as LoadVersion cosmos/iavl#638

Merged

yihuang changed the title ~~feat: add iavl-lazy-loading to enable lazy loading of iavl store~~ feat: add config iavl-lazy-loading to enable lazy loading of iavl store Dec 8, 2022

yihuang mentioned this pull request Dec 8, 2022

feat: speed up rollback command cosmos/iavl#636

Merged

yihuang added a commit to yihuang/cosmos-sdk that referenced this pull request Dec 16, 2022

feat: add iavl-lazy-loading to enable lazy loading of iavl store (b…

255fa63

…ackport cosmos#14189) improve start time of archive nodes. Update CHANGELOG.md

github-actions bot added the stale label Jan 8, 2023

alexanderbez removed the stale label Jan 8, 2023

feat: add iavl-lazy-loading to enable lazy loading of iavl store

cc84e97

improve start time of archive nodes. Update CHANGELOG.md fix test

yihuang force-pushed the iavl-lazy-loading branch from 306a047 to cc84e97 Compare January 29, 2023 21:03

yihuang and others added 3 commits January 30, 2023 07:01

Merge branch 'main' into iavl-lazy-loading

19e2a1e

Merge branch 'main' into iavl-lazy-loading

0ef40a6

feat: add iavl-lazy-loading to enable lazy loading of iavl store

e724036

improve start time of archive nodes. Update CHANGELOG.md fix test

fix build

e9f2adf

Merge branch 'main' into iavl-lazy-loading

1a14a1d

yihuang marked this pull request as ready for review January 31, 2023 00:56

github-prbot requested review from a team, kocubinski and amaury1093 and removed request for a team January 31, 2023 00:56

fix test

df93092

yihuang and others added 2 commits February 1, 2023 00:20

Merge branch 'main' into iavl-lazy-loading

be64ea5

Merge branch 'main' into iavl-lazy-loading

2744e8c

tac0turtle enabled auto-merge (squash) January 31, 2023 21:08

tac0turtle added the A:automerge Automatically merge PR once all prerequisites pass. label Jan 31, 2023

Merge branch 'main' into iavl-lazy-loading

fc62e2b

tac0turtle merged commit b266522 into cosmos:main Jan 31, 2023

yihuang mentioned this pull request Feb 1, 2023

speed up rollback command #13744

Closed

mergify bot mentioned this pull request Feb 1, 2023

feat: add config iavl-lazy-loading to enable lazy loading of iavl store (backport #14189) #14862

Merged

mergify bot mentioned this pull request Feb 1, 2023

feat: add config iavl-lazy-loading to enable lazy loading of iavl store (backport #14189) #14863

Merged

julienrbrt pushed a commit that referenced this pull request Feb 1, 2023

feat: add config iavl-lazy-loading to enable lazy loading of iavl s…

3a1c797

…tore (backport #14189) (#14863) Co-authored-by: yihuang <huang@crypto.com> Co-authored-by: marbar3778 <marbar3778@yahoo.com>

tac0turtle added a commit that referenced this pull request Feb 2, 2023

feat: add config iavl-lazy-loading to enable lazy loading of iavl s…

70589eb

…tore (backport #14189) (#14862) Co-authored-by: yihuang <huang@crypto.com> Co-authored-by: marbar3778 <marbar3778@yahoo.com>

fedekunze mentioned this pull request Feb 9, 2023

chore(deps): Bump SDK to v0.46.9 evmos/cosmos-sdk#6

Merged

19 tasks

DracoLi mentioned this pull request Mar 13, 2023

Upgrade to Cosmos v0.46 Kava-Labs/kava#1477

Merged

tsenart pushed a commit to meka-dev/cosmos-sdk that referenced this pull request Apr 12, 2023

feat: add config iavl-lazy-loading to enable lazy loading of iavl s…

cb9dd65

…tore (cosmos#14189) Co-authored-by: Marko <marbar3778@yahoo.com>

larry0x pushed a commit to larry0x/cosmos-sdk that referenced this pull request May 22, 2023

feat: add config iavl-lazy-loading to enable lazy loading of iavl s…

3d37ad2

…tore (cosmos#14189) Co-authored-by: Marko <marbar3778@yahoo.com>

LampardNguyen234 mentioned this pull request Aug 15, 2023

deps: bump cosmos-sdk v0.46 and evmos v12 AstraProtocol/astra#209

Merged

0xstepit mentioned this pull request Jul 1, 2024

chore(cachemulti): add Copy method evmos/cosmos-sdk#52

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: add config `iavl-lazy-loading` to enable lazy loading of iavl store #14189

feat: add config `iavl-lazy-loading` to enable lazy loading of iavl store #14189

yihuang commented Dec 7, 2022 •

edited

Loading

github-actions bot commented Jan 8, 2023

yihuang commented Jan 8, 2023

tac0turtle commented Jan 30, 2023

yihuang commented Jan 31, 2023 •

edited

Loading

yihuang commented Feb 1, 2023 •

edited

Loading

tac0turtle commented Feb 1, 2023

tac0turtle commented Feb 1, 2023

tac0turtle commented Feb 1, 2023

mergify bot commented Feb 1, 2023

mergify bot commented Feb 1, 2023

feat: add config iavl-lazy-loading to enable lazy loading of iavl store #14189

feat: add config iavl-lazy-loading to enable lazy loading of iavl store #14189

Conversation

yihuang commented Dec 7, 2022 • edited Loading

Description

Author Checklist

Reviewers Checklist

github-actions bot commented Jan 8, 2023

yihuang commented Jan 8, 2023

tac0turtle commented Jan 30, 2023

yihuang commented Jan 31, 2023 • edited Loading

yihuang commented Feb 1, 2023 • edited Loading

tac0turtle commented Feb 1, 2023

tac0turtle commented Feb 1, 2023

tac0turtle commented Feb 1, 2023

mergify bot commented Feb 1, 2023

✅ Backports have been created

mergify bot commented Feb 1, 2023

✅ Backports have been created

feat: add config `iavl-lazy-loading` to enable lazy loading of iavl store #14189

feat: add config `iavl-lazy-loading` to enable lazy loading of iavl store #14189

yihuang commented Dec 7, 2022 •

edited

Loading

yihuang commented Jan 31, 2023 •

edited

Loading

yihuang commented Feb 1, 2023 •

edited

Loading