Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat(en): Make snapshots applier resilient and process storage log chunks in parallel #1036

Merged
merged 9 commits into from
Feb 14, 2024
Merged

feat(en): Make snapshots applier resilient and process storage log chunks in parallel #1036

merged 9 commits into from
Feb 14, 2024

Conversation

tomg10
Copy link
Contributor

@tomg10 tomg10 commented Feb 8, 2024 •
edited by slowli
Loading

What ❔

  • Modifies DB recovery logic so that it retries on failures.
  • Parallelizes DB recovery using a semaphore (i.e., similarly to snapshot recovery in some components).

Why ❔

  • Retries on transient errors reduce the chance of the recovery getting stuck.
  • Parallelizing DB recovery can speed it up.

Checklist

  • PR title corresponds to the body of PR (we generate changelog entries from PRs).
  • Tests for the changes have been added / updated.
  • Documentation comments have been added / updated.
  • Code has been formatted via zk fmt and zk lint.
  • Spellcheck has been run via zk spellcheck.
  • Linkcheck has been run via zk linkcheck.

@tomg10
Added resiliency
c48e25e 
Signed-off-by: tomg10 <lemures64@gmail.com>
@tomg10
zk fmt
54354ae 
Signed-off-by: tomg10 <lemures64@gmail.com>
@tomg10 tomg10 requested a review from slowli February 8, 2024 20:45
@tomg10 tomg10 changed the title Snapshots applier resiliency and parallelized processing storage logs chunks feat: Snapshots applier resiliency and parallelized processing storage logs chunks Feb 8, 2024
popzxc
popzxc approved these changes Feb 14, 2024
View reviewed changes
Copy link
Member

@popzxc popzxc left a comment •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

-1 for PR not having description and no linear task with description linked.
I can't say I'm familiar with snapshot applier, but the diff generally looks good (reasonable, at least).

@slowli slowli changed the title feat: Snapshots applier resiliency and parallelized processing storage logs chunks feat(en): Make snapshots applier resilient and process storage log chunks in parallel Feb 14, 2024
@slowli
Copy link
Contributor

slowli commented Feb 14, 2024

@popzxc I've updated the description and linked the PR to a Linear task.

@popzxc
Copy link
Member

popzxc commented Feb 14, 2024

@slowli thank you! Had a second look on the PR and it does seem to be inline with the description.

@slowli slowli mentioned this pull request Feb 14, 2024
Merged
6 tasks
@slowli slowli added this pull request to the merge queue Feb 14, 2024
Merged via the queue into main with commit 805218c Feb 14, 2024
32 checks passed
@slowli slowli deleted the snapshots-applier-resiliency branch February 14, 2024 15:45
@zksync-era-bot zksync-era-bot mentioned this pull request Feb 14, 2024
Merged
github-merge-queue bot pushed a commit that referenced this pull request Feb 16, 2024
@zksync-era-bot
chore(main): release core 20.7.0 (#1049)
98f87cd 
🤖 I have created a release *beep* *boop*
---


##
[20.7.0](core-v20.6.0...core-v20.7.0)
(2024-02-16)


### Features

* Add input field to CallRequest
([#1069](#1069))
([5087121](5087121))
* **api:** Remove unused and obsolete token info
([#1071](#1071))
([e920897](e920897))
* Better errors for JsonRPC calls
([#1002](#1002))
([079f999](079f999))
* **commitment:** Commitment component
([#1024](#1024))
([60305ba](60305ba))
* **en:** Make snapshots applier resilient and process storage log
chunks in parallel
([#1036](#1036))
([805218c](805218c))
* **node_framework:** Resources and layers for ETH clients
([#1074](#1074))
([776337a](776337a))
* **node_framework:** Support StateKeeper in the framework
([#1043](#1043))
([a80fff2](a80fff2))


### Bug Fixes

* **api:** Return on duplicate earlier
([#1059](#1059))
([cfa5701](cfa5701))
* **contract-verifier:** Use optimizer mode in solidity-single-file
verification
([#1079](#1079))
([fdab638](fdab638))
* Token distribution
([#1051](#1051))
([bd63b3a](bd63b3a))

---
This PR was generated with [Release
Please](https://github.com/googleapis/release-please). See
[documentation](https://github.com/googleapis/release-please#release-please).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@RomanBrodetski RomanBrodetski RomanBrodetski approved these changes

@popzxc popzxc popzxc approved these changes

@slowli slowli Awaiting requested review from slowli

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
4 participants
@tomg10 @slowli @popzxc @RomanBrodetski