Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Accelerate slashing protection import by minification #2354

Closed
michaelsproul opened this issue May 19, 2021 · 0 comments
Closed

Accelerate slashing protection import by minification #2354

michaelsproul opened this issue May 19, 2021 · 0 comments
Labels
A1 database

Comments

@michaelsproul
Copy link
Member

Description

Importing slashing protection data is currently slow for two reasons:

  1. We're using a suboptimal method for parsing the JSON (see Speed up JSON load in slashing protection import #2347)
  2. It involves a lot of database ops, pretty much equivalent to resigning all of the input attestations. For files >1GB this seems to take >1hr, which is unreasonable.

This issue proposes speeding up (2) by safely minifying the input file into one attestation per validator. This minification process could also optionally be applied on export, and could eventually become part of its own CLI tool for working with EIP-3076 JSON files.
@michaelsproul michaelsproul mentioned this issue May 19, 2021
Closed
@michaelsproul michaelsproul mentioned this issue May 31, 2021
Closed
bors bot pushed a commit that referenced this issue Jun 21, 2021
@michaelsproul
Minify slashing protection interchange data (#2380)
6583ce3 
## Issue Addressed

Closes #2354

## Proposed Changes

Add a `minify` method to `slashing_protection::Interchange` that keeps only the maximum-epoch attestation and maximum-slot block for each validator. Specifically, `minify` constructs "synthetic" attestations (with no `signing_root`) containing the maximum source epoch _and_ the maximum target epoch from the input. This is equivalent to the `minify_synth` algorithm that I've formally verified in this repository:

https://github.com/michaelsproul/slashing-proofs

## Additional Info

Includes the JSON loading optimisation from #2347
@mratsim mratsim mentioned this issue Feb 14, 2022
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
A1 database
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@michaelsproul