Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Switch to SSZ eth2 encoding in protobufs #2203

Closed
5 tasks
corverroos opened this issue May 17, 2023 · 0 comments
Closed
5 tasks

Switch to SSZ eth2 encoding in protobufs #2203

corverroos opened this issue May 17, 2023 · 0 comments
Labels
protocol Protocol Team tickets

Comments

@corverroos
Copy link
Contributor

corverroos commented May 17, 2023 •
edited

🎯 Problem to be solved

We currently encode eth2 types to json when sending them over the wire as protobufs. JSON is more than 2x the bandwidth of SSZ. We should decrease our bandwidth requirements to improve the performance of large proposal wire payloads.

Smaller payloads result in less latency in low-throughput clusters, which result in quicker and more robust consensus which result in less missed duties and more rewards.

🛠️ Proposed solution

  • Approved design doc: link
  • Core team consensus on the proposed solution

This is a multi step process, affecting core. UnsignedDataSetToProto and core.UnsignedDataSetFromProto:
1: add support for unmarshalling SSZ encoded values in the proto (in addition to JSON)
2: release v0.16
3: switch to marshalling SSZ encoded protos
4: release v0.17
5: remove support for unmarshalling and marshalling JSON encoded protos.

⚠️⚠️⚠️
This affects backwards compatibility:
V0.17 is only compatible with v0.16

🧪 Tests

  • Tested by new automated unit/integration/smoke tests
  • Manually tested on core team/canary/test clusters
  • Manually tested on local compose simnet
@github-actions github-actions bot added the protocol Protocol Team tickets label May 17, 2023
@corverroos corverroos mentioned this issue May 19, 2023
Merged
obol-bulldozer bot pushed a commit that referenced this issue May 22, 2023
@corverroos
core: support ssz unmarshalling in protos (#2210)
f790cd0 
First step in migrating to SSZ encoding of eth2 objects in protobufs. Support unmarshalling of SSZ. This will reduce bandwidth usage by 2x.

Changes include:
 - `testutil.NewEth2Fuzzer` that can generated valid fuzzed eth2 types.
 - `core.Versioned*Block` ssz marshalling. This is a custom ssz marshaling pattern for types with a version and then one-of-many optional fields.
 - `unmarshal` that supports both SSZ and JSON unmarshalling.
 - `marshal` that will suppoter both SSZ and JSON marshalling once enabled (in v0.17).

category: feature
ticket: #2203
@corverroos corverroos mentioned this issue Jun 6, 2023
Merged
obol-bulldozer bot pushed a commit that referenced this issue Jun 6, 2023
@corverroos
core: enable ssz encoding in protos (#2286)
f1f7318 
Enables SSZ encoding in protos by default. This decreases wire bandwitdh requirements. 

> Note this breaks compatibility with v0.15 and earlier. So v0.17 will only be compatible with v0.16.

category: feature 
ticket: #2203
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
protocol Protocol Team tickets
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@corverroos @boulder225