-
Notifications
You must be signed in to change notification settings - Fork 39.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Serialized protobuf should have stable map order #47701
Serialized protobuf should have stable map order #47701
Conversation
@kubernetes/sig-api-machinery-pr-reviews |
Fuzz test that serializes multiple times and ensures output is stable would be good |
The default marshalers from gogo-protobuf do not order keys, which means successive serializations of the same data would result in same bytes. Kubernetes leverages stable serialization to avoid excessive writes to etcd when body content has not changed, which reduces the amount of watch traffic generated by successive callers. This will have a small memory impact on objects which have maps - allocating a new string array for each map key. All maps must be keys because we serialize those to JSON, so non-string types are not relevant.
276241a
to
c1ff759
Compare
@k8s-bot pull-kubernetes-e2e-gce-etcd3 test this |
1 similar comment
@k8s-bot pull-kubernetes-e2e-gce-etcd3 test this |
LGTM, though I'm unsure whether we want this for 1.7 |
/lgtm This is a small change (diff is only big because generated code). It is backwards compatible. Seems very safe as far as these things go. |
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: lavalamp, smarterclayton Associated issue: 47678 The full list of commands accepted by this bot can be found here.
Needs approval from an approver in each of these OWNERS Files:
You can indicate your approval by writing |
Automatic merge from submit-queue (batch tested with PRs 47669, 40284, 47356, 47458, 47701) |
Leverage
stable_marshaller_all
tag in gogo-protobuf to ensure all of our maps have stable ordering when serialized to protobuf. This preserves the behavior from JSON that we rely on to avoid writing to etcd except when the content has changed.Will slightly increase allocations (1 slice per non-empty map in an object) during Encode, which has a minor impact on CPU. However, avoiding writes when a client issues a blind put results in significantly less CPU across the whole cluster (avoiding a new etcd version for an Endpoints object might save ~1 core/sec on large clusters).
Fixes #47678