Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Increase limit for object size in streaming serializer #72053

Merged
merged 1 commit into from
Dec 15, 2018

Conversation

wojtek-t
Copy link
Member

@wojtek-t wojtek-t commented Dec 14, 2018

Given that apiserver doesn't have validation for object size, and etcd allows for writing objects larger than 1MB (I think 1.5MB is the limit @jpbetz to confirm), we should allow for reading them in the default streaming serializer.

I bumped the limit significantly to also handle situation where we write protobuf to etcd and we watch jsons which are visibly better (it doesn't guarantee it will always work, but it at least leave some slack so that can be possible).

@kubernetes/sig-api-machinery-bugs

Allow for watching objects larger than 1MB given etcd accepts objects of size up to 1.5MB

@wojtek-t wojtek-t added kind/bug Categorizes issue or PR as related to a bug. priority/important-soon Must be staffed and worked on either currently, or very soon, ideally in time for the next release. labels Dec 14, 2018
@k8s-ci-robot k8s-ci-robot added sig/api-machinery Categorizes an issue or PR as relevant to SIG API Machinery. size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. do-not-merge/release-note-label-needed Indicates that a PR should not merge because it's missing one of the release note labels. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. labels Dec 14, 2018
@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: wojtek-t

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Dec 14, 2018
@wojtek-t wojtek-t added release-note-none Denotes a PR that doesn't merit a release note. and removed do-not-merge/release-note-label-needed Indicates that a PR should not merge because it's missing one of the release note labels. labels Dec 14, 2018
@jpbetz
Copy link
Contributor

jpbetz commented Dec 14, 2018

xref #57073 where this issue was encountered

@jpbetz
Copy link
Contributor

jpbetz commented Dec 14, 2018

Confirmed the limit to be 1.5MB on 3.1+:

head -c $(((1024+512)*1024)) /dev/urandom | ETCDCTL_API=3 etcdctl put k
Error:  etcdserver: request is too large
head -c $(((1024+512-1)*1024)) /dev/urandom | ETCDCTL_API=3 etcdctl put k
OK

@jpbetz
Copy link
Contributor

jpbetz commented Dec 14, 2018

/lgtm

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Dec 14, 2018
@fejta-bot
Copy link

/retest
This bot automatically retries jobs that failed/flaked on approved PRs (send feedback to fejta).

Review the full test history for this PR.

Silence the bot with an /lgtm cancel comment for consistent failures.

@k8s-ci-robot k8s-ci-robot merged commit 5a5111f into kubernetes:master Dec 15, 2018
@k8s-ci-robot k8s-ci-robot added release-note Denotes a PR that will be considered when it comes time to generate release notes. and removed release-note-none Denotes a PR that doesn't merit a release note. labels Jan 17, 2019
k8s-ci-robot added a commit that referenced this pull request Jan 17, 2019
…53-upstream-release-1.13

Automated cherry pick of #72053 upstream release 1.13
k8s-ci-robot added a commit that referenced this pull request Jan 18, 2019
…53-upstream-release-1.11

Automated cherry pick of #72053 upstream release 1.11
k8s-ci-robot added a commit that referenced this pull request Jan 22, 2019
…53-upstream-release-1.12

Automated cherry pick of #72053 upstream release 1.12
@wojtek-t wojtek-t deleted the streaming_limit branch July 19, 2019 11:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. kind/bug Categorizes issue or PR as related to a bug. lgtm "Looks good to me", indicates that a PR is ready to be merged. priority/important-soon Must be staffed and worked on either currently, or very soon, ideally in time for the next release. release-note Denotes a PR that will be considered when it comes time to generate release notes. sig/api-machinery Categorizes an issue or PR as relevant to SIG API Machinery. size/XS Denotes a PR that changes 0-9 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

7 participants