Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Limit the batch physical size in batching audit backend #60432

Closed
crassirostris opened this issue Feb 26, 2018 · 2 comments
Closed

Limit the batch physical size in batching audit backend #60432

crassirostris opened this issue Feb 26, 2018 · 2 comments
Labels
area/audit kind/feature Categorizes issue or PR as related to a new feature. priority/important-longterm Important over the long term, but may not be staffed and/or may need multiple releases to complete. sig/api-machinery Categorizes an issue or PR as relevant to SIG API Machinery. sig/auth Categorizes an issue or PR as relevant to SIG Auth.
Milestone
v1.11

Comments

@crassirostris
Copy link

Follow-up from #60076 (comment)

Depending on the type of request, the physical size in bytes can vary a lot, from 10s of bytes to megabytes. Currently buffered audit backend batches requests without taking this into account, which can result in super-heavy requests to the webhook, that have hight chance to timeout and be dropped as a result.

One possible way to implement this would be optionally serializing requests before sending them to the delegate audit backend and measuring the size.

/cc @sttts @ericchiang @CaoShuFeng @tallclair @soltysh
@crassirostris crassirostris added sig/api-machinery Categorizes an issue or PR as relevant to SIG API Machinery. kind/feature Categorizes issue or PR as related to a new feature. sig/auth Categorizes an issue or PR as relevant to SIG Auth. priority/important-longterm Important over the long term, but may not be staffed and/or may need multiple releases to complete. area/audit labels Feb 26, 2018
@crassirostris crassirostris added this to the v1.11 milestone Feb 26, 2018
@crassirostris crassirostris mentioned this issue Feb 26, 2018
Closed
14 tasks
@ericchiang
Copy link
Contributor

/sig auth
/kind feature
/milestone v1.11
/priority important-longterm
/status approved-for-milestone

@tallclair tallclair mentioned this issue Mar 21, 2018
Closed
@crassirostris crassirostris mentioned this issue Mar 26, 2018
Merged
@k8s-github-robot
Copy link

[MILESTONENOTIFIER] Milestone Issue: Up-to-date for process

@crassirostris

Issue Labels
  • sig/api-machinery sig/auth: Issue will be escalated to these SIGs if needed.
  • priority/important-longterm: Escalate to the issue owners; move out of the milestone after 1 attempt.
  • kind/feature: New functionality.
Help

k8s-publishing-bot added a commit to kubernetes/apiserver that referenced this issue Apr 25, 2018
@k8s-publishing-bot
Merge pull request #61711 from crassirostris/audit-size-limiting
0406d05 
Automatic merge from submit-queue (batch tested with PRs 62655, 61711, 59122, 62853, 62390). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Implemented truncating audit backend

Fixes kubernetes/kubernetes#60432

Introduces an optional truncating backend, disabled by default, that estimates the size of audit events and truncates events/split batches based on the configuration.

/cc @sttts @tallclair @CaoShuFeng @ericchiang

```release-note
Introduce truncating audit backend that can be enabled for existing backend to limit the size of individual audit events and batches of events.
```

Kubernetes-commit: bf1974c83fb36ab07c3efe615000d129995fa07f
k8s-publishing-bot added a commit to kubernetes/kube-aggregator that referenced this issue Apr 25, 2018
@k8s-publishing-bot
Merge pull request #61711 from crassirostris/audit-size-limiting
0eed14e 
Automatic merge from submit-queue (batch tested with PRs 62655, 61711, 59122, 62853, 62390). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Implemented truncating audit backend

Fixes kubernetes/kubernetes#60432

Introduces an optional truncating backend, disabled by default, that estimates the size of audit events and truncates events/split batches based on the configuration.

/cc @sttts @tallclair @CaoShuFeng @ericchiang

```release-note
Introduce truncating audit backend that can be enabled for existing backend to limit the size of individual audit events and batches of events.
```

Kubernetes-commit: bf1974c83fb36ab07c3efe615000d129995fa07f
k8s-publishing-bot added a commit to kubernetes/sample-apiserver that referenced this issue Apr 25, 2018
@k8s-publishing-bot
Merge pull request #61711 from crassirostris/audit-size-limiting
9ed9eba 
Automatic merge from submit-queue (batch tested with PRs 62655, 61711, 59122, 62853, 62390). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Implemented truncating audit backend

Fixes kubernetes/kubernetes#60432

Introduces an optional truncating backend, disabled by default, that estimates the size of audit events and truncates events/split batches based on the configuration.

/cc @sttts @tallclair @CaoShuFeng @ericchiang

```release-note
Introduce truncating audit backend that can be enabled for existing backend to limit the size of individual audit events and batches of events.
```

Kubernetes-commit: bf1974c83fb36ab07c3efe615000d129995fa07f
k8s-publishing-bot added a commit to kubernetes/apiextensions-apiserver that referenced this issue Apr 25, 2018
@k8s-publishing-bot
Merge pull request #61711 from crassirostris/audit-size-limiting
0757035 
Automatic merge from submit-queue (batch tested with PRs 62655, 61711, 59122, 62853, 62390). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Implemented truncating audit backend

Fixes kubernetes/kubernetes#60432

Introduces an optional truncating backend, disabled by default, that estimates the size of audit events and truncates events/split batches based on the configuration.

/cc @sttts @tallclair @CaoShuFeng @ericchiang

```release-note
Introduce truncating audit backend that can be enabled for existing backend to limit the size of individual audit events and batches of events.
```

Kubernetes-commit: bf1974c83fb36ab07c3efe615000d129995fa07f
This was referenced May 17, 2018
Closed
Merged
k8s-github-robot pushed a commit that referenced this issue May 18, 2018
Merge pull request #64024 from loburm/truncate
54b3176 
Automatic merge from submit-queue.

Implemented truncating audit backend

Fixes #60432

Introduces an optional truncating backend, disabled by default, that estimates the size of audit events and truncates events/split batches based on the configuration.

```release-note
Introduce truncating audit backend that can be enabled by passing --audit-log-truncate-enabled or --audit-webhook-truncate-enabled flag to the apiserver to limit the size of individual audit events and batches of events.
```

I had to manually remove dependency of original PR #61711, from #60056, that's why automated cherry-pick was not used.
openshift-publish-robot pushed a commit to openshift/kubernetes-sample-apiserver that referenced this issue Jan 14, 2019
@k8s-publishing-bot
Merge pull request #61711 from crassirostris/audit-size-limiting
db62341 
Automatic merge from submit-queue (batch tested with PRs 62655, 61711, 59122, 62853, 62390). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Implemented truncating audit backend

Fixes kubernetes/kubernetes#60432

Introduces an optional truncating backend, disabled by default, that estimates the size of audit events and truncates events/split batches based on the configuration.

/cc @sttts @tallclair @CaoShuFeng @ericchiang

```release-note
Introduce truncating audit backend that can be enabled for existing backend to limit the size of individual audit events and batches of events.
```

Kubernetes-commit: bf1974c83fb36ab07c3efe615000d129995fa07f
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
area/audit kind/feature Categorizes issue or PR as related to a new feature. priority/important-longterm Important over the long term, but may not be staffed and/or may need multiple releases to complete. sig/api-machinery Categorizes an issue or PR as relevant to SIG API Machinery. sig/auth Categorizes an issue or PR as relevant to SIG Auth.
Projects
None yet
Milestone
v1.11
Development

No branches or pull requests
4 participants
@crassirostris @ericchiang @k8s-github-robot @k8s-ci-robot