Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add retrying to audit logging e2e tests #60794

Merged
merged 1 commit into from
Mar 6, 2018
Merged

Add retrying to audit logging e2e tests #60794

merged 1 commit into from
Mar 6, 2018

Conversation

crassirostris
Copy link

@crassirostris crassirostris commented Mar 5, 2018 •
edited
Loading

Fixes #60719

Adds retrying to the audit logging e2e tests so it can work when audit logging is in batch mode and actual writing is delayed.

NONE

/cc @tallclair @liggitt @sttts

@crassirostris crassirostris added kind/bug Categorizes issue or PR as related to a bug. priority/critical-urgent Highest priority. Must be actively worked on as someone's top priority right now. sig/auth Categorizes an issue or PR as relevant to SIG Auth. status/approved-for-milestone area/audit labels Mar 5, 2018
@crassirostris crassirostris added this to the v1.10 milestone Mar 5, 2018
@k8s-ci-robot k8s-ci-robot added release-note-none Denotes a PR that doesn't merit a release note. size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Mar 5, 2018
@thockin
Copy link
Member

thockin commented Mar 5, 2018

This change is Reviewable

@k8s-ci-robot k8s-ci-robot added the cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. label Mar 5, 2018
@k8s-github-robot
Copy link

[MILESTONENOTIFIER] Milestone Pull Request: Up-to-date for process

@crassirostris @liggitt @sttts @tallclair

Pull Request Labels
  • sig/auth: Pull Request will be escalated to these SIGs if needed.
  • priority/critical-urgent: Never automatically move pull request out of a release milestone; continually escalate to contributor and SIG through all available channels.
  • kind/bug: Fixes a bug discovered during the current release.
Help

@crassirostris crassirostris mentioned this pull request Mar 5, 2018
Closed
sttts
sttts reviewed Mar 6, 2018
View reviewed changes
test/e2e/auth/audit.go Outdated
@@ -35,12 +36,18 @@ import (

"github.com/evanphx/json-patch"
. "github.com/onsi/ginkgo"
. "github.com/onsi/gomega"
"k8s.io/apimachinery/pkg/util/wait"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: order

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

sttts
sttts reviewed Mar 6, 2018
View reviewed changes
test/e2e/auth/audit.go Outdated
auditTestUser = "kubecfg"

pollingInterval = 30 * time.Second
pollingTimeout = 5 * time.Minute
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

don't we inline these everywhere else?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

sttts
sttts reviewed Mar 6, 2018
View reviewed changes
test/e2e/auth/audit.go Outdated
@@ -630,7 +637,14 @@ var _ = SIGDescribe("Advanced Audit", func() {
expectedEvents = append(expectedEvents, t.events...)
}

expectAuditLines(f, expectedEvents)
err = wait.Poll(pollingInterval, pollingTimeout, func() (bool, error) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do we have any guarantees about the times?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added a comment

sttts
sttts reviewed Mar 6, 2018
View reviewed changes
test/e2e/auth/audit.go Outdated
@@ -648,33 +662,44 @@ type auditEvent struct {
}

// Search the audit log for the expected audit lines.
func expectAuditLines(f *framework.Framework, expected []auditEvent) {
func expectAuditLines(f *framework.Framework, expected []auditEvent) (bool, error) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should be called searchAuditLines or something like that. Expect implies that it fails on errors.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

crassirostris
crassirostris commented Mar 6, 2018
View reviewed changes
Copy link
Author

@crassirostris crassirostris left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, PTAL

test/e2e/auth/audit.go Outdated
@@ -35,12 +36,18 @@ import (

"github.com/evanphx/json-patch"
. "github.com/onsi/ginkgo"
. "github.com/onsi/gomega"
"k8s.io/apimachinery/pkg/util/wait"
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

test/e2e/auth/audit.go Outdated
auditTestUser = "kubecfg"

pollingInterval = 30 * time.Second
pollingTimeout = 5 * time.Minute
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

test/e2e/auth/audit.go Outdated
@@ -630,7 +637,14 @@ var _ = SIGDescribe("Advanced Audit", func() {
expectedEvents = append(expectedEvents, t.events...)
}

expectAuditLines(f, expectedEvents)
err = wait.Poll(pollingInterval, pollingTimeout, func() (bool, error) {
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added a comment

test/e2e/auth/audit.go Outdated
@@ -648,33 +662,44 @@ type auditEvent struct {
}

// Search the audit log for the expected audit lines.
func expectAuditLines(f *framework.Framework, expected []auditEvent) {
func expectAuditLines(f *framework.Framework, expected []auditEvent) (bool, error) {
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

sttts
sttts reviewed Mar 6, 2018
View reviewed changes
test/e2e/auth/audit.go Outdated
@@ -630,7 +631,18 @@ var _ = SIGDescribe("Advanced Audit", func() {
expectedEvents = append(expectedEvents, t.events...)
}

expectAuditLines(f, expectedEvents)
// The default flush timeout is 30 seconds, therefore 5 minutes should be enough to wait for
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What I really meant with my previous comment: don't we flush regularly even in batched mode? 5min sounds a lot.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, I clarified the comment: we're waiting for 5 minutes to avoid flakes, in most cases, one retry (30 seconds) should be enough

Add retrying to audit logging e2e tests
f327a2a 
Signed-off-by: Mik Vyatskov <vmik@google.com>
@sttts
Copy link
Contributor

sttts commented Mar 6, 2018

/lgtm
/approve

To fix the flakes.

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Mar 6, 2018
@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: crassirostris, sttts

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Mar 6, 2018
@k8s-github-robot
Copy link

/test all [submit-queue is verifying that this PR is safe to merge]

@crassirostris
Copy link
Author

/retest

@k8s-github-robot
Copy link

Automatic merge from submit-queue (batch tested with PRs 60630, 60794). If you want to cherry-pick this change to another branch, please follow the instructions here.

@k8s-github-robot k8s-github-robot merged commit a83aec0 into kubernetes:master Mar 6, 2018
@tallclair tallclair mentioned this pull request Mar 7, 2018
Closed
@crassirostris crassirostris mentioned this pull request Mar 13, 2018
Merged
k8s-github-robot pushed a commit that referenced this pull request Mar 15, 2018
Merge pull request #61134 from crassirostris/automated-cherry-pick-of…
51ce05a 
…-#60794-upstream-release-1.9

Automatic merge from submit-queue.

Automated cherry pick of #60794: Add retrying to audit logging e2e tests

Cherry pick of #60794 on release-1.9.

Fixes #60719, since audit logging behavior has changed in 1.10. Purely e2e change, so no release note

#60794: Add retrying to audit logging e2e tests

```release-note
NONE
```

/cc @sttts @liggitt
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@sttts sttts sttts left review comments

@liggitt liggitt Awaiting requested review from liggitt

@tallclair tallclair Awaiting requested review from tallclair

Assignees

@sttts sttts

@liggitt liggitt

@tallclair tallclair

Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. area/audit cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. kind/bug Categorizes issue or PR as related to a bug. lgtm "Looks good to me", indicates that a PR is ready to be merged. priority/critical-urgent Highest priority. Must be actively worked on as someone's top priority right now. release-note-none Denotes a PR that doesn't merit a release note. sig/auth Categorizes an issue or PR as relevant to SIG Auth. size/M Denotes a PR that changes 30-99 lines, ignoring generated files.
Projects
None yet
Milestone
v1.10
Development

Successfully merging this pull request may close these issues.

None yet
7 participants
@crassirostris @thockin @k8s-github-robot @sttts @k8s-ci-robot @liggitt @tallclair