Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor CloudEvent related tests to use common set of helpers #4387

Merged
merged 1 commit into from
Dec 3, 2021

Conversation

lbernick
Copy link
Member

@lbernick lbernick commented Nov 18, 2021

Prior to this change, the event-related testing helpers were copied
and pasted in several files. The TaskRun Reconcile tests were updated
to not expect cloud events in a particular order, reducing flakiness,
but the PipelineRun reconcile tests and the Event tests did not receive
the same updates.

This commit moves event-related test helpers into a common package
and updates all CloudEvent tests to not expect events to occur in a given order.
This should address some of the causes of flakes in #2992 but not all.

Co-authored-by: Scott Seaward sbws@google.com

Changes

/kind cleanup
/kind flake

Submitter Checklist

As the author of this PR, please check off the items in this checklist:

  • Docs included if any changes are user facing
  • Tests included if any functionality added or changed
  • Follows the commit message standard
  • Meets the Tekton contributor standards (including
    functionality, content, code)
  • Release notes block below has been filled in or deleted (only if no user facing changes)

Release Notes

NONE

@tekton-robot tekton-robot added release-note-none Denotes a PR that doesnt merit a release note. kind/cleanup Categorizes issue or PR as related to cleaning up code, process, or technical debt. kind/flake Categorizes issue or PR as related to a flakey test labels Nov 18, 2021
@tekton-robot tekton-robot added the size/L Denotes a PR that changes 100-499 lines, ignoring generated files. label Nov 18, 2021
@tekton-robot
Copy link
Collaborator

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
pkg/reconciler/events/cloudevent/testing.go Do not exist 76.0%

@tekton-robot
Copy link
Collaborator

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
pkg/reconciler/events/cloudevent/testing.go Do not exist 76.0%

@lbernick
Copy link
Member Author

/remove-kind cleanup
/remove-kind flake

@tekton-robot tekton-robot removed kind/cleanup Categorizes issue or PR as related to cleaning up code, process, or technical debt. kind/flake Categorizes issue or PR as related to a flakey test labels Nov 19, 2021
@lbernick
Copy link
Member Author

/kind cleanup
/kind flake

@tekton-robot tekton-robot added kind/cleanup Categorizes issue or PR as related to cleaning up code, process, or technical debt. kind/flake Categorizes issue or PR as related to a flakey test labels Nov 19, 2021
@lbernick
Copy link
Member Author

/remove-kind cleanup

@tekton-robot tekton-robot removed the kind/cleanup Categorizes issue or PR as related to cleaning up code, process, or technical debt. label Nov 19, 2021
Copy link
Member

@afrittoli afrittoli left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, nice, thank you for this!
/approve

@tekton-robot
Copy link
Collaborator

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: afrittoli

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@tekton-robot tekton-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Nov 24, 2021
@tekton-robot
Copy link
Collaborator

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
pkg/reconciler/events/cloudevent/testing.go Do not exist 76.0%

pkg/reconciler/events/cloudevent/testing.go Outdated Show resolved Hide resolved

// CheckEvents checks that the events received by the FakeRecorder are the same as wantEvents,
// in the same order.
func CheckEvents(t *testing.T, fr *record.FakeRecorder, testName string, wantEvents []string) error {
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

just realised that cloudevents.CheckEvents is not checking cloudevents, which seems like a bit of a misnomer given the package's name. Suggest moving this func to the pkg/reconciler/events package if possible.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree that ideally we'd have CheckEvents in the events package and CheckCloudEvents in the cloudevents package. However it looks like the events package is already doing some cloudevent related things and relies on checkCloudEvents in tests, which would cause a circular import if I split up tests in this way.
It seems like the events package may just need to be refactored but that's probably out of scope for this PR-- I'll create an issue to track.

Prior to this change, the event-related testing helpers were copied
and pasted in several files. The TaskRun Reconcile tests were updated
to not expect cloud events in a particular order, reducing flakiness,
but the PipelineRun reconcile tests and the Event tests did not receive
the same updates.

This commit moves event-related test helpers into a common package
and updates all CloudEvent tests to not expect events to occur in a given order.
This should address some of the causes of flakes in tektoncd#2992 but not all.

Co-authored-by: Scott Seaward sbws@google.com
@tekton-robot
Copy link
Collaborator

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
pkg/reconciler/events/cloudevent/testing.go Do not exist 76.0%

@ghost
Copy link

ghost commented Dec 3, 2021

/lgtm

@tekton-robot tekton-robot assigned ghost Dec 3, 2021
@tekton-robot tekton-robot added the lgtm Indicates that a PR is ready to be merged. label Dec 3, 2021
@tekton-robot tekton-robot merged commit 07adee7 into tektoncd:main Dec 3, 2021
@bobcatfish
Copy link
Collaborator

hey @sbwsg @lbernick it looks like this is adding test code that will be publicly exported as part of the cloudevent package, am i understanding right? if so can we move this so it is either in a testing specific package, or not publicly exported?

@ghost
Copy link

ghost commented Dec 3, 2021

@bobcatfish Sure thing, moved under test/: #4405

@lbernick lbernick deleted the flake branch January 24, 2022 15:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. kind/flake Categorizes issue or PR as related to a flakey test lgtm Indicates that a PR is ready to be merged. release-note-none Denotes a PR that doesnt merit a release note. size/L Denotes a PR that changes 100-499 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants