Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix the race condition on closing eventC #1726

Merged
merged 1 commit into from
Sep 28, 2022

Conversation

sudo-bmitch
Copy link
Contributor

Signed-off-by: Brandon Mitchell git@bmitch.net

Changes

Fixes #1650, alternative to #1722.
This fixes a race condition when eventC is closed before the Informer has stopped sending events. It adds a mutex to stop the races and checks for the stopC channel to be closed before defaulting to send data on eventC. Tests were not added because testing race conditions buried deep in k8s interfaces are difficult to reproduce.

This also cleaned up a minor linter with a redundant include.

Submitter Checklist

These are the criteria that every PR should meet, please check them off as you
review them:

  • Includes tests (if functionality changed/added)
  • Run the code checkers with make check
  • Regenerate the manpages, docs and go formatting with make generated
  • Commit messages follow commit message best practices

See the contribution guide
for more details.

Release Notes

Fix "send on closed channel" panic when viewing logs

Fixes tektoncd#1650

The "defer close(eventC)" results in a race condition with the Informer.
This wraps channel operations with a mutex and checks stopC before
sending on eventC. Adding the mutex is slightly slower than a channel
alone.

Other options include skipping the close on eventC which would leak
memory. Ideally, the Informer would close the channel when stopping.

Signed-off-by: Brandon Mitchell <git@bmitch.net>
@tekton-robot tekton-robot added the release-note Denotes a PR that will be considered when it comes time to generate release notes. label Sep 25, 2022
@tekton-robot tekton-robot added the needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. label Sep 25, 2022
@tekton-robot
Copy link
Contributor

Hi @sudo-bmitch. Thanks for your PR.

I'm waiting for a tektoncd member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@tekton-robot tekton-robot added the size/M Denotes a PR that changes 30-99 lines, ignoring generated files. label Sep 25, 2022
@sudo-bmitch sudo-bmitch mentioned this pull request Sep 25, 2022
4 tasks
@pradeepitm12
Copy link
Contributor

/ok-to-test

@tekton-robot tekton-robot added ok-to-test Indicates a non-member PR verified by an org member that is safe to test. and removed needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels Sep 26, 2022
@sudo-bmitch
Copy link
Contributor Author

Waiting on a fix to tektoncd/pipeline#5537

@piyush-garg
Copy link
Contributor

/retest

2 similar comments
@vdemeester
Copy link
Member

/retest

@pradeepitm12
Copy link
Contributor

/retest

@piyush-garg
Copy link
Contributor

/lgtm
/retest

@tekton-robot tekton-robot added the lgtm Indicates that a PR is ready to be merged. label Sep 28, 2022
@vdemeester
Copy link
Member

/approve

@tekton-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: vdemeester

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@tekton-robot tekton-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Sep 28, 2022
@tekton-robot tekton-robot merged commit d6bdd59 into tektoncd:main Sep 28, 2022
@sudo-bmitch sudo-bmitch deleted the pr-panic-channel branch September 28, 2022 15:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. lgtm Indicates that a PR is ready to be merged. ok-to-test Indicates a non-member PR verified by an org member that is safe to test. release-note Denotes a PR that will be considered when it comes time to generate release notes. size/M Denotes a PR that changes 30-99 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

panic: "send on closed channel" when viewing logs
5 participants