Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Disable keep-alive for cloud-event connections #3201

Merged

Conversation

afrittoli
Copy link
Member

Changes

Disable keep alive forces the HTTP client to drop the connection
once a response is received. This avoids building up large numbers
of idle connections and it fixes the immediate issue.

After this we may want to see how to ensure we can re-use connection
and also set and idle-connection timeout.

Fixes #3190

Submitter Checklist

These are the criteria that every PR should meet, please check them off as you
review them:

  • [n/a] Includes tests (if functionality changed/added)
  • [n/a] Includes docs (if user facing)
  • Commit messages follow commit message best practices
  • Release notes block has been filled in or deleted (only if no user facing changes)

See the contribution guide for more details.

Double check this list of stuff that's easy to miss:

Reviewer Notes

If API changes are included, additive changes must be approved by at least two OWNERS and backwards incompatible changes must be approved by more than 50% of the OWNERS, and they must first be added in a backwards compatible way.

Release Notes

This release fixes an issue with the cloud event delivery that lead to an increasingly high number of open (idle) connections towards the configured target URL. The fix is achieved by disabling keep-alive, which means that - at least for now - we will be setting up a new connection for every cloud event. This is the current behaviour too, only now old connections will be closed immediatly.

/kind bug

@tekton-robot tekton-robot added release-note Denotes a PR that will be considered when it comes time to generate release notes. kind/bug Categorizes issue or PR as related to a bug. labels Sep 10, 2020
@tekton-robot tekton-robot added the size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. label Sep 10, 2020
@afrittoli afrittoli force-pushed the issues/3190-cloudevent-connections branch from f59af3c to 21fcc02 Compare September 10, 2020 23:46
@afrittoli afrittoli added the needs-cherry-pick Indicates a PR needs to be cherry-pick to a release branch label Sep 10, 2020
@afrittoli afrittoli added this to the Pipelines 0.16.1 Bug-fix milestone Sep 10, 2020
@afrittoli afrittoli force-pushed the issues/3190-cloudevent-connections branch from 21fcc02 to fe229e7 Compare September 10, 2020 23:53
@dibyom
Copy link
Member

dibyom commented Sep 11, 2020

This seems fine for now. We were already opening new connections for each cloud event delivery. This ensures that we close those connections instead of keeping them idle. In the future, would be nice to actually reuse the connections.
/lgtm

@tekton-robot tekton-robot added the lgtm Indicates that a PR is ready to be merged. label Sep 11, 2020
Copy link
Member

@vdemeester vdemeester left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm

Disable keep alive forces the HTTP client to drop the connection
once a response is received. This avoids building up large numbers
of idle connections and it fixes the immediate issue.

After this we may want to see how to ensure we can re-use connection
and also set and idle-connection timeout.

Fixes tektoncd#3190
@afrittoli afrittoli force-pushed the issues/3190-cloudevent-connections branch from fe229e7 to 999ad73 Compare September 11, 2020 09:43
@tekton-robot tekton-robot added size/S Denotes a PR that changes 10-29 lines, ignoring generated files. and removed lgtm Indicates that a PR is ready to be merged. size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. labels Sep 11, 2020
@afrittoli
Copy link
Member Author

Follow up feature: #3204

@vdemeester
Copy link
Member

/lgtm

@tekton-robot tekton-robot added the lgtm Indicates that a PR is ready to be merged. label Sep 11, 2020
@imjasonh
Copy link
Member

/approve

@tekton-robot
Copy link
Collaborator

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: ImJasonH

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@tekton-robot tekton-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Sep 11, 2020
@tekton-robot tekton-robot merged commit 7a9a85b into tektoncd:master Sep 11, 2020
@bobcatfish
Copy link
Collaborator

Cherry picking into 0.16.x in #3215

@bobcatfish bobcatfish removed the needs-cherry-pick Indicates a PR needs to be cherry-pick to a release branch label Sep 14, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. kind/bug Categorizes issue or PR as related to a bug. lgtm Indicates that a PR is ready to be merged. release-note Denotes a PR that will be considered when it comes time to generate release notes. size/S Denotes a PR that changes 10-29 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Cloud Events delivery leaks open connections
6 participants