Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

=htc #851 fix leaking AbsorbCancellation #852

Merged
merged 1 commit into from
Feb 16, 2017

Conversation

jrudolph
Copy link
Member

@jrudolph jrudolph commented Feb 8, 2017

Under circumstances that I could not reproduce so far,
the AbsorbCancellation stage introduced in #459 may keep the graph running
if data was already buffered but never read. The reason is that the stage
so far requires completion to reach the stage after cancellation was
absorbed which might be blocked by incoming elements that were never
pulled.

The solution is two-fold. After cancellation:

  1. pull in and ignore all incoming elements to eventually fetch completion
  2. complete the stage after a configurable time

In the HTTP use case, we delay the cancellation by no more than the time specified
in the new linger-timeout setting.

@akka-ci akka-ci added validating PR that is currently being validated by Jenkins needs-attention Indicates a PR validation failure (set by CI infrastructure) and removed validating PR that is currently being validated by Jenkins labels Feb 8, 2017
@akka-ci
Copy link

akka-ci commented Feb 8, 2017

Test FAILed.

# If the network level buffers (including the Akka Stream / Akka IO networking stack buffers)
# contains more data than can be transfered to the client in the given time when the server-side considers
# to be finished with this connection, the client may encounter a connection reset.
linger-timeout = 1 min
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should probably mention how to set it to infinite?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, good point.

@akka-ci akka-ci added validating PR that is currently being validated by Jenkins needs-attention Indicates a PR validation failure (set by CI infrastructure) and removed needs-attention Indicates a PR validation failure (set by CI infrastructure) validating PR that is currently being validated by Jenkins labels Feb 9, 2017
@akka-ci
Copy link

akka-ci commented Feb 9, 2017

Test FAILed.

@akka-ci akka-ci added validating PR that is currently being validated by Jenkins needs-attention Indicates a PR validation failure (set by CI infrastructure) and removed needs-attention Indicates a PR validation failure (set by CI infrastructure) validating PR that is currently being validated by Jenkins labels Feb 9, 2017
@akka-ci
Copy link

akka-ci commented Feb 9, 2017

Test FAILed.

log.debug(s"Stage was canceled after delay of $cancelAfter")
completeStage()
}
case _ ⇒
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I always like to add a // do nothing so it's clear the empty case is intentional

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok

case finite: FiniteDuration ⇒
scheduleOnce(finite) {
log.debug(s"Stage was canceled after delay of $cancelAfter")
completeStage()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok

def absorbCancellation[T]: Flow[T, T, NotUsed] = Flow.fromGraph(new AbsorbCancellationStage)
final class AbsorbCancellationStage[T] extends SimpleLinearGraphStage[T] {
def createLogic(inheritedAttributes: Attributes): GraphStageLogic = new GraphStageLogic(shape) with InHandler with OutHandler with StageLogging {
def delayCancellation[T](cancelAfter: Duration): Flow[T, T, NotUsed] = Flow.fromGraph(new DelayCancellationStage(cancelAfter))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok rename

#
# Set to 'infinite' to disable automatic connection closure (which will risk to leak connections).
linger-timeout = 1 min

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good docs, I'm curious if 1 minute is a good value... I guess it's ok

@ktoso
Copy link
Member

ktoso commented Feb 9, 2017

We have a docs page on timeouts in http, document this there too, so it's the "go-to" place for all timeouts, can be basically a copy of the reference.conf docs.

LGTM once added to docs

@jrudolph
Copy link
Member Author

Added to docs, will merge after validation.

@akka-ci akka-ci added validating PR that is currently being validated by Jenkins and removed needs-attention Indicates a PR validation failure (set by CI infrastructure) labels Feb 16, 2017
contains more data than can be transferred to the client in the given time when the server-side considers
to be finished with this connection, the client may encounter a connection reset.

Set to 'infinite' to disable automatic connection closure (which will risk to leak connections).
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

`infinite`

?

to be finished with this connection, the client may encounter a connection reset.

Set to 'infinite' to disable automatic connection closure (which will risk to leak connections).

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Docs LGTM

@akka-ci akka-ci added needs-attention Indicates a PR validation failure (set by CI infrastructure) and removed validating PR that is currently being validated by Jenkins labels Feb 16, 2017
@akka-ci
Copy link

akka-ci commented Feb 16, 2017

Test FAILed.

Under circumstances that I could not reproduce so far,
the AbsorbCancellation stage introduced in akka#459 may keep the graph running
if data was already buffered but never read. The reason is that the stage
so far requires completion to reach the stage after cancellation was
absorbed which might be blocked by incoming elements that were never
pulled.

The solution is two-fold. After cancellation:

 1) pull in and ignore all incoming elements to eventually fetch completion
 2) complete the stage after a configurable time

In the HTTP use case, we delay the cancellation by no more than the time specified
in the new linger-timeout setting.
@akka-ci akka-ci added validating PR that is currently being validated by Jenkins tested PR that was successfully built and tested by Jenkins and removed needs-attention Indicates a PR validation failure (set by CI infrastructure) validating PR that is currently being validated by Jenkins labels Feb 16, 2017
@akka-ci
Copy link

akka-ci commented Feb 16, 2017

Test PASSed.

@jrudolph jrudolph merged commit ba941e1 into akka:master Feb 16, 2017
@jrudolph jrudolph deleted the w/fix-absorb-cancellation branch February 16, 2017 13:18
@jadhavj
Copy link

jadhavj commented May 30, 2019

Is there an update on this? I've upgraded to Akka-Http 10.1.8 and Akka 2.5.22 but still facing the same issue.

@jrudolph
Copy link
Member Author

jrudolph commented Jun 4, 2019

Is there an update on this? I've upgraded to Akka-Http 10.1.8 and Akka 2.5.22 but still facing the same issue.

Hi @jadhavj, can you explain what issue you are still facing?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
tested PR that was successfully built and tested by Jenkins
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants