New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

InputStreamSource should carry its own dispatcher #17928

Closed
jroper opened this Issue Jul 8, 2015 · 4 comments

Comments

Projects
None yet
5 participants
@jroper
Contributor

jroper commented Jul 8, 2015

From a discussion with @rkuhn:

One thing to be aware of is that the InputStreamSource should be configured to run on a dedicated dispatcher since it is of course blocking.

The problem with doing this is that you can't always control which materializer is used when the source is run. Consider Akka HTTP, a server is a stream of HttpRequests to HttpRespones, and an HttpResponse may have a Source for the entity - Akka HTTP materialises the source, the user has no control over that, so if the user wants to serve a file by returning an InputStreamSource, that is going to be materialised with the same dispatcher that the rest of Akka HTTP uses.

The built in file serving directives used by Akka HTTP actually does carry a dispatcher, it does this by creating its own source:

https://github.com/akka/akka/blob/releasing-akka-stream-and-http-experimental-1.0-RC4/akka-http-core/src/main/scala/akka/http/impl/util/StreamUtils.scala#L181

However, as the comment suggests, InputStreamSource is meant to replace that utility, but since it doesn't take its own dispatcher, it doesn't.

@jrudolph

This comment has been minimized.

Show comment
Hide comment
@jrudolph

jrudolph Jul 8, 2015

Member

Akka HTTP has recently been migrated to use InputStreamSource in #17900 and now also uses the configured dispatcher. So, it is possible to choose a dispatcher but the question is if it is prominently enough enforced for the InputStreamSource where the normal dispatcher should usually never be used to prevent it to become a source of thread starvation issues.

Member

jrudolph commented Jul 8, 2015

Akka HTTP has recently been migrated to use InputStreamSource in #17900 and now also uses the configured dispatcher. So, it is possible to choose a dispatcher but the question is if it is prominently enough enforced for the InputStreamSource where the normal dispatcher should usually never be used to prevent it to become a source of thread starvation issues.

@drewhk drewhk added 1 - triaged and removed 0 - new labels Jul 27, 2015

@drewhk drewhk added this to the http-1.1 milestone Jul 27, 2015

@drewhk

This comment has been minimized.

Show comment
Hide comment
@drewhk

drewhk Jul 27, 2015

Member

You can use the dispatcher attribute to use this. Once you set an attribute on a module, all of its submodules will inherit it, unless they override it themselves, see: http://doc.akka.io/docs/akka-stream-and-http-experimental/1.0/scala/stream-composition.html#Attributes

We should document this for IO that

  • users supposed to provide a dispatcher for blocking IO based sources
  • how attributes inheritance is involved
Member

drewhk commented Jul 27, 2015

You can use the dispatcher attribute to use this. Once you set an attribute on a module, all of its submodules will inherit it, unless they override it themselves, see: http://doc.akka.io/docs/akka-stream-and-http-experimental/1.0/scala/stream-composition.html#Attributes

We should document this for IO that

  • users supposed to provide a dispatcher for blocking IO based sources
  • how attributes inheritance is involved

@rkuhn rkuhn modified the milestones: streams-1.x, http-1.1 Oct 13, 2015

@rkuhn

This comment has been minimized.

Show comment
Hide comment
@rkuhn

rkuhn Dec 20, 2015

Collaborator

This has been done, right @2m @ktoso?

Collaborator

rkuhn commented Dec 20, 2015

This has been done, right @2m @ktoso?

@ktoso ktoso modified the milestones: streams-2.0, streams-backlog Dec 21, 2015

@ktoso

This comment has been minimized.

Show comment
Hide comment
@ktoso

ktoso Dec 22, 2015

Member

Yes, we set those:

    val inputStreamSource = name("inputStreamSource") and IODispatcher
    val outputStreamSource = name("outputStreamSource") and IODispatcher
    val outputStreamSink = name("outputStreamSink") and IODispatcher
    val inputStreamSink = name("inputStreamSink") and IODispatcher
    val fileSink = name("fileSource") and IODispatcher
Member

ktoso commented Dec 22, 2015

Yes, we set those:

    val inputStreamSource = name("inputStreamSource") and IODispatcher
    val outputStreamSource = name("outputStreamSource") and IODispatcher
    val outputStreamSink = name("outputStreamSink") and IODispatcher
    val inputStreamSink = name("inputStreamSink") and IODispatcher
    val fileSink = name("fileSource") and IODispatcher

@ktoso ktoso closed this Dec 22, 2015

@ktoso ktoso removed the 1 - triaged label Dec 22, 2015

@ktoso ktoso modified the milestones: http-2.0, streams-backlog, streams-2.0 Dec 22, 2015

@ktoso ktoso assigned ktoso and unassigned ktoso Dec 22, 2015

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment