New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SourceWithContext and FlowWithContext #25951

Merged
merged 2 commits into from Jan 18, 2019

Conversation

Projects
None yet
10 participants
@RayRoestenburg
Copy link
Contributor

RayRoestenburg commented Nov 19, 2018

Based on earlier work by Johannes

  • More docs
  • Java version

I have added ApiMayChange @patriknw @jrudolph and processed @2m comments. Let me know if anything more is required for now.

@akka-ci

This comment has been minimized.

Copy link
Collaborator

akka-ci commented Nov 19, 2018

Thank you for your pull request! After a quick sanity check one of the team will reply with 'OK TO TEST' to kick off our automated validation on Jenkins. This compiles the project, runs the tests, and checks for things like binary compatibility and source code formatting. When two team members have also manually reviewed and (perhaps after asking for some amendments) accepted your contribution, it should be good to be merged.

For more details about our contributing process, check out CONTRIBUTING.md - and feel free to ask!

@chbatey

This comment has been minimized.

Copy link
Member

chbatey commented Nov 19, 2018

OK TO TEST

@akka-ci akka-ci added validating tested and removed validating labels Nov 19, 2018

@akka-ci

This comment has been minimized.

Copy link
Collaborator

akka-ci commented Nov 19, 2018

Test PASSed.

@akka-ci

This comment has been minimized.

Copy link
Collaborator

akka-ci commented Nov 19, 2018

Test PASSed.

1 similar comment
@akka-ci

This comment has been minimized.

Copy link
Collaborator

akka-ci commented Nov 19, 2018

Test PASSed.

@akka-ci

This comment has been minimized.

Copy link
Collaborator

akka-ci commented Nov 19, 2018

Test PASSed.

@akka-ci

This comment has been minimized.

Copy link
Collaborator

akka-ci commented Nov 19, 2018

Test FAILed.

@akka-ci

This comment has been minimized.

Copy link
Collaborator

akka-ci commented Nov 19, 2018

Test FAILed.

@jrudolph

This comment has been minimized.

Copy link
Member

jrudolph commented Jan 15, 2019

Added a few concrete proposals here: RayRoestenburg#1

@jrudolph

This comment has been minimized.

Copy link
Member

jrudolph commented Jan 17, 2019

I cleaned up things and added scaladoc everywhere. From my side this is ready to go with this first set of supported operations.

@akka-ci akka-ci added validating and removed needs-attention labels Jan 17, 2019

* accessing the delegate
*/
@InternalApi
private[stream] abstract class GraphDelegate[+S <: Shape, +Mat](delegate: Graph[S, Mat]) extends Graph[S, Mat] {

This comment has been minimized.

@RayRoestenburg

RayRoestenburg Jan 17, 2019

Author Contributor

nice!

@RayRoestenburg

This comment has been minimized.

Copy link
Contributor Author

RayRoestenburg commented Jan 17, 2019

@jrudolph nice simplifications and cleanups!

@akka-ci akka-ci added tested and removed validating labels Jan 17, 2019

@akka-ci

This comment has been minimized.

Copy link
Collaborator

akka-ci commented Jan 17, 2019

Test PASSed.

@patriknw patriknw changed the title WIP: Initial SourceWithContext, FlowWithContext, FlowWithContextOps. Initial SourceWithContext, FlowWithContext, FlowWithContextOps. Jan 18, 2019

@patriknw patriknw removed the wip label Jan 18, 2019

@patriknw
Copy link
Member

patriknw left a comment

LGTM, I have asked @RayRoestenburg to squash into 2 commits (Ray/Johannes) and then we can merge this.

@RayRoestenburg RayRoestenburg force-pushed the RayRoestenburg:wip-context-operators branch from 215be72 to a2c811d Jan 18, 2019

@akka-ci

This comment has been minimized.

Copy link
Collaborator

akka-ci commented Jan 18, 2019

Test PASSed.

@RayRoestenburg RayRoestenburg changed the title Initial SourceWithContext, FlowWithContext, FlowWithContextOps. SourceWithContext and FlowWithContext Jan 18, 2019

@patriknw patriknw merged commit a920349 into akka:master Jan 18, 2019

3 checks passed

Jenkins PR Validation Test PASSed. 4625 tests run, 491 skipped, 0 failed.
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details
typesafe-cla-validator All users have signed the CLA
Details
@patriknw

This comment has been minimized.

Copy link
Member

patriknw commented Jan 18, 2019

Great work @RayRoestenburg and @jrudolph

@patriknw patriknw added this to the 2.5.20 milestone Jan 18, 2019

/**
* Apply the given function to each context element (leaving the data elements unchanged).
*/
def mapContext[Ctx2](f: Ctx Ctx2): Repr[Ctx2, Out] =

This comment has been minimized.

@hepin1989

hepin1989 Jan 18, 2019

Contributor

nice too

@RayRoestenburg RayRoestenburg deleted the RayRoestenburg:wip-context-operators branch Jan 18, 2019

@jroper

This comment has been minimized.

Copy link
Contributor

jroper commented Jan 30, 2019

Sorry for the late review, I only just learnt about this.

I suspect a common pattern might be where users don't care about what the context is, they just want a flow that carries whatever context the container (eg, pipelines, or Akka projections) wants to provide. Then, the same flow can be used in multiple situations with different contexts, eg carrying a Kafka context in production, or carrying some dummy context in tests.

An example would be in the Akka projections library, when you define a projection, the context is irrelevant. With this API, I could imagine Akka projections might provide something that looks like this:

trait Projection[In, Out] {
  def projection[Ctx]: FlowWithContext[In, Ctx, Out, Ctx, _]
}

And then the user would implement it like this:

class MyProjection extends Projection[MyIn, MyOut] {
  override def projection[Ctx] = FlowWithContext[MyIn, Ctx].map(myInToMyOut)
}

The question for me then is, can we possibly drop the context type variable somehow? It's not so bad for Scala, but for Java the signatures have the potential to be quite overbearing. I realise that the reason the context type variable is needed is to provide via, but if we made a decision that the context is always opaque within a FlowWithContext, then via could potentially be provided via some mechanism like this:

trait ViaWithContext[In, Out, Mat] {
  def apply[Ctx]: Flow[(In, Ctx), (Out, Ctx), Mat]
}

def via[Out, Mat](flow: ViaWithContext[In, Out, Mat]): FlowWithContext[In, Out, Mat]

And maybe that could be implemented with a lambda because it's a SAM? The same might work in Java? I'm not sure, the ViaWithContext trait only exists essentially as a type constructor, which might be a little confusing to users, but I think the source code should look simple enough.

@jrudolph

This comment has been minimized.

Copy link
Member

jrudolph commented Jan 30, 2019

@jroper

I agree that a FlowWithContext without a fixed context would be a good thing. I had a try at getting that to work but it was non-trivial to get right so we settled for the straight-forward API for now (even if that means extra type parameters).

In any case, we'd like to keep the API surface minimal. If we want a generic FlowWithContext that should be the only API and we should be able to implement it without having to copy all implementations from FlowWithContextOps.

If we can achieve that, I'm all for doing it, it's not to late to iterate on the API (and @raboof also had ideas for improvements he wants to try out at some point).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment