Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WIP: Introduce generalized abstraction over streams #571

Draft
wants to merge 3 commits into
base: master
Choose a base branch
from

Conversation

busti
Copy link
Collaborator

@busti busti commented Jan 19, 2022

This Pull Request aims to introduce a generalized abstraction over streams to outwatch via use of the hummingbird library. While this PR is still a work in progress, it could in theory make outwatch compatible with libraries such as zio and fs2 without the need to introduce their specific features into outwatch itself.

The concept works as if the entire library was wrapped into a trait that defines two abstract type parameters, Rx[-T] and Tx[+T]. These take the role of the Observer and the Observable respectively. The usual set of streaming functions is made available for these types via syntax imports. When working with a specific streaming library, the trait is extended and the abstract types are set to whichever streaming types are supported, i.e. fs2.Stream or zio.ZStream and so on.
While this example is nice to visualize the concept, in reality it is a bit more complicated since the concept would require us to write all of the libraries code into a singe file. Instead this PR creates a class called Env which provides the necessary environment everything is wrapped into it individually.

From a user perspective, this would require changing all imports to something along the lines of
import outwatch.ext.monix.dsl._, which would provide a version of the library with all types set to monix ones.

Even though I am still not done integrating this, I wanted to make this PR to get some feedback on the general idea.
I also expect scala 3 to make this concept a bit more usable, since a lot of the namespace crazyness can probably be avoided using export, trait parameters, opaque types and more.

Right now this requires the hummingbird library to be checked out into the parent folder of this repo, which makes it easier to develop on both libraries at once as opposed to using jitpack.


btw, I do not expect this to get merged as is, since I am wrapping every single file in outwatch in an entire class, which feels pretty insane, I don't like it. I just wanted to see where this concept takes me and this is where I wound up. I hope that I can find a way to manage this entirely via imports, but I don't know if that's possible so I have to ask around more on the scala channels first.

@busti busti changed the title [WIP] Introduce generalized abstraction over streams via environment imports Wip: Introduce generalized abstraction over streams via environment imports Jan 19, 2022
@busti busti changed the title Wip: Introduce generalized abstraction over streams via environment imports WIP: Introduce generalized abstraction over streams via environment imports Jan 19, 2022
@busti busti marked this pull request as draft January 20, 2022 00:34
@busti
Copy link
Collaborator Author

busti commented Jan 20, 2022

Turns out I just found a way to get rid of the wrappers and the Env class as well. It just requires some fiddling inside sbt.
Stay tuned

@busti busti changed the title WIP: Introduce generalized abstraction over streams via environment imports WIP: Introduce generalized abstraction over streams Jan 20, 2022
@fdietze
Copy link
Member

fdietze commented Jan 20, 2022

That whole topic sounds super interesting! Do you have a small code example of usage that you could share?

@busti
Copy link
Collaborator Author

busti commented Jan 20, 2022

@fdietze The goal here is to make outwatch use any streaming libraries types internally and expose those in every function.
This includes the following changes:

  • outwatch would be published for each streaming library individually
    • there would be no "com.github.outwatch" %%% "outwatch" % version anymore
      instead the user would just depend on "com.githuv.outwatch" %%% "outwatch-monix" % version, or "outwatch-zio", etc.
    • all outwatch functions using streaming types would automatically expose the used streaming libraries types.
      • i.e. using outwatch-colibri would expose EmitterBuilder.ofNode[E](create: Observer[E] => VNode) or
        using outwatch-scalarx would expose EmitterBuilder.ofNode[E](create: Var[E] => VNode),
      • Code like the following could work without any workarounds:
        import outwatch._
        import outwatch.dsl._
        
        def datePicker(sink: ZSink[Any, StringParseException, LocalDate, Nothing, Unit]): VNode =
          input(onInput.value.map(DateTime.parse) --> sink)
        The local zio error and environment is still a WIP, it can be done with macros scala 3, but I have no idea how to make it work in scala 2.
        In scala 2 I can right now only define a global zio environment and error. So the same R and E for the whole project.
        Outwatch internally does not do any environment specific stuff at all, so in theory we can just pass it through everywhere, typed errors are a harder issue to make optional.
  • Internally all colibri types are replaced by hummingbird ones.
    • Observer becomes Rx
    • Observable becomes Tx
      Other than that almost all code stays the same.
    • The core project depends on the hummingbird-dummy library.
      • Rx and Tx are type aliases which are set to empty traits in the dummy library.
      • When compiling outwatch artifacts for specific streaming libs the dummy library is replaced with a specific implementation, which overrides all types.
      • Providing a custom zio environment, or fs2 effect type would require users to compile outwatch themselves.
        In scala 3 this can be avoided using macros.
      • Rx and Tx have extension methods defined on them which allows to use generic streaming functions like map, sorted, foldLeft and whatnot on them, although the actual functions in the specific streaming libs might have different names.
      • At compile time all extension syntax methods can be inlined.
  • Support for using multiple streaming libraries at the same time in outwatch itself is dropped.
    • Using outwatch-colibri brings back type mixing since several streaming types can be implicitly converted to colibri types.
      input(onInput.value.flatMap(someScalaRxInstance) --> foo) is still valid because colibriObservable.flatMap(someScalaRxInstance) is valid.

@fdietze
Copy link
Member

fdietze commented Jan 21, 2022

Thank you for the detailed description! This is impressive. 👍

Would all outwatch artifacts still be fully colibri compatible?

@busti
Copy link
Collaborator Author

busti commented Jan 21, 2022

Would all outwatch artifacts still be fully colibri compatible?

No. Only the outwatch-colibri artifact would be compatible. All other artifacts completely switch out the internal library.
If everything goes to plan, the colibri artifact would retain all features and performance characteristics of the original. I cannot say how the others would perform until I have a working build, but chances are that all others will perform worse.
It will probably still be wise to use the colibri version with libraries that support it well, i.e. monix and scala.rx.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants