Join GitHub today
GitHub is home to over 28 million developers working together to host and review code, manage projects, and build software together.Sign up
Improve API for doing parallel execution #229
It would provide a much better user experience if we provide concurrent/parallel semantics out of the box, even with a perf hit of extra scheduling.
As a starting point, any of these changes would greatly improve using IO:
// An infix version of doing manual shift. Less cumbersome to use for a pretty common operation def shifted(implicit timer: Timer[IO]): IO[A] = timer.shift *> this // A `start` that shifts automatically. Typically it's what people would want def fork(implicit timer: Timer[IO]): IO[Fiber[IO, A]] = shifted.start
def raceFork[A, B](fa: IO[A], fb: IO[B])(implicit timer: Timer[IO]): IO[Either[A, B]] = IO.race(fa.shifted, fb)
(Alternatively, we can change the base methods and Concurrent[IO] instance to do that for us. Concurrent[IO] instance will then require Timer[IO])
I think that with a
I do not see an improvement in providing
And here I have a double standard. It is true that in Monix's
With Cats-Effect on the other hand we've got a bigger responsibility to keep the API stable. We cannot retract operations that we introduce so easily, because Cats-Effect will probably end up being used as a transitive dependency in more projects than fs2 or Monix combined.
Personally I don't think that
It's why I started using it in some instances in which I would have used
We must also consider that these problems only come up because of blocking
@alexandru I'll answer to your points in random order, hope you don't mind :)
Not quite. It's similar to what we had discussed previously about
What I see:
val crunchNumbers: IO[Long] = ??? // anything CPU-intensive. List.fill(6)(crunchNumbers).parSequence
What I think: execute things in parallel, using available system resources
List.fill(6)(crunchNumbers).parTraverse(IO.shift *> _)
What I see:
val ioA: IO[String] = ??? val ioB: IO[String] = ??? IO.race(ioA, ioB)
What I think: execute things in parallel, give me the first to complete, cancel the other.
IO.race(IO.shift *> ioA, ioB)
Shifts and cats.effect.IO role
Shifts are great, I think :) Super-handy for switching from polling thread-pool. For making fairness expicit, too. I don't think IO needs to be anyhow "smarter" about fairness.
For concurrency, I almost always want these. Now I think I know all the cases I need to do shifting explicitly, but it took me a while to learn and just today I misstepped with that
Yep. And also
Because newcomers will want to try doing stuff in parallel and then they would have to learn a concept of async boundaries in addition to already trying to learn purely functional IO. And async boundaries are not quite a trivial concept.
It's fine if there is a library with an effect type that does not do shifting for you. I don't think
We can forgo the
IMO this gives us a huge win for intuitiveness of making
So I still don’t like
Requiring Timer in IO’s instances is a solution but I’d would like buyin from fs2 and Http4s, as they are the biggest users of Concurrent. Yes, @SystemFw’s heart helps
Otherwise whatever we agree on is cool.