New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Iterant, version 2 #683

Merged
merged 102 commits into from Jul 4, 2018

Conversation

Projects
None yet
4 participants
@alexandru
Member

alexandru commented Jun 16, 2018

This is a collaboration between myself and @oleg-py and @Avasil for redefining Iterant in a way that is compatible with typelevel/cats-effect#237.

The feature is now feature complete and ready to be merged 🎊

TL;DR — this is a huge PR that modifies Iterant's encoding to be compatible with Cats-Effect's exposed cancellation model.

For example if you specify resources such as this ...

val resource = Iterant[Task].resource {
  Task(new BufferedReader(new FileReader(file)))
} { in =>
  Task(in.close())
}

// ...
resource.flatMap { in => Iterant.repeatEvalF(Task(in.readLine)) }
  .takeWhile(_ != null)
  .foldLeftL("")(_ + _)

Internally the Iterant now deals with resource handling via Bracket and so cancellation of resulting tasks (via Task, IO, etc) is safe and reusable with any effect type that supports the Cats-Effect type classes.

Iterant is a data structure, an ADT with this definition:

  sealed abstract class Iterant[F[_], A]

  case class Next[F[_], A](item: A, rest: F[Iterant[F, A]])
    extends Iterant[F, A]

  case class Last[F[_], A](item: A)
    extends Iterant[F, A]

  case class NextCursor[F[_], A](cursor: BatchCursor[A], rest: F[Iterant[F, A]])
    extends Iterant[F, A]

  case class NextBatch[F[_], A](batch: Batch[A], rest: F[Iterant[F, A]])
    extends Iterant[F, A]

  case class Suspend[F[_], A](rest: F[Iterant[F, A]])
    extends Iterant[F, A]

  case class Halt[F[_], A](e: Option[Throwable])
    extends Iterant[F, A]

  case class Scope[F[_], A, B](
    acquire: F[A],
    use: A => F[Iterant[F, B]],
    release: (A, ExitCase[Throwable]) => F[Unit])
    extends Iterant[F, B]

  case class Concat[F[_], A](lh: F[Iterant[F, A]], rh: F[Iterant[F, A]])
    extends Iterant[F, A] 

What's new with this encoding:

  1. the concept of earlyStop is gone from the nodes, being replaced by Scope
  2. Scope is a new data constructor that mirrors bracket
  3. Concat is a new data constructor that's meant for doing concatenation (++), needed due to the introduction of Scope

In order to keep the sub-project honest, monix-tail (exposing Iterant) no longer depends on monix-eval (exposing Task). This prevents us to take Task-related shortcuts, plus the monix-tail sub-project is now lighter and only depends on monix-execution (for low-level JVM utilities).

Performance has improved across the board due to internal usage of the "Visitor pattern", for eliminating memory allocations when interpreting the Interant data structure and due to introducing the Concat state. In many cases the improvement is ten fold.

oleg-py and others added some commits May 21, 2018

IterantFoldLeft progress
Signed-off-by: Piotr Gawryś <pgawrys2@gmail.com>
Restore fromIndexedSeq, fromIterable, fromList
Signed-off-by: Avasil <pgawrys2@gmail.com>

alexandru added some commits Jun 16, 2018

@alexandru alexandru changed the title from WIP: Iterant, version 2 to Iterant, version 2 Jul 3, 2018

@codecov

This comment has been minimized.

codecov bot commented Jul 3, 2018

Codecov Report

Merging #683 into master will decrease coverage by 5.93%.
The diff coverage is 93.81%.

@@            Coverage Diff             @@
##           master     #683      +/-   ##
==========================================
- Coverage   91.02%   85.09%   -5.94%     
==========================================
  Files         384      386       +2     
  Lines       10494    11645    +1151     
  Branches     1969     2136     +167     
==========================================
+ Hits         9552     9909     +357     
- Misses        942     1736     +794

@alexandru alexandru merged commit e98d64d into master Jul 4, 2018

2 checks passed

continuous-integration/travis-ci/pr The Travis CI build passed
Details
continuous-integration/travis-ci/push The Travis CI build passed
Details
@volth

This comment has been minimized.

Contributor

volth commented Jul 23, 2018

Could you release 3.0.0-RC2 including these changes?

@alexandru

This comment has been minimized.

Member

alexandru commented Jul 23, 2018

@volth I'm back from vacation and working on it.

@alexandru alexandru deleted the iterant2 branch Oct 25, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment