Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Composition of applicative functions #161

Closed
wants to merge 8 commits into from

Conversation

@eed3si9n
Copy link
Contributor

commented Sep 30, 2012

see ML thread.

steps

The private ProductApplicative is a typeclass instance, but it doesn't have a way of retaining itself as a value because it uses pair to hold the data as Applicative[({type λ[α] = (F[α], G[α])})#λ]. So we have half of , but we are missing the (⊗) operator.

what this changes

This adds A => M[B] wrapper called AppFunc that implements andThenA, composeA, and productA methods that implements applicative composition with symbolic aliases @>>>, <<<@, and @&&& respectively.

Using these operators, we can now compose applicative functions:

    // To count characters, treat Int as monoidal applicative
    val countChar = AppFuncU { (c: Char) => 1 }

    // To count lines, treat Int as monoidal applicative
    val countLine = AppFuncU { (c: Char) => test(c === '\n') }

    // To count words, we need to detect transitions from whitespace to non-whitespace.
    val countWord = AppFuncU { (c: Char) =>
      for {
        x <- get[Boolean]
        val y = c =/= ' '
        _ <- put(y)
      } yield test(y && !x)
    } @>>> AppFuncU { (x: Int) => x }

    // Compose applicative functions in parallel
    val countAll = countWord @&&& countLine @&&& countChar

    // ... and execute them in a single traversal 
    val ((wordCountState, lineCount), charCount) = countAll traverse text

The product and composition uses the existing ProductApplicative and CompositionApplicative internally. Since I did not touch existing code other than the word count example, it's unlikely this breaks the existing code.

limitations

Each composition of AppFuncs returns more complex AppFunc, which breaks down the type inference on @&&& or @>>> operators if a composite comes on rhs.

@larsrh

This comment has been minimized.

Copy link
Contributor

commented Sep 30, 2012

I'm on vacation right now, so I don't have much time to review this PR. Here's my cursory opinion:

  • Unsure why AppFunc is there, and why it wraps A => F[B].
  • It partly duplicates efforts in scalaz-typelevel. There's already a KTypeClass type class which abstracts over type classes like Applicative. It retains itself as you described such that it is possible to make a product of multiple instances (there's an example in scalaz-example). Right now, it doesn't handle composition, but there's no reason it couldn't do that as well. In fact, there was this capability in the original design, but I dropped it, because I figured that the existing compose methods are fine. I'm open to revert that, though.

I will try to find some time to investigate. Until then, I'm not in favour of merging.

@eed3si9n

This comment has been minimized.

Copy link
Contributor Author

commented Sep 30, 2012

Apparently, there's a bug in posting Unicode characters, so I am going to substitute some of them with PROD, PRODT, etc.

@larsrh Thanks for your input. The linked ML thread might explain my motivation, but I'll try to explain here again.

Suppose you have two applicative functions f and g:

scala> val f = { (x: Int) => x + 1 }
f: Int => Int = <function1>

scala> val g = { (x: Int) => List(x, 5) }
g: Int => List[Int] = <function1>

Int above is indirectly an applicative too because all monoids are an applicative via Monoid[Int].applicative. My claim is that in Scalaz today it's not possible to compose/product applicative functions the way it is described in The Essence of Iterator Pattern (EIP) paper.

EIP mentions operators that can be used to compose two applicative functions that returns another. These are provided so we can traverse the data structure once and get composed results:

data (m PRODT n) a = Prod { pfst::m a, psnd::n a }
(PROD) :: (Functor m, Functor n) => (a -> m b) -> (a -> n b) -> (a -> (m PRODT n) b)
(f PROD g) x = Prod(f x)(g x)

and

data (m COMPT n) a = Comp { unComp::m (n a) }
(COMP) :: (Functor n, Functor m) => (b -> n c) -> (a -> m b) -> (a -> (m COMPT n) c)
f COMP g = Comp DOT fmap f DOT g

Scalaz is able to calculate the product of two typeclass instances as follows:

scala> Monoid[Int].applicative product Applicative[List]
res0: scalaz.Applicative[[α](Int, List[α])] = scalaz.Applicative$$anon$2@277b9404

But what I am concerned with is composing values of arbitrary applicative functions and deriving types automatically. After all the point of modularity is to be able to treat each modules as black boxes. See for example the existing Word Count example:

    // To count, we traverse with a function returning 0 or 1, and sum the results
    // with Monoid[Int], packaged in a constant monoidal applicative functor.
    val Count = Monoid[Int].applicative

    // Compose the applicative instance for [a]State[Int,a] with the Count applicative
    val WordCount = StateT.stateMonad[Int].compose[({type λ[α] = Int})#λ](Count)

    // Fuse the three applicatives together in parallel...
    val A = Count
      .product[({type λ[α] = Int})#λ](Count)
      .product[({type λ[α] = State[Int, Int]})#λ](WordCount)

    // ... and execute them in a single traversal
    val ((charCount, lineCount), wordCountState) = A.traverse(text)((c: Char) => ((1, test(c == '\n')), atWordStart(c)))

The composition is done at type-level, but implementation is essentially written out at the end. Compare this with the original Word Count example from EIP paper:

clwci :: String -> ((Count PRODT Count) PRODT (M (State Bool) COMPT Count)) [ a ]
clwci = traverse (cciBody PROD lciBody PROD wciBody)

As you can see using (PROD) operator, it's composing functions at the value level, deriving types automatically. Here's an excerpt of my Word Count rewrite:

    // To count characters, treat Int as monoidal applicative
    val countChar = AppFuncU { (c: Char) => 1 }

    ...

    // Compose applicative functions in parallel
    val countAll = countWord @&&& countLine @&&& countChar

    // ... and execute them in a single traversal 
    val ((wordCountState, lineCount), charCount) = countAll traverse text

No more explicitly calling Monoid[Int].applicative or type-level product. I am open to changing implementation to use HList for encoding products instead of Tuple2, but the point of wrapping A => F[B] is to remember the specific Applicative typeclass instance. Tuple2[Int, Int] is by default an applicative, but it behaves different from the product of two applicatives.

scala> List(1, 2, 3) traverseU f
res0: Int = 9

scala> List(1, 2, 3) traverseU g
res1: List[List[Int]] = List(List(1, 2, 3), List(1, 2, 5), List(1, 5, 3), List(1, 5, 5), List(5, 2, 3), List(5, 2, 5), List(5, 5, 3), List(5, 5, 5))

scala> List(1, 2, 3) traverseU (f &&& g)
res2: (Int, List[List[Int]]) = (9,List(List(1, 5), List(2, 5), List(3, 5)))

Here's the AppFunc version of the above:

scala> val f = AppFuncU { (x: Int) => x + 1 }
f: scalaz.AppFunc[scalaz.Unapply[scalaz.Applicative,Int]{type M[X] = Int; type A = Int}#M,Int,Int] = scalaz.AppFuncFunctions$$anon$8@2ad4a8e3

scala> val g = AppFuncU { (x: Int) => List(x, 5) }
g: scalaz.AppFunc[scalaz.Unapply[scalaz.Applicative,List[Int]]{type M[X] = List[X]; type A = Int}#M,Int,Int] = scalaz.AppFuncFunctions$$anon$8@51cd32b5

scala> (f @&&& g) traverse List(1, 2, 3)
res3: (scalaz.Unapply[scalaz.Applicative,Int]{type M[X] = Int; type A = Int}#M[List[Int]], scalaz.Unapply[scalaz.Applicative,List[Int]]{type M[X] = List[X]; type A = Int}#M[List[Int]]) = (9,List(List(1, 2, 3), List(1, 2, 5), List(1, 5, 3), List(1, 5, 5), List(5, 2, 3), List(5, 2, 5), List(5, 5, 3), List(5, 5, 5)))

(f @&&& g) uses the product applicative instance, so the result is the same as traversing f and g independently and composing them together.

@eed3si9n

This comment has been minimized.

Copy link
Contributor Author

commented Oct 1, 2012

I am refactoring AppFunc to Func that's generic, utilizing KTypeClass: https://gist.github.com/3809290

Is this the direction you had in mind? I commented out composeA/<<<@ since you mentioned composition implementation. If it's available somewhere could you make it available on a topic branch or something?

@larsrh

This comment has been minimized.

Copy link
Contributor

commented Oct 1, 2012

Yes, that's a good direction. I'll try to review it as soon as possible.

Regarding composition, it is somewhere in the history and has been removed in d33096b. The assumption was that the type classes which we're interested in (i.e. supporting products) also support composition. I don't know whether this is true.

@eed3si9n

This comment has been minimized.

Copy link
Contributor Author

commented Oct 2, 2012

By assuming that the type classes support both product and composition, Plus, PlusEmpty, and ApplicativePlus will drop out of KTypeClass. This is because there's no plus for Id, which is needed for idCompose.
I think it's a small price to pay for the generic KTypeClass.

@larsrh

This comment has been minimized.

Copy link
Contributor

commented Oct 2, 2012

Now I remember that I found out the same thing some months ago :-)

I'm preparing the patch which re-introduces compose today.

resurrect compose in KTypeClass
The implementation was brought in from
https://github.com/scalaz/scalaz/blob/1af84eadcfd2bb606b565833db35415ee7
fac109/typelevel/src/main/scala/scalaz/typelevel/TypeClasses.scala
@eed3si9n

This comment has been minimized.

Copy link
Contributor Author

commented Oct 2, 2012

I already brought them in. To prioritize implicits, I needed to edit KTypeClass earlier and break implicits into traits like the ones in core. The merge likely would've required hand holding anyway. See eed3si9n@a9d235a.

@eed3si9n

This comment has been minimized.

Copy link
Contributor Author

commented Oct 2, 2012

Merged topic/func refactoring on top of topic/appfunc.

@larsrh

This comment has been minimized.

Copy link
Contributor

commented Oct 5, 2012

Back home. Expect further comments this weekend.

@larsrh

This comment has been minimized.

Copy link
Contributor

commented Oct 6, 2012

I suspect that the @&&& operator has the wrong associativity. From the example:

val (wordCountState :: lineCount :: `hnil`) :: charCount :: `hnil` = countAll traverse text

A nested HList is rarely desired. I'm going to try shuffling things around a bit.

@eed3si9n

This comment has been minimized.

Copy link
Contributor Author

commented Oct 6, 2012

A nested HList is rarely desired.

That's only because typelevel chose HList to encode Tuple2. ProductFunctor etc is still written to support only two items. If calling @&&& for A => M[B] and A => M[C] created A => M[B :: C :: HNil] but calling @&&& for A => M[B :: C :: HNil] and A => M[D] created A => M[B :: C :: D :: HNil] it would be irregular.

@larsrh

This comment has been minimized.

Copy link
Contributor

commented Oct 7, 2012

Current status: I already merged in some parts, see https://github.com/larsrh/scalaz/commit/4946eb93a40c92c59e0020744b034107f68f9fa4. (There are also some additional tests for composition.)

(Minor question: Why was the prioritization of the implicits in KTypeClass necessary? I left this out for the moment, but will happily bring it in if it's needed.)

The concept of Func looks good to me. However, I think @&&& and &&&@ should produce HLists when possible.

That's only because typelevel chose HList to encode Tuple2.

HList is there to obviate the need for any tuple, not just Tuple2. It seems logical to me that Func is able to produce HLists with more than two elements.

@larsrh

This comment has been minimized.

Copy link
Contributor

commented Oct 8, 2012

Formally closing this pull request. I will manually merge the rest of it this week, though.

@larsrh larsrh closed this Oct 8, 2012

@YoEight

This comment has been minimized.

Copy link
Contributor

commented Oct 9, 2012

That's an interesting feature. Well done

@eed3si9n

This comment has been minimized.

Copy link
Contributor Author

commented Oct 15, 2012

What's the status on this? Are you still working on expanding @&&& to :&&&?

@larsrh

This comment has been minimized.

Copy link
Contributor

commented Oct 15, 2012

Yes, I'm working on it. I got a bit side-tracked by university last week, hence the delay.

@larsrh

This comment has been minimized.

Copy link
Contributor

commented Oct 20, 2012

Fully merged as of f2cdd4e.

For now, it's the version you proposed with some minor changes. I wasn't successful in implementing :&&&, so that has to wait. Func is useful nonetheless.

@eed3si9n

This comment has been minimized.

Copy link
Contributor Author

commented Oct 20, 2012

Awesome! If I can implement HListFunc I'll send you a pull req.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
3 participants
You can’t perform that action at this time.