Move Streaming(T) to a separate module

[travisbrown]

I'm opening this issue for discussion, but if there's strong disagreement we can close it right away.

I think it'd make sense to move Streaming and StreamingT to a separate cats-streaming module (or
possibly even a separate project).

These two types currently make up about a sixth of the cats-core codebase (~1,450 lines out of just under 9,000). They're only referred to elsewhere in cats-core to implement toStreaming methods that are redundant anyway, since Streaming has fromFoldable, fromList, fromVector, etc. Most importantly, these two types are the target of a majority of recent bug fixes and are responsible for a large number of open issues and PRs (most recently #856).

I haven't used either yet (and rarely used StreamT in Scalaz). If we care about modularity this seems like an easy way to split a big chunk of less-stable, less-essential code out into a separate module.

[ceedubs]

I'm open to this idea. I questioned the practical usefulness of StreamingT in #657, and nobody has chimed in about it yet. Early on, we made the decision in Cats to support standard library types as opposed to rolling our own (ex: no Maybe or IList), even when those std types have some warts. I like Streaming more than std Stream, but probably not enough to warrant it being in cats-core when something like IList is not.

[non]

I will accept the blame of having decided to add these to Cats (and of course all their bugs as well). It's possible that dogs would be a better home for them at this point.

Trying to reconstruct my original rationale for introducing these to Cats, I guess I would say that having a safe, properly lazy list/stream type seemed important, and the parallels between the concrete stream and the monad transformer seemed compelling enough to warrant its inclusion as well.

Given that we recently de-modularized several Cats modules, I think it would be a good idea to consider moving these types to cut down on the size of cats-core. Arguments against would be people who either want to be able to use these types in cats-core itself (which seems moderately unlikely right now) or a significant fraction who are using (or expect to use) Streaming in their own code.

What do others think?

[stew]

Yes. I agree, and for many reasons. I think that we have had a lot of trouble trying to figure out what the overall shape of cats should be. For instance, how modular should it be? How big should it be? Should it be trying to be "a lean project with just the more important category theoretic type classes" or "A general support library for people trying to do functional programming in scala". I think we haven't quite figured out what we want our eventual goal to be, and I think that this is perfectly understandable.

I think that data-structures in general have been the biggest source of problems for us in trying to figure out what our direction should be. We have eschewed things like Maybe and IList, but we very quickly and aggressively sought out a replacement for the stdlib's Either, and I'm not sure why. There is one design decision that I think is a mistake in the design of Either, which is the lack of right-biased map/flatMap. but we could fix that by just implementing usable Functor/Monad typeclasses, right? The poor design decisions around List/Option is comparatively huge.

I agree with "having a safe, properly lazy list/stream type [is] important", but this is important for doing FP on the jvm, but not necessarily directly related with programming with category related concepts. In some alternate reality where we had an amazing standard library that we were trying to build on, some of this would make more sense, especially if we had some process to contribute back to the standard library in ways where it was lacking, but this is not our current reality.

So this is why I've been experimenting in http://github.com/stew/dogs with trying to figure out what that alternate reality might look like, and try and see what it would feel like to try to create some kind of a better standard library that would sit under cats, taking as many of the "FP supporting" but not necessarily category theory related data structures from cats to sit under cats (and streaming is already in there). I've also publisded a branch of cats (https://github.com/stew/cats/commits/feature/dogs) which sits on top of dogs and uses very little from the standard library. This stuff is a long way from complete (I believe that I got laws compiling but not tests, so none of the laws are actually checked yet). And clearly this is all kinda chucked together and not at all polished yet. Its really still just a thought experiment in my mind.

So then it doesn't take long to realize that just with some of our early failures of modularization, this project of trying to make dogs into an alternate standard library is going to probably remain unsatisfactory, because one of the major categories things I would name as missing from the standard library are typeclasses, and all the good typeclasses are going to remain with cats. Again it will be tough to really judge where to draw this line between "Just used to do FP" and "category theory". Maybe I'll try to pull Foldable down into dogs, because it makes sense to try to abstract over foldLeft / foldRight types of operations, but then I have to remove some very useful methods from it like traverse, foldMap because I don't have the other supporting typeclasses like Monoid and Applicative.

So yeah, I've been thinking a lot about this, obviously and I think this is one small facet of a much bigger issue, and I'd love to hear more and more thoughts about this. I've been trying to prepare myself for what I hope will be a big discussion at the upcoming typelevel summit about these things.

[stew]

sorry, I barfed on the bug a bit.