add comonadic comprehensions (keyword 'cofor')

[shimib]

This is a proposal (POC) to provide a new keyword (cofor) for co-monadic comprehensions in Scala.
The work is based over Dominic Orchard and Alan Mycroft paper: https://www.cl.cam.ac.uk/~dao29/publ/codo-notation-orchard-ifl12.pdf

If you find the implementation flawed, it is me to blame and not them!

the 'cofor' expression returns a function: (T[A] => B).
The function's type MUST be stated explicitly in the call-site. e.g.:
val result: Zipper[Person] => Boolean = cofor...

the syntax:
cofor(inputpatten) {
pattern1 <- gen1
pattern2 <- gen2
...
} yield body

the 'cofor' and generator must use types that provide:
def map(A => B)
def extract: A
def coflatMap(T[A] => B)

implementation is much more complex than 'for' desugaring as stated in the paper.

current implementation supports full Scala patterns in the input and generator patterns.
missing:

desugared tree positioning validation (should not affect surrounding code)
quasiquotes/macros/reification -- this is the next phase in the implementation.
the flag to control the keyword: -Ycofor-extension
if the flag is not enabled, current Scala code should not be affected.

compatibility: adding a new keyword breaks existing use of 'cofor' identifier and it should be back-quoted.

Plz let me know your thoughts on this.

[mandubian]

Nice work :)
We could also think about introducing apfor for applicative for-comprehension which would change how we use Applicatives too...

[shimib]

10x.
Do you know who generates the JavaUniverseForce file?

[milessabin]

@shimib my understanding is that these days it's maintained by hand.

[mandubian]

@shimib yep I confirm what @milessabin said, I had to modify it by hand for my PR

[adriaanm]

Wow! I'm open to a discussion and a proof of concept, but we're going to have to take some time to figure out whether we all agree to add this to the language. Until then, you're welcome to experiment, but the full implementation, including the scanner/parser changes will have to be hidden by a flag, which doesn't seem to be the case.

Instead of a new keyword, could we drive the expansion by the type of the monad?

Could we generalize this pattern to allow macros to customize for comprehension, so that we can have for cover monadic, comonadic and applicative comprehensions?

Since this is a proof of concept, I'm going to schedule this for the WIP milestone.

[tpetricek]

I spent some time working on comonads (see http://tomasp.net/coeffects for more) with Dominic who wrote the paper above - I find it quite funny how hard it is to find useful comonads in contrast to finding useful monads - but there are two things that can make comonads more useful:

• Merging of contexts - with comonads, you can use them in a way that lets a function access context available in the scope in which it is defined - this can give you a mix of lexical and dynamic scoping (see my implicit parameters example above). This is quite nice - with reader monad, you get just dyamic scoping and comonads can give you more

• Indexed (graded) comonads - if you have fancy type system where you can do type-level stuff (I believe Scala has all this), you can annotate comonad C[T] with a monoid and use type like C[r, T] where r is an additional annotation. Your extract function then is C[1, T] -> T which adds a bunch of other use cases (e.g. the Maybe type is not a comonad, but it is an indexed comonad).

To make this even more useful, it might be worth considering whether the language extension can cover these possible use cases.

For merging of contexts, you'd need to change the translation so that cofor inside another cofor is translated using additional operation (merge : C[T1] * C[T2] -> C[T1 * T2]). For indexed (graded) comonads, it just needs to be flexible enough to allow the types to be not just C[T] but also C[r, T].

[shimib]

@adriaanm thanks. Added the flag to parsers (was already in place for scanners).
Btw, i seem to get binary incompatibility errors in the build regarding the additional flag i put in Settings. What's the recommended way of dealing with it?
regarding a more holistic solution, need to think about it.

@tpetricek thanks. i'll take a look at it.

[adriaanm]

Before we get too far into the nitty gritty implementation details, I'd like to have a discussion about adding this as a new language feature, as per my previous comments. Even behind a flag, this needs to be discussed prior to this PR being merged. In other words, we need at least a draft SIP and a thread on https://contributors.scala-lang.org/

Experience has shown that flags don't properly convey that something may not end up becoming part of the language, so, in order to ship with the official compiler, under a flag or not, there has to be some consensus that we eventually want this in the language from a design, implementation and maintenance point of view.

[smarter]

Is this something that could be done as a macro first?

[milessabin]

This is something that would be a good candidate to merge in Typelevel Scala, with or without a SIP. I think it's important that discussion of implementation details happen here early rather than on the TLS repo.

@smarter syntax makes a significant contribution to the usability of this feature and that's something which can't be experimented with practically in a macro or plugin.

[jvican]

@milessabin @smarter I think it can be done in a compiler plugin, just hijack the parser via reflection. Paul did something similar for trailing commas support.

[milessabin]

@jvican it can but it's extra effort and the results are fragile. It's also completely unnecessary given that we can get experience in TLS.

[shimib]

@milessabin i've started a topic at https://contributors.scala-lang.org/t/providing-co-monadic-comprehensions.
I'm going to compose a newbie introduction to Comonads and cofor there.
will also draft a "Pre-SIP" doc.

I'll be happy to discuss implementation details here.

I think that new keywords for "cofor and apfor" will be the most reasonable choice.
I think that reuse of the current "for keyword" can be done for comonads. However, it will be very confusing for programmers.

[edmundnoble]

@shimib It will be necessary eventually to include apfor inside of for so that we can use applicative operations inside monadic comprehensions. For example, this will provide automatic parallelism if the monad in question is Future and automatic error merging if the monad is Either[L, ?]

[shimib]

@edmundnoble sure. I'm waiting to see what will happen with cofor (LBS/TLS) and if it goes well, apfor is next :)

[Atry]

Is cofor really useful? Is it possible to use Cofree + for comprehension to construct any comonadic expressions instead of cofor?

[Atry]

Is this PR compatible with scala.meta?

[shimib]

@Atry Cofree + for and cofor can both express the same computations. However, cofor is more elegant when you want to do deep plumbing in your context (the original article mentions this).

Regarding scala.meta, i don't think this PR is compatible yet, i need to look at it.

Currently, desugaring already works and i'm focusing now on quasi-quotes.

After that, scala.meta is a good candidate.

[Atry]

I did not find any mention to Cofree in the original paper. Could you post a link, please? 2017-02-27 15:26 GMT+08:00 Shimi Bandiel <notifications@github.com>:

…

@Atry <https://github.com/Atry> Cofree + for and cofor can both express the same computations. However, cofor is more elegant when you want to do deep plumbing in your context (the original article mentions this). Regarding scala.meta, i don't think this PR is compatible yet, i need to look at it. Currently, desugaring already works and i'm focusing now on quasi-quotes. After that, scala.meta is a good candidate. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#5725 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAktuhYWEziuDAmNKafMZT3mhNUBjPMTks5rgnq9gaJpZM4MHS50> .

-- 杨博 (Yang Bo)

[shimib]

I assumed you meant arrow notation for Comonads (CoKleisli) which is mentioned in the original paper.
I'll look into the generalization (Cofree) to see if it can be used to provide a more elegant syntax.
Although, i'm not sure that everyone who'll use co-monads will use those abstractions (which is one of the reasons behind the cofor syntax).

[Atry]

Ah, I see. I should mean CoKleisli, not Cofree.
So, is it possible to create nested for comprehension with nested CoKleisli as an alternative to cofor?

[shimib]

@Atry yes it is. However, for complex extractions and context manipulations, cofor provides a more elegant notation.

[shimib]

@milessabin I think everything is in place here at LBS for submitting a PR against TLS and start a discussion there. WDYT?

[milessabin]

@shimib yes, I think so.

[ghik]

@edmundnoble about hybrid monadic-applicative comprehensions: I believe this is exactly what I propose here: https://contributors.scala-lang.org/t/new-syntax-applicative-desugaring-in-for-comprehensions-with-pr/690

[edmundnoble]

@ghik Indeed, I got to take a look. Like it a lot, but I hope this doesn't get Martin off the hook for providing irrefutable patterns without withFilter ;)

[SethTisue]

closing for inactivity. the SIP committee considered a related SIP to be premature

[bblfish]

Pretty interesting work anyway! Lots of good references.