Translation between State & Reader effect doesnt seem to work anymore

[benhutchison]

Some background: back in June I asked if it were possible to weaken a State effect down to a Reader, for pieces of code that only read some state, but do not write it back. IIRC @etorreborre showed me a technique on eff-cats gitter, which I started using a version of. Its requires extending the stack with a new Reader effect , which is syntactically a bit clumsy it did meet the goal.

Since the 2.0 upgrade I cant get it to work, when there are other effects that need be in the stack. The prepend operation seems to hide the other effects in some way. The gist below is an attempt to simplify down a reproducible case of the problem Im hitting.

https://gist.github.com/benhutchison/b5d8267a764a22ca893af078ba3a3a27

[benhutchison]

On further thought I think similar problems would affect operations like modifyReader.

modifyReader takes a stack R1, removes Reader[T, ?] and adds in Reader[S, ?] and returns that as R2. But I can't see how any other Member implicits describing the input stack R1 get threaded through, so as soon as the output stack needs a second effect (like _Option above) post-transform it will error.

For this to work seem to require the compiler to infer:

• If an effect other than Reader[T, ?] is a member if stack R1 then its a member of U (the common "residue" stack)
• If its a member of U then its a member of R2

My guess is that both these cases result from insufficient inference in the Member implicits. Reviewing them I see that there are now ~25 levels of implicit rules in 2.0. That scares me because I dont know if I'll ever understand it properly myself and I wonder if anyone but Eric could maintain that code.

[etorreborre]

@benhutchison I am going to look at this example. I agree that implicits are scary in eff and I don't understand myself yet what's the minimal set of implicits which can make it all work. I have one question though. Do you require def methodWithReadEffect[E: ReadStr :_Option]: Eff[E, Unit] = ???? Could you get away with def methodWithReadEffect[E: ReadStr :_option]: Eff[E, Unit] = ???? (which compiles ok).

[benhutchison]

I really didn't think such a subtle change would get it to compile! Nice :)

TBH I dont know if I need _Option, seems to interpret fine without it. So I get that _option implies weaker MemberIn while _Option implies Member ... but I dont see how interpretation works for a MemberIn Option .. what's the distinction?

[etorreborre]

When you don't need to interpret effects but you just need to create them you use _option which is an alias for MemberIn (short for Inject). This typeclass is a lot less demanding in terms of typechecking because you don't need to know what is the resulting stack U once you remove the effect from the stack R.

Then when we use the Member typeclass, sometimes it is necessary to be explicit about the "output" stack and use Member.Aux[T, R, U] but other times we can get away with just requiring m: Member[T, R] and U is taken as m.Out.

In your example it looks like the implicit using Member.Aux doesn't get triggered but if I define a similar one just defined in terms of Member things are working ok. I need to play with that a bit more before releasing a version fixing the issue, probably by adding more implicit defs...

[benhutchison]

Hi Eric,

I'll include for context another way I tried to write a State-Reader
translator inspired by Eff's ReaderInterpretation.modifyReader, that seemed
syntactically nicer than my other usage. It uses the
"trait-with-apply-method" trick to separate the type param S that must be
specified from the four that are meant to be inferred.

The name "AsState" is because its meant to wrap a Reader, representing its
effect as a State. I'm not sure if it's realistic to hope this could work
when the stacks ER ("effects with Reader") and ES ("effects with State")
contain other effects that must be threaded down:

@ trait AsState[S] {
def apply[ER, ES, U, A](e: Eff[ER, A])(
implicit readerS: Member.Aux[Reader[S, ?], ER, U],
stateS: Member.Aux[State[S, ?], ES, U]): Eff[ES, A] =
transform(e, new ~>[Reader[S, ?], State[S, ?]] {
def apply[X](r: Reader[S, X]): State[S, X] =
State((s: S) => (s, r.run(s)))
})
}
defined trait AsState
@ def AsState[A] = new AsState[A]{}
defined function AsState
@ def methodWithStateEffect[E: StateStr: option]: Eff[E, Unit] = for { <-
AsStateString} yield ()
cmd24.scala:1: No instance found for MemberIn[Option, E].
The effect Option is not part of the stack E
def methodWithStateEffect[E: StateStr: option]: Eff[E, Unit] = for { <-
AsStateString} yield ()

          ^

Compilation Failed

-Ben

On Mon, Sep 12, 2016 at 5:15 PM, Eric Torreborre notifications@github.com
wrote:

When you don't need to interpret effects but you just need to create them
you use _option which is an alias for MemberIn (short for Inject). This
typeclass is a lot less demanding in terms of typechecking because you
don't need to know what is the resulting stack U once you remove the
effect from the stack R.

Then when we use the Member typeclass, sometimes it is necessary to be
explicit about the "output" stack and use Member.Aux[T, R, U] but other
times we can get away with just requiring m: Member[T, R] and U is taken
as m.Out.

In your example it looks like the implicit using Member.Aux doesn't get
triggered but if I define a similar one just defined in terms of Member
things are working ok. I need to play with that a bit more before releasing
a version fixing the issue, probably by adding more implicit defs...

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#16 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAF05EMyrhuepSIu7aQC3asdS3nys81_ks5qpPwhgaJpZM4J5t_D
.

[etorreborre]

I have another proposal Ben. I think your example does not compile because it doesn't define with enough precision the stack E. In order to the use the interpret.transform method you need to know exactly what is the stack resulting from removing the Reader effect (then introduce another type parameter, U and Member.Aux implicits and so on...).

But I suspect that in your case the methodWithStateEffect and methodWithReadEffect are just methods creating those effects and requiring a precise knowledge of the stack is not useful until the moment when you want to interpret those expressions.

So I propose another approach where you can "massage" the MemberIn implicits instead to do the mapping between State and Reader. This is implemented in this spec.

I have published 2.0.0-RC7-20160912142717-c806d04 for your feedback. For me the main issue is that I have also added an extract method on MemberIn which is kind of useful because it is the inverse of inject. However it forces me to require both a natural transformation from Reader[String, ?] to State[String, ?] and vice-versa. This might be restricting in some cases where only one way is available.

I have 2 options here:

• remove extract altogether since it is actually not used at the moment by anything but a law in the spec for MemberIn
• keep extract but add it to a MemberInOut trait which would be between MemberIn and Member

[benhutchison]

Confirming that 2.0.0-RC7-20160912142717-c806d04 seems to work for me in
the original codebase. Thanks :)

I cant yet comment usefully on what to do with extract. Still getting used
to Eff 2.0, Fx, MemberIn etc..

On Tue, Sep 13, 2016 at 12:37 AM, Eric Torreborre notifications@github.com
wrote:

I have another proposal Ben. I think your example does not compile because
it doesn't define with enough precision the stack E. In order to the use
the interpret.transform method you need to know exactly what is the stack
resulting from removing the Reader effect (then introduce another type
parameter, U and Member.Aux implicits and so on...).

But I suspect that in your case the methodWithStateEffect and
methodWithReadEffect are just methods creating those effects and
requiring a precise knowledge of the stack is not useful until the moment
when you want to interpret those expressions.

So I propose another approach where you can "massage" the MemberIn
implicits instead to do the mapping between State and Reader. This is
implemented in this spec
c806d04#diff-f8388679f8bc1f83f685ba205d7c50f4R64
.

I have published 2.0.0-RC7-20160912142717-c806d04 for your feedback. For
me the main issue is that I have also added an extract method on MemberIn
which is kind of useful because it is the inverse of inject. However it
forces me to require both a natural transformation from Reader[String, ?]
to State[String, ?] and vice-versa. This might be restricting in some
cases where only one way is available.

I have 2 options here:

• remove extract altogether since it is actually not used at the
  moment by anything but a law in the spec for MemberIn
• keep extract but add it to a MemberInOut trait which would be
  between MemberIn and Member

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#16 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAF05BbAx5MBF9g0HYEQGORmEQ1tIc84ks5qpWOwgaJpZM4J5t_D
.

[etorreborre]

Cool, I'm closing this then. FYI I added the MemberInOut typeclass. I am not sure how it will be used in the future but I think it fills a gap. With this addition, your example become this, where you just need one Natural Transformation to adapt the MemberIn instance from State to Reader.

[benhutchison]

<=, then |=, and now \=.. Eff keeps growing... I must admit I dont
properly understand the distinction between the 3 types of Member yet, in
terms of what we can/cannot do with each.

But thanks for finding a nice solution to transforming stacks via their
Members, I think I'll use that a fair bit.

On Wed, Sep 14, 2016 at 1:59 AM, Eric Torreborre notifications@github.com
wrote:

Cool, I'm closing this then. FYI I added the MemberInOut typeclass. I am
not sure how it will be used in the future but I think it fills a gap. With
this addition, your example become this
https://github.com/atnos-org/eff-cats/blob/master/jvm/src/test/scala/org/atnos/eff/MemberSpec.scala#L112,
where you just need one Natural Transformation to adapt the MemberIn
instance from State to Reader.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#16 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAF05HkLnv39zbWBilh874Pby8JThxaIks5qpshUgaJpZM4J5t_D
.

[etorreborre]

Maybe I'll try to recap. You mostly need MemberIn and Member and this follows the usual structure of programs with effects where you have distinct parts of your program creating and interpreting effects:

• MemberIn is to be used when you only create effects. The requirements on the effects stack are minimal, we just need to know that the stacks contains the effect, but we don't need to know how exactly
• Member is used when you want to interpret a given effect or transform it to another one in the stack. In that case you need to have a more complete knowledge of the stack and in particular what is the stack resulting from removing the effect

I haven't a real use for MemberInOut yet but I think it can be used to implement catchError in a more straightforward way because we just need to extract an effect (Throwable Xor X) from the stack and we don't need to know what other effects are present in the stack, the end result is in the same stack.

I would like to propose an already finished "theory and practice of effects", but it looks like you are helping me discovering it along the way :-).

[benhutchison]

Right, I think it just clicked into place for me..

If you do your interpreting at the top level, then inside the pure program
you just mention an effect is present, and at the top level interpreter
you'll plug in a materialized stack with final effect types known..?

Whereas if you want to interpret away effects inside the program, eg
convert def foo[R: _Option]: Eff[R, A] to foo2[R]: Eff[R, Option[A]],
you'll need the carry the heavier constraint down to the code site where
you do the interpretation..?

On Wed, Sep 14, 2016 at 4:33 PM, Eric Torreborre notifications@github.com
wrote:

Maybe I'll try to recap. You mostly need MemberIn and Member and this
follows the usual structure of programs with effects where you have
distinct parts of your program creating and interpreting effects:

MemberIn is to be used when you only create effects. The requirements
on the effects stack are minimal, we just need to know that the stacks
contains the effect, but we don't need to know how exactly

Member is used when you want to interpret a given effect or transform
it to another one in the stack. In that case you need to have a more
complete knowledge of the stack and in particular what is the stack
resulting from removing the effect

I haven't a real use for MemberInOut yet but I think it can be used to
implement catchError in a more straightforward way because we just need
to extract an effect (Throwable Xor X) from the stack and we don't need
to know what other effects are present in the stack, the end result is in
the same stack.

I would like to propose an already finished "theory and practice of
effects", but it looks like you are helping me discovering it along the way
:-).

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#16 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAF05ByiwDNmqzvDYBk4mh8eepj9eWIfks5qp5VTgaJpZM4J5t_D
.

[etorreborre]

Yes, that's it.