Writing complex effects

[arybczak]

I asked this on discourse about theseus, but I then started wondering, is it possible with bluefin to write an interpreter for equivalence of the following effect:

data Fork :: Effect where
  AsyncWithUnmask :: ((forall a. m a -> m a) -> m r) -> Fork m (Async r)

that just wraps asyncWithUnmask from the async library?

[tomjaguarpaw]

Yes, the short answer is that you can do it as below.

The longer answer is that this is a bit delicate because withEffToIO_ doesn't ensure that resources are safe to use concurrently. A properly safe version would use withEffToIOCloneHandle, but hopefully this gives a flavour. I also don't think the type of the unmask is quite right, though it seems like it matches the effectful one you provide.

{-# LANGUAGE GHC2021 #-}

import Bluefin.Compound (mapHandle, useImplIn)
import Bluefin.Eff (Eff, (:>), (:&))
import Bluefin.IO (IOE, effIO, withEffToIO_)
import Control.Concurrent.Async qualified as Async

-- Making the new effect "Fork" as described in
-- https://hackage.haskell.org/package/bluefin/docs/Bluefin-Compound.html
newtype Fork e = MkFork (IOE e)

withFork ::
  e1 :> es =>
  IOE e1 ->
  (forall e. Fork e -> Eff (e :& es) r) ->
  Eff es r
withFork io k = useImplIn k (MkFork (mapHandle io))

asyncWithUnmask ::
  e1 :> es =>
  Fork e1 ->
  ((forall b. Eff es b -> Eff es b) -> Eff es r) ->
  Eff es (Async.Async r)
asyncWithUnmask (MkFork io) m =
  withEffToIO_ io $ \runInIO ->
    Async.asyncWithUnmask (\f -> runInIO (m (effIO io . f . runInIO)))

[arybczak]

They are in the link I posted above 🤔

[tomjaguarpaw]

Ah, I see, you want something like runFork1?

https://github.com/haskell-effectful/effectful/blob/963d445f6c2a849def50d05346b7662b814dc42e/effectful/bench/Concurrency.hs#L63-L68

[arybczak]

Yes

[tomjaguarpaw]

OK, so how about the below?

Looks a bit messy with all the foralls

Yes, the newtype wrapping helps (see below), although it is a bit of a pain to have to do that. There are various ways to make that syntactically more lightweight, but I hope this at least shows that it's possible and the resulting API is convenient, even if the plumbing is messy.

For example in polysemy you can't write the interpreter for this effect for two separate reasons

Interesting! I will have to investigate further to understand why it's possible in effectful (unless it's easy for you to summarize the reason for me).

But doesn't effectful have a problem with that feature? For example, I think this causes undefined behaviour, doesn't it, because unmask escapes its scope?

{-# LANGUAGE GHC2021 #-}

import Effectful (Eff, IOE, (:>))
import Effectful.Concurrent.Async (asyncWithUnmask, runConcurrent, wait)

newtype Unmasker es = MkUnmasker (forall b. Eff es b -> Eff es b)

bad :: IOE :> es => Eff es ()
bad = runConcurrent $ do
  as <- asyncWithUnmask $ \unmask -> pure (MkUnmasker unmask)
  MkUnmasker unmask <- wait as

  -- Oh dear I can run the unmask outside its scope!
  unmask (pure ())

  pure ()

I guess we could say it's not effectful's job to catch that. Fair enough. But Bluefin's use of the type system (specifically forall e. Unmasker e -> Eff (e :& es) r) does stop the Unmasker escaping its scope.

---

{-# LANGUAGE GHC2021 #-}
{-# LANGUAGE ViewPatterns #-}

import Bluefin.Compound (Handle, mapHandle, useImpl, useImplIn, useImplUnder, makeOp)
import Bluefin.IO (IOE, withEffToIO_)
import Bluefin.Eff (Eff, (:>), (:&))
import Control.Concurrent.Async qualified as Async

newtype Unmasker e = MkUnmasker (forall b e1. Eff e1 b -> Eff (e1 :& e) b)

newtype WithUnmask r e = MkWithUnmask (forall e1. Unmasker e1 -> Eff (e1 :& e) r)

-- cf. effectful:
--   runFork1 :: IOE :> es => Eff (Fork : es) a -> Eff es a
runFork ::
  e1 :> es =>
  IOE e1 ->
  (forall e. Fork r e -> Eff (e :& es) a) ->
  Eff es a
runFork io k = withEffToIO_ io $ \runInIO ->
  runInIO $ withFork (\w -> asyncWithUnmaskEff io (applyUnmasker w)) k

asyncWithUnmask ::
  e1 :> es =>
  Fork r e1 ->
  (forall e. Unmasker e -> Eff (e :& es) r) ->
  Eff es (Async.Async r)
asyncWithUnmask (MkFork k) f =
  useImplIn (\() -> k (MkWithUnmask f)) ()

---- Implementation

-- These instances are defined according to the recipes in
-- Bluefin.Compound.
instance Handle (WithUnmask r) where
  mapHandle (MkWithUnmask k) = MkWithUnmask (\e -> useImplUnder (k e))

instance Handle Unmasker where
  mapHandle (MkUnmasker k) = MkUnmasker (\e -> useImplUnder (k e))

newtype Fork r e =
  MkFork (forall e3. WithUnmask r e3 -> Eff (e3 :& e) (Async.Async r))

withFork ::
  (forall e. WithUnmask r e -> Eff (e :& es) (Async.Async r)) ->
  (forall e. Fork r e -> Eff (e :& es) a) ->
  Eff es a
withFork f k = useImplIn k (MkFork f)

applyUnmasker ::
  (e1 :> es, e2 :> es) =>
  WithUnmask r e1 ->
  Unmasker e2 ->
  Eff es r
applyUnmasker (mapHandle -> MkWithUnmask w) m = makeOp (w (mapHandle m))

asyncWithUnmaskEff ::
  e1 :> es =>
  IOE e1 ->
  (forall e. Unmasker e -> Eff (e :& es) r) ->
  Eff es (Async.Async r)
asyncWithUnmaskEff io k = withEffToIO_ io $ \runInIO -> do
  Async.asyncWithUnmask $ \unmask ->
    runInIO $ useImplIn k $ MkUnmasker $ \i ->
      withEffToIO_ io $ \runInIO' ->
        unmask (runInIO' (useImpl i))

[arybczak]

Thanks for writing it. To be honest, while I see it can be written, the code is quite verbose and compared to the original has multiple fresh type variables in nested functions, so I would probably fail to write this myself in a reasonable amount of time.

Interesting! I will have to investigate further to understand why it's possible in effectful (unless it's easy for you to summarize the reason for me).

In short, effectful makes it easy to lift/unlift from the local environment in handlers, while polysemy has a convoluted api and a lot of restrictions around it due to its internal design. I suggest having a deeper look at the linked issues though, I think the struggles participants went through trying to make this work are illuminating.

I guess we could say it's not effectful's job to catch that

Indeed, scoping issues of original functions are out of scope ;) for effectful.

But Bluefin's use of the type system (specifically forall e. Unmasker e -> Eff (e :& es) r) does stop the Unmasker escaping its scope.

Yes, but you don't get it for free, now you have to juggle all these nested e type variables in the implementation. While this is a matter of preference, IMO it's not worth the resulted verboseness and additional complexity in types.

[tomjaguarpaw]

Yes, but you don't get it for free, now you have to juggle all these nested e type variables in the implementation. While this is a matter of preference, IMO it's not worth the resulted verboseness and additional complexity in types.

Fine, but you if you prefer a simpler implementation and you're willing to allow the unmask to escape its scope, like effectful, then the code is about the same as runFork1:

asyncWithUnmask ::
  e1 :> es =>
  Fork e1 ->
  ((forall b. Eff es b -> Eff es b) -> Eff es r) ->
  Eff es (Async.Async r)
asyncWithUnmask (MkFork io) m =
  withEffToIO_ io $ \runInIO ->
    Async.asyncWithUnmask (\f -> runInIO (m (effIO io . f . runInIO)))

runFork1 :: IOE :> es => Eff (Fork : es) a -> Eff es a
runFork1 = interpret $ \env -> \case
  ForkWithUnmask m -> withLiftMapIO env $ \liftMap -> do
    localUnliftIO env (ConcUnlift Ephemeral $ Limited 1) $ \unlift -> do
      asyncWithUnmask $ \unmask -> unlift $ m $ liftMap unmask

[tomjaguarpaw]

In short, effectful makes it easy to lift/unlift from the local environment in handlers, while polysemy has a convoluted api and a lot of restrictions around it due to its internal design. I suggest having a deeper look at the linked issues though, I think the struggles participants went through trying to make this work are illuminating.

I see, seems to be at least a bit related to the difficulty of writing MonadControl instances.