Proposal: Existentially hidden query algebra

[natefaubion]

Currently the component query algebra is part of the public interface of a component. This makes it difficult to implement transparent HOCs because the types become infectious. I propose that we existentially hide a component's query algebra.

In #515 I proposed adding a separate library algebra for lifecycle events:

component = H.component { render, eval, lifecycle }
  where
  lifecycle :: H.Lifecycle Input -> HalogenM ... Unit
  lifecycle = case _ of
    H.Initialize -> ...
    H.Receive input-> ...
    H.Finalize -> ...

I'm going to propose we actually just make this eval and add two more constructor so that it looks like this.

data HalogenQ' f g i a
  = Initialize a
  | Finalize a
  | Receive i a
  | Internal (f a)
  | External (g (Maybe a))

data HalogenQ (g :: Type -> Type)  i a

eval :: forall f g i. HalogenQ' f g i ~> HalogenM s f ps o m

This takes the Halogen.Component.Proxy functionality and makes it first class. We separate a public query algebra from an internal query algebra which means type parameters are no longer infectious or invariant. We can even recover the existing component spec definitions in terms of this by making f ~ g.

Using this we can write an encapsulated HOC where no extra types leak.

type QueryI o a
  = Toggle Boolean a
  | Raise o a

type State i =
  { visible :: Boolean
  , input :: i
  }

type ChildSlots g o =
  ( child :: H.Slot g o Unit
  )

toggleHoc :: forall g i o m. Component HH.HTML g i o m -> Component HH.HTML g i o m
toggleHoc inner = component { initialState, render, eval }
  where
  initialState :: i -> State i
  initialState i =
    { visible: true
    , input: i
    }

  render :: State i -> HH.ComponentHTML QueryI (ChildSlots g o) m
  render state =
    HH.div_
      [ HH.button
          [ HP.onClick (HE.input_ (Toggle (not state.visible))) ]
          [ HH.text "Toggle" ]
      , if state.visible
          then HH.slot (SProxy :: SProxy "child") unit inner state.input (HE.input Raise)
          else HH.text ""
      ]

  eval :: HalogenQ' (QueryI o) g i ~> HalogenM (State i) f (ChildSlots g o) m
  eval = case _ of
    Initialize next -> pure next
    Finalize next -> pure next
    Receive input next -> H.modify _ { input = input } $> next
    Internal (Toggle visible next) -> H.modify _ { visible = visible } $> next
    Input (Raise o next) -> H.raise o $> next
    External q -> H.query (SProxy :: SProxy "child") unit q

Note I used External (g (Maybe a)) which means external queries can be partial. This is always the case anyway since a component may not be mounted, but this allows the inner component to control that partiality as well.

[natefaubion]

Here are some more types, and shows how you can recover a backwards compatible spec from this formulation.

data RequestBox g a b = RequestBox (g b) (Maybe b -> a)

data HalogenQ (f :: Type) (g :: Type -> Type) (i :: Type) (a :: Type)
  = Initialize i
  | Receive i
  | Handle f
  | Request (Exists (RequestBox g a))
  | Finalize

data HalogenM f s ps o m a

type ComponentEval f g i s ps o m = HalogenQ f g i ~> HalogenM f s ps o m

type Component' h f g i s ps o m =
  { initialState :: i -> s
  , render :: s -> h (ComponentSlot h ps m f) f
  , eval :: ComponentEval f g i s ps o m
  }

data Component h g i o m

eval
  :: forall f g i s ps o m
   . { initialize :: i -> HalogenM f s ps o m Unit
     , receive :: i -> HalogenM f s ps o m Unit
     , handle :: f -> HalogenM f s ps o m Unit
     , request :: forall b. g b -> HalogenM f s ps o m (Maybe b)
     , finalize :: HalogenM f s ps o m Unit
     }
  -> ComponentEval f g i s ps o m
eval { initialize, receive, handle, request, finalize } = case _ of
  Initialize i -> initialize i
  Receive i -> receive i
  Handle f -> handle f
  Request req -> runExists (\(RequestBox g k) -> k <$> request g) req
  Finalize -> finalize

compatEval
  :: forall g i s ps o m
   . { initializer :: Maybe (g Unit)
     , finalizer :: Maybe (g Unit)
     , receiver :: i -> Maybe (g Unit)
     , eval :: g ~> HalogenM (g Unit) s ps o m
     }
  -> ComponentEval (g Unit) g i s ps o m
compatEval { initializer, finalizer, receiver, eval } = case _ of
  Initialize i -> for_ initializer eval
  Receive i -> for_ (receiver i) eval
  Handle f -> eval f
  Request req -> runExists (\(RequestBox g k) -> k <<< Just <$> request g) req
  Finalize -> for_ finalizer eval

[cryogenian]

I've spent a couple of days working on this proposal and actuall come up with other proposal 😛

data HalogenQ (f :: Type) (i :: Type)
  = Initialize i
  | Receive i
  | Handle f
  | Finalize

data HalogenM f s ps o m a

type ComponentEval f i s ps o m = HalogenQ f i ~> HalogenM f s ps o m

data Component h i o m

The same example from the first comment

newtype Toggle = Toggle Boolean 
newtype Raise o = Raise o

type State i =
  { visible :: Boolean
  , input :: i
  }

type ChildSlots o =
  ( child :: H.Slot o Unit
  )

toggleHoc :: forall i o m. Component HH.HTML i o m -> Component HH.HTML i o m
toggleHoc inner = component { initialState, render, eval }
  where
  initialState :: i -> State i
  initialState i =
    { visible: true
    , input: i
    }

  render :: State i -> HH.ComponentHTML (ChildSlots o) m
  render state =
    HH.div_
      [ HH.button
          [ HP.onClick (HE.send (Toggle (not state.visible))) ]
          [ HH.text "Toggle" ]
      , if state.visible
          then HH.slot (SProxy :: SProxy "child") unit inner state.input Raise
          else HH.text ""
      ]

  eval :: HalogenQ (Raise o) i -> HalogenM (State i) f (ChildSlots o) m Unit
  eval = case _ of
    Initialize _ -> pure unit
    Finalize -> pure unit
    Receive input -> H.modify_ _{ input = input } 
    Handle (Raise o) -> H.raise o 

This approach for component example https://gist.github.com/cryogenian/ededd916867af50f07b639d28449f3d5

[cryogenian]

To be honest, I'd remove i from Initialize case and simply run two commands in interpreter (i.e. Initialize and Receive).

Alternative to removing algebra at all is

data HalogenQ (f :: Type) (r :: Type -> Type) a
  = Initialize i
  | Request (r a)
  | Handle f
  | Finalize

data HalogenM f s ps o m a

type ComponentEval f r s ps o m = HalogenQ f r ~> HalogenM f s ps o m

data Component h r o m

The slot changes to something like (not in HOC case in HOC case it remains the same)

HH.slot (SProxy :: SProxy "child") unit inner (Const state.input) Handle

But I don't quite sure how to formulate profunctor.

BTW, why not make Component a profunctor w/o newtype? It has no any other instances on it

[garyb]

If I understand that first proposal Maxim, you’re saying we should remove the ability to query child components at all? I think we lose something pretty important there, almost the defining feature of what Halogen is.

I’m not sure what the second one is saying, is it not the same as Nate’s proposal, but without the internal/external distinction?

[natefaubion]

If we completely remove the query, then we don't even need a natural transformation, since everything is Unit. I'm not proposing we do that, because we lose a big part of Halogen's functionality as @garyb said.

One "thing" with my proposal is that it means top-level queries through HalogenIO become partial as well. I admit this is awkward, but I don't see a solution for that.

[natefaubion]

I think your second proposal is the same as mine, except I've added the RequestBox wrapper, which is necessary if we want to support HOCs without using halt. Unless we do want to use halt for the partial case. We would need to a way to handle halt when evaluating queries then.

[cryogenian]

I think your second proposal is the same as mine, except I've added the RequestBox wrapper

Not exactly, I propose to remove i parameter as well and use g instead.

we lose a big part of Halogen's functionality as @garyb said

Currently components have

• public query
• public input
• public output
• private state

We want to add private messages and make public query partial. In that case we would have

• public partial query
• public injput
• public output
• private state
• private messages

Since public partial query is needed only for requests, i.e. all actions could be described via input. It's kinda interesting that this g functionality is a copy of pair i * o.

I agree that it seems that request is very important part of halogen, but honestly why do we think so? What will we lost exactly?

[garyb]

I'm strongly against using inputs for actions, I'd rather lose inputs entirely if we were specifically trying to cut down the communication options between components. Trying to make a continuous stream of things discrete is one of my least favourite things about FRP. It's especially weird in Halogen as it ties rendering to action-sending, you end up having to keep the action-that-we-are-currently-sending in the state of the parent, have the action implement Eq so it can be determined when it changes, things like that.

The thing that requests give us is a sensible interface for interacting with 3rd party components. It's impractical to try and keep a copy of the state of a component not under our control in our component's state, so the only way to solve that is to instead use requests to ask for the state instead. There are some cases with just normal HTML that this happens actually, state things that the virtual dom is unaware of like cursor positions in inputs.

You could argue that requests aren't strictly necessary, as you could send an input that is supposed to trigger an output with the reply, but that's unpleasant as it's like programming against event-based interfaces. It makes following the intent difficult as code gets split up between request-making and response handling, and can easily leave the component in broken states unless you're careful to ensure that every response is treated as optional.

Keep in mind we don't need to expose all these options all the time as part of the component interface - if we add these capabilities to make HOC stuff have less overhead, and allow remapping more easily (that was one of Nate's goals with moving the lifecycle algebra into the component), we can still provide a simplified interface for the normal cases. We could even provide the input-output only version for people who like FRP 😉

[cryogenian]

The point of view where g :: Type -> Type is actually public state not public io has much more sense for me, yep.

I'd rather lose inputs entirely if we were specifically trying to cut down the communication options between components

This is really good point. One problem with initializers and receivers that they're sending actions instead of directly use HalogenM. And if we have Receive | Initialize as part of HalogenQ maybe this could work too?

data HalogenQ f (g :: Type -> Type) a
  = Initialize a
  | Rerender a
  | Handle f a
  | Finalize a
  | Request (g a) -- I'm not sure if this should be exist'ed

I.e. we have Rerender function

type ComponentEval s f g ps o m = HalogenQ f g ~> HalogenM s f ps o m
data Component h g o m

type QueryI o 
  = Toggle Boolean 
  | Raise o 

type State =
  { visible :: Boolean
  }

type ChildSlots g o =
  ( child :: H.Slot g o Unit
  )

toggleHoc :: forall g o m. Component HH.HTML g o m -> Component HH.HTML g o m
toggleHoc inner = component { initialState, render, eval }
  where
  initialState :: State
  initialState = { visible: true }

  render :: State -> HH.ComponentHTML QueryI (ChildSlots g o) m
  render state =
    HH.div_
      [ HH.button
          [ HP.onClick (HE.input_ (Toggle (not state.visible))) ]
          [ HH.text "Toggle" ]
      , if state.visible
          then HH.slot (SProxy :: SProxy "child") unit inner (HE.input Raise)
          else HH.text ""
      ]

  eval :: HalogenQ (QueryI o) g ~> HalogenM State f (ChildSlots g o) m
  eval = case _ of
    Initialize next -> pure next
    Finalize next -> pure next
    Rerender next -> pure next
    Handle (Toggle visible) next -> H.modify _ { visible = visible } $> next
    Handle (Raise o) next -> H.raise o $> next
    Request q -> H.query (SProxy :: SProxy "child") unit q

Note, that since we Rerender is called on rerender, there is no need in proxy for this case.

who like FRP

Not me definitely, removing algebras and making communication channels only i * o is much more like true OOP from Erlang processes. But of course FRP could be built on top of it.

Unless we do want to use halt for the partial case

Does it mean we need this?

Request q -> H.query (SProxy :: SProxy "child") unit q *> halt

[natefaubion]

I don't feel like this example illustrates what Rerender is or does since you aren't even using it.

I'd rather lose inputs entirely if we were specifically trying to cut down the communication options between components

Why? How else do you keep something in sync? If we did this, things like halogen-select would cease to work.

[natefaubion]

Not me definitely, removing algebras and making communication channels only i * o is much more like true OOP from Erlang processes.

I mean, that doesn't mean it's good just because someone else does it.

[cryogenian]

Oh, like this

eval = case _ of 
  Rerender next -> do 
    st <- H.get
    H.query (SProxy :: SProxy "child") unit $ H.action $ DoSomething st.usedToBeInput
    pure next
  ...

[natefaubion]

Doesn't that defeat the point of "declarative", if we have to imperatively update everything to keep it in sync?

[natefaubion]

I have no problem adding a Rerender hook, btw. I just don't think it's a great replacement for inputs.

[cryogenian]

Well, querying isn't declarative too :)

Inputs are just queries, right? We send i to receiver, it constructs Query a and then eval is called. This i lives in parent component. All that declarativity is simply sugar propagated to type signature.

I mean, that doesn't mean it's good just because someone else does it.

But this is not so bad idea too. This approach is very old and solid.

What I'm trying to say is that we have too much public interfaces in component. And too much type parameters. If we want to introduce lifecycle algebra, then we could take a chance to remove at least one type parameter. g if we ok with async communication (BTW, it looks like this thing could be synchronous as well) or i if we want to preserve component algebra.

Actually, i functionality could be implemented via g, and g could be implemented via i * o :)

[natefaubion]

Well, querying isn't declarative too :)

I'm not saying that there shouldn't be imperative querying. I'm saying that only having imperative querying is not good, and makes the library strictly less useful. If we are using a pure functional language, why would we make imperative, effectful updates the only means of communication? I just don't see why this would be desirable.

All that declarativity is simply sugar propagated to type signature.

It is sugar, but it's not sugar you can easily recover if you remove it from the library. Could you tell me how I could realistically recover declarative updates if we removed the sugar from the library?

But this is not so bad idea too. This approach is very old and solid.

My point is that it isn't a goal of Halogen to conform to Erlang, so saying "This is how Erlang does it" isn't a motivation or reason to accept a proposal.

If we remove g in favor of always async communication (while it can potentially be synchronous, it is semantically async), then I guarantee you will immediately try to recover requests by piecing together some baroque, out-of-band AVar communication. Decoupling the two is terrible for the use case that requests solve.

[kRITZCREEK]

I agree, having to mess with UUIDs/AVars whatnot to match request/responses is always worse when a synchronous interface is possible instead.

[garyb]

Why? How else do you keep something in sync?

I'm not suggesting we do it at all, I'm saying if we were pushing to cut down the communications options (not that I think we should particularly), I dislike inputs-as-queries enough that I'd rather lose inputs than queries. 😜

[cryogenian]

If we are using a pure functional language, why would we make imperative, effectful updates the only means of communication?

Because communication is effectful and imperative, right?

Could you tell me how I could realistically recover declarative updates if we removed the sugar from the library?

I couldn't of course, because it's almost impossible. I agree that w/o i and Rerender component is less powerful stuff. But with Rerender there is no any difference.

My point is that it isn't a goal of Halogen to conform to Erlang, so saying "This is how Erlang does it" isn't a motivation or reason to accept a proposal.

I didn't say that because it's like in Erlang, I said that because it's not FRP.

while it can potentially be synchronous, it is semantically async

Not necessary

request :: i -> HalogenM f s ps o m (Maybe o) 

eval = case _ of 
  ToggleButton next -> do 
      request CheckButtonState >>= case _ of 
        Just (Response a) -> doSomething
        _ -> pure unit
      pure next

[natefaubion]

Because communication is effectful and imperative, right?

I don't fully understand what point you are trying to make. Can you clarify?

I couldn't of course, because it's almost impossible. I agree that w/o i and Rerender component is less powerful stuff. But with Rerender there is no any difference.

Maybe it's not just sugar then, if it isn't something you can recover? I would say there is a difference. Take for example a list of components. Right now there is a single (declarative) source of truth for coordinating a component's position and parameters. A post-render hook would mean I now have to split that logic into two and make two traversals manually, which just introduces the opportunity for them to drift. I don't agree that these are "the same". You can maybe achieve the same end-result, but post-hook is clearly much more brittle and inefficient.

request :: i -> HalogenM f s ps o m (Maybe o)

This is not equivalent to a request though. Requests have well-typed responses specifically so you don't have to deal with partiality and the bugs it introduces. By only making the result o, you are saying "this could be relevant or not relevant at all". You can't even return a single o, since you can raise any number of times, or not at all. I don't agree that this simplifies anything, it just introduces more edge cases.

[cryogenian]

Because communication is effectful and imperative, right?

Nevermind, I described different concept with word communication. Now I see your point.

Hm... You've convinced me about preserving i. I've just recollected that before i we used to use slots for this type of stuff, and that wasn't cool at all.

request :: i -> HalogenM f s ps o m (Maybe o)

I still think that there is a way of unification g and i * o preserving current semantics. But straightforward removing of one or another wouldn't work, yep. Thanks for discussion, that was really helpful 😃

[garyb]

I'd say there's a semantic difference between what g is for and i * o too:

• No i value is ever guaranteed to raise an o, but with g, aside from infinitely waiting or other "lies" there's no way to avoid providing a reply to a request
• os can arise without any kind of i having happened - they can arise from within the component itself, for example, via intervals, websockets, etc. as well as interactions with the DOM

We could definitely take a more fundamentalist approach to how some of these things work, removing forking and such so that every o would have to arise from an i, so the component has to send is to itself again, but we've been down that road with an older iteration of Halogen and I'm not sure it's any better.

I think simplicity of implementation is a nice goal, but for a library like this, where there are a wide variety of uses and no real strong underlying model that covers all of them naturally, providing an interface that provides easy ways to do things the "right" way (in our opinion - derived from experience of doing it other ways and finding them harder to work with) is the better way to go.

Perhaps some unifications will arise again in the future that result in things being simplified again (I have some ideas that I'd like to experiment with that would reduce or maybe erase the interface difference between elements and child components, for example). But as long as we don't end up with a sprawling interface with many different entirely equivalent ways to solve the same problem, I think we're still good.

[natefaubion]

And FWIW, I agree that it would be nice to have fewer type variables, but with this proposal it's not any more unique parameters than what we currently have since we are removing one by using rows for children. I also agree that we should bikeshed parameter order some.

I also don't think that this fundamentally makes anything more complex. In fact it makes a lot of use cases far simpler since inputs don't have to be functors. With rows for slots, component modules can export their own slot constructors, so it's easy to write simpler Component newtypes (rather than aliases) that can be unpacked in the slot constructor. I don't love having lots of Component variations, but I think it lets us better encapsulate opinionated use cases (in other libraries maybe).

[natefaubion]

In reference to this specific proposal, I'd really like to hear some ideas around handling the partiality of proxied requests. I don't like that it makes top-level requests partial, but it may be inevitable since you can halt anyway.

[garyb]

Could we have External (g a) and then use g ~ MaybeT g' for the non-root case? Or would that introduce too much parameter junk, etc?

[natefaubion]

The problem I have with that is that it special cases the root component. That would mean if I had a root component, and then decided to wrap it in some way, I would have to go back and change it so that it works under MaybeT.

[natefaubion]

One thing we could do is just keep the root request as "total", and insert a halt if the request returns Nothing. I don't think that's any less total than what we have now since anyone can halt at any time.

[garyb]

Yeah, that sounds reasonable to me... so Nothing becomes Halt in the DriverIO query result basically?

[natefaubion]

Yes, but we can also just go ahead and return a Maybe. If someone wants to use note to unwrap it, they can. 🤷‍♂️

[natefaubion]

Bike-shedding parameter order:

data Component g m i o

data HalogenQ g i a

data HalogenM s ps f o m a

data HTML ps f

-- Encourage this type synonym which has parameters in the 
-- same order as HTML and Component.
type HalogenEval s ps f g m i o = HalogenQ g i ~> HalogenM s f ps o m

[garyb]

Don't we need the parameter for the HTML argument still?

I'd vote the m coming pretty close to the front if it's not at the end - maybe like Component h m g i o - that way the things that change within an app are all grouped at the end, and the h m that must be consistent throughout are up front, and could even be synonym'ded; MyAppComponent = Component HTML Aff. I'd also like to keep the argument order consistent in all places that it occurs, to make it somewhat possible to remember / figure out at least.

[natefaubion]

Don't we need the parameter for the HTML argument still?

Yes, of course. I knew I was forgetting something.

I'd also like to keep the argument order consistent in all places that it occurs, to make it somewhat possible to remember / figure out at least.

I would too, but I think the problem is the MonadTrans instance, which requires the m to be second to last. Would we remove that? Or are you just referring to the synonym? Do you have suggestions for the other types?

[garyb]

Damn, you're right. I'll take another look at this later today.

[paulyoung]

I'm excited about this 👍