Polish halogen for public release

[jdegoes]

Halogen's quickly approaching the point where I think we should do a public release / announcement.

Here's some things I'd like to see before we do that:

1. Increase type-safety. Newtypes, phantom types, type classes, etc., wherever applicable.
2. Documentation. Per method, per module, etc., so newbies can get up to speed.
3. Generalize & Parameterize. For example, can we extract out the HTML tags so they can be used to generate HTML strings, etc., similar to blaze (without the monadic sugar, maybe using finally tagless)? Can we use Aff in more places which do not require an immediate return value? Etc.
4. Beef Up Example. A good example, I think, is Pursuit, which uses Ajax and the hash string. Could we do that using Halogen?

[paf31]

1. Can do.
2. Can do.
3. I'll have a think about this one. A String renderer shouldn't be very difficult to implement. I don't think we need to go finally-tagless, since what we have right now is basically a free monad. We can just write another interpreter.
4. Pursuit is a little tricky because it doesn't use a front-end any more. We moved all the HTML rendering to the server side to simplify the deployment process. I could add support for things like AJAX or local storage or hash strings into the todo-mvc demo though. I also have some other ideas for ways in which I could extend the example.
5. This is a little tricky right now. We would probably need a bit of JS code still, but I could pull out the majority of the VirtualDOM code into an NPM module, and then include that via calls to require.

[jdegoes]

Let's hold off on (5) then because I'd rather whatever solution we end up with for PureScript 0.7 not be tied to the bower or NPM ecosystems. As for (4), I do think some AJAX would be nice, or maybe IndexedDB which has async and is global.

[jdegoes]

Oh, on (3), the reason I suggested finally tagless is that it removes the need for intermediate structures. Free involves building elaborate structures which only serve the purpose of interpretation; not sure it's a good idea to abstract using free in building the HTML markup as it's re-built on every change of state in the app.

[paf31]

So here are a few notes after an initial attempt at a finally tagless encoding:

• You lose things like graft, which is a shame.
• Generally, you seem to lose the ability to do transformations on generic representations, like if I wanted to e.g. make all header tags one level bigger.
• Ideally, I think we would want to tie a representation for HTML to a representation for Attribute. It's not very clear to me how to do that yet, without something like associated types.
• It might be tricky to keep the performance of STProps and still have a nice generic API. This might not be a big deal since most attribute lists are small.

[paf31]

Maybe something like

class (Functor (node a), Functor attr) <= HTMLRepr node attr where
  text :: forall a i. String -> node a i
  element :: forall a i. TagName -> attr i -> node a i
  hashed :: forall a i. Hashcode -> (Unit -> node a i) -> node a i
  placeholder :: forall a i. a -> node a i

  attribute :: forall i. AttributeName -> String -> attr i
  eventHandler :: forall i fields. EventHandlerName fields -> (Event fields -> EventHandler i) -> attr i

newtype HTML a i = HTML (forall node attr. (HTMLRepr node attr) => node a i)

I feel like we might end up running into ambiguous type errors with this though.

[paf31]

@jdegoes Finally, regarding your point about data structures, I think we will still have to create a large intermediate data structure - a function taking a dictionary for a representation to the representation itself. I don't really have good evidence in favor of either one in terms of performance, but please see my list of concerns above.

One thing I like about this representation, which you can't do easily in the original one, is the way you can tie the field type to the event handler name in eventHandler. I should be able to accomplish the same thing with Exists in the data structure model, but it will get noisy.

I'm going to be busy for the next few days, but hopefully there's enough here that you can make some progress on porting slamdata over. Just let me know what you think.

[jdegoes]

So here are a few notes after an initial attempt at a finally tagless encoding:

These seem to be pretty fatal errors to me. 😦 Let's just leave it like it is, though it's not generic, it'll be faster than free. We can always go back and generalize it later if we find a performant way or decide that the cost of Free is acceptable.

[jdegoes]

One thing I like about this representation, which you can't do easily in the original one, is the way you can tie the field type to the event handler name in eventHandler. I should be able to accomplish the same thing with Exists in the data structure model, but it will get noisy.

I'll defer to your judgement on whether or not we should pursue this.

Have fun on your vacation!

[paf31]

I was thinking about this last night and came up with this idea:

• Keep the initial (HTML) encoding, but only use it for transformations.
• Use the final encoding for rendering to virtual-dom, String, etc.
• The final encoding could also be "rendered" to the initial encoding "HTML", transformed, and then mapped back for rendering. This would remove the need for something like purescript-reflection to deforest the intermediate HTML structure by creating instances on-the-fly.

This could be a neat solution if it gives a good performance improvement, but it runs the risk of just adding extra complexity if not.

[jdegoes]

Interesting. I'd definitely love to make the ui functions in a halogen app able to generate anything (e.g. Markdown, ASCII rendering, who knows!). Obviously not at the expense of bad performance or cryptic API. So hopefully we'll come up with something.

[paf31]

I had a stab at a final representation in the final branch, but it's a way off yet. I spent some more time on docs and the example as well.

@jdegoes What are your thoughts on the status of the library now?

[paf31]

I've broken up the docs a little more as well, which I think makes them a bit more readable.

[jdegoes]

Well, I'd like to see Aff used in places where callbacks are used now or where an immediate return value is not required, as part of the attempt to generalize it. I haven't specifically looked for places that could be generified through parametricity, though if there are any, it'd be nice to handle that now.

Otherwise, the library is looking really solid -- what sorts of changes do we need to make before '1.0'? Oh, I guess an Ajax + Timer example would be useful (maybe using affjax?).

[paf31]

So, I split the Aff case into a mixin module, because of the need to handle errors. Eff is clunkier, but Aff can be easily layered on top if you use the SupportsErrors class.

Yes, I'll add a timer/AJAX example, probably tomorrow. If it's possible to cut a 0.1.0 of affjax by then, that would be useful. If not, I can code something simple up without it for now.

[paf31]

I've added a couple of additional examples: a counter, to illustrate driving the application with a timer in main, and a simple AJAX demo, which uses the FFI for now. I can switch it to affjax when it gets released.

Did I miss anything? Other than the finally tagless changes, which I think are still a little way off, I think most of the basics are now covered.

[cryogenian]

Could you add Placeholder a example?

[paf31]

Yep, can do.

[paf31]

@cryogenian Done, please let me know what you think.

[cryogenian]

Great! Thank you!

[jdegoes]

@paf31 The placeholder example is good, but it would be nice to include a real widget even if it's just a couple lines of imported JS code.

As far as I can tell, there are a few things still in-progress:

1. Finishing the Bootstrap theme (I'm not sure what remains here).
2. Finally tagless encoding for HTML (I'm not sure if this is going to prove itself or not, though if it does, it will certainly have some interesting applications)
3. Typed CSS, which might become part of a third-party library in conjunction with (2), depending on the approach we end up with.
4. The library docs, organization, and functions bias toward Eff, but I'd prefer biasing towards Aff, unless you think there's a good reason not to. In addition, the SupportsErrors type class is useful, but perhaps not necessary, as if Eff code bombs out, then the application would keep running, so there might be an alternate function we could provide, e.g. runUiAff', which would log errors to the console or something, so users wouldn't have to deal with errors for Aff if they didn't want to.

Additionally, looking at runUI, I feel like there are some abstractions here which could be extracted out, and composed in a limited number of (nonetheless useful) ways.

But honestly, I think what we have is good enough and probably not going to change in completely onerous ways, that we can probably ship a version now, announce it, and start trying to build a community. I think it's probably 0.4 or maybe 0.5, giving us some time to iron out issues before a 1.0 release. What do you think?

[paf31]

It would be nice to include a real widget even if it's just a couple lines of imported JS code.

Agreed, can do.

Finishing the Bootstrap theme

Yes, we can add support for some more components, and I have a short list of prioritized ones, but I don't think this would hold up a release.

Finally tagless encoding for HTML

I can spend some more time on this. My impression is that it will require us to break a few things up, and it will change the way we handle things like traversals and some components, so if we want to do it, I should probably get it done before a release.

Typed CSS

This seems like something which could make up a fairly large independent project. Since we can do all static things with class attributes right now, I suggest we don't let this hold up a release either, and integrate something once a library is available.

bias toward Eff

I can definitely move over to Aff, but I still see Eff as more general:

• We need some way of handling errors - either we log and forget them, or we need to take care of them in types. Adding an Either gets cumbersome when you have other sources of inputs (undo/redo etc.) which is why I added the class. Eff doesn't have this issue because you're forced to accomodate errors in your result.
• Aff doesn't seem to have any kind of callCC. This makes raw Eff or ContT more useful in the (admittedly rare) case where you want to push multiple inputs to the UI in one action (progress updates, for example). I would say "just add callCC" but it seems like a sensible design decision to disallow it.

I feel like there are some abstractions here which could be extracted out

Interesting, anything in particular?

I think it's probably 0.4 or maybe 0.5, giving us some time to iron out issues before a 1.0 release.

That makes sense. I'll have another stab at finally tagless HTML tonight and see where I get to, then maybe we can discuss making a release?

[jdegoes]

I can spend some more time on this. My impression is that it will require us to break a few things up, and it will change the way we handle things like traversals and some components, so if we want to do it, I should probably get it done before a release.

OK, sounds good. If it looks like this is not paying for itself, then let's can it. It may have a lot of potential, but the devil is in the details.

Eff doesn't have this issue because you're forced to accomodate errors in your result.

In my experience, this isn't true. The old nodewebkit version of SlamData would crash all the time in unrecoverable ways. Mainly because Eff code would throw exceptions when it "shouldn't", due to random stuff (e.g. a JS lib modifying a prototype, a browser-specific quirk, or whatever). In practice, Eff code is not strictly typed enough, nearly every FFI Eff should have exceptions and users should be forced to deal with them. But that doesn't happen so we end up with weird behavior.

The current design with Eff will "fail silently". That is, if the handler or the driver throw exceptions, it won't stop the application (as far as I can see), but no errors will be caught or logged (they'll ripple up to the window exception handler, if any). Aff forces you to deal with exceptions through one means or another, and will soon defensibly catch errant exceptions to make it even safer. Most of the things a handler will do are going to be asynchronous, and there is no good way for handling asynchronous errors with Eff; in fact, I'd go one step further and say that such code is semantically broken (the fact that you can throw an unrecoverable exception in a Eff callback and still continue executing breaks the synchronous composition semantics of the Eff monad).

Further, any ContT (Eff ..) stack without ErrorT and religious catching of exceptions is broken for different reasons, namely, the entire application will stop executing on the first Eff that throws an exception not synchronously caught (i.e. any malformed Eff code whatsoever).

After I get through #3, Aff will be safe by default; that is, it won't be possible to stop an application's execution or otherwise cause undefined behavior through mis-typed Eff code or through failure to religiously catch all exceptions in such code; and all such errors will be propagated through the Aff error channel and can be dealt with in all the usual ways.

That's a good point about Aff not having callCC. I do wonder if Queue could serve the same purpose.

In any case, I'm fine punting on this until I do more work on Aff, but I have really bad experiences with callbacks and Eff, and ContT is no better, just more awkward. Further, I'm convinced that most asynchronous code using callbacks and Eff is semantically broken and violates sequential monadic composition, although I can't tell you what the specific effects of that are right now (if any).

Interesting, anything in particular?

Not spending more than 60 seconds, but something like:

data View a r i = View (SF1 i (HTML a (Either i r))) -- Should be These, i.e. i, r, or both i and r?

data Renderer a = Renderer (a -> VTree)

data Effector eff r i = Effector (r -> Queue i)

data UI eff a r i = UI { view :: View a r i, renderer :: Renderer a, effector :: Effector eff r i }

runUI :: forall eff a r i. UI eff a r i -> Eff (HalogenEffects eff) Node

The pieces and the whole compose in various ways, though I need more time to think about how this would be useful. Note I changed Handler a bit, but the above doesn't quite work as Queue doesn't provide enough power at present (but it'd probably be something like Queue / Haskell's MVar that satisfied this use case).

[jdegoes]

That makes sense. I'll have another stab at finally tagless HTML tonight and see where I get to, then maybe we can discuss making a release?

Super duper! I hope your workshop at LambdaConf is going to touch on Halogen a bit, too. 😄

[paf31]

Ok, I'm sold on Aff vs Eff. I'll start moving things over, but I still think we need some way of supporting errors without Either or just logging.

I see what you mean about the various newtypes (View etc.) - I'll pull some of those out and see if I can write any useful instances.

I'm beginning to think I'll retitle my LC talk to be about reactive web apps more generally. Thermite might make a good conceptual overview, but I think I'd enjoy talking about Halogen more :)

[paf31]

I'll break the remaining items out into their own issues, so that I can track them in a milestone, and close this.

[jdegoes]

Ok, I'm sold on Aff vs Eff. I'll start moving things over, but I still think we need some way of supporting errors without Either or just logging.

Agreed. My suggestion to "just log" was only to achieve parity with the Eff case which does not require that i support errors (and logging is better than what happens to errors under Eff, anyway).

One could imagine adding an error handler to the definition of a UI, e.g.

data UI eff a r i = UI { view :: View a r i, renderer :: Renderer a, effector :: Effector eff r i, handleError :: Error -> Aff eff Unit }

Or some such. Not saying it's a good idea, though, but gets the errors out of the i. After thinking about it, I actually think most errors will get logged (via console and maybe AJAX) and propagate to the UI somehow (JS errors are nasty but providing feedback to users is always a good idea).

So many times for SlamData nodewebkit, apps would die and we'd not have any information on what got them into that state. Taking a different approach this time around (every Error is precious and we can't let one slip through the cracks!).

I see what you mean about the various newtypes (View etc.) - I'll pull some of those out and see if I can write any useful instances.

Yeah, if there's nothing useful there (including Zip and imap et all for invariant functors), no need to create out the abstractions (it'll just lead to more boilerplate newtype wrapping / unwrapping), it just looked like there might be ways of composing these things or at least ascribing names to what they are.

I'm beginning to think I'll retitle my LC talk to be about reactive web apps more generally. Thermite might make a good conceptual overview, but I think I'd enjoy talking about Halogen more :)

Super. 😃

[paf31]

The tricky bit with errors is that if we want them to show up in the UI at all, they have to appear in the type of the view, unless we want to handle them completely separately, which seems a little sad. Having errors as bona fide inputs to the state machine seems the cleanest way to me - the question is just how to avoid using something like Either as an input, which requires tidying up with Choice and lmap after the fact. SupportsErrors is lawless rubbish though :)

[paf31]

Is it possible to have the type of Aff distinguish between the cases where errors can be thrown, and where all errors are handled? That would be nice, because it would force users to use catchError or whatever to deal with errors.

[jdegoes]

SupportsErrors is lawless rubbish though :)

I don't mind it, like you say, it's a way of working around "extensible coproducts" or some such while minimizing boilerplate.

The errors probably do need to be inputs to the state machine because I can't imagine a scenario where you wouldn't want the user to know about the error. Presumably the view would log them (i.e. using a request handled by the Handler function) as well as display them.

So I don't think there's anything wrong with the current approach.

Is it possible to have the type of Aff distinguish between the cases where errors can be thrown, and where all errors are handled?

Sure, via phantom types, for example, but then you can't compose them using >>=. Maybe helper functions here? A phantom type could indicate 'this Aff cannot generate errors because they've already been caught', or 'it's possible this Aff can generate errors.'