diff --git a/blogdata/content/2012/12/9/heist-0.10-released.md b/blogdata/content/2012/12/9/heist-0.10-released.md new file mode 100644 index 0000000..116b256 --- /dev/null +++ b/blogdata/content/2012/12/9/heist-0.10-released.md @@ -0,0 +1,65 @@ +| title: Heist 0.10...now with a >3000x performance improvement +| author: Doug Beardsley +| published: 2012-12-09T20:55:00+0100 +| updated: 2012-12-09T20:55:00+0100 +| summary: Release notes for Heist 0.10 + +The Snap team is excited to announce the release of Heist 0.10. For this +massively backwards-incompatible release we went back to the drawing board and +re-wrote Heist from the ground up with performance in mind. But was Heist +actually slow? Nobody has actually complained about it, but yes, Heist was +slow. Well...it was until now. + +![](/media/img/heist-perf.png) + +Note that this chart is log scale. The difference is so drastic that if the +chart was linear scale, the bars for Heist 0.10 wouldn't even show up! The +three pages benchmarked are taken directly from snapframework.com, so they are +real-world examples. Heist 0.10 gives an improvement of more than 3000x for +snaplets.tpl, 2000x for about.tpl, and 700x for faq.tpl. The improvement for +faq.tpl is smaller because most its content is statically generated by pandoc, +so Heist only has to traverse a small DOM compared to the other two templates. + +To understand what happened here, we need to go back to the beginning. When +we originally wrote Heist, speed was not our goal. Instead we wanted to see +how far a simple concept like binding Haskell code to HTML nodes could take us +in solving the problem with HTML boilerplate. Now, two and a half years +later, we've discovered that it's an incredibly powerful and enjoyable model +for web programming. + +Heist achieves this by traversing the DOM, executing splices to generate +replacement nodes, and recursively traversing those nodes until all splices +have been executed. A significant part of Heist's power lies in the fact that +this transformation happens at runtime every time a template is served, so the +splice computations are running in your web server monad and have access to +all the data from the HTTP request. Once this transformation is complete, the +resulting DOM is rendered to a ByteString and served back in the HTTP +response. This is really slow when compared with concatenative template +systems like StringTemplate that just blast out interleaved static and dynamic +chunks of data. Therefore it should be no surprise that splicing is slow, as +it allows for much more expressiveness and power. + +However, we realized that a lot of the transformations could be done at load +time and be preprocessed to an intermediate representation. This consists of +static ByteStrings interleaved with deferred dynamic computations that still +give you the power of being able to access the web server's runtime monad. +This "compiled splice" as we call it works quite a bit differently from the +old model of splices. It is also strictly less powerful, so we kept the +ability to use the old "interpreted splice" model if you really need the power +and can afford the performance penalty. + +While we were making backwards incompatible changes, we decided to clean up +everything. This included reorganizing the modules and any other changes we +felt would improve the overall quality and readability of the code base. It +will be a bit of work to migrate existing code to Heist 0.10, but the old +interpreted splices are still available and allow you do a gradual transition. +We also added a new feature called attribute splices that allows you to +abstract attributes in ways that weren't possible before. + +For more detailed information about the new features, check out the tutorials +in the [heist section](/docs#heist) of our docs page. After you have read +those and are ready to upgrade your site, you might want to follow our [migration +guide](https://github.com/snapframework/heist/wiki/Migrating-Snap-Applications-to-Heist-0.10) +to help you with the transition to 0.10. The migration guide is a github +wiki, so feel free to help us make it clearer and more complete. + diff --git a/snap-website.cabal b/snap-website.cabal index 90779c1..a38625a 100644 --- a/snap-website.cabal +++ b/snap-website.cabal @@ -20,10 +20,10 @@ Executable snap-website blaze-html, bytestring, containers, - data-lens-template >= 2.1 && < 2.2, directory, filepath, heist >= 0.10 && < 0.11, + lens >= 3.7 && < 3.8, MonadCatchIO-transformers >= 0.2 && < 0.4, mtl >= 2 && <3, process, diff --git a/snaplets/heist/templates/docs.tpl b/snaplets/heist/templates/docs.tpl index 6f7bff3..8ec410d 100644 --- a/snaplets/heist/templates/docs.tpl +++ b/snaplets/heist/templates/docs.tpl @@ -3,64 +3,67 @@
-

Tutorials

+

Snap

Quick Start
- - - + A guide to getting Snap installed.
Snap API Introduction
- - - + A quick tutorial on the Snap API. Covers installation, the “snap” command-line tool, and a walkthough of the Snap starter application.
-
Heist Template Tutorial
-
- - - - A tutorial for the Heist HTML templating library.
-
Snaplets Tutorial
- - - + Guide to using snaplets to build reusable web components.
Snaplets Design
- - - + Description of the snaplets internal design and motivation.
+ +

Heist

+
+
+
Heist Template Tutorial
+
+ + A tutorial for the Heist HTML templating library. This tutorial + applies to Heist 0.8 and earlier.
+ +
Compiled Splices Tutorial
+
+ + Discusses compiled splices, which were introduced in Heist 0.10.
+ +
Attribute Splices Tutorial
+
+ + How to use attribute splices (also introduced in Heist 0.10).
+ +
+
-

Resources

+

Resources

Haskell Style Guide
- - - + A guide to the Haskell source style we're using for the project.
Benchmarks
- - - + Some benchmark results comparing Snap to several other web frameworks.
@@ -70,7 +73,7 @@
-

API Documentation

+

API Documentation

diff --git a/snaplets/heist/templates/docs/tutorials/AttributeSplices.lhs b/snaplets/heist/templates/docs/tutorials/AttributeSplices.lhs new file mode 100644 index 0000000..bd06216 --- /dev/null +++ b/snaplets/heist/templates/docs/tutorials/AttributeSplices.lhs @@ -0,0 +1,58 @@ +Attribute Splices +================= + +Attribute splices are new in Heist 0.10. They solve the problem of wanting to +be able to dynamically make empty attributes appear or disappear with a splice +without binding a splice to the whole tag. This issue comes up most +frequently when dealing with empty attributes such as HTML's "disabled" or +"checked". + +> module Heist.Tutorial.AttributeSplices where +> import Heist.Tutorial.Imports + +Consider a page with several radio buttons. You want the correct one to be +selected based on the value of a parameter in the HTTP request. The HTML +would look something like this: + + Red + Green + Blue + +We want to automatically generate the "checked" attribute appropriately. This +could be done with a splice bound to the input tag, but there might be a +number of other input tags on the page, so your splice would at best be +executed on more tags than necessary and at worst not have the granularity +necessary to work properly. The ${} syntax for splices inside of attribute +values also won't work because it can only affect an attribute's value. It +can't make the attribute disappear entirely. This problem can be solved +nicely with attribute splices that have the following type: + +< type AttrSplice m = Text -> m [(Text, Text)] + +An attribute splice is a computation in the runtime monad that takes the value +of the attribute it is bound to as its argument and returns a list of +attributes to substitute back into the tag. Here's how we might implement a +splice to solve the above problem. + +> autocheckedSplice :: Text -> StateT Text IO [(Text, Text)] +> autocheckedSplice v = do +> val <- get -- app-specific retrieval of the appropriate value here +> let checked = if v == val +> then [("checked","")] +> else [] +> return $ ("value", v) : checked + +In this toy example we are using `StateT Text IO` as our "runtime" monad where +the Text state holds the value of the radio button that should be checked. We +assume that the current value we're checking against is passed as the bound +attribute's value, so we compare that against the value to be checked. Then +we return a list with the appropriate value and the checked attribute if +necessary. We bind this splice to the "autocheck" attribute by adding it to +the hcAttributeSplices list in HeistConfig. + +To make everything work we use the following markup for our radio buttons: + + Red + Green + Blue + diff --git a/snaplets/heist/templates/docs/tutorials/CompiledSplices.lhs b/snaplets/heist/templates/docs/tutorials/CompiledSplices.lhs new file mode 100644 index 0000000..ba6e5eb --- /dev/null +++ b/snaplets/heist/templates/docs/tutorials/CompiledSplices.lhs @@ -0,0 +1,269 @@ +Introduction to Compiled Heist +============================== + +Before version 0.10, Heist has essentially been an interpreter. It loads your +templates and "runs" them whenever a page is served. This is relatively +inefficient since a lot of document transformations happen every time the +template is requested. For Heist version 0.10 we completely rethought +everything with performance in mind. We call it "compiled Heist". The main +idea is to do most of your splice processing up front at load time. There is +still a mechanism for rendering dynamic information at runtime, but it is +faster than the fully interpreted approach that Heist started with. + +It should also be mentioned that the old "interpreted Heist" is not gone. You +can still use the old approach where all the transformations happen at +render time. This allows you to upgrade without making sweeping changes to +your code, and gradually convert your application to the more performant +compiled approach as you see fit. + +Before we continue it should be mentioned that you are reading real live +literate Haskell code from our test suite. All the code you see here is +compiled into our test suite and the results automatically checked by our +buildbot. So first we need to get some boilerplate and imports out of the way. + +> {-# LANGUAGE NoMonomorphismRestriction #-} +> module Heist.Tutorial.CompiledSplices where +> import Heist +> import qualified Heist.Compiled as C +> import Heist.Tutorial.Imports + +As a review, normal (interpreted) Heist splices are defined like this. + +< type Splice m = HeistT m m [Node] + +The type parameter `m` is the runtime execution monad (in a Snap application +this will usually be `Handler` or `Snap`). Don't worry about why the `m` is +there twice right now. We'll get to that later. The splice's return value is +a list of nodes that is substituted back into the document wherever the +spliced node was. + +This kind of splice proccessing involves traversing the DOM, which is +inefficient. Compiled Heist is designed so that all the DOM traversals happen +once at load time in the IO monad. This is the "compile" phase. The type +signature for compiled splices is this. + +< type Splice n = HeistT n IO (DList (Chunk n)) + +We see that where Heist splices ran in the m monad, compiled splices run in the +IO monad. This also explains why HeistT now has two monad type parameters. +The first parameter is a placeholder for the runtime monad and the second +parameter is the monad that we're actually running in now. + +But the key point of the compiled splice type signature is the return value. +They return a DList of Chunks. DList is a list that supports efficient +insertion to both the front and back of the list. The Chunk type is not +exposed publicly, but there are three ways to construct a Chunk. + +< yieldPure :: Builder -> DList (Chunk m) +< yieldRuntime :: RuntimeSplice m Builder -> DList (Chunk m) +< yieldRuntimeEffect :: Monad m => RuntimeSplice m () -> DList (Chunk m) + +If your splice output can be calculated at load time, then you should use +`yieldPure` or one of its variants. When you do this, Heist can concatenate +all adjacent pure chunks into a single precalculated ByteString that can be +rendered very efficiently. If your template needs a value that has to be +calculated at runtime, then you should use the `yieldRuntime` constructor and +supply a computation in the RuntimeSplice monad transformer that is +parameterized by `m` which we saw above is the runtime monad. Occasionally +you might want to run a runtime side effect that doesn't actually insert any +data into your template. The `yieldRuntimeEffect` function gives you that +capability. + +An Example +========== + +With that background, let's get to a real example. + +> stateSplice :: C.Splice (StateT Int IO) +> stateSplice = return $ C.yieldRuntimeText $ do +> val <- lift get +> return $ pack $ show (val+1) + +Here we see that our splice's runtime monad is `StateT Int IO`. This makes +for a simple example that can clearly demonstrate the different contexts that +we are operating in. To make things more clear, here's a version with some +print statements that clarify the details of which monad is executed when. + +> stateSplice2 :: C.Splice (StateT Int IO) +> stateSplice2 = do +> -- :: C.Splice (StateT Int IO) +> lift $ putStrLn "This executed at load time" +> let res = C.yieldRuntimeText $ do +> -- :: RuntimeSplice (StateT Int IO) a +> lift $ lift $ putStrLn "This executed at run/render time" +> val <- lift get +> return $ pack $ show (val+1) +> lift $ putStrLn "This also executed at load time" +> return res + +Note here that even though the type parameter to C.Splice is a monad, it is not +a monad transformer. RuntimeSplice, however, is. Now let's look at a simple +load function that sets up a default HeistState and loads templates from a +directory with compiled splices. + +> load :: MonadIO n +> => FilePath +> -> [(Text, C.Splice n)] +> -> IO (HeistState n) +> load baseDir splices = do +> tmap <- runEitherT $ do +> templates <- loadTemplates baseDir +> let hc = HeistConfig [] defaultLoadTimeSplices splices [] templates +> initHeist hc +> either (error . concat) return tmap + +Here's a function demonstrating all of this in action. + +> runWithStateSplice :: FilePath +> -> IO ByteString +> runWithStateSplice baseDir = do +> hs <- load baseDir [ ("div", stateSplice) ] +> let runtime = fromJust $ C.renderTemplate hs "index" +> builder <- evalStateT (fst runtime) 2 +> return $ toByteString builder + +First this function loads the templates with the above compiled splice. You +have to specify all the compiled splices in the call to loadTemplates because +loadTemplates takes care of compiling all the templates up front. If you were +able to bind compiled splices later, then all the templates would have to be +recompiled, a potentially expensive operation. Next, the function renders the +template called "index" using a runtime (StateT Int IO) seeded with a value of +2 and returns the resulting ByteString. + +Now let's look at a more complicated example. We want to render a data +structure with a compiled splice. + +> data Person = Person +> { pFirstName :: Text +> , pLastName :: Text +> , pAge :: Int +> } +> +> personSplice :: (Monad n) +> => C.Promise Person +> -> HeistT n IO (RuntimeSplice n Builder) +> personSplice = C.promiseChildrenWithText +> [ ("firstName", pFirstName) +> , ("lastName", pLastName) +> , ("age", pack . show . pAge) +> ] +> +> peopleSplice :: (Monad n) +> => n [Person] +> -> C.Splice n +> peopleSplice getPeople = C.mapPromises personSplice getPeople +> +> allPeopleSplice :: C.Splice (StateT [Person] IO) +> allPeopleSplice = peopleSplice get +> +> personListTest :: FilePath +> -> IO ByteString +> personListTest baseDir = do +> hs <- load baseDir [ ("people", allPeopleSplice) ] +> let runtime = fromJust $ C.renderTemplate hs "people" +> builder <- evalStateT (fst runtime) +> [ Person "John" "Doe" 42 +> , Person "Jane" "Smith" 21 +> ] +> return $ toByteString builder + + +Disadvantages of Compiled Heist +=============================== + +Compiled Heist is faster than the original interpreted approach, but as with +most things in computing there is a tradeoff. Compiled Heist is strictly less +powerful than interpreted Heist. There are two things that compiled Heist +loses: the ability to bind new splices on the fly at runtime and splice +recursion/composability. + +The first point follows immediately from the definition of compiled Heist. +When you decide to do all your splice DOM traversals once at load time you're +unavoidably limited to only those splices that you defined at load time. But +this seems to be a good pattern to use in general because debugging your +splices will be easier if you don't have to consider the possibility that +the handler that binds them didn't run. + +The loss of recursion/composability happens because of the change in the type +signature of splices. Interpreted splices are a essentially function `[Node] +-> m [Node]`. This means that the output of one splice can be the input of +another splice (including itself). Compiled splices are a function `[Node] -> +IO (DList (Chunk m))`. Therefore, once a splice processes some nodes, the +output is no longer something that can be passed into other splices. + +This composability turns out to be a very powerful feature. Head merging is +one feature that can't be done without it. Head merging allows you to put +`` tags anyhere in any template and have them all merged into a single +`` tag at the top of your HTML document. This is useful because it allows +you to keep concerns localized. For instance, you can have a template +represent a small piece of functionality that uses a less common javascript or +CSS file. Instead of having to depend on that resource being included in the +top-level `` tag, you can include it in a `` tag right where you're +using it. Then it will only be included on your pages when you are using the +markup that needs it. + +Our implementation of head merging uses a splice bound to the `` tag. +This splice removes all the `` nodes from its children, combines them, and +inserts them as its first child. This won't work unless the `` splice +first runs all its children to make sure all `` and `` tags have +happened first. And that is impossible to do with compiled splices. + +To get around this problem we added the concept of load time splices. Load +time splices are just interpreted splices that are completely executed at load +time. If interpreted splices have type `[Node] -> m [Node]` where m is the +runtime monad, then load time splices have type `[Node] -> IO [Node]`, where +IO is the monad being executed at load time. Load time splices give you the +power and composability of interpreted splices as long as they are performing +transformations that don't require runtime data. All of the built-in splices +that we ship with Heist work as load time splices. So you can still have head +merging by including our html splice in the load time splice list in your +HeistConfig. + + +A More Involved Example +======================= + +The person example above is a very common and useful pattern for using dynamic +data in splices. But it has the simplification that it always generates +output the same way. Sometimes you might want a splice's output to have one +form in some cases and a different form in other cases. A simple example is a +splice that reads some kind of a key from a request parameter then looks that +key up in some kind of map. If the key is present the splice uses its child +nodes as a view for the retrieved value, otherwise it outputs an error message. + +This pattern is a little tricky because you're making decisions about what to +render based on runtime data, but the actual rendering of child nodes has to +be done at load time. To bridge the gap and allow communication between load +time and runtime processing we provide the Promise data type. A Promise is +kind of like an IORef except that operations on them are restricted to the +appropriate Heist context. You create a new empty promise in the HeistT n IO +(load time) monad, and you operate on it in the RuntimeSplice monad. + +Here's an example of how to use a promise manually to render a splice +differently in the case of failure. + +< failingSplice :: MonadSnap m => C.Splice m +< failingSplice = do +< children <- childNodes <$> getParamNode +< promise <- C.newEmptyPromise +< outputChildren <- C.promiseChildrenWithNodes splices promise +< return $ C.yieldRuntime $ do +< -- :: RuntimeSplice m Builder +< mname <- lift $ getParam "username" +< let err = return $ fromByteString "Must supply a username" +< single name = do +< euser <- liftIO $ lookupUser $ decodeUtf8 name +< either (return . fromByteString . encodeUtf8 . T.pack) +< doUser euser +< where +< doUser value = do +< C.putPromise promise (name, value) +< outputChildren +< maybe err single mname +< +< +< splices :: [(Text, (Text, Text) -> [Node])] +< splices = [ ("user", (:[]) . TextNode . T.pack . fst) +< , ("value", (:[]) . TextNode . T.pack . snd) +< ] + diff --git a/snaplets/heist/templates/docs/tutorials/attribute-splices.tpl b/snaplets/heist/templates/docs/tutorials/attribute-splices.tpl new file mode 100644 index 0000000..9c68f9f --- /dev/null +++ b/snaplets/heist/templates/docs/tutorials/attribute-splices.tpl @@ -0,0 +1,6 @@ +: Attribute Splices Tutorial + +
+ +
+ diff --git a/snaplets/heist/templates/docs/tutorials/compiled-splices.tpl b/snaplets/heist/templates/docs/tutorials/compiled-splices.tpl new file mode 100644 index 0000000..7b3e40c --- /dev/null +++ b/snaplets/heist/templates/docs/tutorials/compiled-splices.tpl @@ -0,0 +1,6 @@ +: Compiled Splices Tutorial + +
+ +
+ diff --git a/snaplets/heist/templates/docs/tutorials/heist.md b/snaplets/heist/templates/docs/tutorials/heist.md index 43f3cb9..e2365e1 100644 --- a/snaplets/heist/templates/docs/tutorials/heist.md +++ b/snaplets/heist/templates/docs/tutorials/heist.md @@ -203,8 +203,8 @@ called `default.tpl`: ~~~~~~~~~~~~~~~ -The `` tag "pulls in" the page content from the calling -template and inserts it into the content `
`. +The `` tag "pulls in" the page content from inside the apply +tag in the calling template and inserts it into the content `
`. Now we have a template for our home page called home.tpl: @@ -253,9 +253,9 @@ illustrates the power of a simple concept like `apply`. What if, in the above example, we decided that the contents of the header div should be different for different pages? To do this, we need a way to pass multiple parameters into a template. Heist -provides this capability with the `` tag. Inside the body of a +provides this capability with the `` tag. Inside the body of an `` tag, you can have multiple bind tags surrounding data to be -passed as separate parameters. Each `` tag must have a `tag` +passed as separate parameters. Each `` tag must have a `tag` attribute that provides a name for its contents just as described above. Then, inside the template, those tags will be substituted with the appropriate data. @@ -283,33 +283,33 @@ to allow multiple parameters. ~~~~~~~~~~~~~~~ -And `home.tpl` uses the `` tag with a name attribute to define +And `home.tpl` uses the `` tag with a name attribute to define values for the `
` and `
` tags: ~~~~~~~~~~~~~~~ {.html} - +

XYZ Inc.

- + Some in-between text. - +

Home Page

Welcome to XYZ Inc

- + ~~~~~~~~~~~~~~~ The result template for this example is the same as the previous example. -NOTE: In this example the `` tag is still bound as described -above. The `` tag is always bound to the complete contents -of the calling `apply` tag. However, any `bind-content` tags inside the apply +NOTE: In this example the `` tag is still bound as described +above. The `` tag is always bound to the complete contents +of the calling `apply` tag. However, any `bind` tags inside the apply will disappear. If we changed `default.tpl` to the following: ~~~~~~~~~~~~~~~ {.html} - + ~~~~~~~~~~~~~~~ diff --git a/src/Main.hs b/src/Main.hs index 40da532..48a0d33 100644 --- a/src/Main.hs +++ b/src/Main.hs @@ -7,11 +7,11 @@ module Main where import Control.Applicative import Control.Exception (SomeException) +import Control.Lens import Control.Monad import Control.Monad.CatchIO import Control.Monad.Trans import qualified Data.ByteString.Char8 as B -import Data.Lens.Template import Data.Maybe import qualified Data.Text as T import Data.Text.Encoding @@ -28,7 +28,6 @@ import Snap.Util.FileServe import Snap.Util.GZip import Text.Blaze.Html5 (toHtml) import qualified Text.Blaze.Html5 as H -import Heist import Heist.Interpreted data App = App @@ -36,7 +35,7 @@ data App = App , _blog :: Snaplet StaticPages } -makeLenses [''App] +makeLenses ''App instance HasHeist App where heistLens = subSnaplet heist @@ -69,9 +68,9 @@ appInit = makeSnaplet "snap-website" description Nothing $ do setCache :: MonadSnap m => m a -> m () -setCache act = do +setCache action = do pinfo <- liftM rqPathInfo getRequest - act + action when ("media" `B.isPrefixOf` pinfo) $ do expTime <- liftM (+604800) $ liftIO epochTime s <- liftIO $ formatHttpTime expTime diff --git a/static/media/img/heist-perf.png b/static/media/img/heist-perf.png new file mode 100644 index 0000000..91de63a Binary files /dev/null and b/static/media/img/heist-perf.png differ