Make it possible to run unit-test assertions with hspec on a reflex-dom application.

[Wizek]

Examples:

import Reflex.Dom.Testing
import Reflex.Dom.Testing (childButton, doClick)

hspec $ do
  specify "constant rendering" $ do
    testRender (text "test1") `shouldBe` "test1"
    testRender (el "div" $ text "test2") `shouldBe` "<div>test2</div>"
    testRender (dynText $ constDyn "test3") `shouldBe` "test3"

  specify "dynamic rendering" $ do
    (e1, trigger) <- newTriggerEvent
    tw1 <- createTestWidget $ do
      dy1 <- holdDyn "test4" e1
      dynText dy1 
      
    testRender tw1 `shouldBe` "test4"
    testRender tw1 `shouldBe` "test4"
    trigger "test5"
    testRender tw1 `shouldBe` "test5"
    testRender tw1 `shouldBe` "test5"

  specify "interactive rendering" $ do
    tw1 <- createTestWidget $ do
      ev1 <- button "button1"
      dy1 <- holdDyn "test6" (const "test7" <$> ev1)
      dynText dy1 
    
    testRender tw1 `shouldBe` "<button>button1</button>test6"
    testRender tw1 `shouldBe` "<button>button1</button>test6"
    tw1 ^. childButton . doClick 
    testRender tw1 `shouldBe` "<button>button1</button>test7"
    testRender tw1 `shouldBe` "<button>button1</button>test7"

Maybe we can build an API that is similarly convenient as outlined above, or perhaps even better.

I've created this issue so this can be a space for @ryantrinkle, @dalaing, myself, and any others to collaborate, share ideas, or subscribe to be notified about updates on adding testability to reflex(+dom).

[Wizek]

So, to begin, I think the first two specify blocks might be relatively easy to implement with a static ByteString renderer as Ryan pointed out on IRC. But I am a bit unsure about the last one, yet I'm quite sure it is crucial for proper testability. @ryantrinkle, or others, how hard additional work do you think it would be to implement it after the first two specify blocks are passing?

And until we have an idea how to do it properly, perhaps it can be worked around for some time like so:

widget1 mockButtonClick = do
      ev1 <- button "button1" <> mockButtonClick
      dy1 <- holdDyn "test6" (const "test7" <$> ev1)
      dynText dy1 

...

  specify "interactive rendering" $ do
    (ev0, trigger) <- newTriggerEvent
    tw1 <- createTestWidget $ widget1 ev0
    
    testRender tw1 `shouldBe` "<button>button1</button>test6"
    testRender tw1 `shouldBe` "<button>button1</button>test6"
    trigger () 
    testRender tw1 `shouldBe` "<button>button1</button>test7"
    testRender tw1 `shouldBe` "<button>button1</button>test7"

[dalaing]

I've got something here that might be interesting. To be useful it would need the ability to kill a mainWidget from GHC code, and would need helper functions to read / modify the state of the various reflex-dom widgets.

I've got something similar for reflex in the same repository, but it also needs a bit of work.

[Wizek]

@dalaing Glad to see a bit of activity on this ticket, thanks for sharing! I've also been thinking about this on and off since then. I also came to the conclusion that we need a way to construct/destroy widgets under test on the fly.

My current thinking is: why not make the API of the testing code such that it accepts any and all MonadWidget t m => m (), appends it as a child to the mainWidget for the duration of a single specify block, then removes it immediately at the end? Maybe using dyn, or similar? What do you think of that approach?

[dalaing]

I've used something like that for Hedgehog and Criterion integration in my older testing repository. It works, but if you can't kill the WebView then your test executable isn't going to exit at the end :/

At the moment my next focus in this area is going to be working on helpers that can be used to query / modify the reflex-dom widgets - I think those pieces will be usable in a lot of different contexts, and some of them might be a bit fiddly to get going. Might be a while before I have time to tidy these things up though :)

[Wizek]

Continuing:

and would need helper functions to read / modify the state of the various reflex-dom widgets.

Having had a cursory glance at the code that you've shared, I see that you are using Reflex.Dom.mainWidget. Does that mean that these tests run in a real webkit2gtk browser instance? If so, couldn't the easiest and most correct way for testing be if we used JS/JSaddle to interface with the DOM elements directly? E.g.

inpEl.value = 'testVal'
inpEl.dispatchEvent(new Event('change'))

[Wizek]

but if you can't kill the WebView then your test executable isn't going to exit at the end :/

Oh, that sounds like an easy issue, why not use System.Exit from base?

[Wizek]

Good news everyone, this experimental proof of concept has turned out to work quite well:

https://github.com/Wizek/reflex-dom-testing/blob/d1100d8/frontend/src/Main.hs#L155-L178

Can be easily tried out by running

$ cd ./frontend/
$ nix-shell ../default.nix -A shells.ghc --run "ghcid -W -c 'cabal new-repl' -T main"

it should print

OK: Just "0"
OK: Just "1"
OK: Just "0"
OK: Just "1"

[Wizek]

Currently the API is a bit clunky, but with some work I believe it could be cleaned up like:

main = runTests 3196 $ do
  testWidget widget1
  doc~.getElementById "output".innerHTML `shouldBeJS` "0"

  doc~.getElementsByTagName "button".js "0".click
  doc~.getElementById "output".innerHTML `shouldBeJS` "1"

widget1 :: forall t m. MonadWidget t m =>  m ()
widget1 = do
  bClick <- button "Increment"
  cnt <- count bClick
  elAttr "div" ("id" =: "output") $ do
    display cnt

[Wizek]

@dalaing JSaddle was also causing me grief with exceptions and being able to exit, but I believe, together with @kevroletin we've been able to outwit it like this.

[Wizek]

Next up:

• I'll be using this in some of my reflex projects, seeing if it stands up to being used in the wild.
  • I'd be very glad to receive usage reports of others too, as this version should already work well for some simple testing; does it work well for you?
• HSpec integration
• HSpec-given integration
• Investigate mock-ability.
  • @dalaing, do you have experience with mocking sub-widgets and/or function dependencies in the context of reflex/reflex-dom/haskell testing? E.g. if a widget depends on the current time or uses delay.
  • Perhaps we can try to use hs-di here. Or mtl-style mocking.

[dfordivam]

This should land pretty soon in reflex-dom. Ref https://github.com/reflex-frp/reflex-dom/blob/e42df1bdc5ea3afd4709f1f847141c792c38df24/reflex-dom-core/test/MountedEvent.hs

[matthewbauer]

PR is #305