Datascript example

[ouvanous]

Hi,
I wonder how i can use datascript with re-frame ?
Is there an example somewhere ?
Many thanks,

Samuel

[mike-thompson-day8]

I'm not aware of any example.

[mike-thompson-day8]

Just to be clear, you'd need to rewrite the reference implementation, rethinking the subscriptions process, etc. There'd be a bit in it. But an interesting exercise!!

[ouvanous]

Ok many thanks

[rmoehn]

Anyone who sees this thread and thinks, 'oh, now I'll go away and rewrite the reference implementation': actually you only need to swap in a Datascript database for the re-frame app-db, use something like https://gist.github.com/allgress/11348685 for the subscriptions and re-think (ooh!) the pure middleware.

[hkjels]

Has anyone actually done this yet? I'm trying, but having problems triggering reactions.
The Re-frame docs, says you should put your query inside subscription-handlers reaction, but even though I can test that the value has changed, the bind function from that example does not re-evaluate.

Note that I'm new to everything clojure, so I might just be missing something obvious.

[rmoehn]

Check out this: https://github.com/rmoehn/theatralia/blob/compl-recon/src/cljs/theatralia/thomsky.cljs

Tomorrow (Japanese time) I'll be able to help more if you need. It would also make it easier for me if you posted the relevant snippets of your code.

[hkjels]

OK. So I've had a look now at thomsky, and think I have a better idea of whats missing for these reactions to take place. I'll have a go tomorrow and see if things pan out 😀
Thank you..

[rmoehn]

I've just realized that it doesn't work in all cases (unless I'm mistaken). I haven't thought about which cases those are, but probably enough that I wouldn't use the approach. See my new comment on https://gist.github.com/allgress/11348685. I don't know yet what I'll do with Thomsky.

[hkjels]

OK, I'm starting to feel some time-pressure, so I'm going to just maintain another atom based on the data from datascript. I have a better idea of how to cause those reactions to fire then.

[hkjels]

Still, this should be researched further. Something tells me that using datascript and some kind of sync-layer with datomic would be really future-proof. Add in declerative queries like with atomic and datomic-junk and your already way into the future 😀

[rmoehn]

There are alternatives:

• The transaction listener could run the query over the whole database (instead of :tx-data) to see if the result changed. – Less performant, but probably the easiest fix.
• One could implement what Dylan Butman outlines in https://groups.google.com/d/msg/clojurescript/o0W57ptvPc8/ArrJ6wKi-9gJ.
  • Maybe tonsky's Cat Chat is yet another approach, but I haven't gotten my head around that yet. Shouldn't be too hard, though; I'm just confused right now.

David Nolen talks about the sync thing in his teaser for Om Next. He also says he wrote a Datascript adapter in a few lines of ClojureScript. I'm curious what approach he chose.

[pupeno]

Has there been any progress in making re-frame work seamlessly with datascript?

[rmoehn]

I've changed my approach to querying the whole database instead of the :tx-data. That does work seamlessly and is complete. Only performance might be not so good if your database is big, but that has to be measured.

thomsky.cljs is everything you need in order to use Datascript from re-frame. Since it's MIT license software, you can just copy and paste it into your code (a Credits line would be nice) and see how it performs. It's fairly well documented, I'd say. You'll find examples for event handlers using it in theatralia.handlers and for subscription handlers in theatralia.subs.</self-promotion>

[pupeno]

@rmoehn I asked about an alternative to DataScript on Slack and @tonsky said that we could use Datascript not in an atom, but directly as a data structure. That seems like it would work with less workarounds. Have you tried that?

[rmoehn]

I just noticed that Datascript has moved on quite a bit since I wrote thomsky.cljs. You'll have to make some adjustments.

@pupeno Can you give me a link to the relevant bits on Slack? Because I don't know how he meant that. I can't imagine how it should be easier, but I'm not very familiar with Datascript.

[pupeno]

@rmoehn I hope this link works: https://clojurians.slack.com/archives/clojurescript/p1444029537000112

[hkjels]

So, you would use db-with instead of real transactions?
I guess it would allow a pretty straight forward way of using Datascript with Re-frame, but you would have to manually create more transactional data would you not?

I'd really like a setup whereas I can fetch initial load from Datomic, work with that data realtime in Datascript and have a web-socket layer that tries to keep them both in sync. Such eventual consistency would be good enough for 99% of the projects I work on I guess.

[rmoehn]

@pupeno Apparently I can't look into Slack archives, because I'm not on Slack. :-( Why do people communicate about open source projects on a closed platform?

So the point was to use db-with? Yeah, right, this way you can return whole Datascript DBs in the event handlers and re-frame swaps them into app-db. One problem is that you lose the ability to listen to transactions. That aside, you would make the subscription handlers query the db
parameter with d/q, right? And put that in a reaction. Then you have another problem, because all components that use these reactions get re-rendered every time app-db changes. If you don't see why, I can explain it, but it's lengthy.

All in all, you if you don't use the bind, you get more re-frame-y code, but more re-renders. The question is how the performance impact of the re-renders compares to that of all the DataScript querying and by-value comparisons.

@hkjels Why don't you program the synchronization stuff? Or help programming it? I'm pretty sure there are already projects on the way. Om Next is not far I think, and it might have a good deal of these ideas already supported. (Er, sorry for advertising Om Next in a re-frame issue.)

[hkjels]

I've tried a few approaches already, but I'm new to all of these technologies, so you might say I'm having some difficulties.

[dmkolobov]

@rmoehn Can you shed some light on why components in the db-with scenario would all re-render whenever app-db changes? Subscriptions which dereference the datascript DB atom would be rerun, but wouldn't the result-set of the query be cached?

[rmoehn]

Sorry, it's been a while, so I need some time in order to answer this properly. I'll get around to it on 25th or 26th. (Although if your cost of delay is high, I could fit it in today.)

[dmkolobov]

No hurry at all, it can definitely wait until then. Thank you.

[rmoehn]

Thanks for your patience! I'm still struggling to get into my head from four months ago and I'm not so confident in my statements about re-renders anymore. However, here a two examples that might shed some light.

The first is the usual case without DataScript:

cljs.user=>   (require '[reagent.core :as r]
                       '[reagent.ratom :as ratom :include-macros true])
nil
cljs.user=>   (def map-atom (r/atom {}))
#'cljs.user/map-atom
cljs.user=>   (def r2 (ratom/reaction (get @map-atom :x1 nil)))
#'cljs.user/r2
cljs.user=>   (swap! map-atom assoc :x1 {:name "bla"})
{:x1 {:name "bla"}}
cljs.user=>   (def val3 @r2)
#'cljs.user/val3
cljs.user=>   (swap! map-atom assoc :x2 {:name "blu"})
{:x1 {:name "bla"}, :x2 {:name "blu"}}
cljs.user=>   (def val4 @r2)
#'cljs.user/val4
cljs.user=>   val3
{:name "bla"}
cljs.user=>   (identical? val3 val4)
true

You see that even though the map as a whole inside the ratom is a different one after the second swap!, the subtree beneath :x1 hasn't changed. It is the same object, thanks to immutable data structures with sharing. Reagent would see this and know that it doesn't need to re-render components which depend on r2 only.

The second example is with DataScript:

cljs.user=> (require '[datascript.core :as d]
                     '[reagent.core :as r]
                     '[reagent.ratom :as ratom :include-macros true])
nil
cljs.user=>   (def conn (d/create-conn))
#'cljs.user/conn
cljs.user=>   (def r1 (ratom/reaction (d/q '[:find ?x :where [?x :name "bla"]] @conn)))
#'cljs.user/r1
  (d/transact! conn [{:id -1
                      :name "bla"}])
[…]
cljs.user=>   (def val1 @r1)
#'cljs.user/val1
  (d/transact! conn [{:id -1
                      :name "blu"}])
[…]
cljs.user=>   (def val2 @r1)
#'cljs.user/val2
cljs.user=>   val1
#{[1]}
cljs.user=>   (identical? val1 val2)
false
cljs.user=>   (= val1 val2)
true

Here, the results you get are not identical, even though they are equal, because DataScript allocates new data structures for every query's return value. I'm not yet sure, but I think the current stable version of Reagent re-renders everytime the results are not identical and in the next release it will be changed to re-render only for non-equal results. Therefore, whenever re-frame does a with-db and swaps the result into the app-db ratom, Reagent will re-render all components, because all queries return values not identical to the previous ones.

As I said, I'm not sure about these. If you know more about the inner workings of Reagent, I would be happy to be enlightened by you. Otherwise I will dig around some more.

[dmkolobov]

Your comments about re-renders were indeed correct, thank you for your thorough response. I'm afraid I don't know much about Reagent's inner workings, and your explanation makes sense. In any case, I tested query reactions with Reagent 0.6.0-alpha, and I can confirm that the new equality based reactions solve the issue! My backup solution was to rewrite the bind function using the new when-let macro's finally clause to remove the watch, removing the need for unbind.

Also, to address the loss of transaction reports in the immutable DB value case, I wrote a middleware factory, which returns a middleware that wraps a function from DataScript DB values and event vectors to transaction data. The :listeners keyword argument provides listen!-like functionality. It is inspired by the source of transact! in the DataScript codebase.

(defn transact-mid
  "Create a middleware wrapping a DataScript transaction."
  [& {:keys [listeners]}]
  (fn [handler]
    (fn [db v]
      (let [tx-data (handler db v)
            report  (ds/with db tx-data)]
        (doseq [cb listeners] (cb report))
        (:db-after report)))))

;; example

(def tx-mid
 (transact-mid 
   ;; The listener handler will be called synchronously 
   ;; with the transaction, and will take the transaction
   ;; report as its sole argument.
   :listeners [(partial println "tx-report:")])

(defn make-named-thing
  [datascript-db-value [_ name]]
  [{:db/id -1 :thing/name name}])

(register-handler
  :make-named-thing
  [tx-mid] ;; We assume that app-db is a DataScript DB value
  make-named-thing)

(dispatch [:make-named-thing "foobar"])
;; adds a thing/name fact and prints a tx-report to the console.

The body oftransact-with is inspired by the transact! source from the DataScript codebase. This middleware can be re-used across event handlers, allowing us to attach the same sequence of listeners to each transaction. If the DataScript DB is part of a larger data structure, then we can use another middleware to focus updates:

(defn update-ds 
   [handler]
   (fn [db v]
     (update db :datascript-db handler v)))

(register-handler
  :make-named-thing
  [update-ds tx-mid]
  make-named-thing)

I'm not entirely sure how good or useful this solution is. It may feel limiting to only return tx-data in these handlers. Also, unlike DataScript's transact!, the listeners are called synchronously with the state change, though this might be addressed by a combination of dispatching in listeners and the techniques discussed in the CPU hog section of the wiki. I'll try to post a more thorough example this week. Comments welcome!

[rmoehn]

Cool! Looks like you've dug re-frame much more than me!

[eggsyntax]

@mike-thompson-day8 et al -- anyone know if anyone has gone further down this road in the meantime? We're making some front-end architectural decisions, and I'd love to be able to put re-frame + datascript on the table.

[rmoehn]

I suggest putting it on the table with all the other reasonable choices, and evaluating them for what they are.

[eggsyntax]

Sure, but there's a difference between 1) re-frame + datascript and 2) reframe + we-rewrite-re-frame-for-datascript. It'll go on the table either way, but it's a much easier sell if there's already an implementation out there that works well with datascript.

[rmoehn]

But why would you want to sell it if it's not adequate? That's like »see here's this hammer. I really like it, because it feels good in my hand and has a great picture of a scruffy blacksmith on it. We should use it. By the way, it's also good for cooling a black eye.« Rather than »see, we have to fasten this picture to the wall. How solid does it have to be? Can we make permanent changes or do we have to remove it when we move out? Any other considerations? Which tools are available? To what degree do they fulfil the criteria we set when analysing the problem?«

[rmoehn]

Sorry, I'm missing your point. You just wanted to know whether there are new developments and I can't answer that question. No developments on my table at least.

[thenonameguy]

It should be trivial to implement a Datascript based db with pure-frame.

[eggsyntax]

@rmoehn -- thanks for all the input. I definitely don't want to try to sell it if it's the wrong choice. Maybe I didn't communicate effectively -- we've got some investment already in datascript (we're syncing with a datomic backend), so we probably wouldn't consider re-frame if we couldn't use it with datascript. Of course we could write our own re-frame implementation from scratch that used datascript, but that would require a substantially larger time commitment, which is why it would be a more difficult sell.

@thenonameguy, awesome! I wasn't aware of pure-frame -- very cool. That does seem like it'd make it way easier to swap out the data store.

[Conaws]

You might find this helpful

mpdairy/posh#4

On Saturday, March 19, 2016, Egg Syntax (Davis) notifications@github.com
wrote:

@rmoehn https://github.com/rmoehn -- thanks for all the input. I
definitely don't want to try to sell it if it's the wrong choice. Maybe I
didn't communicate effectively -- we've got some investment already in
datascript (we're syncing with a datomic backend), so we probably wouldn't
consider re-frame if we couldn't use it with datascript. Of course we could
write our own re-frame implementation from scratch that used datascript,
but that would require a substantially larger time commitment, which is why
it would be a more difficult sell.

@thenonameguy https://github.com/thenonameguy, awesome! I wasn't aware
of pure-frame -- very cool. That does seem like it'd make it way easier to
swap out the data store.

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub
#36 (comment)

Sent from Gmail Mobile

[eggsyntax]

@Conaws thanks!

[denistakeda]

Here an alternative solution

[denistakeda]

I've just released a library that allows combining re-frame and DataScript together DataFrame