Tips & tricks

Jeaye Wilkerson edited this page May 2, 2018 · 13 revisions

In all examples below assuming DataScript version 0.13.0 or newer aliased to d

Representing enums

As of 0.13.0, DataScript does not support idents (this may change in the future). Instead, if you need to represent enums, use keywords as values. E.g. instead of

;; setting attribute type to reference
@(datomic.api/transact conn [[:db/add <attr> :db/valueType :db.type/ref]])

;; registering enum value
@(datomic.api/transact conn [[:db/add #db/id[:db.part/db] :db/ident :some-enum-value]])

;; transacting data
@(datomic.api/transact conn [[:db/add <eid> <attr> :some-enum-value]])

just do

(d/transact! conn [[:db/add <eid> <attr> :some-enum-value]])

Referencing entities via ident codes

Sometimes you may need idents anyway. Say, you want to transparently stream Datomic data to client without any additional conversion. Or you want to specify :db/ident on some special entities and reference them in transactions later.

At the moment, you can create :db/ident as a regular attribute on your own, and reference special entities with idents via lookup refs:

;; registering in schema
(def schema {:db/ident {:db/unique :db.unique/identity}})

;; specifying ident on an entity
(d/transact! conn [[:db/add <eid> :db/ident :unique-key]])

;; looking up entity by ident value
(d/entity @conn [:db/ident :unique-key])

(d/transact! conn [[:db/add [:db/ident :unique-key] <attr> <value>]])

(d/q '[:find ?a ?v
       :where [[:db/ident :unique-key] ?a ?v]]
     @conn)

EDN serialization

DataScript DB support EDN serialization natively:

(pr-str (d/empty-db)) => "#datascript/DB {:schema {} :datoms[]}"

Do read database from a reader,

In Clojure use datascript.core/data-readers:

(clojure.edn/read-string
  {:readers d/data-readers}
  "#datascript/DB {:schema {} :datoms[]}")

In ClojureScript #datascript/DB handler is installed globally when using DataScript, so read-string just works:

(cljs.reader/read-string "#datascript/DB {:schema {} :datoms[]}")

Once the db is read, load it into a connection with d/conn-from-db. A round-trip on the JVM would look like this.

(def read-db #(clojure.edn/read-string {:readers datascript.core/data-readers} %))
(def conn (-> (d/create-conn {})
              d/db
              pr-str
              read-db
              d/conn-from-db)

Transit serialization

See datascript-transit

Filtering database entities

datascript.core/filter filters individual datoms, but what if you want to keep whole entities? For example, you want to filter a DB leaving out only persons whose name is "Ivan":

(d/filter db
  (fn [db datom]
    (or
      ;; leaving all datoms that are not about :person/* as-is
      (not= "person" (namespace (:a datom)))
      ;; for :person/* attributes take entity id
      ;; and check :person/name on that entity using db value
      (let [eid    (:e datom)
            entity (d/entity db eid)]
        (= "Ivan" (:person/name entity))))))

Querying on a missing attribute

Until DataScript supports not clause, use a missing? predicate to find out entities which do not have some attribute:

(d/q '[:find ?e
       :where [?e :document/id _]
              [(missing? $ ?e :document/users)]]
     db)

Getting top 10 entities by some attribute

If you have a big database and want to render 10 first posts on a page, instead of doing a query (which will have to materialize everything you have in a result set), use index directly:

(take 10 (d/datoms :avet :post/timestamp))

This will return 10 datoms with the smallest :post/timestamp value. You can then (mapv :e ...) to get their entity ids and pass them down to a query/pull-many call.

Index is already sorted and stored in ascending order. (take 10 ...) will return 10 smallest values very fast.

To get 10 biggest values, reverse the index:

(take 10 (reverse (d/datoms :avet :post/timestamp)))

Reverse return a special view on an index that allows walking it in the reverse direction. This operation is allocation free and about as fast as direct index walking.

Preserving order

DataScript makes it easy to store unordered data. If you need to keep order, you have several options.

By convention :db.cardinality/many attributes are unordered. Sorting datoms by transaction id will give you insertion order:

;; schema
{ :aliases { :db/cardinality :db.cardinality/many } }

;; query
(->>
  (d/q '[:find ?v ?tx
         :in $ ?e
         :where [?e :aliases ?v ?tx]]
       db eid)
  (sort-by second)
  (map first))

;; index lookup
(->> (d/datoms :aevt eid :aliases)
     (sort-by :tx)
     (map :v))

You can store vector of values as a :db.cardinality/one value. Because it will be just a single value, you can keep inside any structure you want, including order:

;; schema
{ :aliases { :db/cardinality :db.cardinality/one } }

;; transaction
(d/transact! conn [[:db/add eid :aliases ["X" "Y" "Z"]]])

;; entity lookup
(:aliases (d/entity @conn eid)) ;; => ["X" "Y" "Z"]

You can introduce attribute you want to sort by, and store references in an :db.cardinality/many attribute:

;; schema
{ :aliases { :db/cardinality :db.cardinality/many
             :db/valueType   :db.type/ref } }

;; transaction
(d/transact! conn [{:db/id eid
                    :aliases [-1 -2 -3]}
                   {:db/id -1, :name "X", :order 3}
                   {:db/id -2, :name "Y", :order 7}
                   {:db/id -3, :name "Z", :order 1})

;; query
(->>
  (d/q '[:find [(pull ?a [:name :order]) ...]
         :in   $ ?e
         :where [?e :aliases ?a]
       db eid)
  (sort-by :order)
  (map :name))

Querying a schema

Unlike Datomic, in DataScript schema is just a map kept separate from database datoms. It means you cannot query it as you would in Datomic, but you can query it just as any other collection. Use [[?attr [[?aprop ?avalue] ...]] ...] destructuring form to convert nested maps into a flat collection of facts.

For example, following query will return all datoms from a DB which attribute has cardinality many:

(def schema
  { :entry/id           {:db/unique      :db.unique/identity}
    :entry/child        {:db/cardinality :db.cardinality/many
                         :db/valueType   :db.type/ref}
    :entry/first-child  {:db/valueType   :db.type/ref} })
    
(def db (-> (d/empty-db schema)
            (d/db-with [[:db/add 1 :entry/id "a"]
                        [:db/add 1 :entry/child 2]
                        [:db/add 1 :entry/child 3]
                        [:db/add 1 :entry/first-child 2]])))

(d/q '[:find ?entity ?attr ?value
       :in $ [[?attr [[?aprop ?avalue] ...]] ...]
       :where [(= ?avalue :db.cardinality/many)]
              [?entity ?attr ?value]]
     db (:schema db))

=> #{[1 :entry/child 3] [1 :entry/child 2]}

Editing the schema

During development it can be useful to change the schema while preserving the state of you datascript connections.

(def schema {...})
;; Global ds connection. The only one in the app.
(defonce conn (d/create-conn schema))
;; We update to a new schema on the fly:
(when-not (= schema (:schema @conn))
  (set! conn (d/conn-from-datoms (d/datoms @conn :eavt) schema)))

See also this issue for its limitations..

You can’t perform that action at this time.
You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session.
Press h to open a hovercard with more details.