non-Datomic persisted backend

[afhammad]

I'm very interested in using DataScript but wondering what your thoughts are when you are populating it with data from a backend database such as Mongo where the attributes are named without a prefix (i.e name vs room/name), is there a performant or idiomatic way of handling this without having to a) changing the naming convention in your backend db or b) convert names as you sync data which can be a performance hit.

Would love to hear thoughts on this matter if anyone has tackled this.

[tonsky]

Converting data is a performance hit, of course. But DS does not require namespaced attributes, :name or :title would work fine. It’s convenient to use namespaced attrs so you can differentiate entity types by attribute name (e.g. :person/name is certainly a person, but :name could be person, car, dog, anything). If you don’t have this ambiguity, you’ll do fine without namespace. Another option is to have special “:type” attribute.

Biggest penalty comes not from attributes, but from managing cross-entities references. In DS, entities should reference each other by :db/id attribute, it should be a number and it should be unique among all entities. So you cannot directly assign entities ids = PK from SQL database, as in different tables there may be overlapping PKs.

There’re two solution to this. First is to let DS assign ids and store PKs in external id attribute. This is how we convert to DS format in acha-acha: https://github.com/clojurecup2014/acha/blob/clojurecup/src-cljs/acha.cljs#L418

Second is to tune your DB so that different tables have non-overlapping id ranges. It allows for PK to be used directly as :db/id in DataScript. We moved to this solution in Acha-acha, as it eliminates lookup by external id at conversion step and makes server pushes much easier to handle. Here’s an example for SQLite https://github.com/clojurecup2014/acha/blob/0.2.0/src-clj/acha/db.clj#L67

[afhammad]

Thanks for the informative answer @tonsky.