Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fails with future returned by Datomic. #17

Closed
benkamphaus opened this issue Feb 20, 2015 · 21 comments
Closed

Fails with future returned by Datomic. #17

benkamphaus opened this issue Feb 20, 2015 · 21 comments
Assignees
Labels
Milestone

Comments

@benkamphaus
Copy link

Hi I've been playing around with ultra and enjoying it a lot. Unfortunately, most of the work I do at the repl is with Datomic and there's presently an issue with handling the deref of the future returned by a datomic transaction. If you include Datomic free in a library, this is a simple reproduce case:

(require '[datomic.api :as d])

(def db-uri "datomic:mem://test")
(d/create-database db-uri)
(def conn (d/connect db-uri))
@(d/transact conn [{:db/id (d/tempid :db.part/user) :db/doc "hello world"}])

Stacktrace:

java.lang.RuntimeException: Unable to convert: class datomic.btset.BTSet to Object[]
                                                                  ...
                                                clojure.core/to-array                core.clj:  333
                                                    clojure.core/sort                core.clj: 2828
                                                                  ...
                                                   clojure.core/apply                core.clj:  626
                                              clojure.core/partial/fn                core.clj: 2468
                                                                  ...
                                           puget.printer/sort-entries             printer.clj:  131
                                            puget.printer/eval2715/fn             printer.clj:  236
                                                                  ...
                               puget.printer/canonize-map/canonize-kv             printer.clj:  253
                                                  clojure.core/map/fn                core.clj: 2559
                                                                  ...
                                                     clojure.core/seq                core.clj:  133
                                           clojure.core/interleave/fn                core.clj: 3973
                                                                  ...
                                                     clojure.core/seq                core.clj:  133
                                               clojure.core/drop/step                core.clj: 2646
                                                 clojure.core/drop/fn                core.clj: 2650
                                                                  ...
                                                     clojure.core/seq                core.clj:  133
                                                  clojure.core/map/fn                core.clj: 2551
                                                                  ...
                                                     clojure.core/seq                core.clj:  133
                                                   clojure.core/apply                core.clj:  624
                                                  clojure.core/mapcat                core.clj: 2586
                                                                  ...
                                               fipp.printer/serialize             printer.clj:   28
                                                  clojure.core/map/fn                core.clj: 2557
                                                                  ...
                                                     clojure.core/seq                core.clj:  133
                                                   clojure.core/apply                core.clj:  624
                                                  clojure.core/mapcat                core.clj: 2586
                                                                  ...
                                               fipp.printer/serialize             printer.clj:   28
                                             fipp.printer/eval2406/fn             printer.clj:   67
                                                                  ...
                                               fipp.printer/serialize             printer.clj:   31
                                                  clojure.core/map/fn                core.clj: 2557
                                                                  ...
                                                     clojure.core/seq                core.clj:  133
                                                   clojure.core/apply                core.clj:  624
                                                  clojure.core/mapcat                core.clj: 2586
                                                                  ...
                                               fipp.printer/serialize             printer.clj:   28
                                             fipp.printer/eval2394/fn             printer.clj:   55
                                                                  ...
                                               fipp.printer/serialize             printer.clj:   31
                                                  clojure.core/map/fn                core.clj: 2557
                                                                  ...
                                                     clojure.core/seq                core.clj:  133
                                                   clojure.core/apply                core.clj:  624
                                                  clojure.core/mapcat                core.clj: 2586
                                                                  ...
                                               fipp.printer/serialize             printer.clj:   28
                                             fipp.printer/eval2377/fn             printer.clj:   44
                                                                  ...
                                               fipp.printer/serialize             printer.clj:   31
                                                  clojure.core/map/fn                core.clj: 2557
                                                                  ...
                                                     clojure.core/seq                core.clj:  133
                                                   clojure.core/apply                core.clj:  624
                                                  clojure.core/mapcat                core.clj: 2586
                                                                  ...
                                               fipp.printer/serialize             printer.clj:   28
                                             fipp.printer/eval2377/fn             printer.clj:   44
                                                                  ...
                                               fipp.printer/serialize             printer.clj:   31
                                                  clojure.core/map/fn                core.clj: 2559
                                                                  ...
                                                    clojure.core/next                core.clj:   64
                                           clojure.core/concat/cat/fn                core.clj:  701
                                                                  ...
                                                     clojure.core/seq                core.clj:  133
                                           clojure.core/concat/cat/fn                core.clj:  694
                                                                  ...
                                                     clojure.core/seq                core.clj:  133
                                           clojure.core/concat/cat/fn                core.clj:  694
                                                                  ...
                                                     clojure.core/seq                core.clj:  133
                                           clojure.core/concat/cat/fn                core.clj:  694
                                                                  ...
                                                     clojure.core/seq                core.clj:  133
                                           clojure.core/concat/cat/fn                core.clj:  694
                                                                  ...
                                                     clojure.core/seq                core.clj:  133
                                           clojure.core/concat/cat/fn                core.clj:  694
                                                                  ...
                                                     clojure.core/seq                core.clj:  133
                                               clojure.core/concat/fn                core.clj:  685
                                                                  ...
                                                     clojure.core/seq                core.clj:  133
                                           clojure.core/concat/cat/fn                core.clj:  694
                                                                  ...
                                                     clojure.core/seq                core.clj:  133
                                           clojure.core/concat/cat/fn                core.clj:  694
                                                                  ...
                                                     clojure.core/seq                core.clj:  133
                                           clojure.core/concat/cat/fn                core.clj:  694
                                                                  ...
                                                     clojure.core/seq                core.clj:  133
                                           clojure.core/concat/cat/fn                core.clj:  694
                                                                  ...
                                              clojure.core/chunk-next                core.clj:  667
                                            clojure.core.protocols/fn           protocols.clj:  101
                                          clojure.core.protocols/fn/G           protocols.clj:   19
                                    clojure.core.protocols/seq-reduce           protocols.clj:   31
                                            clojure.core.protocols/fn           protocols.clj:   54
                                          clojure.core.protocols/fn/G           protocols.clj:   13
                      clojure.core.reducers/reducer/reify/coll-reduce            reducers.clj:  112
                      clojure.core.reducers/reducer/reify/coll-reduce            reducers.clj:  112
                      clojure.core.reducers/reducer/reify/coll-reduce            reducers.clj:  112
                                                  clojure.core/reduce                core.clj: 6289
                                              transduce.reducers/each            reducers.clj:   44
                                      fipp.printer/pprint-document/fn             printer.clj:  215
                                         fipp.printer/pprint-document             printer.clj:  209
                                                 puget.printer/pprint             printer.clj:  389
                                          puget.printer/pprint-str/fn             printer.clj:  398
                                             puget.printer/pprint-str             printer.clj:  397
                                            whidbey.render/render-str              render.clj:   15
                                                                  ...
clojure.tools.nrepl.middleware.render-values/wrap-renderer/reify/send       render_values.clj:   35
     clojure.tools.nrepl.middleware.interruptible-eval/evaluate/fn/fn  interruptible_eval.clj:   79

With ultra disabled, the output of the deref is:

{:db-before datomic.db.Db@133188bf, :db-after datomic.db.Db@28cd0b35, :tx-data [#datom[13194139534312 50 #inst "2015-02-20T14:48:52.680-00:00" 13194139534312 true] #datom[17592186045417 62 "hello world" 13194139534312 true]], :tempids {-9223350046623220288 17592186045417}}
@venantius
Copy link
Owner

I believe this is essentially the same issue as #8 - I'm hoping to make some headway on both of these in the near future.

@venantius
Copy link
Owner

Hi @benkamphaus - what version of datomic free are you using? I can't seem to replicate your initial output.

@venantius
Copy link
Owner

user=> (require '[datomic.api :as d])
nil
user=> (def db-uri "datomic:mem://test")
#'user/db-uri
user=> (d/create-database db-uri)
true
user=> (def conn (d/connect db-uri))
#'user/conn
user=> @(d/transact conn [{:db/id (d/tempid :db.part/user) :db/doc "hello world"}])
{:t 1000}

@benkamphaus
Copy link
Author

I see this behavior on the latest version (0.9.5130) on both free and pro, Lein dependencies as:

[com.datomic/datomic-free "0.9.5130"]

or

[com.datomic/datomic-pro "0.9.5130"]

@venantius
Copy link
Owner

@benkamphaus Would you mind seeing if this behavior is still present with the latest version of Ultra (0.3.0)? I believe this should be resolved.

@venantius
Copy link
Owner

Better make that 0.3.2, since I'm an idiot.

@benkamphaus
Copy link
Author

I'm still seeing it with 0.3.2. I've observed the same behavior (as listed above) with:

java.lang.RuntimeException: Unable to convert: class datomic.btset.BTSet to Object[]

as the exception (stacktrace is identical).

Tested on:

Ubuntu 14.04
Mac OS X Yosemite

Environment isolated to this as only lein plugin.

JVMs:

Oracle JDK 1.7.0_67
Oracle JDK 1.7.0_60
Oracle JDK 1.8.0_31

Various Datomic versions tested, including:

0.9.5130
0.9.5067
0.9.4899

At least as far back as:

0.9.4572

Not sure if this info helps or is just superfluous.

@venantius
Copy link
Owner

What version of Clojure are you using?

@benkamphaus
Copy link
Author

1.6.0

@venantius
Copy link
Owner

Great, and which version of Leiningen?

@benkamphaus
Copy link
Author

2.4.2 on the Ubuntu VM.
2.5.0 on Yosemite.

@venantius
Copy link
Owner

Ah, great - I've been able to successfully reproduce this. Not quite sure how this sidestepped the fix introduced in 0.3.0, but I'll get to the bottom of it now.

@venantius
Copy link
Owner

@greglook I'm afraid this is boils down to another Puget issue; datomic.db.Db objects satisfy clojure.lang.IPersistentMap, but have custom print-methods defined. In this case the error is thrown because deep within the nested datomic.db.Db object is an object that Puget/Fipp can't render - but even if it could, Datomic's intention is not to print all of the information involved because the nested map is quite large.

It makes me wonder if, rather than testing for interface satisfaction, Puget should check for class membership, at least when it comes to printing documents that Puget expects to be core Clojure types.

I don't know if you have thoughts on this. I'm fine with building an escape hatch in Ultra for Datomic in the short term as well.

@venantius venantius self-assigned this Mar 3, 2015
@venantius venantius added the bug label Mar 3, 2015
venantius pushed a commit that referenced this issue Mar 3, 2015
This commit adds a new namespace, `ultra.printer.escape-hatch` to allow
me to define custom methods for Puget's `format-doc` multimethod. In
this case I'm using it to add an escape hatch for `datomic.db.Db`
objects; it's a temporary workaround until I can figure out a better
long term solution with @greglook, though it'll work fine for now.

This commit also adds a bugfix for colorscheme selection that was
introduced in `0.3.0`.

This commit is therefore intended to resolve both
#23 and
#17
@venantius
Copy link
Owner

@benkamphaus - I've added a temporary escape hatch for datomic.db.Db objects in my development branch for 0.3.3 - it seems to resolve this issue. Would you mind taking a look at it? You can pull down a copy for installation here: https://github.com/venantius/ultra/tree/0.3.3

In the longer run I think @greglook and I will want to figure out something in Puget to address this, but I think this sort of escape hatch may be a necessary device in the near-to-mid term for Ultra to provide support for popular libraries like Datomic where I don't want to leave users out in the cold.

@venantius venantius modified the milestone: Ultra 0.3.3 Mar 3, 2015
@venantius
Copy link
Owner

Hm. This escape hatch issue creates problems for projects that don't use Datomic. There's probably an easy way around this, too, but it warrants additional thinking.

@greglook
Copy link

greglook commented Mar 4, 2015

It makes me wonder if, rather than testing for interface satisfaction, Puget should check for class membership, at least when it comes to printing documents that Puget expects to be core Clojure types.

Hmm, that might work, though it would be a bit more brittle that way. I think we're starting to bump into Puget's origins as a canonical serialization library rather than a colorizing pretty-printer.

I wonder if there's a way to check if a class is loaded without blowing up if it's not present? If so you could curate some format-doc extensions for Puget which only take effect when those classes are present.

@venantius
Copy link
Owner

That's sort of what I had in mind for my next task at this - grab the class, cast to string, check if that string matches a list of known class strings and if so, use the object's built-in print-method. Feels very monkey-patch-y, but it's also relatively painless to backdoor into Puget from Ultra.

@venantius
Copy link
Owner

The latest commit (2fded31) on the 0.3.3 branch covers this.

@venantius
Copy link
Owner

Closing as this will be resolved once 0.3.3 ships.

@venantius
Copy link
Owner

@benkamphaus 0.3.3 is up on Clojars now; datomic should be happy working with it. Let me know if you find this not to be the case.

@benkamphaus
Copy link
Author

Works beautifully! Thanks!

<clj> [user]$  @(d/transact conn [{:db/id (d/tempid :db.part/user) :db/doc "hello world"}])
{:db-after datomic.db.Db@5fbf819c
 :db-before datomic.db.Db@1eb71cfe
 :tempids {-9223350046623220288 17592186045417}
 :tx-data [#datom[13194139534312 50 #inst "2015-03-16T17:13:57.415-00:00" 13194139534312 true] #datom[17592186045417 62 "hello world" 13194139534312 true]]}

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

3 participants