history query improvements

[dpetran]

Fixes #357

Changes shape of response to use json-ld iri keys and include the assert/retract subject @ids within each assert/retract entry.

Also adds timestamp support to the from/to t parameters.

[zonotope]

This will be good with me after a few minor changes.

[zonotope]

This might not matter all that much in practice, but it's faster to use group-by instead of sort-by + partition-by, unless you care that the groups are also sorted by subject in the end.

(->> assert-flakes
     (group-by flake/s)
     vals
     async/to-chan!)

[zonotope]

I think it makes sense to refactor this repeated logic into its own function.

[zonotope]

<? can throw errors, but s-asserts-json-ch will either fail to close or close without an error. you should use <! instead. The same goes for the line below.

[aaj3f]

@dpetran not sure if this should be another ticket/PR or not, but I do see some odd behavior w/ the :t { :from } results:

  (def ledger-name "test/historyPR")

  (def conn @(fluree/connect {:method       :file
                              :storage-path "data"
                              :defaults     {:context {:id     "@id"
                                                       :type   "@type"
                                                       :schema "http://schema.org/"
                                                       :ex     "http://example.org/ns/"
                                                       :rdfs   "http://www.w3.org/2000/01/rdf-schema#"
                                                       :rdf    "http://www.w3.org/1999/02/22-rdf-syntax-ns#"
                                                       :xsd    "http://www.w3.org/2001/XMLSchema#"
                                                       :f      "https://ns.flur.ee/ledger#"}}}))


  (def ledger @(fluree/create conn ledger-name))

  (def db (fluree/db ledger))

  (def db1 @(fluree/stage db {:graph [{:id   :ex/andrew
                                       :type :schema/Person
                                       :schema/name "Andrew"
                                       :ex/favNum 8}]}))
  
  (def committed-db @(fluree/commit! ledger db1))
  
  (def db2 @(fluree/stage (fluree/db ledger) {:graph [{:id :ex/andrew
                                        :ex/favNum 10}]}))
  
  (def committed-db @(fluree/commit! ledger db2)) 
  
  @(fluree/history ledger {:history :ex/andrew})
  ;; => [#:f{:t 2, :assert [{:ex/favNum 10, :id :ex/andrew}], :retract [{:ex/favNum 8, :id :ex/andrew}]}
  ;;     #:f{:t 1, :assert [{:id :ex/andrew, :rdf/type [:schema/Person], :schema/name "Andrew", :ex/favNum 8}], :retract []}]
  
  @(fluree/history ledger {:history :ex/andrew
                           :t {:from 2}})
  ;; => [#:f{:t 2, :assert [{:ex/favNum 10, :id :ex/andrew}], :retract []}
  ;;     #:f{:t 1, :assert [{:id :ex/andrew, :rdf/type [:schema/Person], :schema/name "Andrew"}], :retract []}]

The first call to fluree/history looks great, results exactly as I'd expect.

The second call with :t { :from 2 } seems odd: it (1) includes the :t 1 assertions and (2) oddly leaves out the :t 2 retraction of the original :ex/favNum 10 value (which, coincidentally, is also missing from the :t 1 assertions in this second result body as well)

[dpetran]

@aaj3f that's really strange - I have a test for exactly that case and it works fine:

db/test/fluree/db/query/history_test.clj

Line 120 in 6fb8d24

(testing "from-t"

I'll experiment with your example to see if I can reproduce it.

[dpetran]

That was a tricky case - the problem was in the caching, and my tests exercised a path with more transactions and therefore no cache for the specific index node that we were filtering for flakes. This last commit adds a unique history cache key so that won't happen, and a test that fails without the fix.

[cap10morgan]

🌺