Skip to content

Commit

Permalink
feat: babashka pod (#630)
Browse files Browse the repository at this point in the history
* feat: babashka pod

* fixup! feat: babashka pod

* fixup! feat: babashka pod

* fixup! feat: babashka pod

* fixup! feat: babashka pod

* format: auto format

* chore: only using one json lib

* chore: remove comment blocks

* docs: babashka pod

* Revert "chore: bump deps (#627)"

This reverts commit f31b4bf.

* fixup! feat: babashka pod

* Revert "Revert "chore: bump deps (#627)""

This reverts commit c389ba9.

* fixup! feat: babashka pod

* fixup! feat: babashka pod

* fixup! feat: babashka pod

* fixup! feat: babashka pod

* fixup! feat: babashka pod

* fixup! feat: babashka pod

* fixup! feat: babashka pod

* fixup! feat: babashka pod

* fixup! feat: babashka pod

* fixup! feat: babashka pod

* Revert "fixup! feat: babashka pod"

This reverts commit d1225b2.

* fixup! feat: babashka pod

* fixup! feat: babashka pod

* fixup! feat: babashka pod
  • Loading branch information
TimoKramer committed Aug 3, 2023
1 parent 549e122 commit 9f08580
Show file tree
Hide file tree
Showing 9 changed files with 608 additions and 46 deletions.
16 changes: 16 additions & 0 deletions .circleci/config.yml
Original file line number Diff line number Diff line change
Expand Up @@ -156,6 +156,17 @@ jobs:
cd /home/circleci/replikativ
bb test native-image
no_output_timeout: 5m
bb-pod-test:
executor: tools/clojurecli
steps:
- attach_workspace:
at: /home/circleci
- run:
name: Run bb-pod test
command: |
cd /home/circleci/replikativ
bb test bb-pod
no_output_timeout: 5m
backward-compatibility-test:
executor: tools/clojurecli
steps:
Expand Down Expand Up @@ -248,6 +259,11 @@ workflows:
requires:
- build
- native-image
- bb-pod-test:
context: dockerhub-deploy
requires:
- build
- native-image
- backward-compatibility-test:
context: dockerhub-deploy
requires:
Expand Down
62 changes: 31 additions & 31 deletions bb.edn
Original file line number Diff line number Diff line change
Expand Up @@ -14,18 +14,18 @@
:init (do (def config-file "config.edn")
(def config (edn/read-string (slurp config-file)))
(def deps (edn/read-string (slurp "deps.edn"))))

; tools

bench {:doc "Run benchmarks"
:task (clojure "-M:benchmark" "measure")}
ffix {:doc "Format source files"
:task (clojure "-M:benchmark" "measure")}

ffix {:doc "Format source files"
:task (clojure "-M:ffix")}


; checks

format {:doc "Test formatting"
:task (clojure "-M:format")}

Expand All @@ -39,29 +39,29 @@

kaocha {:doc "Run kaocha with arbitrary arguments"
:task (apply test/kaocha *command-line-args*)}

outdated {:doc "Find outdated libraries"
:task (clojure "-M:outdated")}

check {:doc "Run all checks"
:depends [test format lint outdated]}


; build and release
inc {:doc "Increment the project version: [major|minor] <version>"

inc {:doc "Increment the project version: [major|minor] <version>"
:task (apply version/inc config-file *command-line-args*)}

tag {:doc "Return current version as a tag"
:task (println (version/as-tag config))}

clean {:doc "Remove build files"
clean {:doc "Remove build files"
:task (build/clean (-> config :build :clj))}

jcompile {:doc "Compile java classes"
:depends [clean]
:task (build/compile-java (-> config :build :clj))}

ccompile {:doc "Compile clojure namespaces"
:task (build/compile-clojure (-> config :build :clj))}

Expand All @@ -70,7 +70,7 @@

jar {:doc "Build jar"
:depends [jcompile pom]
:task (build/jar config (-> config :build :clj))}
:task (build/jar config (-> config :build :clj))}

install {:doc "Install jar locally"
:task (deploy/local config (-> config :build :clj))}
Expand All @@ -82,36 +82,36 @@
:depends [jar]
:task (let [jar (build/jar-path config (-> config :build :clj))]
(release/gh-release jar config))}

;; native image





ni-check {:doc "Check for 'native-image' program"
:task (try (shell "which" "native-image")
(println "Program native-image found!")
(catch Exception _
(println "PATH does not contain native-image! Make sure to add your GraalVM to it.")
(System/exit 1)))}
:task (try (shell "which" "native-image")
(println "Program native-image found!")
(catch Exception _
(println "PATH does not contain native-image! Make sure to add your GraalVM to it.")
(System/exit 1)))}

ni-cli {:doc "Build native image cli"
:depends [jcompile ni-check]
:task (clojure "-M:native-cli" "--no-fallback" "-J-Xmx4g")}
:depends [jcompile ni-check]
:task (clojure "-M:native-cli" "--no-fallback" "-J-Xmx4g")}

ni-ccompile {:doc "Create native cpp library"
:task (build/compile-clojure (-> config :build :native))}

ni-uber {:doc "Build native image uber jar"
:depends [jcompile ccompile ni-ccompile]
:task (build/uber config (-> config :build :native))}
ni-compile {:doc "Create native cpp library"

ni-compile {:doc "Create native cpp library"
:depends [ni-uber]
:task (build/native-compile config (-> config :build :native))}
ni-release {:doc "Build and release native assets to GitHub"

ni-release {:doc "Build and release native assets to GitHub"
:depends [ni-compile]
:task (let [{:keys [artifact target-dir]} (-> config :build :native)]
(fs/zip artifact [target-dir])
(fs/zip artifact [target-dir])
(release/gh-release artifact config)
(fs/delete artifact))}}}
161 changes: 161 additions & 0 deletions bb/resources/native-image-tests/run-bb-pod-tests.clj
Original file line number Diff line number Diff line change
@@ -0,0 +1,161 @@
#!/usr/bin/env bb

(require '[babashka.pods :as pods]
'[clojure.test :refer [run-tests deftest testing is]])
(import '[java.util Date])

(pods/load-pod ["clojure" "-M:pod" "-m" "datahike.cli"])

(require '[datahike.pod :as d])

(def config {:keep-history? true,
:search-cache-size 10000,
:index :datahike.index/persistent-set,
:store {:id "inexpensive-red-fox", :backend :mem},
:store-cache-size 1000,
:attribute-refs? false,
:writer {:backend :self},
:crypto-hash? false,
:schema-flexibility :read,
:branch :db})

(deftest pod-workflow

(testing "delete-database"
(is (= {}
(d/delete-database config))))

(testing "create-database"
(is (= {:keep-history? true
:search-cache-size 10000
:index :datahike.index/persistent-set
:store {:id "inexpensive-red-fox", :backend :mem}
:store-cache-size 1000
:attribute-refs? false
:writer {:backend :self}
:crypto-hash? false
:schema-flexibility :read
:branch :db}
(d/create-database config))))

(testing "database-exists?"
(is (= true
(d/database-exists? config))))

(let [conn (d/connect config)]
(testing "connect"
(is (= "conn:-1033483489"
conn)))
(testing "transact"
(is (= [:tempids :db-before :db-after :tx-meta :tx-data]
(keys (d/transact conn [{:name "Alice", :age 20}
{:name "Bob", :age 30}
{:name "Charlie", :age 40}
{:age 15}])))))
(testing "transact with bad arg"
(is (thrown? clojure.lang.ExceptionInfo
(keys (d/transact
"foo"
[{:name "Alice", :age 20}
{:name "Bob", :age 30}
{:name "Charlie", :age 40}
{:age 15}])))))
(testing "with-db"
(is (= #{[2 "Bob" 30] [1 "Alice" 20] [3 "Charlie" 40]}
(d/with-db [db (d/db conn)]
(d/q {:query '{:find [?e ?n ?a]
:where
[[?e :name ?n]
[?e :age ?a]]}
:args [(d/db conn)]})))))
(testing "release-db"
(let [db (d/db conn)]
(is (= {}
(d/release-db db)))))
(testing "q"
(is (= #{[2 "Bob" 30] [1 "Alice" 20] [3 "Charlie" 40]}
(d/q {:query '{:find [?e ?n ?a]
:where
[[?e :name ?n]
[?e :age ?a]]}
:args [(d/db conn)]})
(d/q '[:find ?e ?n ?a
:where
[?e :name ?n]
[?e :age ?a]]
(d/db conn)))))
(let [timestamp (Date.)]
(d/transact conn {:tx-data [{:db/id 3 :age 25}]})
(d/transact conn [{:name "FOO" :age "BAR"}])
(testing "pull"
(is (= {:db/id 1, :age 20, :name "Alice"}
(d/pull (d/db conn) '[*] 1))))
(testing "pull-many"
(is (= [{:db/id 1, :age 20, :name "Alice"}
{:db/id 2, :age 30, :name "Bob"}
{:db/id 3, :age 25, :name "Charlie"}]
(d/pull-many (d/db conn) '[*] [1 2 3]))))
(testing "metrics"
(is (= {:per-attr-counts {:age 5, :name 4, :db/txInstant 3}
:per-entity-counts {1 2, 2 2, 3 2, 4 1, 5 2, 536870913 1, 536870914 1, 536870915 1}
:count 12, :avet-count 0, :temporal-count 11, :temporal-avet-count 0}
(d/metrics (d/db conn)))))
(testing "as-of tx-id"
(is (= #{[3 "Charlie" 25] [2 "Bob" 30] [5 "FOO" "BAR"] [1 "Alice" 20]}
(d/q '[:find ?e ?n ?a
:where
[?e :name ?n]
[?e :age ?a]]
(d/as-of (d/db conn) 536870916)))))
(testing "as-of timestamp"
(is (= #{[2 "Bob" 30] [1 "Alice" 20] [3 "Charlie" 40]}
(d/q '[:find ?e ?n ?a
:where
[?e :name ?n]
[?e :age ?a]]
(d/as-of (d/db conn) timestamp)))))
(testing "since tx-id"
(is (= #{[5 "FOO" "BAR"]}
(d/q '[:find ?e ?n ?a
:where
[?e :name ?n]
[?e :age ?a]]
(d/since (d/db conn) 536870914)))))
(testing "since timestamp"
(is (= #{[5 "FOO" "BAR"]}
(d/q '[:find ?e ?n ?a
:where
[?e :name ?n]
[?e :age ?a]]
(d/since (d/db conn) timestamp)))))
(testing "history"
(is (= #{[3 "Charlie" 25] [2 "Bob" 30] [5 "FOO" "BAR"] [1 "Alice" 20] [3 "Charlie" 40]}
(d/q '[:find ?e ?n ?a
:where
[?e :name ?n]
[?e :age ?a]]
(d/history (d/db conn))))))
(testing "datoms"
(is (= '((1 :age 20 536870913 true))
(d/datoms (d/db conn) :eavt 1 :age 20)))
(is (= '((2 :age 30 536870913 true) (2 :name "Bob" 536870913 true))
(d/datoms (d/db conn) {:index :eavt :components [2]}))))
(testing "schema"
(d/transact conn {:tx-data [{:db/ident :name :db/valueType :db.type/string :db/unique :db.unique/identity
:db/index true :db/cardinality :db.cardinality/one}
{:db/ident :age :db/valueType :db.type/long :db/cardinality :db.cardinality/one}]})
(is (= {:name {:db/ident :name, :db/valueType :db.type/string, :db/unique :db.unique/identity
:db/index true, :db/cardinality :db.cardinality/one, :db/id 6}
:age {:db/ident :age, :db/valueType :db.type/long, :db/cardinality :db.cardinality/one, :db/id 7}}
(d/schema (d/db conn))))
(testing "entity"
(is (= {:age "BAR" :name "FOO"}
(d/entity (d/db conn) 5))))))))

(defn -main [& _args]
(let [{:keys [fail error]} (run-tests)]
(when (and fail error (pos? (+ fail error)))
(System/exit 1))))

(when (= *file* (System/getProperty "babashka.file"))
(apply -main *command-line-args*))
9 changes: 8 additions & 1 deletion bb/src/tools/test.clj
Original file line number Diff line number Diff line change
Expand Up @@ -49,19 +49,26 @@
(p/shell "./bb/resources/native-image-tests/run-native-image-tests")
(println "Native image cli missing. Please run 'bb ni-cli' and try again.")))

(defn bb-pod []
(if (fs/exists? "./dhi")
(p/shell "./bb/resources/native-image-tests/run-bb-pod-tests.clj")
(println "Native image cli missing. Please run 'bb ni-cli' and try again.")))

(defn specs []
(kaocha "--focus" "specs" "--plugin" "kaocha.plugin/orchestra"))

(defn all [config]
(kaocha "--skip" "specs")
(specs)
(back-compat config)
(native-image))
(native-image)
(bb-pod))

(defn -main [config & args]
(if (seq args)
(case (first args)
"native-image" (native-image)
"bb-pod" (bb-pod)
"back-compat" (back-compat config)
"specs" (specs)
(apply kaocha "--focus" args))
Expand Down
19 changes: 14 additions & 5 deletions deps.edn
Original file line number Diff line number Diff line change
Expand Up @@ -6,10 +6,9 @@
io.replikativ/konserve {:mvn/version "0.7.311"}
persistent-sorted-set/persistent-sorted-set {:mvn/version "0.2.3"}
environ/environ {:mvn/version "1.2.0"}
cheshire/cheshire {:mvn/version "5.11.0"}
com.taoensso/timbre {:mvn/version "6.1.0"}
io.replikativ/superv.async {:mvn/version "0.3.46"}
io.replikativ/datalog-parser {:mvn/version "0.2.28"}
com.taoensso/timbre {:mvn/version "5.2.1"}
io.replikativ/superv.async {:mvn/version "0.3.43"}
io.replikativ/datalog-parser {:mvn/version "0.2.25"}
io.replikativ/zufall {:mvn/version "0.2.9"}
junit/junit {:mvn/version "4.13.2"}
medley/medley {:mvn/version "1.4.0"}
Expand Down Expand Up @@ -69,7 +68,17 @@
{clj.native-image/clj.native-image
{:git/url "https://github.com/taylorwood/clj.native-image.git"
:sha "7708e7fd4572459c81f6a6b8e44c96f41cdd92d4"}
org.clojure/data.json {:mvn/version "2.4.0"}}}
com.cognitect/transit-clj {:mvn/version "1.0.333"}
nrepl/bencode {:mvn/version "1.1.0"}
babashka/pods {:git/url "https://github.com/babashka/pods"
:git/sha "8b717eb001811bc5da5d15d1163565de00b4ffa4"}
cheshire/cheshire {:mvn/version "5.11.0"}}}

:pod {:extra-deps {com.cognitect/transit-clj {:mvn/version "1.0.333"}
nrepl/bencode {:mvn/version "1.1.0"}
babashka/pods {:git/url "https://github.com/babashka/pods"
:git/sha "8b717eb001811bc5da5d15d1163565de00b4ffa4"}
cheshire/cheshire {:mvn/version "5.11.0"}}}

;; Checks

Expand Down
10 changes: 10 additions & 0 deletions doc/babashka_pod.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
# Datahike Babashka Pod

This doc is about using Datahike from a [babashka pod](https://babashka.org/).

## Why Datahike as a babashka pod?

A babashka pod is a way to use Datahike in babashka without Datahike being built into the babashka binary. When you
don't want to run a whole JVM for a little script you can use babashka and write your script in your preferred language.
Babashka already has ways to persist data to other databases but Datahike adds the possibility to write to a durable
datalog database.

0 comments on commit 9f08580

Please sign in to comment.