Skip to content

Commit

Permalink
enhance: make bb dev:db-transact usable for any task
Browse files Browse the repository at this point in the history
closes LOG-2647. This task was used to update multiple graphs
that were now invalid with the removal of :block/type "object"
  • Loading branch information
logseq-cldwalker committed Nov 16, 2023
1 parent cca80e3 commit f64f146
Show file tree
Hide file tree
Showing 2 changed files with 53 additions and 18 deletions.
13 changes: 7 additions & 6 deletions deps/outliner/script/transact.cljs
Expand Up @@ -12,10 +12,10 @@
["os" :as os]))

(defn -main [args]
(when (< (count args) 2)
(println "Usage: $0 GRAPH-DIR QUERY")
(when (< (count args) 3)
(println "Usage: $0 GRAPH-DIR QUERY TRANSACT-FN")
(js/process.exit 1))
(let [[graph-dir query*] args
(let [[graph-dir query* transact-fn*] args
dry-run? (contains? (set args) "-n")
[dir db-name] (if (string/includes? graph-dir "/")
((juxt node-path/dirname node-path/basename) graph-dir)
Expand All @@ -24,15 +24,16 @@
conn (sqlite-cli/read-graph db-name)
;; find blocks to update
query (into (edn/read-string query*) [:in '$ '%]) ;; assumes no :in are in queries
transact-fn (edn/read-string transact-fn*)
blocks-to-update (mapv first (d/q query @conn (rules/extract-rules rules/db-query-dsl-rules)))
;; TODO: Make this configurable
update-tx (mapv #(vector :db.fn/retractEntity %)
;; TODO: Use sci eval when it's available in nbb-logseq
update-tx (mapv (fn [id] (eval (list transact-fn id)))
blocks-to-update)]
(if dry-run?
(do (println "Would update" (count blocks-to-update) "blocks with the following tx:")
(prn update-tx)
(println "With the following blocks updated:")
(prn (map #(into {} (d/entity @conn %)) blocks-to-update)))
(prn (map #(select-keys (d/entity @conn %) [:block/name :block/content]) blocks-to-update)))
(do
(persist-graph/add-listener conn db-name)
(d/transact! conn update-tx)
Expand Down
58 changes: 46 additions & 12 deletions docs/dev-practices.md
Expand Up @@ -313,18 +313,6 @@ point out:
bb dev:validate-repo-config-edn deps/common/resources/templates/config.edn
```

* `dev:validate-db` - Validates a DB graph's datascript schema

```sh
# One time setup
$ cd deps/db && yarn install && cd -
# One or more graphs can be validated e.g.
$ bb dev:validate-db test-db schema -c -g
Read graph test-db with 1572 datoms, 220 entities and 13 properties
Valid!
Read graph schema with 26105 datoms, 2320 entities and 3168 properties
Valid!
```

* `dev:publishing` - Build a publishing app for a given graph dir. If the
publishing frontend is out of date, it builds that first which takes time.
Expand All @@ -343,6 +331,52 @@ There are also some tasks under `nbb:` which are useful for inspecting database
changes in realtime. See [these
docs](https://github.com/logseq/bb-tasks#logseqbb-tasksnbbwatch) for more info.

#### DB Graph Tasks

These tasks are specific to database graphs. For these tasks there is a one time setup:

```sh
$ cd deps/db && yarn install && cd -
```

* `dev:validate-db` - Validates a DB graph's datascript schema

```sh
# One or more graphs can be validated e.g.
$ bb dev:validate-db test-db schema -c -g
Read graph test-db with 1572 datoms, 220 entities and 13 properties
Valid!
Read graph schema with 26105 datoms, 2320 entities and 3168 properties
Valid!
```

* `dev:db-query` - Query a DB graph

```sh
$ bb dev:db-query woot '[:find (pull ?b [*]) :where (block-content ?b "Dogma")]'
DB contains 833 datoms
[{:block/tx-id 536870923, :block/link #:db{:id 100065}, :block/uuid #uuid "65565c26-f972-4400-bce4-a15df488784d", :block/updated-at 1700158508564, :block/left #:db{:id 100051}, :block/refs [#:db{:id 100064}], :block/created-at 1700158502056, :block/format :markdown, :block/tags [#:db{:id 100064}], :block/content "Dogma #~^65565c2a-b1c5-4dc8-a0f0-81b786bc5c6d", :db/id 100090, :block/path-refs [#:db{:id 100051} #:db{:id 100064}], :block/parent #:db{:id 100051}, :block/page #:db{:id 100051}}]
```
* `dev:db-transact` - Run a `d/transact!` against the queried results of a DB graph
```sh
# The second arg is a datascript like with db-query. The third arg is a fn that is applied to each query result to generate transact data
$ bb dev:db-transact
Usage: $0 GRAPH-DIR QUERY TRANSACT-FN

# First use the -n flag to see a dry-run of what would happen
$ bb dev:db-transact test-db '[:find ?b :where [?b :block/type "object"]]' '(fn [id] (vector :db/retract id :block/type "object"))' -n
Would update 16 blocks with the following tx:
[[:db/retract 100137 :block/type "object"] [:db/retract 100035 :block/type "object"] [:db/retract 100128 :block/type "object"] [:db/retract 100049 :block/type "object"] [:db/retract 100028 :block/type "object"] [:db/retract 100146 :block/type "object"] [:db/retract 100144 :block/type "object"] [:db/retract 100047 :block/type "object"] [:db/retract 100145 :block/type "object"] [:db/retract 100046 :block/type "object"] [:db/retract 100045 :block/type "object"] [:db/retract 100063 :block/type "object"] [:db/retract 100036 :block/type "object"] [:db/retract 100044 :block/type "object"] [:db/retract 100129 :block/type "object"] [:db/retract 100030 :block/type "object"]]
With the following blocks updated:
...

# When the transact looks good, run it without the flag
$ bb dev:db-transact test-db '[:find ?b :where [?b :block/type "object"]]' '(fn [id] (vector :db/retract id :block/type "object"))'
Updated 16 block(s) for graph test-db!
```
### Dev Commands
In the app, you can enable Dev commands under `Settings > Advanced > Developer
Expand Down

0 comments on commit f64f146

Please sign in to comment.