Merge pull request #5 from scicloj/ordered-value-representations

Ordered value representations

CHANGELOG.md

@@ -3,6 +3,12 @@ All notable changes to this project will be documented in this file. This change
 
 ## [1-alpha4-SNAPSHOT]
 - basic change detection (#4)
+- special value representations are defined in an ordered fashion
+- moved all is clauses to one test
+- added skip options
+- taking care of requires in the body of the notebook
+- catching exceptions
+- allowing nil as a valid value representation
 
 ## [1-alpha3]
 - shifting to shapshot-based workflow

README.md

@@ -5,24 +5,23 @@ Generating tests from Clojure notes
 [![Clojars Project](https://img.shields.io/clojars/v/org.scicloj/note-to-test.svg)](https://clojars.org/org.scicloj/note-to-test)
 
 ## Status
-Initial draft
+Experiental. Please expect breaking changes.
 
 ## Intro
 
 This is a tiny library for testable-documentation / literate-testing in Clojure.
 
 It can automatically generate tests:
 - from code examples in namespaces - already supported
-- from code examples in docstrings - coming soon
+- from code examples in docstrings - coming soon 
 
 Tests are created by running code examples and remembering their outputs. The person writing the documentation is responsible for checking that these outputs are sensible. The tests are responsible for checking these outputs remain the same on future versions.
 
 ### Test files
 Tests are written in standard `clojure.test` files. Each namespace of code examples has its own generated test file. 
-* The top-level forms in the test namespace correspond to all runnable top-level forms in the source namespace. That is, all top-level forms except for Rich `(comment ...)` blocks. 
-* The namespace definition form is adapted to the test namespace's needs. 
-* Forms that result in a var (e.g., `def`, `defn`, `defonce`) are kept as-is. 
-* All other forms are turned in to `(deftest ...)` test definitions.
+* The test namespace has one `(deftest ...)` clause with many `is` clauses.
+* The `is` clauses corresond to all runnable top-level forms in the source namespace. That is, all top-level forms except for the `(ns ...)` definition and Rich `(comment ...)` blocks. 
+* The namespace definition form is adapted to the test namespace's needs, including not only the source namespace `require`s, but also those which appear in the body of the source namespace.
 
 ## Usage
 
@@ -37,11 +36,25 @@ Assume you have a namespace, say `dum.dummy` in the file [notebooks/dum/dummy.cl
 ```clj
 (note-to-test/gentest! "notebooks/dum/dummy.clj")
 ```
-would generate a test namespace 'dum.dummy-generated-test' in the file [test/dum/dummy_generated_test.clj](test/dum/dummy_generated_test.clj) with `clojure.test` tests verifying that those code examples actually return the values they had at the time we executed that `gentest!` call.
+would generate a test namespace 'dum.dummy-generated-test' in the file [test/dum/dummy_generated_test.clj](test/dum/dummy_generated_test.clj) with checks verifying that those code examples actually return the values they had at the time we executed that `gentest!` call.
+
+The call to `gentest!` can also accept an options map.
+Options:
+- `accept` - boolean - default `false` - should we accept overriding an existing test file which has changed?
+- `verbose` - boolean - default `false` - should we report whether an existing test file has changed?
+
+E.g.:
+```clj
+(note-to-test/gentest! "notebooks/dum/dummy.clj"
+                       {:accept true
+                        :verbose true})
+```
+
+There is also a `gentests!` function that handles a list of directories, rather than one source file.
 
 ### Command Line
 
-See [scicloj.note-to-test.v1.main/-main](src/scicloj/note_to_test/v1/main)
+See [scicloj.note-to-test.v1.main/-main](src/scicloj/note_to_test/v1/main).
 
 For a deps.edn project, merge the following alias
 
@@ -63,53 +76,68 @@ Add `gentests!` to your test function in `build.clj`.
 
 ### Handling special values
 
-For plain Clojure data structures, pretty printing the structure results in a readable string that can be used for the test code.
+For plain Clojure vectors, maps, sets, and primitive values, pretty printing the structure results in a readable string that can be used for the test code as is.
 
-Other values may need some care to be represented in the test. Defining such representations can be done using `define-value-representation!`.
+Other values may need some care to be represented in the test. Defining such representations can be done using `define-value-representations!`. Applying the data representations is done internally by the library, but can also be done directly using `represent-value`. This is useful for defining recursive representations.
 
-For example, let us add support for [tech.ml.dataset](https://github.com/techascent/tech.ml.dataset) datasets, that we will use through [Tablecloth](https://scicloj.github.io/tablecloth/).
+The definition is a vector of maps, where each map has a predicate and a representation function. Each value is checked through the maps in order and represented by the representation function corresponding to the first predicate that applies to it.
 
+Toy example:
 ```clj
-(require '[tablecloth.api :as tc])
-(note-to-test/define-value-representation!
-  "tech.ml.dataset dataset"
-  {:predicate tc/dataset?
-   :representation (fn [ds]
-                     (-> ds
-                         (update-vals vec)
-                         (->> (into {}))))})
+(note-to-test/define-value-representations!
+  [{:predicate #(> % 20)
+    :representation (partial * 100)}
+   {:predicate #(> % 10)
+    :representation (partial * 10)}])
+
+(note-to-test/represent-value 9) ; => 9, no predicate applies
+(note-to-test/represent-value 19) ; => 190, second predicate applies
+(note-to-test/represent-value 29) ; => 2900, first predicate applies
 ```
 
-Now, a code example like
+For a more practical example, let us define our representations so that all values and keys of a map are represented recursively, and that long sequential structures are shortened (to keep the test file not too large) and turned into vectors (to make the printed value valid to evaluate).
+
 ```clj
-(-> {:x [1 2 3]}
-    tc/dataset
-    (tc/map-columns :y [:x] (partial * 10)))
+(note-to-test/define-value-representations!
+  [{:predicate map?
+    :representation (fn [m]
+                      (-> m
+                          (update-keys represent-value)
+                          (update-vals represent-value)))}
+   {:predicate sequential?
+    :representation (fn [v]
+                      (->> v
+                           (take 5)
+                           vec))}])
+
+(note-to-test/represent-value
+ {:x (range 99)
+  :y 9}) ; => {:x [0 1 2 3 4], :y 9}
 ```
-will result in a test like
+### Skipping tests
+Tesks will be skipped if either original form has the `^:note-to-test/skip` metadata, e.g.
+
 ```clj
-(deftest test-4
-  (is (->
-       (-> {:x [1 2 3]}
-           tc/dataset
-           (tc/map-columns :y [:x] (partial * 10)))
-       note-to-test/represent-value
-       (=
-        {:x [1 2 3], :y [10 20 30]}))))
+^:note-to-test/skip
+(+ 2 3) ; will be skipped
+```
+or the represented value is `:note-to-test/skip`, e.g.
+```clj
+(note-to-test/define-value-representations!
+  [{:predicate #{5}
+    :representation (constantly :note-to-test/skip)}])
+
+(+ 2 3) ; will be skipped
 ```
 
 ## Wishlist
-- support running from command line
-- support an alternative plain-data output format to help seeing the output changes explicitly
-- make clear error messages:
-- when outputs change
-- when the code fails
-- support code examples in comment blocks?
-- support metadata to skip certain forms
-- remove nil outputs?
-- support docstrings
 - generate tests from code examples in docstrings
 - explore generate docstrings in a structured way (marking code examples explicitly)
+- support an alternative plain-data output format to help seeing the output changes explicitly
+- make clearer error messages
+  - when outputs change
+  - when the code fails
+- support code examples in comment blocks?
 - support approximate comparisons of values for floating-point computations
 - support automatic regeneration (say, using file watch)
 
@@ -127,3 +155,4 @@ Public License, v. 2.0 are satisfied: GNU General Public License as published by
 the Free Software Foundation, either version 2 of the License, or (at your
 option) any later version, with the GNU Classpath Exception which is available
 at https://www.gnu.org/software/classpath/license.html.
+

notebooks/dum/dummy.clj

@@ -2,24 +2,33 @@
   (:require [scicloj.note-to-test.v1.api :as note-to-test]
             [tablecloth.api :as tc]))
 
-(note-to-test/define-value-representation!
-  "tablecloth dataset"
-  {:predicate tc/dataset?
-   :representation (fn [ds]
-                     (-> ds
-                         (update-vals vec)
-                         (->> (into {}))))})
+:note-to-test/skip
+(note-to-test/define-value-representations!
+  [{:predicate var?
+    :representation (constantly :var)}
+   {:predicate tc/dataset?
+    :representation (fn [ds]
+                      (-> ds
+                          (update-vals vec)
+                          (->> (into {}))))}
+   {:predicate (partial = 5)
+    :representation (constantly :five)}])
+
+(+ 2 3)
 
 (+ 4
    5
    6)
 
+:note-to-test/skip
+(+ 1 2)
 
 (defn f [x]
   (+ x 19))
 
 (f 12)
 
+(require 'clojure.java.io)
 
 (-> {:x [1 2 3]}
     tc/dataset
@@ -28,6 +37,10 @@
 (f 13)
 
 
+{:x 9}
+
 (comment
-  (note-to-test/gentest! "notebooks/dum/dummy.clj")
+  (note-to-test/gentest! "notebooks/dum/dummy.clj"
+                         {:accept true
+                          :verbose true})
   ,)

src/scicloj/note_to_test/v1/api.clj

@@ -8,18 +8,40 @@
 (defn gentest!
   "Generate a clojure.test file for the code examples in the file at `source-path`.
 
-  Example:
+  Options:
+  - `accept` - boolean - default `false` - should we accept overriding an existing test file which has changed?
+  - `verbose` - boolean - default `false` - should we report whether an existing test file has changed?
+
+  Examples:
   ```clj
   (gentest! \"notebooks/dum/dummy.clj\")
   ```
-  "
+  ```clj
+  (gentest! \"notebooks/dum/dummy.clj\"
+            {:accept true
+             :verbose true})
+  ```
+   "
   [source-path options]
   (-> source-path
       impl/prepare-context
       (impl/write-tests! options)))
 
 (defn gentests!
-  "Generate tests for all source files discovered in [dirs]."
+  "Generate tests for all source files discovered in [dirs].
+
+  Options: like in `gentest!`.
+
+  Examples:
+  ```clj
+  (gentests! [\"notebooks\"])
+  ```
+  ```clj
+  (gentests! [\"notebooks\"]
+            {:accept true
+             :verbose true})
+  ```
+  "
   ([dirs] (gentests! dirs {}))
   ([dirs options]
    (let [{:keys [verbose]} options]
@@ -30,27 +52,28 @@
              :when (impl/clojure-source? file)]
        (when verbose (println "Loading file" (str file)))
        (cond-> (gentest! file options)
-               verbose (println))))
+         verbose (println))))
    [:success]))
 
-(defn define-value-representation!
-  "Define a data representation for special values. Outputs in test code will be represented this way.
+(defn define-value-representations!
+  "Define how values should be represented in the tests.
+  The definition is a vector of maps, where each map has a predicate and a representation function. Each value is checked through the maps in order and represented by the representation function corresponding to the first predicate that applies to it.
 
-  For example, let us add support for [tech.ml.dataset](https://github.com/techascent/tech.ml.dataset) datasets, that we will use through [Tablecloth](https://scicloj.github.io/tablecloth/).
+  Example:
   ```clj
-  (require '[tablecloth.api :as tc])
-  (note-to-test/define-value-representation!
-    \"tech.ml.dataset dataset\"
-    {:predicate tc/dataset?
-     :representation (fn [ds]
-                       `(tc/dataset ~(-> ds
-                                         (update-vals vec)
-                                         (->> (into {})))))})
+  (define-value-representations!
+    [{:predicate #(> % 20)
+      :representation (partial * 100)}
+     {:predicate #(> % 10)
+      :representation (partial * 10)}])
+
+  (represent-value 9) ; => 9, no predicate applies
+  (represent-value 19) ; => 190, second predicate applies
+  (represent-value 29) ; => 2900, first predicate applies
   ```
   "
-  [name spec]
-  (impl/define-value-representation! name spec))
-
+  [representations]
+  (impl/define-value-representations! representations))
 
 (defn represent-value
   "Represent a given value `v` using the extensible definitions of special value representations."