WIP 6

backend/specs/Readme.org

@@ -31,45 +31,154 @@ Lumen is a Component Oriented System, it uses [[https://github.com/duct-framewor
 Main idea behind component is ~Managed lifecycle of stateful objects in Clojure~ (IoC or DI)
 
 ;; WIP #+INCLUDE: "components.org" 
-* config                                                             :config:
+* config                                                   
 Configuration of lumen backend is done through [[https://github.com/akvo/akvo-lumen/blob/specs-lp/backend/resources/akvo/lumen/system.edn#L1][system.edn]]
  #+INCLUDE: "config.org" 
-* db                                                                     :db:
+* db                                                       
 #+INCLUDE: "db.org" 
-* DELEGATED endpoints                                             :endpoints:
+* DELEGATED endpoints                                        
   CLOSED: [2018-06-20 Wed 23:28]
 these endopoints functions should be specified as [[https://clojure.org/guides/spec#_higher_order_functions][HOF]]
 
 ;; #+INCLUDE: "endpoints.org" 
 * DELEGATED imports
   CLOSED: [2018-06-20 Wed 23:28]
 ;; #+INCLUDE: "imports.org" 
-* libs                                                                 :libs:
+* libs                                                      
 #+INCLUDE: "libs.org" 
-* transformations                                           :transformations:
+* transformations                                           
 #+INCLUDE: "transformations.org" 
 
 * FAQs
 
-** TODO explain where to put this stuff :) ?
- explain why choose ~backend/src/akvo/lumen/specs/~ as org folder and why generated html has to live in ~docs~ folder
-** FEEDBACK ~[org.clojure/test.check "0.9.0" :scope "test"]~
-   only reason to have this ~test~ dependency in ~prod~ side is to be able to generate samples inside custom generator
+
+** where to put this stuff :) ?                      
+ these org files live in akvo-lumen/backend/specs as far as specs relates to backend side  
+ the compiled clojure source is located at ~backend/src/akvo/lumen/specs/~  to be available in ~prod~ environment
+and generated html documentation live in ~docs~ folder as [[https://pages.github.com/][requirement]] to be available from github pages
+** FEEDBACK Why is ~[org.clojure/test.check "0.9.0" :scope "test"]~ in ~prod~ dependencies?
+   the only reason to have this ~test~ dependency in ~prod~ side is to be able to [[https://clojure.org/guides/spec#_sampling_generators][generate samples]] inside custom generator. 
 
-** DONE investigate if clojure.spec can check value returned by function spec(ed)
-   CLOSED: [2018-06-20 Wed 22:28]
-clojure.spec only can check/instrument args in function call but no the value returned by the function. 
+** can ~clojure.spec~ check value returned by function spec(ed)?   
+clojure.spec only can check/instrument args in function call but *no* the value returned by the function. 
 
 Following this [[https://groups.google.com/forum/#!topic/clojure/JU6EmjtbRiQ][thread]] is not possible (using plain clojure.spec lib) but there is other option to test it https://github.com/jeaye/orchestra 
 
+** how to naming specs namespaces and aliases?
+Specs usually live in same namespace as the data they are referring
+
+Below a sample data inside a sample namespace
+
+#+BEGIN_SRC clojure :exports code :results no
+(ns example.one)
+
+(def d {:id "12345abcde"})
+
+#+END_SRC
+
+And a sample spec namespace that defines specs for previous namespace
+
+#+BEGIN_SRC clojure :exports code :results no
+(ns example.specs
+  (:require [example.one :as one]
+            [clojure.spec.alpha :as s]))
+
+(s/def ::one/id string?)
+(s/def ::one/d (s/keys :req-un [::one/id]))
+
+#+END_SRC
+
+As you can see, specs live in its own namespace (~example.specs~ in the example provided) and they refers to the data to specify using the alias namespace (~::one/....~ again in the example provided)
+
+By default, we'll need to create a spec namespace for the ~src~ namespace to specify adding ~.specs~ suffix 
+
+** creating namespaces to solve qualified keywords spec ids
+spec ids need to be [[https://clojuredocs.org/clojure.core/qualified-keyword_q][qualified keywords]] , sometimes this requirement we'll force us to create new namespaces to allocate the specs. 
+this is an [[https://github.com/akvo/akvo-lumen/blob/org-specs/backend/src/akvo/lumen/specs/aggregation/query.clj#L1][example of creating a ns to allocate spec]]: in this case [[https://github.com/akvo/akvo-lumen/blob/6567c0a907807d586d6744a855d0b26aa1638c84/backend/src/akvo/lumen/lib/aggregation.clj#L14][query]] as data doesn't have its own namespace but it is used from other nss as aggregation.clj 
+
+** aliasing namespaces 
+as you surely have realised, soon we could need a lot of typing to refere specs using qualified keywords ...
+
+#+BEGIN_SRC clojure :exports code :results no
+
+(s/def ::akvo.lumen.library.example.one/d (s/keys :req-un [::akvo.lumnen.library.one/id]))
+
+#+END_SRC
+
+so aliasing seems to be pretty important here
+
+#+BEGIN_SRC clojure :exports code :results no
+(require '[akvo.lumen.library.example.one :as one])
+
+(s/def ::one/d (s/keys :req-un [::one/id]))
+
+#+END_SRC
+
+But, a new problem appears thus if we have a medium size file, we'll soon loose the meaning of aliasing 
+
+
+#+BEGIN_SRC clojure :exports code :results no
+(require '[akvo.lumen.library.js.engine :as engine])
+(require '[akvo.lumen.library.example.one :as one])
+;; ... 100 LOC after
+;; .....
+
+(s/def ::one/d (s/keys :req-un [::one/id ::engine/type]))
+;; ups, I totally forgot the origin of ::engine at this point :(
+
+#+END_SRC
+
+
+and/or we could face conflict naming problems
+
+#+BEGIN_SRC clojure :exports code :results no
+(require '[akvo.lumen.library.js.one :as one])
+(require '[akvo.lumen.library.example.one :as one])
+
+;; ups two clj files in differents folders with same aliasing pattern
+
+#+END_SRC
+
+
+so ... following  the first approach/patter to aliasing specs 
+
+#+BEGIN_SRC clojure :exports code :results no
+(ns akvo.lumen.specs.aggregation
+  (:require [akvo.lumen.component.tenant-manager :as tenant-manager]
+	    [akvo.lumen.lib :as lib]
+	    [akvo.lumen.lib.aggregation :as lib.aggregation]
+	    [akvo.lumen.lib.aggregation.filter :as l.aggregation.filter]
+	    [akvo.lumen.lib.aggregation.pie :as l.aggregation.pie]
+	    [akvo.lumen.lib.aggregation.pivot :as l.aggregation.pivot]
+	    [akvo.lumen.lib.aggregation.utils :as l.aggregation.utils]
+	    [akvo.lumen.specs.aggregation.pivot.row :as a.pivot.row.s]
+	    [akvo.lumen.specs.aggregation.query :as aggregation.query.s]
+	    [akvo.lumen.specs.core :as lumen.s]
+	    [akvo.lumen.specs.dataset :as dataset.s]
+	    [akvo.lumen.specs.dataset.column :as dataset.column.s]
+	    [akvo.lumen.specs.db :as db.s]
+	    [akvo.lumen.specs.libs]
+	    [clojure.spec.alpha :as s]))
+
+#+END_SRC
+
++ all aliases ignore common project domain => ~akvo.lumen~ 
+is also ignored
+~[akvo.lumen.lib :as lib]~
+
+
++ if namespace contains only one or two nss then we use full nss
+~[akvo.lumen.lib :as lib]~
+~[akvo.lumen.lib.aggregation :as lib.aggregation]~
 
++ if namespace contains more than two nss then we contract parent ones 
 
++ if namespace is a ~spec~ namespace, we ignored  ~akvo.lumen.specs~ and aliasing ends in ~.s~
+=> ~[akvo.lumen.specs.aggregation.query :as aggregation.query.s]~
+=> ~[akvo.lumen.specs.aggregation.pivot.row :as a.pivot.row.s]~
 
-** TODO [[https://github.com/akvo/akvo-lumen/issues/1441#issuecomment-398687412][how to naming schemas ns and aliases]]
-** TODO how to aliasing namespaces
-the other problem to think about is how long this alias should be ... and which pattern to apply when we need to reduce naming
 
-* tools                                                               :index:
+* tools                                                               
 
 ** dev
    #+INCLUDE: "dev.org"

backend/src/akvo/lumen/lib/aggregation/utils.clj

@@ -7,4 +7,4 @@
       (throw (ex-info "No such column" {:columnName column-name})))))
 
 (defn load-columns [data columns]
-  (map #(assoc % :column (find-column columns (:column %))) data))
+  (map #(assoc % :column (find-column columns (:columnName %))) data))