Permalink
Browse files

Initial commit

  • Loading branch information...
0 parents commit 570be211ee28469b4af25e75f73449f24d4a3876 David Leatherman committed Nov 14, 2012
Showing with 363 additions and 0 deletions.
  1. +8 −0 .gitignore
  2. +120 −0 README.md
  3. +2 −0 etc/config.clj
  4. +10 −0 project.clj
  5. +135 −0 src/carica/core.clj
  6. +70 −0 test/carica/test/core.clj
  7. +14 −0 test/config.clj
  8. +4 −0 test/config.json
@@ -0,0 +1,8 @@
+/pom.xml
+*jar
+/lib
+/classes
+/native
+/.lein-failures
+/checkouts
+/.lein-deps-sum
120 README.md
@@ -0,0 +1,120 @@
+# Carica
+
+Carica is a flexible configuration library.
+
+It offers:
+* a simple lookup syntax
+* support for both Clojure and JSON config files
+* config file merging (if you have more than one config file)
+ * Even if one is a Clojure file and the other is JSON
+* code evaluation in Clojure files
+* runtime override capabilities for testing
+* easy default config file names (config.clj and config.json)
+ * ability to override the defaults
+
+## Setup
+
+Carica looks for the config files on the classpath.
+
+In your project.clj, add a directory to your resources-path, for these
+examples, I'll be using "etc":
+
+```clojure
+:resources-path "etc"
@rboyd

rboyd Dec 8, 2012

This didn't work for me. Should it be:

:resource-paths ["etc"]

(which did work)?

@leathekd

leathekd Dec 8, 2012

Contributor

Thanks for pointing this out. I believe it's a lein1 vs lein2 issue. I'll verify and update the docs.

+```
+
+Now, create an "etc" directory at the root of your project. Create
+and open "etc/config.clj" in your favorite editor.
+
+```clojure
+{:foobar-timeout 300 #_"In seconds"
+ :favorite-hour-of-day 8 #_"0-23"
+ :blacklist nil
+ :export-dir "/mnt/export"
+ :timeout-ms (* 20 60 1000) #_"20 minutes"
+ :db {:classname "org.postgresql.Driver"
+ :subprotocol "postgresql"
+ :subname "//localhost/test"
+ :username "cosmo"
+ :password "toomanysecrets"}}
+```
+
+(If you're wondering about the #_"" comments, we've found that they're
+less prone to errors in the configuration files. That is, from
+accidental newlines or from pulling up the closing brace into a line
+with a ;; comment.)
+
+## Usage
+
+Now, with all of that in place, open a new REPL session:
+
+```clojure
+(use '[carica.core])
+
+(config :export-dir)
+;;=> "/mnt/export"
+
+(config :db :username)
+;;=> "cosmo"
+
+(config :blacklist)
+;;=> nil
+
+(config :non-existent-key)
+;;=> nil (with a warning message logged)
+```
+
+That's it!
+
+## Overriding the defaults
+
+Maybe you already have a config file with a different name, or a
+config.clj that you use for a different purpose. No problem. To
+override what files Carica loads you can create your own `config`
+function using the `configurer` function.
+
+```clojure
+(ns my-proj.config
+ (:require [carica.core :refer [configurer
+ resources]]))
+
+(def config (configurer (resources "proj_config.clj")))
+```
+
+Calling `my-proj.config/config` will work the same as calling
+`carica.core/config` except that it will use your config file.
+
+## Testing
+
+Sometimes, during tests, it's handy to be able to override config
+values:
+
+```clojure
+(with-redefs [config (override-config :db :password "swordfish")]
+
+ (config :db :password)
+ ;;=> "swordfish"
+
+ (config :db :username))
+ ;;=> "cosmo"
+```
+
+Or:
+
+```clojure
+(with-redefs [config (override-config :db {:username "wagstaff"
+ :password "swordfish"})]
+ (config :db :password)
+ ;;=> "swordfish"
+
+ (config :db :username))
+ ;;=> "wagstaff"
+```
+
+Only the provided values will be overwritten.
+
+## License
+
+Copyright (C) 2012 Sonian, Inc.
+
+Distributed under the Eclipse Public License, the same as Clojure.
@@ -0,0 +1,2 @@
+{:from-etc true
+ :merged-val "etc"}
@@ -0,0 +1,10 @@
+(defproject sonian/carica "1.0.0"
+ :description "A flexible configuration library"
+ :dependencies [[cheshire "4.0.4"]
+ [org.clojure/tools.logging "0.2.3"]]
+ :profiles {:dev
+ {:resource-paths ["etc"],
+ :dependencies [[org.clojure/clojure "1.4.0"]]}}
+ ;; For Lein 1
+ :dev-dependencies [[org.clojure/clojure "1.4.0"]]
+ :dev-resources-path "etc")
@@ -0,0 +1,135 @@
+(ns carica.core
+ (:use [clojure.java.io :only [reader]])
+ (:require [clojure.tools.logging :as log]
+ [clojure.walk :as walk]
+ [cheshire.core :as json]))
+
+(defn resources
+ "Search the classpath for resources matching the given path"
+ [path]
+ (when path
+ (reverse
+ (enumeration-seq
+ (.getResources
+ (.getContextClassLoader
+ (Thread/currentThread))
+ path)))))
+
+(defn merge-nested [v1 v2]
+ (if (and (map? v1) (map? v2))
+ (merge-with merge-nested v1 v2)
+ v2))
+
+(defmulti load-config (comp second
+ (partial re-find #"\.([^..]*?)$")
+ (memfn getPath)))
+
+(defmethod load-config "clj" [resource]
+ (try
+ (eval
+ (try
+ (read-string (slurp resource))
+ (catch Throwable t
+ (log/warn t "error reading config" resource)
+ (throw
+ (Exception. (str "error reading config " resource) t)))))
+ (catch Throwable t
+ (log/warn t "error evaling config" resource)
+ (throw
+ (Exception. (str "error evaling config " resource) t)))))
+
+(defmethod load-config "json" [resource]
+ (with-open [s (.openStream resource)]
+ (-> s reader (json/parse-stream true))))
+
+(defn get-configs
+ "Takes a data structure of config resources (URLs) in priority order and
+ merges them together. The resources can be a simple list where first-in wins.
+ Additionally the structure may contain maps where the key becomes the
+ effective namespace of the resources in the value.
+
+ Each node is handled by type:
+ - resources (URL): load the config
+ - collections (except for maps): merge the members
+ - all others, return as is
+
+ E.g., the following:
+ [#<URL file:/some/path1>
+ {:ns1 [#<URL file:/some/path2> #<URL file:/some/path3>]}]
+
+ would become:
+ {<keys and values from /some/path>
+ :ns1 {<the merged keys and value from path2 and path3>}}"
+ [resources]
+ (walk/postwalk
+ (fn [n]
+ (cond (= java.net.URL (class n))
+ (load-config n)
+ (map? n)
+ n
+ ;; don't include vectorized maps
+ (and (coll? n) (coll? (first n)))
+ (apply merge-with merge-nested (reverse n))
+ (nil? n)
+ {}
+ :else
+ n))
+ resources))
+
+(defn config*
+ "Looks up the keys in the maps. If not found, log and return nil."
+ [m ks]
+ (let [v (get-in m ks ::not-found)]
+ (if (= v ::not-found)
+ (log/warn ks "isn't a valid config")
+ v)))
+
+(defn configurer
+ "Given a the list of resources in the format expected by get-configs,
+ return a function that can be used to search the configuration files
+ in the following manner"
+ [resources]
+ (let [configs (get-configs resources)]
+ (fn [& ks]
+ (config* configs ks))))
+
+(def ^:dynamic config
+ "The default config function. It searches for carica.clj and carica.json
+ on the classpath (with json taking preference) and returns a fuction with
+ the signature of (fn [& ks] ...)
+
+ To retrieve a config value in the following configuration...
+
+ {:name \"bob\"
+ :address {:street \"42 Main St.\" :city \"...\" ...}}
+
+ ...one would call (config :address :street) to retrieve \"42 Main St.\""
+ (configurer (concat (resources "config.json")
+ (resources "config.clj"))))
+
+(defn reduce-into-map [overrides]
+ (let [[val & keys] (reverse overrides)]
+ (reduce (fn [v k] (hash-map k v)) val keys)))
+
+(defn overrider* [cfg-fn-var]
+ (fn [& overrides]
+ (let [c (merge-nested (cfg-fn-var) (reduce-into-map overrides))]
+ (fn [& ks]
+ (config* c ks)))))
+
+(defmacro overrider [cfg-fn]
+ `(overrider* (var ~cfg-fn)))
+
+(def override-config
+ "Useful for testing, override-config enables overriding config
+ values. It takes a series of keys and a replacement value.
+
+ E.g., these are all equivalent:
+ (with-redefs [config (override-config {:address {:street \"42 Broadway\"}})
+ (with-redefs [config (override-config :address {:street \"42 Broadway\"})
+ (with-redefs [config (override-config :address :street \"42 Broadway\")
+
+ It isn't possible to remove any values, though they can be replaced with nil.
+ E.g.,
+ (with-redefs [config (override-config nil)])"
+ (overrider config))
@@ -0,0 +1,70 @@
+(ns carica.test.core
+ (:require [carica.core :refer :all]
+ [clojure.test :refer :all]
+ [clojure.tools.logging.impl :refer [write!]]))
+
+(deftest config-test
+ (testing "config"
+ (testing "should offer a map of settings"
+ (is (map? (config :nested-one-clj)))
+ (testing "with the ability to get at nested values"
+ (is (= "test-clj" (config :nested-one-clj :test-clj)))))
+ (testing "should merge all maps on the classpath"
+ (is (= true (config :from-etc)))
+ (testing "but the first on the classpath should win"
+ (is (= "etc" (config :merged-val)))))
+ (testing "should be overridden with override-config"
+ (with-redefs [config (override-config
+ {:nested-multi-json {:test-json {:test-json 21}
+ :hello :world}})]
+ (is (= :world (config :nested-multi-json :hello)))
+ (is (= 21 (config :nested-multi-json :test-json :test-json)))))
+ (testing "even if it's all made up"
+ (with-redefs [config (override-config :common :apply :hash-map)]
+ (is (= :hash-map (config :common :apply)))))
+ (testing "should return nil and warn if a key isn't found"
+ (let [called (atom false)]
+ (with-redefs [write! (fn [& _] (do (reset! called true) nil))]
+ (is (nil? (config :test-multi-clj :non-existent-key)))
+ (is @called))))
+ (testing "should return nil and not warn if a key has a nil value"
+ (let [called (atom false)]
+ (with-redefs [write! (fn [& _] (do (reset! called true) nil))]
+ (is (nil? (config :nil-val)))
+ (is (not @called)))))))
+
+(deftest test-dynamic-config
+ (testing "config should be dynamic, for runtime repl overrides"
+ (is (.isDynamic #'config)))
+ (testing "also, it should work when rebound"
+ ;; this is the same test as 'should be overridden with
+ ;; override-config' above, only rewritten to use binding
+ (binding [config (override-config
+ {:nested-multi-json {:test-json {:test-json 21}
+ :hello :world}})]
+ (is (= :world (config :nested-multi-json :hello)))
+ (is (= 21 (config :nested-multi-json :test-json :test-json))))
+ (binding [config (override-config :common :apply :hash-map)]
+ (is (= :hash-map (config :common :apply))))))
+
+(deftest test-json-config
+ (is (= 42 (config :json-only :nested))))
+
+(deftest test-jar-json-config-loading
+ ;; make sure we get the missing jar exception, not the exception
+ ;; that comes from calling `file` on a jar: URL
+ (is (thrown? java.io.IOException
+ (load-config (java.net.URL. "jar:file:///foo.jar!/bar.json")))))
+
+(deftest nested-config-redefs-are-okay
+ (with-redefs [config (override-config :foo {:baz 42 :quux :x})]
+ (with-redefs [config (override-config :foo {:quux :y})]
+ (is (= (config :foo) {:baz 42 :quux :y})))))
+
+(deftest nested-missing-keys-are-acceptable
+ (with-redefs [write! (fn [& _] nil)]
+ (is (not (config :nested-multi-clj :missing :missing :missing :missing)))))
+
+(deftest nil-resources-are-handled
+ (is (= (get-configs [(resources "config.clj")])
+ (get-configs [nil (resources "config.clj") nil [nil nil]]))))
@@ -0,0 +1,14 @@
+{:from-etc false
+ :from-test true
+
+ :nil-val nil
+
+ :merged-val "test"
+
+ :test-clj "test-clj"
+ :nested-one-clj {:test-clj "test-clj"}
+ :nested-multi-clj {:test-clj {:test-clj "test-clj"}}
+
+ :test-json "test-clj"
+ :nested-one-json {:test-json "test-clj"}
+ :nested-multi-json {:test-json {:test-json "test-clj"}}}
@@ -0,0 +1,4 @@
+{"test-json": "test-json",
+ "nested-one-json": {"test-json": "test-json"},
+ "nested-multi-json": {"test-json": {"test-json": "test-json"}},
+ "json-only": {"nested": 42}}

0 comments on commit 570be21

Please sign in to comment.