managing Clojure and ClojureScript app state since (reset)
Clojure HTML

README.md

I think that it's extraordinarily important that we in computer science keep fun in computing

Alan J. Perlis from Structure and Interpretation of Computer Programs

mount

module branch status
mount master Circle CI
mount 0.1.12-SNAPSHOT Circle CI

Clojars Project

any questions or feedback: #mount clojurians slack channel (or just open an issue)

Table of Contents generated with DocToc

Why?

Clojure is

  • powerful
  • simple
  • and fun

Depending on how application state is managed during development, the above three superpowers can either stay, go somewhat, or go completely.

If Clojure REPL (i.e. lein repl, boot repl) fired up instantly, the need to reload application state inside the REPL would go away. But at the moment, and for some time in the future, managing state by making it reloadable within the same REPL session is important to retain all the Clojure superpowers.

Here is a good breakdown on the Clojure REPL startup time, and it is not because of JVM.

mount is here to preserve all the Clojure superpowers while making the application state enjoyably reloadable.

There is another Clojure superpower that mount is made to retain: Clojure community. Pull request away, let's solve this thing!

Differences from Component

mount is an alternative to the component approach with notable differences.

How

(require '[mount.core :refer [defstate]])

Creating State

Creating state is easy:

(defstate conn :start (create-conn))

where the create-conn function is defined elsewhere, can be right above it.

In case this state needs to be cleaned / destroyed between reloads, there is also :stop

(defstate conn :start (create-conn)
               :stop (disconnect conn))

That is pretty much it. But wait, there is more.. this state is a top level being, which means it can be simply required by other namespaces or in REPL:

dev=> (require '[app.nyse :refer [conn]])
nil
dev=> conn
#object[datomic.peer.LocalConnection 0x1661a4eb "datomic.peer.LocalConnection@1661a4eb"]

Using State

For example let's say an app needs a connection above. No problem:

(ns app
  (:require [above :refer [conn]]))

where above is an arbitrary namespace that defines the above state / connection.

Dependencies

If the whole app is one big application context (or system), cross dependencies with a solid dependency graph is an integral part of the system.

But if a state is a simple top level being, these beings can coexist with each other and with other namespaces by being required instead.

If a managing state library requires a whole app buy-in, where everything is a bean or a component, it is a framework, and dependency graph is usually quite large and complex, since it has everything (every piece of the application) in it.

But if stateful things are kept lean and low level (i.e. I/O, queues, threads, connections, etc.), dependency graphs are simple and small, and everything else is just namespaces and functions: the way it should be.

Talking States

There are of course direct dependencies that mount respects:

(ns app.config
  (:require [mount.core :refer [defstate]]))

(defstate config
  :start (load-config "test/resources/config.edn"))

this config, being top level, can be used in other namespaces, including the ones that create states:

(ns app.database
  (:require [mount.core :refer [defstate]]
            [app.config :refer [config]]))

(defstate conn :start (create-connection config))

here is an example of a web server that "depends" on a similar config.

Value of values

Lifecycle functions start/stop can take both functions and values. This is "valuable" and also works:

(defstate answer-to-the-ultimate-question-of-life-the-universe-and-everything :start 42)

While it would be useful in REPL and for testing, real application states would usually have start / stop logic, in other words, the real lifecycle.

Besides scalar values, lifecycle functions can take anonymous functions, partial functions, function references, etc.. Here are some examples:

(defn f [n]
  (fn [m]
    (+ n m)))

(defn g [a b]
  (+ a b))

(defn- pf [n]
  (+ 41 n))

(defn fna []
  42)

(defstate scalar :start 42)
(defstate fun :start #(inc 41))
(defstate with-fun :start (inc 41))
(defstate with-partial :start (partial g 41))
(defstate f-in-f :start (f 41))
(defstate f-no-args-value :start (fna))
(defstate f-no-args :start fna)
(defstate f-args :start g)
(defstate f-value :start (g 41 1))
(defstate private-f :start pf)

Check out fun-with-values-test for more details.

The Importance of Being Reloadable

mount has start and stop functions that will walk all the states created with defstate and start / stop them accordingly: i.e. will call their :start and :stop defined functions. Hence the whole application state can be reloaded in REPL e.g.:

dev=> (require '[mount.core :as mount])

dev=> (mount/stop)
dev=> (mount/start)

While it is not always necessary, mount lifecycle can be easily hooked up to tools.namespace, to make the whole application reloadable with refreshing the app namespaces. Here is a dev.clj as an example, that sums up to:

(defn go []
  (start)
  :ready)

(defn reset []
  (stop)
  (tn/refresh :after 'dev/go))

the (reset) is then used in REPL to restart / reload application state without the need to restart the REPL itself.

Start and Stop Order

Since dependencies are "injected" by requireing on the namespace level, mount trusts the Clojure compiler to maintain the start and stop order for all the defstates.

The "start" order is then recorded and replayed on each (reset).

The "stop" order is simply (reverse "start order"):

dev=> (reset)
08:21:39.430 [nREPL-worker-1] DEBUG mount - << stopping..  nrepl
08:21:39.431 [nREPL-worker-1] DEBUG mount - << stopping..  conn
08:21:39.432 [nREPL-worker-1] DEBUG mount - << stopping..  config

:reloading (app.config app.nyse app.utils.datomic app)

08:21:39.462 [nREPL-worker-1] DEBUG mount - >> starting..  config
08:21:39.463 [nREPL-worker-1] DEBUG mount - >> starting..  conn
08:21:39.481 [nREPL-worker-1] DEBUG mount - >> starting..  nrepl
:ready

You can see examples of start and stop flows in the example app.

Composing States

Besides calling (mount/start) there are other useful ways to start an application:

While all of these are great by themselves, sometimes it is really handy to compose these super powers. For example to start an application with only certain states, swapping a couple of them for new values, while passing runtime arguments.

Composer's Toolbox

Each "tool" has a single responsibility and can be composed with other tools in any combination and order.

  • only will return only states that it is given + exist (seen by mount) in the application
  • except will return all the states that it is given except a given set
  • swap will take a map with keys as states and values as their substitute values
  • swap-states will take a map with keys as states and values with {:start fn :stop fn} as their substitute states
  • with-args will take a map that could later be accessed by (mount/args)

All these functions take one or two arguments. If called with two arguments, the first one will be treated as the universe of states to work with. If called with one argument, it will work with all known to mount states.

None of these functions start or stop the application states, they merely serve as transformations from the initial set of states to the one that will later be passed to (mount/start).

Be Composing

All of the above is much easier to understand by looking at examples:

(-> (only #{#'foo/a
            #'foo/b
            #'foo/c
            #'bar/d
            #'baz/e})
    (except [#'foo/c
             #'bar/d])
    (with-args {:a 42})
    mount/start)

This would start off from 5 states, even though the whole application may have many more states available. It would then exclude two states (i.e. #'foo/c and #'bar/d), then it will pass runtime arguments {:a 42}, and finally it will start the remaining three states: #'foo/a, #'foo/b, #'baz/e.

You may notice that only takes a set, while except takes a vector in this example. This is done intentionally to demonstrate that both these functions can take any collection of states. set would make more sense for most cases though.

Here is a more "involved" example:

(-> (only #{#'foo/a
            #'foo/b
            #'foo/c
            #'bar/d
            #'baz/e})
    (with-args {:a 42})
    (except [#'foo/c
             #'bar/d])
    (swap-states {#'foo/a {:start #(create-connection test-conf)
                           :stop #(disconnect a)}})
    (swap {#'baz/e {:datomic {:uri "datomic:mem://composable-mount"}}})
    mount/start)

This will do the same thing as the previous example plus it would swap #'foo/a with #'test/a state and #'baz/e with {:datomic {:uri "datomic:mem://composable-mount"}} value before starting the application.

Start and Stop Parts of Application

In REPL or during testing it is often very useful to work with / start / stop only a part of an application, i.e. "only these two states".

mount's lifecycle functions, i.e. start/stop, can optionally take states as vars (i.e. prefixed with their namespaces):

(mount/start #'app.config/config #'app.nyse/conn)
...
(mount/stop #'app.config/config #'app.nyse/conn)

which will only start/stop config and conn (won't start/stop any other states).

Here is an example test that uses only two namespaces checking that the third one is not started.

Start an Application Without Certain States

Whether it is in REPL or during testing, it is often useful to start an application without certain states. These can be queue listeners that are not needed at REPL time, or a subset of an application to test.

The start-without function can do just that:

(mount/start-without #'app.feeds/feed-listener
                     #'app/nrepl)

which will start an application without starting feed-listener and nrepl states.

Here is an example test that excludes Datomic connection and nREPL from an application on start.

Swapping Alternate Implementations

During testing it is often very useful to mock/stub certain states. For example running a test against an in memory database vs. the real one, running with a publisher that publishes to a test core.async channel vs. the real remote queue, etc.

Swapping States with Values

The start-with function takes values as substitutes.

Say we have a send-sms state:

(ns app.sms)
;; ...
(defstate send-sms :start (create-sms-sender
                            (:sms config)))

When running tests it would be great not to send the real text messages, but rather send them all to a local core.async channel instead:

(let [sms-ch (chan)
      send-sms (fn [sms] (go (>! sms-ch sms)))]
  (mount/start-with {#'app.sms/send-sms send-sms})   ;; <<<< swapping the "send-sms" state with a test function
  ;; testing.. checking "sms-ch" channel
  (mount/stop))

start-with takes a map of states with their substitutes. For example #'app.sms/send-sms here is the real deal SMS sender that is being substituted with a send-sms test function.

Swapping States with States

The start-with-states function takes values in a form of {:start fn :stop fn} as substitutes:

(mount/start-with-states {#'app.neo/db        {:start #(connect test-config)
                                               :stop #(disconnect db)}
                          #'app.neo/publisher {:start #(create-pub test-config)
                                               :stop #(close-pub publisher)}})

start-with-states takes a map of states with their substitutes. For example #'app.nyse/db here is the real deal (remote) DB that is being substituted with #(connect test-config) function, which could endup being anything, a map, an in memory DB, etc.

The :stop functions of substitutes can be anything, and could refer to the original state references. As in the example above: db and publisher are real references. They would need to be accessible from the namespace of course, so you might need to (:require [app.neo :refer [db]]) in order to use db in :stop #(disconnect db) example above.

One thing to note is whenever

(mount/stop)

is run after start-with/start-with-states, it rolls back to an original "state of states", i.e. #'app.neo/db is #'app.neo/db again. So subsequent calls to (mount/start) or even to (mount/start-with {something else}) will start from a clean slate.

Here is an example test that starts an app with mocking Datomic connection and nREPL.

Stop an Application Except Certain States

Calling (mount/stop) will stop all the application states. In case everything needs to be stopped besides certain ones, it can be done with (mount/stop-except).

Here is an example of restarting the application without bringing down #'app.www/nyse-app:

dev=> (mount/start)
14:34:10.813 [nREPL-worker-0] INFO  mount.core - >> starting..  config
14:34:10.814 [nREPL-worker-0] INFO  mount.core - >> starting..  conn
14:34:10.814 [nREPL-worker-0] INFO  app.db - creating a connection to datomic: datomic:mem://mount
14:34:10.838 [nREPL-worker-0] INFO  mount.core - >> starting..  nyse-app
14:34:10.843 [nREPL-worker-0] DEBUG o.e.j.u.component.AbstractLifeCycle - STARTED SelectChannelConnector@0.0.0.0:4242
14:34:10.843 [nREPL-worker-0] DEBUG o.e.j.u.component.AbstractLifeCycle - STARTED org.eclipse.jetty.server.Server@194f37af
14:34:10.844 [nREPL-worker-0] INFO  mount.core - >> starting..  nrepl
:started

dev=> (mount/stop-except #'app.www/nyse-app)
14:34:47.766 [nREPL-worker-0] INFO  mount.core - << stopping..  nrepl
14:34:47.766 [nREPL-worker-0] INFO  mount.core - << stopping..  conn
14:34:47.766 [nREPL-worker-0] INFO  app.db - disconnecting from  datomic:mem://mount
14:34:47.766 [nREPL-worker-0] INFO  mount.core - << stopping..  config
:stopped
dev=>

dev=> (mount/start)
14:34:58.673 [nREPL-worker-0] INFO  mount.core - >> starting..  config
14:34:58.674 [nREPL-worker-0] INFO  app.config - loading config from test/resources/config.edn
14:34:58.674 [nREPL-worker-0] INFO  mount.core - >> starting..  conn
14:34:58.674 [nREPL-worker-0] INFO  app.db - creating a connection to datomic: datomic:mem://mount
14:34:58.693 [nREPL-worker-0] INFO  mount.core - >> starting..  nrepl
:started

Notice that the nyse-app is not started the second time (hence no more accidental java.net.BindException: Address already in use). It is already up and running.

Recompiling Namespaces with Running States

Mount will detect when a namespace with states (i.e. with (defstate ...)) was reloaded/recompiled, and will check every state in this namespace whether it was running at the point of recompilation. If it was, it will restart it:

  • if a state has a :stop function, mount will invoke it on the old version of state (i.e. cleanup)
  • it will call a "new" :start function after this state is recompiled/redefined

Mount won't keep it a secret, it'll tell you about all the states that had to be restarted during namespace reload/recompilation:

same is true for recompiling and reloading (figwheel, boot-reload, etc.) namespaces in ClojureScript:

Providing a :stop function is optional, but in case a state needs to be cleaned between restarts or on a system shutdown, :stop is highly recommended.

:on-reload

By default a state will be restarted on its redefinition or a namespace recompilation. However it is not always a desired behavior. Sometimes it's ok to have stale references during REPL sessions / development, other times all that is needed is not a "restart", but just a "stop".

This behavior could be controlled with an optional :on-reload meta attribute when defining a state.

In case nothing needs to be done to a running state on reload / recompile / redef, set :on-reload to :noop:

(defstate ^{:on-reload :noop}
          mem-db :start (connect config)
                 :stop (disconnect mem-db))

When a running state needs to be just "stopped" on reload, set :on-reload to :stop:

(defstate ^{:on-reload :stop}
          mem-db :start (connect config)
                 :stop (disconnect mem-db))

Again, by default, if no :on-reload meta is added, internally it would be set to :restart, in which case a running state will be restarted on a redef / a namespace reload.

Note that ^{:on-reload :noop} will disable stopping or starting the state on namespace recompilation but it will still obey (mount/start) / (mount/stop) calls. This means that if any of the namespaces with (mount/start) / (mount/stop) calls are reloaded or these calls are explicitely executed (i.e. somewhere in the dev namespace or in an :after clause), the state's start/stop functions will still be called.

Cleaning up Deleted States

Mount will detect when a state was renamed/deleted from a namespace, and will do two things:

  • if a state had a :stop function, mount will invoke it on the old version of state (i.e. cleanup)
  • will remove any knowledge of this state internally

Here is an example:

dev=> (defstate won't-be-here-long :start (println "I am starting... ")
                                   :stop (println "I am stopping... "))
#'dev/won't-be-here-long
dev=>

dev=> (mount/start #'dev/won't-be-here-long)
INFO  app.utils.logging - >> starting..  #'dev/won't-be-here-long
I am starting...
{:started ["#'dev/won't-be-here-long"]}
dev=>

"deleting" it from REPL, and starting all the states:

dev=> (ns-unmap 'dev 'won't-be-here-long)
nil
dev=> (mount/start)

"<< stopping.. #'dev/won't-be-here-long (it was deleted)"
I am stopping...

INFO  app.utils.logging - >> starting..  #'app.conf/config
INFO  app.utils.logging - >> starting..  #'app.db/conn
INFO  app.utils.logging - >> starting..  #'app.www/nyse-app
INFO  app.utils.logging - >> starting..  #'app.example/nrepl
{:started ["#'app.conf/config" "#'app.db/conn" "#'app.www/nyse-app" "#'app.example/nrepl"]}

Mount detected that #'dev/won't-be-here-long was deleted, hence:

<< stopping.. #'dev/won't-be-here-long (it was deleted)

Packaging

Since mount relies on the Clojure/Script Compiler to learn about all the application states, before mount/start is called all the namespaces that have defstates need to be compiled.

At the development time this requirement is mostly transparent, since these namespaces are compiled with nREPL, or refreshed with "tools.namespace", etc. But it becomes important when packaging an application or when starting a web application via lein-ring's or boot-http's :init hooks.

Depending on a structure and a kind of an application, this means that these namespaces need to be :required prior to a call to mount/start when packaging the app as a stand alone JAR or a WAR.

This can be easily done with choosing an application entry point, which could be a web handler namespace with routes or just an arbitrary app namespace (i.e. my.app). In this app entry point namespace all other namespaces that have defstate would be :required and a call to the mount/start function would be defined:

(ns my.app
  (:require [a]
            [b]
            [c]
            [mount.core :as mount]))

(defn rock-n-roll []                   ;; or (defn -main [args].. )
  (mount/start))

this would ensure that at the time (rock-n-roll) is called, all the namespaces with states were compiled (i.e. mount knows about them). (rock-n-roll) can be used in/as a -main function or as a web hook such as :init.

In practice only a few namespaces need to be :required, since others will be brought in transitively (i.e. by already required namespaces). From the my.app example above, say we had namespaces d, e and f that are required by a, and g and h that are required by b. They (d, e, f, g and h) won't need to be required by my.app, since a and b would "bring" them in.

Affected States

Every time a lifecycle function (start/stop) is called mount will return all the states that were affected:

dev=> (mount/start)
{:started [#'app.config/config
           #'app.nyse/conn
           #'app/nrepl]}
dev=> (mount/stop)
{:stopped [#'app/nrepl
           #'app.nyse/conn
           #'app.config/config]}

An interesting bit here is a vector vs. a set: all the states are returned in the order they were affected.

Logging

All the mount examples have >> starting.. / << stopping.. logging messages, but when I develop an application with mount I don't see them.

Valid question. It was a conscious choice not to depend on any particular logging library, since there are few to select from, and this decision is best left to the developer who may choose to use mount.

Since mount is a library it should not bring any dependencies unless its functionality directly depends on them.

But I still these logging statements in the examples.

The way this is done is via an excellent robert hooke. Example applications live in test, so does the utility that adds logging to all the mount's lifecycle functions on start in dev.clj.

Clojure Version

Since mount supports both Clojure and ClojureScript, it relies on Reader Conditionals that were introduced in Clojure 1.7. mount's code is not precompiled (i.e. AOT) and distributed in .cljc sources, hence it currently requires Clojure 1.7 and above.

Mount and Develop!

Besides a a collection of sample mount applications, mount sources come with two sample apps:

  • Clojure app
  • ClojureScript app

You can clone mount, jump into a REPL and start playing with these built in apps.

Below is an example of the Clojure app that comes with mount.

The app has 4 states:

  • config, loaded from the files and refreshed on each (reset)
  • datomic connection that uses the config to create itself
  • nyse web app which is a web server with compojure routes (i.e. the actual app)
  • nrepl that uses config to bind to host/port

Running New York Stock Exchange

To try it out, clone mount, get to REPL (boot repl or lein repl) and switch to (dev):

$ boot repl

user=> (dev)
#object[clojure.lang.Namespace 0xcf1a0cc "dev"]

start/restart/reset everything using (reset):

dev=> (reset)

:reloading (mount.tools.macro mount.core app.utils.logging app.conf app.db app.utils.datomic app.nyse app.www app.example dev)
INFO  app.utils.logging - >> starting..  #'app.conf/config
INFO  app.conf - loading config from dev/resources/config.edn
INFO  app.utils.logging - >> starting..  #'app.db/conn
INFO  app.db - conf:  {:datomic {:uri datomic:mem://mount}, :www {:port 4242}, :h2 {:classname org.h2.Driver, :subprotocol h2, :subname jdbc:h2:mem:mount, :user sa, :password }, :rabbit {:api-port 15672, :password guest, :queue r-queue, :username guest, :port 5672, :node jabit, :exchange-type direct, :host 192.168.1.1, :vhost /captoman, :auto-delete-q? true, :routing-key , :exchange foo}, :nrepl {:host 0.0.0.0, :port 7878}}
INFO  app.db - creating a connection to datomic: datomic:mem://mount
INFO  app.utils.logging - >> starting..  #'app.www/nyse-app
INFO  app.utils.logging - >> starting..  #'app.example/nrepl
dev=>

everything is started and can be played with:

dev=> (add-order conn {:ticker "GOOG" :bid 665.51M :offer 665.59M :qty 100})
dev=> (add-order conn {:ticker "GOOG" :bid 665.50M :offer 665.58M :qty 300})

dev=> (find-orders conn "GOOG")
({:db/id 17592186045418, :order/symbol "GOOG", :order/bid 665.51M, :order/qty 100, :order/offer 665.59M}
 {:db/id 17592186045420, :order/symbol "GOOG", :order/bid 665.50M, :order/qty 300, :order/offer 665.58M})

since there is also a web server running, we can add orders with HTTP POST (from a different terminal window):

$ curl -X POST -d "ticker=TSLA&qty=100&bid=232.38&offer=232.43" "http://localhost:4242/nyse/orders"

{"added":{"ticker":"TSLA","qty":"100","bid":"232.38","offer":"232.43"}}
dev=> (find-orders conn "TSLA")
({:db/id 17592186045422, :order/symbol "TSLA", :order/bid 232.38M, :order/qty 100, :order/offer 232.43M})

once something is changed in the code, or you just need to reload everything, do (reset).

note: a simple (mount/stop) / (mount/start) will also work, (reset) is for "convenience + ns refresh":

dev=> (reset)
INFO  app.utils.logging - << stopping..  #'app.example/nrepl
INFO  app.utils.logging - << stopping..  #'app.www/nyse-app
INFO  app.utils.logging - << stopping..  #'app.db/conn
INFO  app.db - disconnecting from  datomic:mem://mount
INFO  app.utils.logging - << stopping..  #'app.conf/config

:reloading (app.conf app.db app.nyse app.www app.example dev)

INFO  app.utils.logging - >> starting..  #'app.conf/config
INFO  app.conf - loading config from dev/resources/config.edn
INFO  app.utils.logging - >> starting..  #'app.db/conn
INFO  app.db - conf:  {:datomic {:uri datomic:mem://mount}, :www {:port 4242}, :h2 {:classname org.h2.Driver, :subprotocol h2, :subname jdbc:h2:mem:mount, :user sa, :password }, :rabbit {:api-port 15672, :password guest, :queue r-queue, :username guest, :port 5672, :node jabit, :exchange-type direct, :host 192.168.1.1, :vhost /captoman, :auto-delete-q? true, :routing-key , :exchange foo}, :nrepl {:host 0.0.0.0, :port 7878}}
INFO  app.db - creating a connection to datomic: datomic:mem://mount
INFO  app.utils.logging - >> starting..  #'app.www/nyse-app
INFO  app.utils.logging - >> starting..  #'app.example/nrepl
:ready

notice that it stopped and started again.

In app.db connection :stop calls a disconnect function where a database is deleted. Hence after (reset) was called the app was brought its starting point: database was created by the :start that calls a new-connection function, and db schema is created by nyse.app.

But again no orders:

dev=> (find-orders conn "GOOG")
()
dev=> (find-orders conn "TSLA")
()

hence the app is in its "clean" state, and ready to rock and roll as right after the REPL started:

dev=> (add-order conn {:ticker "TSLA" :bid 232.381M :offer 232.436M :qty 250})

dev=> (find-orders conn "TSLA")
({:db/id 17592186045418, :order/symbol "TSLA", :order/bid 232.381M, :order/qty 250, :order/offer 232.436M})

New York Stock Exchange Maintenance

Say we want to leave the exchange functioning, but would like to make sure that no one can hit it from the web. Easy, just stop the web server:

dev=> (mount/stop #'app.www/nyse-app)
INFO  app.utils.logging - << stopping..  #'app.www/nyse-app
{:stopped ["#'app.www/nyse-app"]}
dev=>
$ curl localhost:4242
curl: (7) Failed to connect to localhost port 4242: Connection refused

everything but the web server works as before:

dev=> (find-orders conn "TSLA")
({:db/id 17592186045420, :order/symbol "TSLA", :order/bid 232.381M, :order/qty 250, :order/offer 232.436M})
dev=>

once we found who DDoSed us on :4242, and punished them, we can restart the web server:

dev=> (mount/start #'app.www/nyse-app)
INFO  app.utils.logging - >> starting..  #'app.www/nyse-app
{:started ["#'app.www/nyse-app"]}
dev=>
$ curl localhost:4242
welcome to the mount sample app!

Web and Uberjar

There is an uberjar branch with an example webapp and it's uberjar sibling. Before trying it:

$ git checkout uberjar
Switched to branch 'uberjar'

The documentation is here.

Runtime Arguments

There is an with-args branch with an example app that takes command line params

$ git checkout with-args
Switched to branch 'with-args'

The documentation is here.

License

Copyright © 2016 tolitius

Distributed under the Eclipse Public License either version 1.0 or (at your option) any later version.