I’m in the process of abandoning Yo-yo - while it was a good experiment, it seems in practice that this isn’t an easy way to write readable code, and that refactoring Yo-yo based code is more difficult than ‘normal’ Clojure.
Thanks for checking it out, and for your help and feedback in implementing it!
James
Yo-yo is a lightweight library for composing ‘Components’ in a functional style, in Clojure and ClojureScript, using Clojure’s ’Cats’ monad library.
(Unless specified otherwise, all of the below works in both Clojure and ClojureScript)
[jarohen/yoyo "0.0.6-beta11"]There will likely be many breaking changes until 0.1.0!
Yo-yo came into existence after a few threads of conversation on the Clojure mailing list, Twitter, and in real life (I know!). It seemed that a number of people whose opinions I respect highly weren’t quite sold on Component and its derivatives (openly, including Phoenix) - a few even said that Phoenix reminded them of Spring.
This obviously won’t do!
So we did what Clojurians do: going to look at other languages to see what they do, and pinching the best ideas; mostly Haskell, but a couple of others as well - and Yo-yo is the result of those discussions.
- A means of composing stoppable ‘Components’
- Optionally, (via
yoyo.system) a means of composing Components that depend on other Components. - That’s it!
- A configuration library
- A Leiningen/Boot plugin
- A source of painful Java/OO/Spring memories ;)
- …
- A module for starting/stopping Aleph/http-kit web servers.
- A module for compiling/building CLJS
- A module for starting/stopping JDBC connection pools
- Templates for creating webapps and REST APIs
(There are a couple of Lein templates - ‘yoyo-webapp’ and ‘yoyo-api’, but here follows a fuller explanation!)
I’m presuming you’ve added the Yo-yo dependency. It’s at the top. Go have a look and copy it into your project.clj/build.boot - I’ll wait :)
Yo-yo is based on two main data types: Components and Dependents.
Components are just a pair - a value, and (optionally) a means of
‘stopping’ the value (whatever that may mean for the value in
question), created using yoyo.core/->component. For example, for a
database pool, you could create a Yo-yo Component as follows:
(:require [yoyo.core :as yc])
(defn open-db-pool! [db-config]
(let [db-pool (start-db-pool! db-config)]
(yc/->component db-pool
(fn []
(stop-db-pool! db-pool)))))The interesting side to Components is that they compose very simply - when we compose two Components together, we’d expect a combined Component to have a combination of the two values, and a stop function consisting of the two stop functions, in reverse order.
So, let’s say we wanted to compose a database pool with a scheduler that required a database pool, we’d want to compose them as follows:
(defn start-scheduler! [db-pool]
(let [scheduled-task {:times ...
:schedule-fn (fn []
(run-job! ... {:db-pool db-pool}))}
stop-scheduler! (schedule! scheduled-task)]
(yc/->component scheduled-task
(fn []
(stop-scheduler!)))))
(defn combine-components [component f]
;; ...
)
(defn start-combined-component! [db-config]
(let [db-pool-component (open-db-pool! db-config)]
(combine-components db-pool-component
(fn [db-pool]
(start-scheduler! db-pool)))))We’d expect start-combined-component! to return a Component with the
combined value, and a stop-function that first stops the scheduler,
then the database pool, as specified. This combined Component can then
be combined with other Components (combined or otherwise), using the
same combine-components function, to form a larger system.
What I’ve just described, is the monadic ‘bind’ function, over Components - which, if it were written in Haskell, would have the following type:
bind :: Component a -> (a -> Component b) -> Component bWe can then use all of the support in Cats to build up our systems. I
won’t duplicate its documentation here, but one macro in particular is
very useful: mlet.
mlet is structured similarly to Clojure’s let bindings, except all
of the values on the right-hand-side are monadic values (in this case,
Components) which are then extracted and bound to the symbols on the
left, like Haskell’s do notation:
(:require [cats.core :as c])
(defn start-combined-component! [db-config]
(c/mlet [db-pool (open-db-pool! db-config)
scheduled-task (start-scheduler! db-pool)]
(yc/->component {:db-pool db-pool
:scheduled-task scheduled-task})))The value returned by the mlet is itself a monadic value, and hence
can itself be combined again into higher-level Components.
As users of Yo-yo, we don’t have to worry about combining the
stop-functions of the two Components - the bind functionality,
implemented by Yo-yo and called by mlet, handles all of
that. Likewise, the Yo-yo bind implementation includes error handling
so that, if a subsequent Component fails, the earlier Components are
stopped - you aren’t left with a half-started system.
(Here, we’re using the 1-arg version of yc/->component, because the
combined Component doesn’t require any ‘stop’ behaviour of its own,
above the stop-functions of the two individual Components.)
Whatever you like! Vanilla maps, records, reify‘d protocols,
functions, objects, you name it…
Components can be tested on their own, or as part of a combined
Component, using Yo-yo’s yc/with-component function:
(deftest test-component
(yc/with-component (open-db-pool! {...})
(fn [db-pool]
;; test away!
)))
(deftest test-combination
(yc/with-component (start-combined-component! {...})
(fn [{:keys [db-pool scheduled-task]}]
;; test away!
)))with-component passes the started Component to the given function,
and stops it when the function returns.
Yo-yo has a few REPL utilities in the top-level yoyo namespace:
yoyo/start!, yoyo/stop! and yoyo/reload! - these allow you to
quickly start, stop and reload your system from the REPL. To set these
up, call yoyo/set-system-fn!, passing it a 0-arg function returning
a Component, and then REPL away to your heart’s content.
yoyo/reload!, by default, will stop the system, reload any
changed namespaces using clojure.tools.namespace, then restart the
system.
My -main functions, therefore, usually look something like this:
(ns myapp.main
(:require [cats.core :as c]
[yoyo :as y]))
(defn make-system []
(c/mlet [db-pool (open-db-pool! {...})
...]
...))
(defn -main []
(y/set-system-fn! #'make-system)
(y/start!))It’s often helpful to store a reference to the started system, to
introspect for debugging purposes. You can wrap the system in
yc/with-system-put-to, as follows:
(:require [cats.core :as c]
[yoyo.core :as yc])
(defn make-system []
(-> (c/mlet [db-pool (open-db-pool! {...})
...]
...)
(yc/with-system-put-to 'user/foo-system)))The started system is then available to query at the REPL, at
user/foo-system. When the system is stopped, the reference is
cleared.
(In CLJ, this can be a symbol or an atom; in CLJS, just an atom (for now?))
Yo-yo’s dependency injection is based on a system map - a map of values, identified by a dependency key.
Yo-yo will, given a set of values that declare their dependencies, construct this system map in the correct order and, when required, stop the system in the opposite order.
Yo-yo’s second data type, therefore, is the Dependent - a value that depends on another value. A Dependent has two possible instance types - firstly, another pair, this time consisting of:
- the dependency key of the value that it depends on, and
- a function that, given a system with that key, returns another Dependent
There’s also a simple base case, where we’ve resolved all the dependencies that we need - this is a simple wrapper around the resolved value.
Again, if we were writing Haskell, we might write the type out like this:
Dependent a = Resolved a
| Dependent (DependencyKey, (System -> Dependent a))(The fact that there are two types here is, in fact, completely transparent to users of Yo-yo, but is included here for interest!)
These Dependent values are also easily composed: given two values, each with a dependency, we can compose them into a single Dependent that requests the first dependency but then, when it’s function is given the first value, returns a Dependent depending on the second value.
Of course, given that we can compose two Dependents in this way, we can compose arbitrarily many.
It shouldn’t be much of a surprise to readers who’ve made it this far,
but the Dependent value is also monadic, and can therefore be bound,
fmap‘d, and mlet‘d as before.
Dependents are quite similar to the Reader monad, with the main difference that each value declares its dependency in advance - this is so that Yo-yo’s dependency resolution can construct dependencies in the correct order.
Yo-yo, therefore, provides two main constructors for the Dependent type:
(:require [yoyo.system :as ys])
;; 'return' - yields a 'Resolved' dependent
(ys/->dep <value>)
;; 'ask' - returns a Dependent depending on the given key
(ys/ask :db-pool)
;; 'ask' also takes a path, if the value that you're depending happens
;; to be a map
(ys/ask :config :aws :secret-key)We can then combine Dependents together easily - let’s say, to fetch a user from a database:
(:require [yoyo.system :as ys]
[cats.core :as c]
[clojure.java.jdbc :as jdbc])
;; prefixing with 'm-' because we're returning a value of type
;; 'Dependent User', not a 'User'
(defn m-get-user [user-id]
(c/mlet [db-pool (ys/ask :db-pool)]
(ys/->dep
(jdbc/query db-pool
["SELECT * FROM users WHERE user_id = ?" user-id]))))Callers of m-get-user then don’t need to know/worry that it depends
on the database pool - they can compose with it all the same:
(defn m-send-confirmation-email! [user-id]
(c/mlet [{:keys [email-address] :as user} (m-get-user user-id)]
(ys/->dep
(send-email! email-address (format-confirmation-email user)))))The return value of that function is still a Dependent on the
:db-pool, but there’s no reason for m-send-confirmation-email! to
know!
Before we get into the dependency injection side of Yo-yo system, it’s
still possible to evaluate a Dependent, by providing a pre-constructed
system map to ys/run. This, again, is particularly useful for testing:
(:require [yoyo.system :as ys])
(deftest test-the-config
(-> (c/mlet [{:keys [access-key secret-key]} (ys/ask :config :aws)]
(ys/->dep
(is (= secret-key "very-secret"))))
(ys/run {:config {:aws {:secret-key "maybe-this-isnt-so-secret-really"}}})))ys/run, here, is taking a Dependent and a system map, and returning
the resolved result of the Dependent - in this case, the return value
of (is (= secret-key "very-secret"))
This is all very well and good, if you have a system to hand!
To create a system, with Yo-yo, we provide a set of Dependencies -
Dependents with names. We create these with ys/named - passing it:
- a function returning a
Dependent (Component a)- a value that depends on other values, that can also be ‘stopped’ - a dependency key
(defn make-config []
(-> (fn []
(ys/->dep
(yc/->component (read-config (io/file "config-file")))))
(ys/named :config)))
(defn make-db-pool []
(-> (fn []
(c/mlet [db-config (ys/ask :config :db)]
(ys/->dep
(let [db-pool (open-db-pool! db-config)]
(yc/->component db-pool
(fn []
(close-db-pool! db-pool)))))))
(ys/named :db-pool)))We then pass these two Dependencies to ys/make-system which returns
a started, combined Component:
(defn make-system []
(ys/make-system #{(make-config)
(make-db-pool)}))ys/make-system then looks at what each dependency ys/ask-s for,
directly or indirectly, and determines the required startup order.
Systems constructed in multiple namespaces can be combined either by
referring to individual functions, like make-config, or by
clojure.set/union-ing over multiple subsets.
Notice that, at the top level, we don’t have to provide a dependency
set for each component - we pass ys/make-system an unordered set.
This is done automatically based on what each component has
ys/ask-ed for. This means that there’s no need for the top-level to
change if, one day, a component requires an extra dependency - just
ys/ask for it at the level of abstraction that it’s needed, and
Yo-yo’s dependency resolution will automatically take it into account.
We can then use our make-system function, as before, to create a
live application:
(ns myapp.main
(:require [cats.core :as c]
[yoyo :as y]
[yoyo.system :as ys]))
;; ...
(defn make-system []
(ys/make-system #{(make-config)
(make-db-pool)}))
(defn -main []
(y/set-system-fn! #'make-system)
(y/start!))We can also test the whole system, in the same way as before:
;; ...
(defn make-system []
(ys/make-system #{(make-config)
(make-db-pool)}))
(deftest test-system
(yc/with-component (make-system)
(fn [{:keys [config db-pool]}]
;; test away!
)))(This part of Yo-yo, in particular, is the part I’m not so convinced of - please let me know if you have ideas around this, whether it be approach, implementation, or even a better explanation!)
Sometimes, we don’t always know the dependencies of a dependent when the system is starting. For example, we have to provide a web handler function to a web server at startup, but we may not know its dependencies until it is called with a request map.
In particular, we have to provide a function of type Request ->
Response, but we’d like to write a function of type Request ->
Dependent Response:
;; what we'd like to write:
(defn open-web-server! []
(c/mlet [server-opts (ys/ask :config :web-server)]
(ys/->dep
(let [web-server (start-server! {:handler (fn [req]
(c/mlet [...]
(ys/->dep
{:status 200
:body ...})))
:server-opts server-opts})]
(yc/->component web-server
(fn []
(stop-server! web-server)))))))This won’t work, because the :handler provided returns a Dependent
Response, rather than a Response.
We’d rather not:
- have the web handler depend on the whole started system
- have the web handler declare and close over its dependencies outside
the function - we’d like to keep the
ys/ask-s at the same level of abstraction as the values are used
So, Yo-yo provides the means to ask for an ‘environment’ - which can satisfy
dependencies from a different thread. In the main thread, we wrap the body in
the ys/mgo macro, to capture the environment. Then, in other threads, we
evaluate Dependents by calling either <!! (synchronous, throws an exception in
CLJS if a Dependency is not satisfied) or <ch (asynchronous, returning a
core.async channel). <!! and <ch can only be used within an mgo block,
like core.async’s <! and >! operators, because they rely on mgo’s captured
environment.
;; what we actually write:
(defn open-web-server! []
(ys/mgo ;; wrap with `mgo`
(c/mlet [server-opts (ys/ask :config :web-server)]
(ys/->dep
(let [web-server (start-server! {:handler (fn [req]
;; Wrap Dependents with `<!!`
;; to evaluate them
(ys/<!! (c/mlet [...]
(ys/->dep
{:status 200
:body ...}))))
:server-opts server-opts})]
(yc/->component web-server
(fn []
(stop-server! web-server))))))))In this case, we wrap the handler function with ys/<!!, which turns a Request
-> Dependent Response function into a Request -> Response function. As the
!! in its name suggests, <!! will block waiting for a dependency if it is
requested on another thread without having yet been started on the main system
thread.
In ClojureScript, we don’t have the luxury of blocking, so there is an
equivalent <ch function, which turn a Dependent into a core.async channel
returning the satisfied value.
There are a couple of Leiningen templates that’ll get you up and
running quickly - yoyo-webapp and yoyo-api. Run (e.g.) lein new
yoyo-app your-app-name to get started!
Yes please! Yo-yo’s still in its infancy, so I’d be particularly interested to hear what you think - are we on the right lines here?
I can be contacted via Twitter, Github, e-mail (on my profile), Slack, Gitter, you name it!
Yes please to these too! Please submit through Github in the traditional manner.
A big thanks, in particular, to Kris Jenkins - who’s provided a lot of time, thoughts, advice and inspiration for the ideas behind and around Yo-yo. Cheers Kris!
Thanks also to those involved in discussions about Component which helped to shape Yo-yo, including (but not limited to)
- Michael Griffiths
- @mccraigmccraig
- Daniel Neal
- Yodit Stanton
- Neale Swinnerton
- Martin Trojer
Cheers!
James
Copyright © 2015 James Henderson
Yo-yo, and all modules within this repo, are distributed under the Eclipse Public License - either version 1.0 or (at your option) any later version.