Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP
Newer
Older
100644 401 lines (280 sloc) 13.838 kb
8a359a4 @stuartsierra Update README for 0.2.0 (still work-in-progress)
stuartsierra authored
1 clojure.tools.namespace
2 ========================================
d331440 @abedra Initial conversion
abedra authored
3
9afdb08 @stuartsierra Large addition to usage instructions in README
stuartsierra authored
4 Tools for managing namespaces in Clojure. Parse `ns` declarations from
5 source files, extract their dependencies, build a graph of namespace
6 dependencies within a project, update that graph as files change, and
7 reload files in the correct order.
d331440 @abedra Initial conversion
abedra authored
8
7872428 @stuartsierra Update README
stuartsierra authored
9 This is only about namespace dependencies *within* a single project.
10 It has nothing to do with Leiningen, Maven, JAR files, or
11 repositories.
12
d331440 @abedra Initial conversion
abedra authored
13
8a359a4 @stuartsierra Update README for 0.2.0 (still work-in-progress)
stuartsierra authored
14 Releases and Dependency Information
15 ========================================
d331440 @abedra Initial conversion
abedra authored
16
3d5f7bb @stuartsierra Prepare README for release 0.2.0
stuartsierra authored
17 * [Latest stable release is 0.2.0](https://github.com/clojure/tools.namespace/tree/tools.namespace-0.2.0)
d331440 @abedra Initial conversion
abedra authored
18
8a359a4 @stuartsierra Update README for 0.2.0 (still work-in-progress)
stuartsierra authored
19 * [All Released Versions](http://search.maven.org/#search%7Cgav%7C1%7Cg%3A%22org.clojure%22%20AND%20a%3A%22tools.namespace%22)
d331440 @abedra Initial conversion
abedra authored
20
3d5f7bb @stuartsierra Prepare README for release 0.2.0
stuartsierra authored
21 [Leiningen](https://github.com/technomancy/leiningen) dependency information:
22
23 [org.clojure/tools.namespace "0.2.0"]
24
25 [Maven](http://maven.apache.org/) dependency information:
26
27 <dependency>
28 <groupId>org.clojure</groupId>
29 <artifactId>tools.namespace</artifactId>
30 <version>0.2.0-SNAPSHOT</version>
31 </dependency>
32
33
34 Development Snapshots
35 ---------------------
36
37 * Git master branch is **0.2.1-SNAPSHOT**
38
39 * [Development Snapshot Versions](https://oss.sonatype.org/content/groups/public/org/clojure/tools.namespace/)
40
41 * [Instructions for Development Snapshot Repositories](http://dev.clojure.org/display/doc/Maven+Settings+and+Repositories)
8a359a4 @stuartsierra Update README for 0.2.0 (still work-in-progress)
stuartsierra authored
42
43
9afdb08 @stuartsierra Large addition to usage instructions in README
stuartsierra authored
44 Usage
8a359a4 @stuartsierra Update README for 0.2.0 (still work-in-progress)
stuartsierra authored
45 ========================================
46
9afdb08 @stuartsierra Large addition to usage instructions in README
stuartsierra authored
47 [API Documentation](http://clojure.github.com/tools.namespace/)
48
49 tools.namespace consists of several parts:
50
51 **clojure.tools.namespace.parse:** A parser for namespace declarations
52 in Clojure source files. Given a stream of characters from a Clojure
53 source file, it can find the `ns` declaration and parse the `:require`
54 and `:use` clauses to find the names of other namespaces that file
55 depends on. This is all syntactic analysis: it does not
56 evaluate any code.
57
58 **clojure.tools.namespace.find:** Utilities to search for Clojure
59 namespaces on the filesystem, in directories or JAR files. Combined
60 with [java.classpath](http://clojure.github.com/java.classpath/), it
61 can search for namespaces on the Java classpath. This namespace
62 contains most of the functions in clojure.tools.namespace version
63 0.1.x.
64
65 **clojure.tools.namespace.repl:** Utilities to load and reload code
357cd29 @stuartsierra README: Explain broken namespaces after error during reload
stuartsierra authored
66 based on the namespace dependency graph. This takes some explaining,
67 see below. c.t.n.repl is built out of smaller parts:
9afdb08 @stuartsierra Large addition to usage instructions in README
stuartsierra authored
68
69 * c.t.n.dependency - generic dependency graph data structure
70 * c.t.n.track - namespace dependency tracker
71 * c.t.n.file - file-reader extension to tracker
72 * c.t.n.dir - directory-scanner extension to tracker
73 * c.t.n.reload - namespace-reloading extension to tracker
74
75 You can recombine these parts in other ways, but c.t.n.repl is the
357cd29 @stuartsierra README: Explain broken namespaces after error during reload
stuartsierra authored
76 primary public entry-point to their functionality.
9afdb08 @stuartsierra Large addition to usage instructions in README
stuartsierra authored
77
78
79 Reloading Code: Motivation
80 ----------------------------
81
82 c.t.n.repl is a smarter way to reload code.
83
84 The traditional way to reload Clojure code without restarting the JVM
85 is `(require ... :reload)` or `:reload-all` or an editor/IDE feature
86 that does the same thing. This has several problems:
87
88 * If you modify two namespaces which depend on each other, you must
89 remember to reload them in the correct order to avoid compilation
90 errors.
91
92 * If you remove definitions from a source file and then reload it,
93 those definitions are still available in memory. If other code
94 depends on those definitions, it will continue to work but will
95 break the next time you restart the JVM.
96
97 * If the reloaded namespace contains `defmulti`, you must also reload
98 all of the associated `defmethod` expressions.
99
100 * If the reloaded namespace contains `defprotocol`, you must also
101 reload any records or types implementing that protocol and create
102 new instances.
103
357cd29 @stuartsierra README: Explain broken namespaces after error during reload
stuartsierra authored
104 * If the reloaded namespace contains macros, you must also reload any
105 namespaces which use those macros.
106
9afdb08 @stuartsierra Large addition to usage instructions in README
stuartsierra authored
107 * If the running program contains functions which close over values in
108 the reloaded namespace, those closed-over values are not updated.
357cd29 @stuartsierra README: Explain broken namespaces after error during reload
stuartsierra authored
109 (This is common in web applications which construct the "handler
110 stack" as a composition of functions.)
9afdb08 @stuartsierra Large addition to usage instructions in README
stuartsierra authored
111
357cd29 @stuartsierra README: Explain broken namespaces after error during reload
stuartsierra authored
112 Often the only surefire way to reload Clojure code is to restart the
113 JVM. A large Clojure application can take 20 seconds or more just to
114 compile. I wrote tools.namespace to help speed up this development
115 cycle.
9afdb08 @stuartsierra Large addition to usage instructions in README
stuartsierra authored
116
117
118 Reloading Code: Usage
119 -----------------------
120
5bec71f @stuartsierra README updates; document disable-reload/unload
stuartsierra authored
121 There's only one important function, `refresh`:
9afdb08 @stuartsierra Large addition to usage instructions in README
stuartsierra authored
122
123 user=> (use '[clojure.tools.namespace.repl :only (refresh)])
357cd29 @stuartsierra README: Explain broken namespaces after error during reload
stuartsierra authored
124 nil
125
9afdb08 @stuartsierra Large addition to usage instructions in README
stuartsierra authored
126 user=> (refresh)
357cd29 @stuartsierra README: Explain broken namespaces after error during reload
stuartsierra authored
127 :reloading (com.example.util com.example.app com.example.app-test)
128 :ok
9afdb08 @stuartsierra Large addition to usage instructions in README
stuartsierra authored
129
130 The `refresh` function will scan all the directories on the classpath
131 for Clojure source files, read their `ns` declarations, build a graph
132 of their dependencies, and load them in dependency order. (You can
133 change the directories it scans with `set-refresh-dirs`.)
134
135 Later on, after you have changed and saved a few files in your editor,
136 run it again:
137
138 user=> (refresh)
357cd29 @stuartsierra README: Explain broken namespaces after error during reload
stuartsierra authored
139 :reloading (com.example.app com.example.app-test)
140 :ok
9afdb08 @stuartsierra Large addition to usage instructions in README
stuartsierra authored
141
142 Based on file modification timestamps and the graph of dependencies,
143 the `refresh` function will reload *only* the namespaces that have
144 changed, in dependency order. But first, it will *unload* the
145 namespaces that changed to clear out any old definitions.
146
357cd29 @stuartsierra README: Explain broken namespaces after error during reload
stuartsierra authored
147 This is quite unlike `(require ... :reload)`. Calling `refresh` will
148 *blow away your old code*. Sometimes this is helpful: it can catch
149 trivial mistakes like deleting a function that another piece of code
150 depends on. But sometimes it hurts when you have built-up application
151 state stored in a Var that got deleted by `refresh`.
8a359a4 @stuartsierra Update README for 0.2.0 (still work-in-progress)
stuartsierra authored
152
9afdb08 @stuartsierra Large addition to usage instructions in README
stuartsierra authored
153 This brings us to the next section:
154
155
156 Reloading Code: Preparing Your Application
157 --------------------------------------------
158
159 Being able to safely destroy and reload namespaces without breaking
160 your application requires some discipline and careful design. It won't
161 "just work" on any Clojure project.
162
163 ### No Global State
164
165 The first rule for making your application reload-safe is **no global
357cd29 @stuartsierra README: Explain broken namespaces after error during reload
stuartsierra authored
166 state**. That means you should avoid things like this:
9afdb08 @stuartsierra Large addition to usage instructions in README
stuartsierra authored
167
168 (def state-of-world (ref {}))
357cd29 @stuartsierra README: Explain broken namespaces after error during reload
stuartsierra authored
169 (def object-handle (atom nil))
9afdb08 @stuartsierra Large addition to usage instructions in README
stuartsierra authored
170
357cd29 @stuartsierra README: Explain broken namespaces after error during reload
stuartsierra authored
171 c.t.n.repl/refresh will destroy those Vars when it reloads the
172 namespace (even if you used `defonce`).
9afdb08 @stuartsierra Large addition to usage instructions in README
stuartsierra authored
173
357cd29 @stuartsierra README: Explain broken namespaces after error during reload
stuartsierra authored
174 Instead of storing your state in global Vars, store it *locally* in an
175 object that represents the running state of your application. Then
176 provide a constructor function to initialize that state:
9afdb08 @stuartsierra Large addition to usage instructions in README
stuartsierra authored
177
178 (defn create-application []
179 {:state-of-world (ref {})
357cd29 @stuartsierra README: Explain broken namespaces after error during reload
stuartsierra authored
180 :object-handle (atom nil)})
9afdb08 @stuartsierra Large addition to usage instructions in README
stuartsierra authored
181
182 You can choose what representation works best for your application:
183 map, vector, record, or even just a single Ref by itself.
184
5bec71f @stuartsierra README updates; document disable-reload/unload
stuartsierra authored
185 Typically you'll still need one global `def` somewhere, perhaps in the
186 REPL itself, to hold the current application instance. See "Managing
187 Reloads" below.
188
9afdb08 @stuartsierra Large addition to usage instructions in README
stuartsierra authored
189 ### Managed Lifecycle
190
191 The second rule for making your application reload-safe is having a
192 consistent way to **start and stop the entire system**. I like to do
193 this with a protocol implemented by each major component in the
194 system:
195
196 (defprotocol Lifecycle
197 (start [component])
198 (stop [component]))
199
357cd29 @stuartsierra README: Explain broken namespaces after error during reload
stuartsierra authored
200 Smaller applications can probably get along fine with just a pair of
9afdb08 @stuartsierra Large addition to usage instructions in README
stuartsierra authored
201 functions.
202
203 The point is that you need a convenient way to destroy all the
204 built-up state of your application and then *recreate it from
205 scratch*. The "stop" function should stop any running processes,
206 release all external resources, and zero-out any internal state. The
207 "start" function should create internal state, acquire resources, and
208 start processes.
209
210 It might take a few tries to get it right, but once you have a working
211 start/stop you can have a workflow like this:
212
7cb1409 @stuartsierra Fix for GitHub markdown formatting
stuartsierra authored
213 Step 1. Start up a REPL.
9afdb08 @stuartsierra Large addition to usage instructions in README
stuartsierra authored
214
7cb1409 @stuartsierra Fix for GitHub markdown formatting
stuartsierra authored
215 Step 2. Load the app:
9afdb08 @stuartsierra Large addition to usage instructions in README
stuartsierra authored
216
969693f @stuartsierra README: More clarification of refresh usage
stuartsierra authored
217 user=> (use 'clojure.tools.namespace.repl)
9afdb08 @stuartsierra Large addition to usage instructions in README
stuartsierra authored
218 user=> (refresh)
219 user=> (def my-app (create-application))
220 user=> (start my-app)
221
7cb1409 @stuartsierra Fix for GitHub markdown formatting
stuartsierra authored
222 Step 3. Test it out.
9afdb08 @stuartsierra Large addition to usage instructions in README
stuartsierra authored
223
7cb1409 @stuartsierra Fix for GitHub markdown formatting
stuartsierra authored
224 Step 4. Modify some source files.
9afdb08 @stuartsierra Large addition to usage instructions in README
stuartsierra authored
225
7cb1409 @stuartsierra Fix for GitHub markdown formatting
stuartsierra authored
226 Step 5. Restart:
9afdb08 @stuartsierra Large addition to usage instructions in README
stuartsierra authored
227
228 user=> (stop my-app)
229 user=> (refresh)
230 user=> (def my-app (create-application))
231 user=> (start my-app)
232
0f192e2 @stuartsierra README: Add warnings about helper functions
stuartsierra authored
233 (You could also combine all those steps in a single utility function,
234 but see warnings below.)
357cd29 @stuartsierra README: Explain broken namespaces after error during reload
stuartsierra authored
235
9afdb08 @stuartsierra Large addition to usage instructions in README
stuartsierra authored
236 After that, you've got a squeaky-clean new instance of your app
237 running, in a fraction of the time it takes to restart the JVM.
238
357cd29 @stuartsierra README: Explain broken namespaces after error during reload
stuartsierra authored
239 ### Handling Errors
240
241 If an exception is thrown while loading a namespace, `refresh` stops,
242 prints the namespace that caused the exception, and returns the
243 exception. You can print the rest of the stacktrace with
244 `clojure.repl/pst`; the exception itself is bound to `*e`.
245
246 user=> (refresh)
247 :reloading (com.example.app com.example.app-test)
248 :error-while-loading com.example.app
249 #<IllegalArgumentException java.lang.IllegalArgumentException:
250 Parameter declaration cond should be a vector>
251
252 user=> (clojure.repl/pst)
253 IllegalArgumentException Parameter declaration cond should be a vector
254 clojure.core/assert-valid-fdecl (core.clj:6567)
255 clojure.core/sigs (core.clj:220)
256 clojure.core/defn (core.clj:294)
257 clojure.lang.Var.invoke (Var.java:427)
258 ...
259
260 Remember that any namespaces which depend on the namespace that caused
261 the exception *do not exist* at this point: they have been removed but
262 not yet reloaded.
263
264 After you fix the problem, call `refresh` again and it will resume
969693f @stuartsierra README: More clarification of refresh usage
stuartsierra authored
265 reloading where it left off.
357cd29 @stuartsierra README: Explain broken namespaces after error during reload
stuartsierra authored
266
267 user=> (refresh)
268 :reloading (com.example.app com.example.app-test)
269 :ok
270
969693f @stuartsierra README: More clarification of refresh usage
stuartsierra authored
271 If your current REPL namespace is one of those that has not yet been
272 reloaded, then you will need to call `refresh` by its fully-qualified
273 name `clojure.tools.namespace.repl/refresh`.
274
5bec71f @stuartsierra README updates; document disable-reload/unload
stuartsierra authored
275 ### Managing Reloads
276
277 Some projects have a "project REPL" or a "scratch" namespace where you
278 want keep state during development. You can use the functions
279 `disable-unload!` and `disable-reload!` in
280 `clojure.tools.namespace.repl` to prevent `refresh` from automatically
281 un/reloading those namespaces.
282
283 Use this feature sparingly: it exists as a development-time
357cd29 @stuartsierra README: Explain broken namespaces after error during reload
stuartsierra authored
284 convenience, not a work-around for code that is not reload-safe. Also,
285 see the warnings about aliases, below. Aliases to reloaded namespaces
286 will break if the namespace *containing* the alias is not reloaded
287 also.
5bec71f @stuartsierra README updates; document disable-reload/unload
stuartsierra authored
288
9afdb08 @stuartsierra Large addition to usage instructions in README
stuartsierra authored
289
290 Warnings
291 --------------------
8a359a4 @stuartsierra Update README for 0.2.0 (still work-in-progress)
stuartsierra authored
292
5bec71f @stuartsierra README updates; document disable-reload/unload
stuartsierra authored
293 Be careful when reloading the namespace in which you run your REPL.
294 Because namespaces are removed when reloading, all your past
357cd29 @stuartsierra README: Explain broken namespaces after error during reload
stuartsierra authored
295 definitions are lost. Either keep your REPL in a namespace which has
296 no file associated with it, such as `user`, or put all your REPL
297 definitions in a "scratch" namespace that can be reloaded.
5bec71f @stuartsierra README updates; document disable-reload/unload
stuartsierra authored
298
299 Be careful when using fully-qualified symbol names without namespace
357cd29 @stuartsierra README: Explain broken namespaces after error during reload
stuartsierra authored
300 aliases (`require` with no `:as`). If the namespace happens to be
301 loaded already, it will not necessarily cause an error if you forget
302 to `require` it, but the dependency graph of namespaces will be
5bec71f @stuartsierra README updates; document disable-reload/unload
stuartsierra authored
303 incorrect.
304
305 Beware of code which has references to old definitions, especially
306 references to things you created in the REPL.
8a359a4 @stuartsierra Update README for 0.2.0 (still work-in-progress)
stuartsierra authored
307
9afdb08 @stuartsierra Large addition to usage instructions in README
stuartsierra authored
308 If you create your own instance of the dependency tracker, do not
309 store it in a namespace which gets reloaded.
8a359a4 @stuartsierra Update README for 0.2.0 (still work-in-progress)
stuartsierra authored
310
0f192e2 @stuartsierra README: Add warnings about helper functions
stuartsierra authored
311 ### Warnings for Helper Functions
312
313 Be careful defining a helper function in a namespace which calls
314 `refresh` if that namespace also could get reloaded. For example, you
315 might try to combine the stop-refresh-start code from the "Managed
316 Lifecycle" section into a single function:
317
318 (defn restart []
319 (stop my-app)
320 (refresh)
321 (def my-app (create-application))
322 (start my-app))
323
324 This won't work if the namespace containing `restart` could get
325 reloaded. After `refresh`, the namespace containing `restart` has been
326 dropped, but the function continues to run in the *old* namespace.
327
4209eea @stuartsierra README: warn about broken aliases
stuartsierra authored
328 ### Warnings for Aliases
329
330 Namespace aliases created at the REPL will still refer to the *old* namespace after `refresh`. For example:
331
332 user=> (require '[com.example.foo :as foo])
333
334 user=> foo/bar
335
336 user=> (refresh)
337 :reloading (com.example.foo)
338 :ok
339
340 user=> foo/bar ; this is the *old* foo/bar
341
342 If you try to recreate the alias with the new namespace, you will get an error:
343
344 user=> (require '[com.example.foo :as foo])
345 IllegalStateException Alias foo already exists in
346 namespace user, aliasing com.example.foo
347 clojure.lang.Namespace.addAlias (Namespace.java:224)
348
349 The only way out is to remove the alias before recreating it:
350
351 user=> (ns-unalias *ns* 'foo)
352 nil
353 user=> (alias 'foo 'com.example.foo)
354
9afdb08 @stuartsierra Large addition to usage instructions in README
stuartsierra authored
355 ### Warnings for Protocols
8a359a4 @stuartsierra Update README for 0.2.0 (still work-in-progress)
stuartsierra authored
356
357 When reloading namespaces which contain protocols, be careful that you
358 do not leave any old instances of records or types implementing those
359 protocols.
360
361 For example, if you have a namespace like this:
362
363 (ns com.example.foo)
364
365 (defprotocol IFoo
366 (foo [this]))
367
368 (defrecord FooRecord []
369 IFoo (foo [this] nil))
370
371 And you do something like the following at the REPL:
372
373 user=> (def my-foo (->FooRecord))
374 user=> (clojure.tools.namespace.repl/refresh)
375 user=> (foo my-foo)
376
377 You will get a confusing error message like this:
378
379 IllegalArgumentException
380 No implementation of method: :foo
381 of protocol: #'com.example.foo/IFoo
382 found for class: com.example.foo.FooRecord
383 clojure.core/-cache-protocol-fn (core_deftype.clj:527)
384
385 That's because `my-foo` is an **instance** of the **old** version of
386 `FooRecord`, implementing the **old** version of `IFoo`. As far as the
387 JVM is concerned, the old `IFoo` and the new `IFoo` are completely
388 different classes.
d331440 @abedra Initial conversion
abedra authored
389
9afdb08 @stuartsierra Large addition to usage instructions in README
stuartsierra authored
390 To avoid this problem, always create new instances of records after a
391 refresh.
d331440 @abedra Initial conversion
abedra authored
392
393
5bec71f @stuartsierra README updates; document disable-reload/unload
stuartsierra authored
394 Copyright and License
395 ========================================
d331440 @abedra Initial conversion
abedra authored
396
9afdb08 @stuartsierra Large addition to usage instructions in README
stuartsierra authored
397 Copyright © 2012 Stuart Sierra
d331440 @abedra Initial conversion
abedra authored
398
9afdb08 @stuartsierra Large addition to usage instructions in README
stuartsierra authored
399 Licensed under the [Eclipse Public License Version 1.0](https://github.com/clojure/tools.namespace/blob/master/epl.html).
8a359a4 @stuartsierra Update README for 0.2.0 (still work-in-progress)
stuartsierra authored
400
Something went wrong with that request. Please try again.