Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with
or
.
Download ZIP
Newer
Older
100644 726 lines (572 sloc) 32.81 kB
b10fbea @gcv Working on README file.
authored
1 The appengine-magic library attempts to abstract away the infrastructural nuts
2 and bolts of writing a Clojure application for the Google App Engine platform.
b81b8a9 @gcv Added barebones steps to the README file.
authored
3
b10fbea @gcv Working on README file.
authored
4 The development environment of Google App Engine for Java expects pre-compiled
5 classes, and generally does not fit well with Clojure's interactive development
6 model. appengine-magic attempts to make REPL-based development of App Engine
7 applications as natural as any other Clojure program.
b81b8a9 @gcv Added barebones steps to the README file.
authored
8
b10fbea @gcv Working on README file.
authored
9 1. Programs using appengine-magic just need to include appengine-magic as a
10 Leiningen dev-dependency.
11 2. appengine-magic takes a Ring handler and makes it available as a servlet for
12 App Engine deployment.
13 3. appengine-magic is also a Leiningen plugin, and adds several tasks which
14 simplify preparing for App Engine deployment.
b81b8a9 @gcv Added barebones steps to the README file.
authored
15
7efad8a @gcv Updated documentation for the new datastore support.
authored
16 Using appengine-magic still requires familiarity with Google App Engine. This
17 README file tries to describe everything you need to know to use App Engine with
18 Clojure, but does not explain the details of App Engine semantics. Please refer
19 to Google's official documentation for details.
20
780d876 @gcv Fixed Markdown link syntax.
authored
21 Please read the [HISTORY](HISTORY.md) file to learn what changed in recent
6ac52ad @gcv Updated documentation.
authored
22 releases.
23
b81b8a9 @gcv Added barebones steps to the README file.
authored
24
25
b10fbea @gcv Working on README file.
authored
26 ## Dependencies
b81b8a9 @gcv Added barebones steps to the README file.
authored
27
b10fbea @gcv Working on README file.
authored
28 * Clojure 1.2.0
29 * Leiningen 1.3.1
30 * Google App Engine SDK 1.3.7
31 * swank-clojure 1.2.1 (optional)
32
33
34
f3bdc91 @gcv Documentation updates.
authored
35 ## Overview
36
37 To use appengine-magic effectively, you need the following:
38
39 1. The `appengine-magic` jar available on the project classpath.
40 2. A Ring handler for your main application. You may use any Ring-compatible
41 framework to make it. If your application does not yet have a `core.clj`
42 file, then the `lein appengine-new` task creates one for you with a simple
43 "hello world" Ring handler.
44 3. A var defined by passing the Ring handler to the
45 `appengine-magic.core/def-appengine-app` macro. This makes the application
46 available both to interactive REPL development, and to App Engine itself.
47 4. An entry point servlet. REPL development does not use it, but the standard
48 App Engine SDK `dev_appserver.sh` mode and production deployment both
49 do. This servlet must be AOT-compiled into a class file. This servlet
50 defaults to the name `app_servlet.clj`, and the `lein appengine-new` task
51 creates one for your project. The servlet must refer to the var defined by
52 `def-appengine-app`.
53 5. Web application resources. This primarily includes web application
54 descriptors. `lein appengine-new` generates those and places them in the
6ac52ad @gcv Updated documentation.
authored
55 `resources/WEB-INF/` directory. You should also place all static files that
56 your application uses in `resources/`.
f3bdc91 @gcv Documentation updates.
authored
57
fad6998 @gcv Added Compojure example.
authored
58 Here is a sample `core.clj`, using Compojure (other Ring-compatible frameworks,
780d876 @gcv Fixed Markdown link syntax.
authored
59 such as [Moustache](https://github.com/cgrand/moustache), also work):
fad6998 @gcv Added Compojure example.
authored
60
61 (ns simple-example.core
62 (:use compojure.core)
63 (:require [appengine-magic.core :as ae]))
64
65 (defroutes simple-example-app-handler
66 (GET "/" req
67 {:status 200
68 :headers {"Content-Type" "text/plain"}
69 :body "Hello, world!"})
70 (GET "/hello/:name" [name]
71 {:status 200
72 :headers {"Content-Type" "text/plain"}
73 :body (format "Hello, %s!" name)})
74 (ANY "*" _
75 {:status 200
76 :headers {"Content-Type" "text/plain"}
77 :body "not found"}))
78
79 (ae/def-appengine-app simple-example-app #'simple-example-app-handler)
80
f3bdc91 @gcv Documentation updates.
authored
81
82
b10fbea @gcv Working on README file.
authored
83 ## Getting Started
84
f3bdc91 @gcv Documentation updates.
authored
85
b10fbea @gcv Working on README file.
authored
86 ### Project setup
87
88 You need a copy of the Google App Engine SDK installed
89 somewhere. appengine-magic cannot replace its `dev_appserver.sh` and `appcfg.sh`
90 functionality.
91
92 1. `lein new` <project-name>
f3bdc91 @gcv Documentation updates.
authored
93 2. Optional: `rm src/<project-name>/core.clj` to clean out the default
94 `core.clj` file created by Leiningen. You need to do this so that
95 appengine-magic can create a default file which correctly invokes the
96 `def-appengine-app` macro.
437e6f7 @gcv Version 0.3.3.
authored
97 3. Edit `project.clj`: add `[appengine-magic "0.3.3"]` to your
56a3c79 @gcv Documentation updates.
authored
98 `:dev-dependencies`.
f3bdc91 @gcv Documentation updates.
authored
99 4. `lein deps`. This fetches appengine-magic, and makes its Leiningen plugin
100 tasks available.
101 5. `lein appengine-new`. This sets up four files for your project: `core.clj`
102 (which has a sample Ring handler and uses the `def-appengine-app` macro),
b10fbea @gcv Working on README file.
authored
103 `app_servlet.clj` (the entry point for the application),
6ac52ad @gcv Updated documentation.
authored
104 `resources/WEB-INF/web.xml` (a servlet descriptor), and
105 `resources/WEB-INF/appengine-web.xml` (an App Engine application
f3bdc91 @gcv Documentation updates.
authored
106 descriptor). These files should contain reasonable starting defaults for your
b10fbea @gcv Working on README file.
authored
107 application.
108
56a3c79 @gcv Documentation updates.
authored
109 With regard to AOT-compilation, if your project needs it, then you must include
110 `<project>.app_servlet` in Leiningen's `:aot` directive. Otherwise, omit the
111 `:aot` directive altogether. The `lein appengine-prepare` task will take care of
112 AOT-compiling the entry point servlet and cleaning up afterwards.
113
b10fbea @gcv Working on README file.
authored
114 The default `.gitignore` file produced by Leiningen works well with the
f3bdc91 @gcv Documentation updates.
authored
115 resulting project, but do take a careful look at it. In particular, you should
6ac52ad @gcv Updated documentation.
authored
116 avoid checking in `resources/WEB-INF/lib/` or `resources/WEB-INF/classes/`: let
117 Leiningen take care of managing those directories.
b10fbea @gcv Working on README file.
authored
118
6d69583 @gcv Documented a couple of pain points.
authored
119 NB: When editing the Leiningen `project.clj` file, do not point `:compile-path`
120 or `:library-path` to `resources/WEB-INF/classes/` and
121 `resources/WEB-INF/lib/`. This will interfere with deployment.
122
b10fbea @gcv Working on README file.
authored
123
124 ### Development process
125
126 Launch `lein swank` or `lein repl`, whichever you normally use. Once you have a
f3bdc91 @gcv Documentation updates.
authored
127 working REPL, compile your application's `core.clj` (or whatever other entry
128 point file you use).
b10fbea @gcv Working on README file.
authored
129
130 The key construct provided by appengine-magic is the
131 `appengine-magic.core/def-appengine-app` macro. It takes a Ring handler and
f3bdc91 @gcv Documentation updates.
authored
132 defines a new `<project-name>-app` var. If you want to rename this var, remember
133 to update `app_servlet.clj`. That's it: you may now write your application using
134 any framework which produces a Ring-compatible handler. Then, just pass the
135 resulting Ring handler to `def-appengine-app`.
b10fbea @gcv Working on README file.
authored
136
137 To test your work interactively, you can control a Jetty instance from the REPL
138 using `appengine-magic.core/start` and `appengine-magic.core/stop`. Examples
36c7502 @gcv Removed all the "suggested alias" nonsense from the documentation.
authored
139 (assuming you are in your application's `core` namespace and your application is
140 named `foo`):
141
142 (require '[appengine-magic.core :as ae])
b10fbea @gcv Working on README file.
authored
143
885faf9 @gcv Changed the new preferred alias for the appengine-magic library to "ae".
authored
144 (ae/start foo-app)
145 (ae/stop)
146 (ae/start foo-app :port 8095)
147 (ae/stop)
b10fbea @gcv Working on README file.
authored
148
149 Recompiling the functions which make up your Ring handler should produce
150 instantaneous results.
151
152
81d01b7 @gcv Updated the README file.
authored
153 ### Testing with dev_appserver.sh
b10fbea @gcv Working on README file.
authored
154
506555c @gcv Documented resource duplication in jar files.
authored
155 1. `lein appengine-prepare`. This AOT-compiles the entry point servlet, makes a
156 jar of your application, and copies it, along with all your library
157 dependencies, to your application's `resources/WEB-INF/lib/` directories.
6ac52ad @gcv Updated documentation.
authored
158 2. Run `dev_appserver.sh` with a path to your application's `resources/`
b10fbea @gcv Working on README file.
authored
159 directory.
160
161
162 ### Static resources
163
6ac52ad @gcv Updated documentation.
authored
164 Just put all static files into your application's `resources/` directory. If you
165 put a file called `index.html` there, it will become a default welcome file.
166
167
168 ### Classpath resources
169
170 Put all classpath resources you expect to need at runtime in `resources/`. You
012292f @gcv Updated documentation.
authored
171 can then access them using the `appengine-magic.core/open-resource-stream`,
172 which returns a `java.io.BufferedInputStream` instance. Please note that, by
173 default, App Engine then makes these resources available as static files. To
174 change this behavior, you need to modify `appengine-web.xml` file. See [Google
6ac52ad @gcv Updated documentation.
authored
175 documentation](http://code.google.com/appengine/docs/java/config/appconfig.html)
176 for details.
b10fbea @gcv Working on README file.
authored
177
46c0317 @gcv Emphasized the correct way to access classpath resources.
authored
178 Do not use direct methods like `java.io.File` or
179 `ClassLoader/getSystemClassLoader` to access classpath resources; they do not
180 work consistently across all App Engine environments.
181
b10fbea @gcv Working on README file.
authored
182
183 ### Deployment to App Engine
184
fc15ca6 @gcv Updated README file.
authored
185 1. First of all, be careful. You must manually maintain the version field in
f3bdc91 @gcv Documentation updates.
authored
186 `appengine-web.xml` and you should understand its implications. Refer to
187 Google App Engine documentation for more information.
6ac52ad @gcv Updated documentation.
authored
188 2. `lein appengine-prepare` prepares the `resources/` directory with the latest
b10fbea @gcv Working on README file.
authored
189 classes and libraries for deployment.
190 3. When you are ready to deploy, just run `appcfg.sh update` with a path to your
6ac52ad @gcv Updated documentation.
authored
191 application's `resources/` directory.
192
193
012292f @gcv Updated documentation.
authored
194 ### Checking the runtime environment
195
196 It is sometimes useful to know if the current execution environment is the
197 production App Engine, `dev_appserver.sh`, or the interactive REPL. For example,
198 you may wish to return more detailed error messages and stack traces in
199 non-production mode. `appengine-magic.core/appengine-environment-type` returns a
200 keyword corresponding to the current environment: `:production`,
201 `:dev-appserver`, and `:interactive`.
202
203
6ac52ad @gcv Updated documentation.
authored
204 ### Automatic testing code
205
206 The `clojure.test` system works well for testing `appengine-magic` applications,
207 but all tests must bootstrap App Engine services in order to run. The
208 `appengine-magic.testing` namespace provides several functions usable as
209 `clojure.test` fixtures to help you do so. The easiest way to get started is:
210
211 (use 'clojure.test)
212 (require '[appengine-magic.testing :as ae-testing])
213
214 (use-fixtures :each (ae-testing/local-services :all))
215
216 Then, write `deftest` forms normally; you can use App Engine services just as you
217 would in application code.
b10fbea @gcv Working on README file.
authored
218
219
91248c2 @gcv Updated documentation.
authored
220 ### File uploads and multipart forms
221
222 A Ring application requires the use of middleware to convert the request body
223 into something useful in the request map. Ring comes with
224 `ring.middleware.multipart-params/wrap-multipart-params` which does this;
225 unfortunately, this middleware uses classes restricted in App Engine. To deal
226 with this, `appengine-magic` has its own middleware.
227
228 `appengine-magic.multipart-params/wrap-multipart-params` works just like the
229 Ring equivalent, except file upload parameters become maps with a `:bytes` key
230 (instead of `:tempfile`). This key contains a byte array with the upload data.
231
232 A full Compojure example (includes features from the Datastore service):
233
234 (use 'compojure.core
235 '[appengine-magic.multipart-params :only [wrap-multipart-params]])
236
237 (require '[appengine-magic.core :as ae]
238 '[appengine-magic.services.datastore :as ds])
239
240 (ds/defentity Image [^:key name, content-type, data])
241
242 (defroutes upload-images-demo-app-handler
243 ;; HTML upload form
244 (GET "/upload" _
245 {:status 200
246 :headers {"Content-Type" "text/html"}
247 :body (str "<html><body>"
248 "<form action=\"/done\" "
249 "method=\"post\" enctype=\"multipart/form-data\">"
250 "<input type=\"file\" name=\"file-upload\">"
251 "<input type=\"submit\" value=\"Submit\">"
252 "</form>"
253 "</body></html>")})
254 ;; handles the uploaded data
255 (POST "/done" _
256 (wrap-multipart-params
257 (fn [req]
258 (let [img (get (:params req) "file-upload")
259 img-entity (Image. (:filename img)
260 (:content-type img)
261 (ds/as-blob (:bytes img)))]
262 (ds/save! img-entity)
263 {:status 200
264 :headers {"Content-Type" "text/plain"}
265 :body (with-out-str
266 (println (:params req)))}))))
267 ;; hit this route to retrieve an uploaded file
268 (GET ["/img/:name", :name #".*"] [name]
269 (let [img (ds/retrieve Image name)]
270 (if (nil? img)
271 {:status 404}
272 {:status 200
273 :headers {"Content-Type" (:content-type img)}
274 :body (.getBytes (:data img))}))))
275
276 (ae/def-appengine-app upload-images-demo-app #'upload-images-demo-app-handler)
277
278 Please note that you do not need to use this middleware with the Blobstore
279 service. App Engine takes care decoding the upload in its internal handlers, and
280 the upload callbacks do not contain multipart data.
281
282
b10fbea @gcv Working on README file.
authored
283
81d01b7 @gcv Updated the README file.
authored
284 ## App Engine Services
285
286 appengine-magic provides convenience wrappers for using App Engine services from
287 Clojure.
288
289
290 ### User service
291
36c7502 @gcv Removed all the "suggested alias" nonsense from the documentation.
authored
292 The `appengine-magic.services.user` namespace provides the following functions
293 for handling users.
b10fbea @gcv Working on README file.
authored
294
81d01b7 @gcv Updated the README file.
authored
295 - `current-user`: returns the `com.google.appengine.api.users.User` for the
296 currently logged-in user.
38de8f5 @gcv Updated the README.
authored
297 - `user-logged-in?`
298 - `user-admin?`
81d01b7 @gcv Updated the README file.
authored
299 - `login-url` (optional keyword: `:destination`): returns the Google
300 authentication servlet URL, and forwards the user to the optional destination.
301 - `logout-url` (optional keyword: `:destination`): performs logout, and forwards
302 the user to the optional destination.
303
304
7dee3d5 @gcv Updated the README file with memcache service documentation.
authored
305 ### Memcache service
306
36c7502 @gcv Removed all the "suggested alias" nonsense from the documentation.
authored
307 The `appengine-magic.services.memcache` namespace provides the following
308 functions for the App Engine memcache. See App Engine documentation for detailed
309 explanations of the underlying Java API.
7dee3d5 @gcv Updated the README file with memcache service documentation.
authored
310
311 - `statistics`: returns the current memcache statistics.
77559be @gcv Renamed destructive memcache interface functions to use trailing ! ch…
authored
312 - `clear-all!`: wipes the entire cache for all namespaces.
7dee3d5 @gcv Updated the README file with memcache service documentation.
authored
313 - `contains? <key>` (optional keyword: `:namespace`): checks if the given key
314 exists in the cache.
77559be @gcv Renamed destructive memcache interface functions to use trailing ! ch…
authored
315 - `delete! <key>` (optional keywords: `:namespace`, `:millis-no-readd`): removes
7dee3d5 @gcv Updated the README file with memcache service documentation.
authored
316 the key from the cache, optionally refraining from adding it for the given
317 number of milliseconds. If the key argument is sequential, deletes all the
318 named keys.
319 - `get <key>` (optional keyword: `:namespace`): returns the value for the given
320 key, but if the key argument is sequential, returns a map of key-value pairs
321 for each supplied key.
6ac52ad @gcv Updated documentation.
authored
322 - `put! <key> <value>` (optional keywords: `:namespace`, `:expiration`,
7dee3d5 @gcv Updated the README file with memcache service documentation.
authored
323 `:policy`): saves the given value under the given key; expiration is an
324 instance of `com.google.appengine.api.memcache.Expiration`; policy is one of
325 `:always` (the default), `:add-if-not-present`, or `:replace-only`.
6ac52ad @gcv Updated documentation.
authored
326 - `put-map! <key-value-map>` (optional keywords: `:namespace`, `:expiration`,
7dee3d5 @gcv Updated the README file with memcache service documentation.
authored
327 `:policy`): writes the key-value-map into the cache. Other keywords same as
328 for `put`.
6ac52ad @gcv Updated documentation.
authored
329 - `increment! <key> <delta>` (optional keywords: `:namespace`, `:initial`):
7dee3d5 @gcv Updated the README file with memcache service documentation.
authored
330 atomically increments long integer values in the cache; if key is sequential,
331 it increments all keys by the given delta.
6ac52ad @gcv Updated documentation.
authored
332 - `increment-map! <key-delta-map>` (optional keywords: `:namespace`, `:initial`):
7dee3d5 @gcv Updated the README file with memcache service documentation.
authored
333 atomically increments long integer values by deltas given in the argument map.
334
81d01b7 @gcv Updated the README file.
authored
335
7efad8a @gcv Updated documentation for the new datastore support.
authored
336 ### Datastore
337
36c7502 @gcv Removed all the "suggested alias" nonsense from the documentation.
authored
338 The `appengine-magic.services.datastore` namespace provides a fairly complete
339 interface for the App Engine datastore.
7efad8a @gcv Updated documentation for the new datastore support.
authored
340
98b75dc @gcv Tweaked the README file.
authored
341 A few simple examples:
7efad8a @gcv Updated documentation for the new datastore support.
authored
342
36c7502 @gcv Removed all the "suggested alias" nonsense from the documentation.
authored
343 (require '[appengine-magic.services.datastore :as ds])
344
345 (ds/defentity Author [^:key name, birthday])
346 (ds/defentity Book [^:key isbn, title, author])
7efad8a @gcv Updated documentation for the new datastore support.
authored
347
348 ;; Writes three authors to the datastore.
349 (let [will (Author. "Shakespeare, William" nil)
350 geoff (Author. "Chaucer, Geoffrey" "1343")
351 oscar (Author. "Wilde, Oscar" "1854-10-16")]
352 ;; First, just write Will, without a birthday.
36c7502 @gcv Removed all the "suggested alias" nonsense from the documentation.
authored
353 (ds/save! will)
7efad8a @gcv Updated documentation for the new datastore support.
authored
354 ;; Now overwrite Will with an entity containing a birthday, and also
355 ;; write the other two authors.
36c7502 @gcv Removed all the "suggested alias" nonsense from the documentation.
authored
356 (ds/save! [(assoc will :birthday "1564"), geoff, oscar]))
7efad8a @gcv Updated documentation for the new datastore support.
authored
357
358 ;; Retrieves two authors and writes book entites.
36c7502 @gcv Removed all the "suggested alias" nonsense from the documentation.
authored
359 (let [will (first (ds/query :kind Author :filter (= :name "Shakespeare, William")))
360 geoff (first (ds/query :kind Author :filter [(= :name "Chaucer, Geoffrey")
361 (= :birthday "1343")]))]
362 (ds/save! (Book. "0393925870" "The Canterbury Tales" geoff))
363 (ds/save! (Book. "143851557X" "Troilus and Criseyde" geoff))
364 (ds/save! (Book. "0393039854" "The First Folio" will)))
7efad8a @gcv Updated documentation for the new datastore support.
authored
365
366 ;; Retrieves all Chaucer books in the datastore, sorting by descending title and
367 ;; then by ISBN.
36c7502 @gcv Removed all the "suggested alias" nonsense from the documentation.
authored
368 (let [geoff (ds/retrieve Author "Chaucer, Geoffrey")]
369 (ds/query :kind Book
370 :filter (= :author geoff)
371 :sort [[title :dsc] :isbn]))
7efad8a @gcv Updated documentation for the new datastore support.
authored
372
373 ;; Deletes all books by Chaucer.
36c7502 @gcv Removed all the "suggested alias" nonsense from the documentation.
authored
374 (let [geoff (ds/retrieve Author "Chaucer, Geoffrey")]
375 (ds/delete! (ds/query :kind Book :filter (= :author geoff))))
7efad8a @gcv Updated documentation for the new datastore support.
authored
376
377 - `defentity` (optional keyword: `:kind`): defines an entity record type
378 suitable for storing in the App Engine datastore. These entities work just
379 like Clojure records. Internally, they implement an additional protocol,
380 EntityProtocol, which provides the `save!` method. When defining an entity,
381 you may specify `^:key` metadata on any one field of the record, and the
382 datastore will use this as the primary key. Omitting the key will make the
383 datastore assign an automatic primary key to the entity. Specifying the
384 optional `:kind` keyword (a string value) causes App Engine to save the entity
385 under the given "kind" name — like a datastore table. This allows kinds to
386 remain disjoint from entity record types.
387 - `new*`: instantiates a datastore entity record. You may also use standard
388 Clojure conventions to instantiate entity records, but creating entities
389 destined for entity groups requires using `new*`. To put the new entity into a
390 group, use the `:parent` keyword with the parent entity. Instantiating an
391 entity does not automatically write it to the datastore.
392 - `get-key-object`: this returns the primary Key object of the given entity. For
393 a newly-instantiated entity lacking an explicit primary key, this method
394 returns nil. Entities properly brought under entity groups using `new*` will
395 have hierarchical keys. You should rarely need to use this explicitly.
396 - `save!`: calling this method on an entity writes it to the datastore, using
397 the primary key returned by calling `get-key-object` on the entity. May be
398 called on a sequence of entities.
399 - `delete!`: removes an entity. May be called on a sequence of entities.
400 - `retrieve <entity-record-type> <primary-key>` (optional keywords: `:parent`,
401 `:kind`): this is a low-level entity retrieval function. It returns a record
402 of the given type with the given primary key value. If the target entity
403 belongs to an entity group, specify the parent using the optional keyword. If
404 the target entity was stored with a different kind from the entity record
bf99661 @gcv Documentation update.
authored
405 type, specify the actual kind using the optional keyword. This function
406 returns `nil` if the given key of the given kind does not exist.
c070f52 @gcv Documentation updates for the addition of exists? to the Datastore API.
authored
407 - `exists? <entity-record-type> <primary-key>` (optional keywords the same as
408 for `retrieve`): used exactly like `retrieve`, but returns `true` if the given
409 entity exists and `false` otherwise.
7efad8a @gcv Updated documentation for the new datastore support.
authored
410 - `query` (optional keywords: `:kind`, `:ancestor`, `:filter`, `:sort`,
411 `:keys-only?`, `:count-only?`, `:in-transaction?`, `:limit`, `:offset`,
412 `:prefetch-size`, `:chunk-size`, `:entity-record-type`): runs a query with the
413 given parameters.
414 * `:kind`: primarily identifies the App Engine entity kind. If given as an
415 entity record type (recommended), the query returns a sequence of entity
416 records of that type. If given as a string, it then checks to see if
417 `:entity-record-type` is given, and uses that type if so; otherwise, the
418 query returns generic `EntityBase` records.
419 * `:filter`: one filter clause, or a list of clauses. Each consists of a
420 symbol specifying the filter operation, a property name, and a target
98b75dc @gcv Tweaked the README file.
authored
421 property value. See example.
7efad8a @gcv Updated documentation for the new datastore support.
authored
422 * `:sort`: one sort criterion, or a list of criteria. Each specified criterion
423 defaults to ascending sort order, but may also sort in descending order.
424 - `with-transaction <body>`: wraps the body in a transaction. Can be
425 nested. (Keep the limitations of App Engine's transaction system in mind when
426 using this.)
427 - `init-datastore-service`: not normally needed. Only use this method if you
428 want to modify the the read consistency and implicit transaction policies of
429 the datastore service.
91248c2 @gcv Updated documentation.
authored
430 - Type conversion functions: these help cast your data into a Java type which
431 receives special treatment from App Engine.
432 * `as-blob`: casts a byte array to `com.google.appengine.api.datastore.Blob`.
433 * `as-short-blob`: casts a byte array to
434 `com.google.appengine.api.datastore.ShortBlob`.
435 * `as-blob-key`: casts a string to
436 `com.google.appengine.api.blobstore.BlobKey`.
437 * `as-text`: casts a string to `com.google.appengine.api.datastore.Text`.
438 * `as-link`: casts a string to `com.google.appengine.api.datastore.Link`.
7efad8a @gcv Updated documentation for the new datastore support.
authored
439
440
c9fa431 @gcv Added preliminary Blobstore documentation.
authored
441 ### Blobstore
442
36c7502 @gcv Removed all the "suggested alias" nonsense from the documentation.
authored
443 The `appengine-magic.services.blobstore` namespace helps with the App Engine
444 Blobstore service, designed for hosting large files. Note that the production
445 App Engine only enables the Blobstore service for applications with billing
446 enabled.
c9fa431 @gcv Added preliminary Blobstore documentation.
authored
447
448 Using the Blobstore generally requires three components: an upload session, an
449 HTTP `multipart/form-data` file upload (usually initiated through an HTML form),
450 and an upload callback.
451
452 1. Your application must first initiate an upload session; this gives it a URL
453 to use for the corresponding HTTP POST request.
454 2. Your application must provide a proper upload form, with the `action`
455 pointing to the URL of the upload session, the `method` set to `post`, and
456 `enctype` set to `multipart/form-data`; each uploaded file must have a `name`
457 attribute.
458 3. Your application must provide an upload callback URL. App Engine will make an
459 HTTP POST request to that URL once the file upload completes. This callback's
460 request will contain information about the uploaded files. The callback
461 should save this data in some way that makes sense for the application. The
462 callback implementation must end with an invocation of the
463 `callback-complete` function. Do not attempt to return a Ring response map
464 from an upload handler.
465 4. A Ring handler which serves up a blob must end with an invocation of the
466 `serve` function. Do not attempt to return a Ring response map from a
467 blob-serving handler.
468
469 NB: In the REPL environment and in `dev_appserver.sh`, using the Blobstore
470 writes entities into the datastore: `__BlobInfo__` and
471 `__BlobUploadSession__`. This does not happen in the production environment.
472
473 - `upload-url <success-path>`: initializes an upload session and returns its
474 URL. `success-path` is the URL of the upload callback.
475 - `delete! <blob-keys>`: deletes the given blobs by their keys.
476 - `serve <ring-request-map> <blob-key>`: modifies the given Ring request map to
477 serve up the given blob.
478 - `callback-complete <ring-request-map> <destination>`: redirects the uploading
479 HTTP client to the given destination.
1a11063 @gcv Documentation updates.
authored
480 - `uploaded-blobs <ring-request-map>`: returns a map of form upload name fields
481 to blob keys.
c9fa431 @gcv Added preliminary Blobstore documentation.
authored
482
6c25ea1 @gcv Added Blobstore example.
authored
483 This is confusing, but a Compojure example will help.
484
485 (use 'compojure.core)
486
487 (require '[appengine-magic.core :as ae]
36c7502 @gcv Removed all the "suggested alias" nonsense from the documentation.
authored
488 '[appengine-magic.services.datastore :as ds]
489 '[appengine-magic.services.blobstore :as blobs])
6c25ea1 @gcv Added Blobstore example.
authored
490
36c7502 @gcv Removed all the "suggested alias" nonsense from the documentation.
authored
491 (ds/defentity UploadedFile [^:key blob-key])
6c25ea1 @gcv Added Blobstore example.
authored
492
493 (defroutes upload-demo-app-handler
494 ;; HTML upload form; note the upload-url call
495 (GET "/upload" _
496 {:status 200
497 :headers {"Content-Type" "text/html"}
498 :body (str "<html><body>"
499 "<form action=\""
36c7502 @gcv Removed all the "suggested alias" nonsense from the documentation.
authored
500 (blobs/upload-url "/done")
6c25ea1 @gcv Added Blobstore example.
authored
501 "\" method=\"post\" enctype=\"multipart/form-data\">"
502 "<input type=\"file\" name=\"file1\">"
503 "<input type=\"file\" name=\"file2\">"
504 "<input type=\"file\" name=\"file3\">"
505 "<input type=\"submit\" value=\"Submit\">"
506 "</form>"
507 "</body></html>")})
508 ;; success callback
509 (POST "/done" req
1a11063 @gcv Documentation updates.
authored
510 (let [blob-map (blobs/uploaded-blobs req)]
36c7502 @gcv Removed all the "suggested alias" nonsense from the documentation.
authored
511 (ds/save! [(UploadedFile. (.getKeyString (blob-map "file1")))
512 (UploadedFile. (.getKeyString (blob-map "file2")))
513 (UploadedFile. (.getKeyString (blob-map "file3")))])
514 (blobs/callback-complete req "/list")))
6c25ea1 @gcv Added Blobstore example.
authored
515 ;; a list of all uploaded files with links
516 (GET "/list" _
517 {:status 200
518 :headers {"Content-Type" "text/html"}
519 :body (apply str `["<html><body>"
520 ~@(map #(format " <a href=\"/serve/%s\">file</a>"
521 (:blob-key %))
36c7502 @gcv Removed all the "suggested alias" nonsense from the documentation.
authored
522 (ds/query :kind UploadedFile))
6c25ea1 @gcv Added Blobstore example.
authored
523 "</body></html>"])})
524 ;; serves the given blob by key
525 (GET "/serve/:blob-key" {{:strs [blob-key]} :params :as req}
36c7502 @gcv Removed all the "suggested alias" nonsense from the documentation.
authored
526 (blobs/serve req blob-key)))
6c25ea1 @gcv Added Blobstore example.
authored
527
528 (ae/def-appengine-app upload-demo-app #'upload-demo-app-handler)
529
c9fa431 @gcv Added preliminary Blobstore documentation.
authored
530
54ee717 @gcv Added documentation for task queues.
authored
531 ### Mail service
b79d57b @gcv Updated documentation for the Mail service.
authored
532
533 The `appengine-magic.services.mail` namespace provides helper functions for
534 sending and receiving mail in an App Engine application.
535
536 To send an mail message, construct it using `make-message` and `make-attachment`
537 functions, and send it using the `send` function.
538
539 To receive incoming mail, first read and understand the relevant section in
540 (Google's official
541 documentation)[http://code.google.com/appengine/docs/java/mail/receiving.html]. You
542 need to modify your application's `appengine-web.xml`, and you should add a
543 security constraint for `/_ah/mail/*` URLs in your `web.xml`. In your
544 application add a Ring handler for POST methods for URLs which begin with
545 `/_ah/mail`.
546
547 - `make-attachment <filename> <bytes>`: constructs an attachment object for a
548 file with the given filename and consisting of the given bytes.
549 - `make-message`: this function has many keyword parameters, and constructs a
550 message object. The parameters are self-explanatory: `:from`, `:to` (takes a
551 string or a vector), `:subject`, `:cc` (takes a string or a vector), `:bcc`
552 (takes a string or a vector), `:reply-to` (takes a string or a vector),
553 `:text-body`, `:html-body`, and `:attachments` (takes a vector).
554 - `send <msg>`: sends the given message.
1a11063 @gcv Documentation updates.
authored
555 - `parse-message <ring-request-map>`: returns a Clojure record of type
b79d57b @gcv Updated documentation for the Mail service.
authored
556 `appengine-magic.services.mail.MailMessage`. Call this function inside the
557 POST handler for `/_ah/mail/*`, and it will return the message sent in the
558 given HTTP request.
559
560 NB: With Compojure, the only route which seems to work in the production App
561 Engine for handling mail is `/_ah/mail/*`.
562
563 (use 'compojure.core)
564
565 (require '[appengine-magic.core :as ae]
566 '[appengine-magic.services.mail :as mail])
567
568 (defroutes mail-demo-app-handler
569 ;; sending
570 (GET "/mail" _
571 (let [att1 (mail/make-attachment "hello.txt" (.getBytes "hello world"))
572 att2 (mail/make-attachment "jk.txt" (.getBytes "just kidding"))
573 msg (mail/make-message :from "one@example.com"
574 :to "two@example.com"
575 :cc ["three@example.com" "four@example.com"]
576 :subject "Test message."
577 :text-body "Sent from appengine-magic."
578 :attachments [att1 att2])]
579 (mail/send msg)
580 {:status 200
581 :headers {"Content-Type" "text/plain"}
582 :body "sent"}))
583 ;; receiving
584 (POST "/_ah/mail/*" req
585 (let [msg (mail/parse-message req)]
586 ;; use the resulting MailMessage object
587 {:status 200})))
588
589 (ae/def-appengine-app mail-demo-app #'mail-demo-app-handler)
590
591
54ee717 @gcv Added documentation for task queues.
authored
592 ### Task Queues service
593
594 The `appengine-magic.services.task-queues` namespace has helper functions for
595 using task queues. As always, read [Google's documentation on task
596 queues](http://code.google.com/appengine/docs/java/taskqueue/overview.html), in
597 particular the sections on configuring `queue.xml`, and on securing task URLs in
598 `web.xml`. In addition, [the section on scheduled
599 tasks](http://code.google.com/appengine/docs/java/config/cron.html) (`cron.xml`)
600 is useful.
601
602 Use the `add!` function to add a new task to a queue, and provide a callback URL
603 which implements the actual work performed by the task. The current App Engine
604 SDK does not seem to have any API calls for removing tasks from a queue, but
605 does support this from the administration console.
606
607 - `add! :url <callback-url>` (optional keywords: `:queue`,
608 `:join-current-transaction?`, `:params`, `:headers`, `:payload`, `:method`,
609 `:countdown-ms`, `:eta-ms`, `:eta`). The `:url` keyword is required.
610 * `:queue`: name of the queue to use; if omitted, uses the system default
611 queue. If provided, the queue must be defined in `queue.xml`.
612 * `:join-current-transaction?`: defaults to false. If true, and if this occurs
613 inside a datastore transaction context, then only adds this task to the
614 queue if the transaction commits successfully.
615 * `:params`: a map of form parameter key-value pairs for the callback. Do not
616 combine with the `:payload` keyword.
617 * `:headers`: a map of extra HTTP headers sent to the callback.
618 * `:payload`: provides data for the callback. Can be a string, a vector of the
619 form `[<string> <charset>]`, or a vector of the form `[<byte-array>
620 <content-type>]`.
621 * `:method`: supports `:post`, `:delete`, `:get`, `:head`, and `:put`. Default
622 is `:post`.
623 * `:countdown-ms`, `:eta-ms`, and `:eta`: scheduling parameters. Only one of
624 these may be used at a time. `:countdown-ms` schedules a task for the given
625 number of milliseconds from the time the `add!` function ran. `:eta-ms`
626 schedules a task for the given number of milliseconds from the beginning of
627 the epoch. `:eta` schedules execution for the time given by the a
628 `java.util.Date` object.
629
630
105c989 @gcv Added documentation for URL Fetch.
authored
631 ### URL Fetch service
632
7538a18 @gcv Renamed urlfetch to url-fetch, for consistency with other service names.
authored
633 `appengine-magic.services.url-fetch` lets App Engine applications send arbitrary
105c989 @gcv Added documentation for URL Fetch.
authored
634 HTTP requests to external services.
635
636 - `fetch <url>` (optional keywords: `:method`, `:headers`, `:payload`,
637 `:allow-truncate`, `:follow-redirects`, `:deadline`).
638 * `:method`: `:get` (default), `:post`, `:delete`, `:head`, or `:put`.
639 * `:headers`: a map from header name (string) to value (string).
640 * `:payload`: a Java byte array.
641 * `:allow-truncate`: if true, allow App Engine to truncate a large response
642 without an error; if false, throws an exception instead.
643 * `:follow-redirects`: if true (default), follows request redirects.
644 * `:deadline`: deadline for the requst, in seconds, expressed as a double.
645 - `fetch-async <url>` (optional keywords same as `fetch`): works like `fetch`,
646 but returns a future-like object. May block when derefed if it has not yet
647 finished loading.
648
649
506555c @gcv Documented resource duplication in jar files.
authored
650
81d01b7 @gcv Updated the README file.
authored
651 ## Limitations
b10fbea @gcv Working on README file.
authored
652
506555c @gcv Documented resource duplication in jar files.
authored
653
6d69583 @gcv Documented a couple of pain points.
authored
654 ### Using App Engine API calls
655
656 Most App Engine services do not work when invoked without an initialized App
657 Engine context. For the time being, this context only exists (1) inside an
658 application's Ring handlers, and (2) in the automatic testing environment
659 provided by `appengine-magic.testing`. This means that you cannot directly
660 invoke most App Engine API functions from the REPL.
661
662
506555c @gcv Documented resource duplication in jar files.
authored
663 ### Incomplete features
664
f3bdc91 @gcv Documentation updates.
authored
665 When using the interactive REPL environment, some App Engine services are more
666 limited than in `dev_appserver.sh` or in deployment. Because the App Engine
667 SDK's jars are a mess, and many are not available in Maven repositories,
668 providing the same functionality in an interactive Clojure environment is tricky
669 and error-prone. In particular, the administration console, `/_ah/admin` is not
670 available in the REPL environment.
671
672 The following Google services are not yet tested in the REPL environment:
81d01b7 @gcv Updated the README file.
authored
673
674 - Images
675 - Multitenancy
676 - OAuth
677 - XMPP
b10fbea @gcv Working on README file.
authored
678
f3bdc91 @gcv Documentation updates.
authored
679 They may still work, but appengine-magic does not provide convenient Clojure
680 interfaces for them, and may lack mappings for any necessary supporting URLs.
681
b10fbea @gcv Working on README file.
authored
682
506555c @gcv Documented resource duplication in jar files.
authored
683 ### Resource duplication
684
685 The `appengine-prepare` task currently copies all your static files and other
686 resources into the jar file containing your application. This means that these
687 resources deploy to App Engine both as separate files, and inside the jar. This
688 should not cause problems for the time being (except for increased space), and
689 will be fixed when Leiningen 1.4 comes out (which supports a `:jar-exclusions`
690 project property).
691
692
b10fbea @gcv Working on README file.
authored
693
fc15ca6 @gcv Updated README file.
authored
694 ## Warning
695
f3bdc91 @gcv Documentation updates.
authored
696 Google App Engine maintains a whitelist of permitted classes in Java's standard
697 library. Other classes will cause your application to fail to deploy. Examples
698 include threads and sockets. If you use those in your application, it will not
699 work. This means that you cannot use Clojure's agents or futures. In addition,
700 if one of your dependencies uses those, your application will also not work. For
701 example, `clojure.java.io` (and its fore-runner, duck-streams from
702 `clojure-contrib`), uses `java.net.Socket`, a forbidden class.
703
704 Whenever you add a new dependency, no matter how innocuous, you should make sure
705 your app still works. `dev_appserver.sh` is a good place to start, but you must
706 also test in the main App Engine. The two do not always load classes the same
707 way.
fc15ca6 @gcv Updated README file.
authored
708
709
710
1a11063 @gcv Documentation updates.
authored
711 ## Contributors
712
713 Many thanks to:
714
715 * Brian Gruber
716 * Marko Kocić
717 * Conrad Barski
2f799fe @gcv Documentation updates for version 0.3.2.
authored
718 * Alex Bolodurin
437e6f7 @gcv Version 0.3.3.
authored
719 * Masashi Iizuka
1a11063 @gcv Documentation updates.
authored
720
721
722
b10fbea @gcv Working on README file.
authored
723 ## License
724
725 appengine-magic is distributed under the MIT license.
Something went wrong with that request. Please try again.