Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with
or
.
Download ZIP
Newer
Older
100644 566 lines (423 sloc) 21.203 kB
84bd80a @jgm Renamed README -> README.markdown.
authored
1 `yst` - static websites from YAML and string templates
2 ======================================================
3
4 `yst` is a tool for generating a static website by filling [string
c875993 @sigurdmeldgaard README: Added documentation about the SQLite database access.
sigurdmeldgaard authored
5 template][]s with data taken from [YAML][] or [CSV][] text files or
6 [SQLite3][] file based databases. This approach
84bd80a @jgm Renamed README -> README.markdown.
authored
7 combines the speed, security, and ease of deployment of a static
8 website with the flexibility and maintainability of a dynamic site that
9 separates presentation and data.
10
11 Installing yst
12 --------------
13
14 `yst` is written in Haskell. The easiest way to install `yst` is by using
15 Haskell's [cabal install][] tool. The best way to get this tool is to
16 install the [Haskell platform][], which includes a complete installation
17 of the [GHC][] compiler and the `cabal` executable.
18
19 Once you have `cabal`, you can install yst with two commands:
20
21 cabal update
22 cabal install yst
23
24 (Note that by default, `cabal` installs the `yst` executable into
25 a special directory: `~\.cabal\bin` on unix systems. You will need
26 to make sure that directory is in your system path.)
27
b5ded41 @jgm Added note in README about workaround for th dependency.
authored
28 If you get the error "yst-0.2.3 depends on template-haskell-2.4.0.0
29 which failed to install," then try the following:
30
31 cabal install syb-with-class-0.6
32 cabal install yst
33
84bd80a @jgm Renamed README -> README.markdown.
authored
34 Getting started
35 ---------------
36
37 To get started with `yst`, use the command:
38
39 yst create mysite
40
41 This will create a directory `mysite` and populate it with the files
42 needed for a sample site. Change to this directory and run `yst` with
43 no arguments to create the site:
44
45 cd mysite
46 yst
47
48 The site will be created in the `site` directory. Open up `site/index.html`
49 to take a look.
50
51 The configuration file `index.yaml` tells `yst` which pages to build,
52 and from which templates and data files. Let's take a look, so we can see
53 what went into `index.html`. The file is a YAML list of YAML hashes
54 (name/value pairs). The first item is
55
56 - url : index.html
57 title : Home
58 template : index.st
59 requires : event.st
60 data :
6d9bb30 @jgm Changed data: field to allow data to be specified directly.
authored
61 recentevents : FROM events.yaml ORDER BY date DESC LIMIT 2
84bd80a @jgm Renamed README -> README.markdown.
authored
62
63 This says: build the page `index.html` from the string template `index.st`
64 (and subtemplate `event.st`) and data from `events.yaml`. Sort this data
65 (a list of events) by date in descending order, and discard all but the
66 first two items. Put the resulting data in the string template attribute
67 `recentevents`. Give the page the title "Home."
68
69 Now take a look at `events.yaml`, the data source. Again it is a YAML
70 list of YAML hashes, each item in the list representing one event.
71 The first is:
72
73 - date: 2009-06-28
74 speaker: Sam Smith
75 title: Building a static website
76
77 Pretty self-explanatory! Try adding a new event, then rebuild the
78 site by typing `yst` and see how it looks.
79
80 If you have problems, make sure `events.yaml` is a valid YAML file.
81 Structure is shown through indentation, so make sure things line
82 up right. And occasionally you may need to use quotation marks
83 around string values---for example, when a title contains a colon.
84
85 There's one more ingredient in our recipe---the string templates.
86 Take a look at `index.st`, the template from which `index.html`
87 is built:
88
89 # Welcome
90
91 Here's our website. Have a look around.
92
93 Our last two events:
94
95 $recentevents:event()$
96
97 For a complete list of events, see the [events](events.html) page.
98
99 The first thing to notice is that this is in [markdown][] format (or,
100 to be precise, markdown with [pandoc][] extensions). So, for example,
101 the first line is a level-one header, and there is a hyperlink
102 to the events page on the last line.
103
104 The second thing to notice is the bit between dollar signs.
105 This is a placeholder for some formatted data. The rendered
106 template will include the list `recentevents` (remember, this
107 was populated from `events.yaml` after some transformations---see
108 above). And each element of this list will be formatted by
109 a subtemplate called `event.st`:
110
111 - $if(it.speaker)$$it.speaker; separator=" and "$, $endif$ *$it.title$*.
112
113 Let's break this down. The whole line will print a bulleted list
114 item. `it` here refers to the event that is being processed by the
115 template. So the first part says: if this event has a `speaker` field,
116 print the speaker, or, if the `speaker` field is a list, print all the
117 speakers separated by the word "and", followed by a comma. And the
118 second part says, print the contents of the `title` field, surrounding
119 it with asterisks (which is the markdown way of making it *emphasized*).
120
121 (See the [string template][] documentation for details of template syntax,
122 and examples.)
123
124 If you look at `index.html`, you'll see this rough structure, but in
125 an HTML file, not a markdown file. `yst` converts the rendered markdown
126 template to HTML (using pandoc), and inserts it into a "layout" file
127 called `layout.html.st`. If you look at this file, you'll see that it's
128 an HTML file with blanks for `$contents$` and `$nav$`. The `$contents$`
129 blank gets filled by the rendered template, converted to HTML, and
130 the `$nav$` blank gets filled by an HTML navigation menu (an unordered
131 list with links).
132
133 To summarize our example, then: `yst` sorts and filters the data in
134 `events.yaml` according to our instructions; inserts this data into
135 the `events.st` template, formatting each item using the `event.st`
136 template; uses pandoc to convert the rendered template to HTML;
137 constructs a navigation menu; and puts the contents and navigation
138 menu in the layout template `layout.html.st`. The result is our page,
139 `index.html`.
140
141 Reference
142 ---------
143
144 ### The `yst` command
145
146 Synopsis:
147
148 yst # rebuilds site, using default config.yaml
149 yst -f myconf.yaml # rebuilds site, using myconf.yaml as config
150 yst create newsite # creates a starter (demo) site in newsite directory
151
152 When run without arguments, `yst` looks at `index.yaml` to determine
153 the dependencies of each page, and rebuilds only the pages whose
154 dependencies have changed since the last build.
155
156 In order for this to work properly, you must be sure to list all
157 subtemplates included recursively in the main page template using
158 the `requires` field. This field takes a single filename or a
159 YAML list, which may be in either of two formats:
160
161 requires: [event.st, date.st]
162
163 or
164
165 requires:
166 - event.st
167 - date.st
168
169 If you don't list all the subtemplates needed to render a page
170 under `requires`, `yst` will still work, but it might sometimes
171 fail to rebuild a page when one of these subtemplates has been changed.
172
173 ### `config.yaml`
174
175 The configuration file specifies the following:
176
177 - `indexfile`: the filename of the index file (default: `index.yaml`)
178 - `title`: the title of the whole site
88ba034 @league Update documentation regarding directory lists for `sourcedir` etc.
league authored
179 - `sourcedir`: list of directories containing templates and page sources
2175d5b Documentation updates (thanks to deepakjois).
John MacFarlane authored
180 (default: `.`)
88ba034 @league Update documentation regarding directory lists for `sourcedir` etc.
league authored
181 - `datadir`: list of directories containing yaml data files (default: `.`)
182 - `filesdir`: list of directories containing static files (default: `files`)
2175d5b Documentation updates (thanks to deepakjois).
John MacFarlane authored
183 - `layout`: the default layout template for the site, relative to
184 `sourcedir` (default: `layout.html.st`)
84bd80a @jgm Renamed README -> README.markdown.
authored
185
88ba034 @league Update documentation regarding directory lists for `sourcedir` etc.
league authored
186 The directories specified by `sourcedir` and `datadir` are searched in
187 order to find source/template or data files, respectively. This allows
188 for a `../templates` directory to be shared among multiple sites, for
189 example. Static files are merged from the contents of all directories
190 in `filesdir`. All of these accept a string as a singleton list.
191
84bd80a @jgm Renamed README -> README.markdown.
authored
192 ### `index.yaml` and submenus
193
194 The index file is a YAML list of pages. Each page may have the following
195 fields:
196
197 - `url`: the relative URL of the page to be built
198 - `title`: the title of the page
199 - `template`: the main string template from which the page will be built
200 - `source`: the markdown source from which the page will be built
201 - `requires`: other files changes to which should trigger a page rebuild
202 (primarily subtemplates of the main page template)
203 - `data`: string template attributes, data sources, and transformations
204 (see below)
205 - `layout`: a layout template to use, if other than the site default
206 - `inmenu`: if 'no', the page will not be included in the site navigation
207 menu
208
209 Each page must have at least `url`, `title`, and either `template` or
2175d5b Documentation updates (thanks to deepakjois).
John MacFarlane authored
210 `source`. Values for `template`, `source`, and `layout` are relative to
211 `sourcedir` specified in `config.yaml`.
84bd80a @jgm Renamed README -> README.markdown.
authored
212
213 The pages may be organized into a tree-like hierarchy, which will be
214 reflected in the site navigation menu. It is easiest to see how this
215 works by example:
216
217 - Rooms:
218 - url : room101.html
219 title : Room 101
220 source : room101.txt
221
222 - url : room203.html
223 title : Room 203
224 source : room203.txt
225
226 Here we have a subtree called "Rooms" with two pages under it.
227 Subtrees can contain other subtrees. Just be consistent about indentation.
228
229 ### The `data` field
230
c875993 @sigurdmeldgaard README: Added documentation about the SQLite database access.
sigurdmeldgaard authored
231 The `data` field in `index.yaml` can populate any number of
232 stringtemplate attributes with data from YAML or CSV files or SQLite3
233 databases. The syntax is easiest to explain by example (note that the
234 keywords do not have to be in ALL CAPS, although they may, and the
235 query doesn't have to end with a semicolon, though it may):
84bd80a @jgm Renamed README -> README.markdown.
authored
236
237 data:
6d9bb30 @jgm Changed data: field to allow data to be specified directly.
authored
238 events: from events.yaml order by date desc group by title then location
239 people: from people.csv order by birthday then lastname where
84bd80a @jgm Renamed README -> README.markdown.
authored
240 birthstate = 'CA' limit 5
c875993 @sigurdmeldgaard README: Added documentation about the SQLite database access.
sigurdmeldgaard authored
241 beststudents: from students.sqlite
242 query "select * from students where grade > 5"
243 order by name
84bd80a @jgm Renamed README -> README.markdown.
authored
244
245 First we have the name of the stringtemplate attribute to be populated
246 (say, `events`). Then, after the colon, we have the data source
c875993 @sigurdmeldgaard README: Added documentation about the SQLite database access.
sigurdmeldgaard authored
247 (`events.yaml`). If the data source is an SQLite3 database, it should be
248 followed by a query that is a quoted string.
249
250 The data source is followed by one or more *transformations*, which will
84bd80a @jgm Renamed README -> README.markdown.
authored
251 be applied in order. Here are the possible transformations. In
252 what follows, brackets denote an optional component, `|` denotes
253 alternatives, and `*` indicates that the component may be repeated
254 several times:
255
256 `ORDER BY field [ASC|DESC] [THEN field [ASC|DESC]]*`
257
258 > Sorts a list by comparing the value of `field`. `ASC`
aed5ad1 @jgm Minor formatting changes to README.
authored
259 (the default) means "ascending", and `DESC` means "descending".
260 The keyword `THEN` is used to separate fields that will be
261 compared in order. So, if we are ordering by `birthday then lastname`,
262 we will compare birthdays, and if these are equal, we will break
263 the tie by comparing last names.
84bd80a @jgm Renamed README -> README.markdown.
authored
264
265 `GROUP BY field [THEN field]*`
266
267 > Converts a list into a list of lists, where each sublist contains
aed5ad1 @jgm Minor formatting changes to README.
authored
268 only items with the same value for `field`. So, for example,
269 `group by date` takes a list of events and produces a list of
270 lists of items, where each sublist contains events occuring at
271 a single date. `GROUP BY date THEN venue` would produce a list
272 of lists of lists, and so on.
84bd80a @jgm Renamed README -> README.markdown.
authored
273
274 `LIMIT n`
275
276 > Removes all but the *n* top items from a list. *n* must be a number.
277
278 `WHERE condition`
279
280 > Selects only items that meet a condition.
281
282 > A *condition* in a `WHERE` statement is a Boolean combination (using
aed5ad1 @jgm Minor formatting changes to README.
authored
283 `NOT`, `AND`, `OR`, and parentheses for disambiguation) of *basic
284 conditions*. A *basic condition* is of the form `value op value`,
285 where `value` may be either a fieldname or a constant. Note that
286 all constants must be enclosed in quotes. `op` may be one of the
c29f6a3 @ohad Added the Contains basic operation to the documentation.
ohad authored
287 following: `=`, `>=`, `<=`, `>`, `<` and 'contains'.
288
289 > The basic condition `arg1 contains arg2` succeeds if and only if
290 `arg1` is a fieldname whose value is a list containing the value of
291 `arg2`.
84bd80a @jgm Renamed README -> README.markdown.
authored
292
293 Note that the order of transformations is significant. You can get
294 different results if you use `LIMIT` before or after `ORDER BY`,
295 for example.
296
6d9bb30 @jgm Changed data: field to allow data to be specified directly.
authored
297 If you want to specify an attribute's value directly, rather than
298 reading it from a file, just omit the "FROM":
299
1f3b3f4 @jgm Fixed typo in README.
authored
300 data:
6d9bb30 @jgm Changed data: field to allow data to be specified directly.
authored
301 deadline: 11/20/2009
302
303 Any YAML value can be given to an attribute in this way.
304
84bd80a @jgm Renamed README -> README.markdown.
authored
305 ### Static files
306
307 Any file or subdirectory in the `files` directory (or whatever is
308 the value of `filesdir` in `config.yaml`) will be copied verbatim to
309 the site. So this is the place to put javascripts, css files, images,
310 PDFs, and the like.
311
312 ### Date fields
313
314 `yst` will recognize date fields in data files automatically, if the
315 dates are in one of the following formats:
316
aed5ad1 @jgm Minor formatting changes to README.
authored
317 - the locale's standard date format
318 - MM/DD/YYYY (e.g. 04/28/1953)
319 - MM/DD/YY (e.g. 04/28/53)
320 - YYYY-MM-DD (e.g. 1953-04-28)
321 - DD MON YYYY (e.g. 28 Apr 1953)
84bd80a @jgm Renamed README -> README.markdown.
authored
322
323 Dates may be formatted in templates using a stringtemplate "format"
324 directive. There's an example in the demo file `date.st`:
325
326 $it; format="%B %d, %Y"$
327
328 The following codes may be used in format strings (taken from
329 Haskell's `Date.Time.Format` documentation):
330
331 - `%D` : same as `%m/%d/%y`
332 - `%F` : same as `%Y-%m-%d`
333 - `%x` : as dateFmt locale (e.g. `%m/%d/%y`)
334 - `%Y` : year
335 - `%y` : last two digits of year, 00 - 99
336 - `%C` : century (being the first two digits of the year), 00 - 99
337 - `%B` : month name, long form (fst from months locale), January - December
338 - `%b, %h` : month name, short form (snd from months locale), Jan - Dec
339 - `%m` : month of year, leading 0 as needed, 01 - 12
340 - `%d` : day of month, leading 0 as needed, 01 - 31
341 - `%e` : day of month, leading space as needed, 1 - 31
342 - `%j` : day of year for Ordinal Date format, 001 - 366
343 - `%G` : year for Week Date format
344 - `%g` : last two digits of year for Week Date format, 00 - 99
345 - `%f` : century (first two digits of year) for Week Date format, 00 - 99
346 - `%V` : week for Week Date format, 01 - 53
347 - `%u` : day for Week Date format, 1 - 7
348 - `%a` : day of week, short form (snd from wDays locale), Sun - Sat
349 - `%A` : day of week, long form (fst from wDays locale), Sunday - Saturday
350 - `%U` : week number of year, where weeks start on Sunday (as sundayStartWeek), 00 - 53
351 - `%w` : day of week number, 0 (= Sunday) - 6 (= Saturday)
352 - `%W` : week number of year, where weeks start on Monday (as mondayStartWeek), 00 - 53
353
354 ### Lists as values
355
356 In some cases, a field may have one or several values. For example, an
357 event might occur at a date or a date range, and an article may have one
358 author or a list of authors.
359
360 An elegant way to handle these cases is to let the field take either a
361 scalar or a list value, and use stringtemplate's "separator" directive
362 to format the result appropriately. So, for example, in our `events.yaml`
363 we have:
364
365 - date: 2009-06-28
366 speaker: Sam Smith
367 title: Building a static website
368
369 - date: 2009-04-15
370 speaker:
371 - Sam Smith
372 - '[Jim Jones](http://foo.bar/baz)'
373 title: Advantages of static websites
374
375 - date:
376 - 2009-04-20
377 - 2009-04-22
378 title: Seminar on web security
379
380 - date: 2009-04-15
381 speaker: Jim Jones
382 title: XSS attacks for dummies
383
384 Note that the `date` field is sometimes a single date, sometimes a
385 list (with start and end date of a range), and the `speaker` field is
386 sometimes a single speaker, and sometimes a range.
387
388 Here is how we handle the date in `eventgroup.st`:
389
390 **$first(it).date:date(); separator=" - "$**
391
392 Here `first(it).date` is the raw data, which may be a single date
393 or a list. `first(it).date:date()` is the result of formatting each
394 date using the `date.st` template (discussed above). And
395 `first(it).date:date; separator=" - "` is the result of taking this
396 list of formatted dates and concatenating them, separated by a hyphen.
397 When there is just one date, we just get a date. When there are two,
398 we get a date range.
399
400 We can use the same trick in the case of `speaker`. If `it` is
401 an event record, then `it.speaker; separator=" and "` will be
402 either a single speaker (if the value is not a list) or a list
403 of speakers separated by "and" (if it is a list).
404
405 In sorting lists with `order by`, `yst` compares two lists by
406 comparing the first members, then (in case of a tie) the second
407 members, and so on. If one item is a list and the other a scalar,
408 the scalar is compared to the first item of the list. So, in
409 the example above, `Seminar on web security` will be sorted
410 an earlier than an event with date `2009-04-21`, and later than
411 an event with date range `2009-04-20 - 2009-04-21`.
412
413 ### YAML gotchas
414
415 If you have a colon in a YAML value, be sure to enclose it in quotes,
416 or you'll get an error. So,
417
418 title: "Cheever: A Life"
419
420 not
421
422 title: Cheever: A Life
423
424 Or (especially if the string is long), use `>` or `|` for a wrapped
425 or unwrapped multiline string literal:
426
427 title: |
428 A very long string that
429 goes on and on.
430
431 You can even have blank lines,
432 but be sure to maintain indentation.
433
434 ### Using CSV files instead of YAML
435
436 If you like, you can use a CSV file instead of YAML for your data source.
437 Just give it the extension `.csv`. In `index.yaml`, you'd have:
438
439 data:
6d9bb30 @jgm Changed data: field to allow data to be specified directly.
authored
440 events: from events.csv order by date desc
84bd80a @jgm Renamed README -> README.markdown.
authored
441
442 This can be handy if you're using existing data, because spreadsheets
443 and databases can easily be dumped to CSV. In the case of a SQL
444 database, you can use a query like this to get the CSV:
445
446 SELECT * INTO OUTFILE 'result.csv'
447 FIELDS TERMINATED BY ',' OPTIONALLY ENCLOSED BY '"'
448 LINES TERMINATED BY '\n'
449 FROM my_table;
450
451 (Thanks to
452 <http://www.terminally-incoherent.com/blog/2006/07/20/dump-mysql-table-into-csv-file/>.)
453
c875993 @sigurdmeldgaard README: Added documentation about the SQLite database access.
sigurdmeldgaard authored
454 ### Using a SQLite database as data source
455
456 You can also get the data directly from a database. Just give the file
457 name from the database followed by a query quoted in "'s
458
459 In this way you can do joins and other advanced operations on your
460 data before handing them over to yst:
461
462 data:
463 meetings : FROM data.sqlite
464 QUERY "select * from meetings
465 left outer join persons
466 on meetings.speaker = persons.name"
467
84bd80a @jgm Renamed README -> README.markdown.
authored
468 ### Using HTML in the templates
469
470 Markdown allows raw HTML to be used, so you can embed HTML in templates.
471 Pandoc's extended markdown is different from standard markdown in that
472 it parses text within HTML block elements as markdown. So, for example,
473 you can include a section in `<div>` tags, or use raw `<ul>` and `<li>`
474 tags instead of markdown formatting.
475
476 ### Layout templates
477
478 Layout files are also string templates, but they are not treated as
479 markdown by default. They should use a double extension to indicate
480 the format. So, for example, an HTML layout could be `standard.html.st`,
481 and a LaTeX layout could be `printed.tex.st`. `yst` will convert the
482 page contents appropriately for the format of the layout template.
483 Here are the supported formats and extensions:
484
485 - HTML: `.html.st`, `.xhtml.st`
486 - LaTeX: `.tex.st`, `.latex.st`
487 - ConTeXt: `.context.st`
488 - Groff man: `.`1`.st`
489 - Rich text format: `.rtf.st`
490 - Texinfo: `.texi.st`
491 - DocBook: `.db.st`
492 - OpenDocument XML: `.fodt.st`
493 - Plain text (markdown): `.txt.st`, `.markdown.st`
494
495 The demo site shows how you can use `yst` to produce a LaTeX document
496 from the same data sources you use to produce HTML pages.
497
498 The following stringtemplate attributes are defined when layouts
499 are rendered:
500
501 - `$contents$`: the result of rendering the page and converting to the layout's format
502 - `$nav$`: an HTML navigation menu created from `index.yaml`
503 - `$gendate$`: the date the page was generated
504 - `$sitetitle$`: the site title from `config.yaml`
505 - `$pagetitle$`: the page title as defined in `index.yaml`
36ca407 @jgm Changed 'base' -> 'root'.
authored
506 - `$root$`: the path to the website's root, relative to the page being
507 rendered. So, for example, if we are rendering `rooms/room503.html`,
508 `$root$` will have the value `../`. Put `$root$` in front of relative URLs
509 in your layout file, so that the links aren't broken on pages in
510 subdirectories.
84bd80a @jgm Renamed README -> README.markdown.
authored
511
3bb2144 @jgm Added note about previewing site with maid.
authored
512 ### Previewing a site
513
514 If you use only relative URLs in your site, you can preview it by
515 opening any of the HTML files in site in your web browser. If you use
516 absolute links, this won't work, but you can use Jinjing Wang's simple
517 static web server `maid`:
518
519 cabal update
520 cabal install maid
521
522 To use maid to preview your site, just change to the site directory and
523 start `maid`:
524
525 cd site
526 maid
527
528 The site will appear at <http://localhost:3000>. If you want to serve it
529 at another port, just pass the port number as an argument to `maid`:
530
531 maid 5999
532
5a642df @jgm Added Development section to README.
authored
533 ## Development
534
535 ### Source code
536
537 yst's source code lives on github at <http://github.com/jgm/yst/tree/master>.
538 You can clone the repository with
539
540 git://github.com/jgm/yst.git
541
542 To install the development code once you've checked it out, just do
543
544 cabal install
545
546 (But please stick to the released version if you don't like things to break
547 unexpectedly!)
548
549 ### Reporting bugs
550
551 If you find a bug, please report it using
552 [the issue tracker on yst's github page](http://github.com/jgm/yst/issues).
553
84bd80a @jgm Renamed README -> README.markdown.
authored
554
555 [string template]: http://www.stringtemplate.org/
556 [YAML]: http://www.yaml.org/
557 [CSV]: http://en.wikipedia.org/wiki/Comma-separated_values
c875993 @sigurdmeldgaard README: Added documentation about the SQLite database access.
sigurdmeldgaard authored
558 [SQLite3]: http://www.sqlite.org/
84bd80a @jgm Renamed README -> README.markdown.
authored
559 [cabal install]: http://hackage.haskell.org/trac/hackage/wiki/CabalInstall
560 [Haskell platform]: http://hackage.haskell.org/platform/
561 [markdown]: http://daringfireball.com/markdown
562 [pandoc]: http://johnmacfarlane.net/pandoc/
563 [vim]: http://www.vim.org/
eb776e5 @jgm Added link to GHC compiler.
authored
564 [GHC]: http://www.haskell.org/ghc/
84bd80a @jgm Renamed README -> README.markdown.
authored
565
Something went wrong with that request. Please try again.