forked from piskvorky/gensim
-
Notifications
You must be signed in to change notification settings - Fork 6
/
dist_lsi.html
309 lines (272 loc) · 24.4 KB
/
dist_lsi.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<title>Distributed Latent Semantic Analysis — gensim</title>
<link rel="stylesheet" href="_static/default.css" type="text/css" />
<link rel="stylesheet" href="_static/pygments.css" type="text/css" />
<script type="text/javascript">
var DOCUMENTATION_OPTIONS = {
URL_ROOT: '',
VERSION: '0.8.4',
COLLAPSE_INDEX: false,
FILE_SUFFIX: '.html',
HAS_SOURCE: true
};
</script>
<script type="text/javascript" src="http://ajax.googleapis.com/ajax/libs/jquery/1.4.2/jquery.min.js"></script>
<script type="text/javascript" src="_static/underscore.js"></script>
<script type="text/javascript" src="_static/doctools.js"></script>
<link rel="author" title="About these documents" href="about.html" />
<link rel="top" title="gensim" href="index.html" />
<link rel="up" title="Distributed Computing" href="distributed.html" />
<link rel="next" title="Distributed Latent Dirichlet Allocation" href="dist_lda.html" />
<link rel="prev" title="Distributed Computing" href="distributed.html" />
<!-- twitter search widget
<script type="text/javascript" src="_static/widget.js"></script>
-->
<meta property="og:title" content="#gensim" />
<meta property="og:description" content="Efficient topic modelling in Python" />
<script type="text/javascript">
var _gaq = _gaq || [];
_gaq.push(['_setAccount', 'UA-24066335-1']);
_gaq.push(['_trackPageview']);
(function() {
var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true;
ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js';
var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s);
})();
</script>
</head>
<body>
<div class="related">
<h3>Navigation</h3>
<ul>
<li class="right" style="margin-right: 10px">
<a href="genindex.html" title="General Index"
accesskey="I">index</a></li>
<li class="right" >
<a href="py-modindex.html" title="Python Module Index"
>modules</a> |</li>
<li class="right" >
<a href="dist_lda.html" title="Distributed Latent Dirichlet Allocation"
accesskey="N">next</a> |</li>
<li class="right" >
<a href="distributed.html" title="Distributed Computing"
accesskey="P">previous</a> |</li>
<li><a href="index.html">Gensim home</a>| </li>
<li><a href="tutorial.html">Tutorials</a>| </li>
<li><a href="http://groups.google.com/group/gensim">Support</a>| </li>
<li><a href="https://github.com/piskvorky/gensim/wiki">Contribute</a>| </li>
<li><a href="apiref.html">API reference</a>»</li>
<li><a href="distributed.html" accesskey="U">Distributed Computing</a> »</li>
</ul>
</div>
<div class="sphinxsidebar">
<div class="sphinxsidebarwrapper">
<h3><a href="index.html">Table Of Contents</a></h3>
<ul>
<li><a class="reference internal" href="#">Distributed Latent Semantic Analysis</a><ul>
<li><a class="reference internal" href="#setting-up-the-cluster">Setting up the cluster</a></li>
<li><a class="reference internal" href="#running-lsa">Running LSA</a><ul>
<li><a class="reference internal" href="#distributed-lsa-on-wikipedia">Distributed LSA on Wikipedia</a></li>
</ul>
</li>
</ul>
</li>
</ul>
<h4>Previous topic</h4>
<p class="topless"><a href="distributed.html"
title="previous chapter">Distributed Computing</a></p>
<h4>Next topic</h4>
<p class="topless"><a href="dist_lda.html"
title="next chapter">Distributed Latent Dirichlet Allocation</a></p>
<div id="searchbox" style="display: none">
<h3>Quick search</h3>
<form class="search" action="search.html" method="get">
<input type="text" name="q" size="24" />
<input type="submit" value="Go" />
<input type="hidden" name="check_keywords" value="yes" />
<input type="hidden" name="area" value="default" />
</form>
<p class="searchtip" style="font-size: 90%">
Enter search terms or a module, class or function name.
</p>
</div>
<script type="text/javascript">$('#searchbox').show(0);</script>
</div>
</div>
<div class="document">
<div class="documentwrapper">
<div class="bodywrapper">
<div class="body">
<div class="section" id="distributed-latent-semantic-analysis">
<span id="dist-lsi"></span><h1>Distributed Latent Semantic Analysis<a class="headerlink" href="#distributed-latent-semantic-analysis" title="Permalink to this headline">¶</a></h1>
<div class="admonition note">
<p class="first admonition-title">Note</p>
<p class="last">See <a class="reference internal" href="distributed.html"><em>Distributed Computing</em></a> for an introduction to distributed computing in <cite>gensim</cite>.</p>
</div>
<div class="section" id="setting-up-the-cluster">
<h2>Setting up the cluster<a class="headerlink" href="#setting-up-the-cluster" title="Permalink to this headline">¶</a></h2>
<p>We will show how to run distributed Latent Semantic Analysis by means of an example.
Let’s say we have 5 computers at our disposal, all on the same network segment (=reachable
by network broadcast). To start with, install <cite>gensim</cite> and <cite>Pyro</cite> on each computer with:</p>
<div class="highlight-python"><pre>$ sudo easy_install gensim[distributed]</pre>
</div>
<p>and run Pyro’s name server on exactly one of the machines (doesn’t matter which one):</p>
<blockquote>
<div>$ python -m Pyro4.naming -n 0.0.0.0 &</div></blockquote>
<p>Let’s say our example cluster consists of dual-core computers with loads of
memory. We will therefore run <strong>two</strong> worker scripts on four of the physical machines,
creating <strong>eight</strong> logical worker nodes:</p>
<div class="highlight-python"><pre>$ python -m gensim.models.lsi_worker &</pre>
</div>
<p>This will execute <cite>gensim</cite>‘s <cite>lsi_worker.py</cite> script (to be run twice on each of the
four computer).
This lets <cite>gensim</cite> know that it can run two jobs on each of the four computers in
parallel, so that the computation will be done faster, while also taking up twice
as much memory on each machine.</p>
<p>Next, pick one computer that will be a job scheduler in charge of worker
synchronization, and on it, run <cite>LSA dispatcher</cite>. In our example, we will use the
fifth computer to act as the dispatcher and from there run:</p>
<div class="highlight-python"><pre>$ python -m gensim.models.lsi_dispatcher &</pre>
</div>
<p>In general, the dispatcher can be run on the same machine as one of the worker nodes, or it
can be another, distinct computer (within the same broadcast domain). The dispatcher
won’t be doing much with CPU most of the time, but pick a computer with ample memory.</p>
<p>And that’s it! The cluster is set up and running, ready to accept jobs. To remove
a worker later on, simply terminate its <cite>lsi_worker</cite> process. To add another worker, run another
<cite>lsi_worker</cite> (this will not affect a computation that is already running, the additions/deletions are not dynamic).
If you terminate <cite>lsi_dispatcher</cite>, you won’t be able to run computations until you run it again
(surviving worker processes can be re-used though).</p>
</div>
<div class="section" id="running-lsa">
<h2>Running LSA<a class="headerlink" href="#running-lsa" title="Permalink to this headline">¶</a></h2>
<p>So let’s test our setup and run one computation of distributed LSA. Open a Python
shell on one of the five machines (again, this can be done on any computer
in the same <a class="reference external" href="http://en.wikipedia.org/wiki/Broadcast_domain">broadcast domain</a>,
our choice is incidental) and try:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="gp">>>> </span><span class="kn">from</span> <span class="nn">gensim</span> <span class="kn">import</span> <span class="n">corpora</span><span class="p">,</span> <span class="n">models</span><span class="p">,</span> <span class="n">utils</span>
<span class="gp">>>> </span><span class="kn">import</span> <span class="nn">logging</span>
<span class="gp">>>> </span><span class="n">logging</span><span class="o">.</span><span class="n">basicConfig</span><span class="p">(</span><span class="n">format</span><span class="o">=</span><span class="s">'</span><span class="si">%(asctime)s</span><span class="s"> : </span><span class="si">%(levelname)s</span><span class="s"> : </span><span class="si">%(message)s</span><span class="s">'</span><span class="p">,</span> <span class="n">level</span><span class="o">=</span><span class="n">logging</span><span class="o">.</span><span class="n">INFO</span><span class="p">)</span>
<span class="gp">>>> </span><span class="n">corpus</span> <span class="o">=</span> <span class="n">corpora</span><span class="o">.</span><span class="n">MmCorpus</span><span class="p">(</span><span class="s">'/tmp/deerwester.mm'</span><span class="p">)</span> <span class="c"># load a corpus of nine documents, from the Tutorials</span>
<span class="gp">>>> </span><span class="n">id2word</span> <span class="o">=</span> <span class="n">corpora</span><span class="o">.</span><span class="n">Dictionary</span><span class="o">.</span><span class="n">load</span><span class="p">(</span><span class="s">'/tmp/deerwester.dict'</span><span class="p">)</span>
<span class="gp">>>> </span><span class="n">lsi</span> <span class="o">=</span> <span class="n">models</span><span class="o">.</span><span class="n">LsiModel</span><span class="p">(</span><span class="n">corpus</span><span class="p">,</span> <span class="n">id2word</span><span class="o">=</span><span class="n">id2word</span><span class="p">,</span> <span class="n">num_topics</span><span class="o">=</span><span class="mi">200</span><span class="p">,</span> <span class="n">chunksize</span><span class="o">=</span><span class="mi">1</span><span class="p">,</span> <span class="n">distributed</span><span class="o">=</span><span class="bp">True</span><span class="p">)</span> <span class="c"># run distributed LSA on nine documents</span>
</pre></div>
</div>
<p>This uses the corpus and feature-token mapping created in the <a class="reference internal" href="tut1.html"><em>Corpora and Vector Spaces</em></a> tutorial.
If you look at the log in your Python session, you should see a line similar to:</p>
<div class="highlight-python"><pre>2010-08-09 23:44:25,746 : INFO : using distributed version with 8 workers</pre>
</div>
<p>which means all went well. You can also check the logs coming from your worker and dispatcher
processes — this is especially helpful in case of problems.
To check the LSA results, let’s print the first two latent topics:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="gp">>>> </span><span class="n">lsi</span><span class="o">.</span><span class="n">print_topics</span><span class="p">(</span><span class="n">num_topics</span><span class="o">=</span><span class="mi">2</span><span class="p">,</span> <span class="n">num_words</span><span class="o">=</span><span class="mi">5</span><span class="p">)</span>
<span class="go">topic #0(3.341): 0.644*"system" + 0.404*"user" + 0.301*"eps" + 0.265*"time" + 0.265*"response"</span>
<span class="go">topic #1(2.542): 0.623*"graph" + 0.490*"trees" + 0.451*"minors" + 0.274*"survey" + -0.167*"system"</span>
</pre></div>
</div>
<p>Success! But a corpus of nine documents is no challenge for our powerful cluster...
In fact, we had to lower the job size (<cite>chunksize</cite> parameter above) to a single document
at a time, otherwise all documents would be processed by a single worker all at once.</p>
<p>So let’s run LSA on <strong>one million documents</strong> instead:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="gp">>>> </span><span class="c"># inflate the corpus to 1M documents, by repeating its documents over&over</span>
<span class="gp">>>> </span><span class="n">corpus1m</span> <span class="o">=</span> <span class="n">utils</span><span class="o">.</span><span class="n">RepeatCorpus</span><span class="p">(</span><span class="n">corpus</span><span class="p">,</span> <span class="mi">1000000</span><span class="p">)</span>
<span class="gp">>>> </span><span class="c"># run distributed LSA on 1 million documents</span>
<span class="gp">>>> </span><span class="n">lsi1m</span> <span class="o">=</span> <span class="n">models</span><span class="o">.</span><span class="n">LsiModel</span><span class="p">(</span><span class="n">corpus1m</span><span class="p">,</span> <span class="n">id2word</span><span class="o">=</span><span class="n">id2word</span><span class="p">,</span> <span class="n">num_topics</span><span class="o">=</span><span class="mi">200</span><span class="p">,</span> <span class="n">chunksize</span><span class="o">=</span><span class="mi">10000</span><span class="p">,</span> <span class="n">distributed</span><span class="o">=</span><span class="bp">True</span><span class="p">)</span>
<span class="gp">>>> </span><span class="n">lsi1m</span><span class="o">.</span><span class="n">print_topics</span><span class="p">(</span><span class="n">num_topics</span><span class="o">=</span><span class="mi">2</span><span class="p">,</span> <span class="n">num_words</span><span class="o">=</span><span class="mi">5</span><span class="p">)</span>
<span class="go">topic #0(1113.628): 0.644*"system" + 0.404*"user" + 0.301*"eps" + 0.265*"time" + 0.265*"response"</span>
<span class="go">topic #1(847.233): 0.623*"graph" + 0.490*"trees" + 0.451*"minors" + 0.274*"survey" + -0.167*"system"</span>
</pre></div>
</div>
<p>The log from 1M LSA should look like:</p>
<div class="highlight-python"><pre>2010-08-10 02:46:35,087 : INFO : using distributed version with 8 workers
2010-08-10 02:46:35,087 : INFO : updating SVD with new documents
2010-08-10 02:46:35,202 : INFO : dispatched documents up to #10000
2010-08-10 02:46:35,296 : INFO : dispatched documents up to #20000
...
2010-08-10 02:46:46,524 : INFO : dispatched documents up to #990000
2010-08-10 02:46:46,694 : INFO : dispatched documents up to #1000000
2010-08-10 02:46:46,694 : INFO : reached the end of input; now waiting for all remaining jobs to finish
2010-08-10 02:46:47,195 : INFO : all jobs finished, downloading final projection
2010-08-10 02:46:47,200 : INFO : decomposition complete</pre>
</div>
<p>Due to the small vocabulary size and trivial structure of our “one-million corpus”, the computation
of LSA still takes only 12 seconds. To really stress-test our cluster, let’s do
Latent Semantic Analysis on the English Wikipedia.</p>
<div class="section" id="distributed-lsa-on-wikipedia">
<h3>Distributed LSA on Wikipedia<a class="headerlink" href="#distributed-lsa-on-wikipedia" title="Permalink to this headline">¶</a></h3>
<p>First, download and prepare the Wikipedia corpus as per <a class="reference internal" href="wiki.html"><em>Experiments on the English Wikipedia</em></a>, then load
the corpus iterator with:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="gp">>>> </span><span class="kn">import</span> <span class="nn">logging</span><span class="o">,</span> <span class="nn">gensim</span><span class="o">,</span> <span class="nn">bz2</span>
<span class="gp">>>> </span><span class="n">logging</span><span class="o">.</span><span class="n">basicConfig</span><span class="p">(</span><span class="n">format</span><span class="o">=</span><span class="s">'</span><span class="si">%(asctime)s</span><span class="s"> : </span><span class="si">%(levelname)s</span><span class="s"> : </span><span class="si">%(message)s</span><span class="s">'</span><span class="p">,</span> <span class="n">level</span><span class="o">=</span><span class="n">logging</span><span class="o">.</span><span class="n">INFO</span><span class="p">)</span>
<span class="gp">>>> </span><span class="c"># load id->word mapping (the dictionary)</span>
<span class="gp">>>> </span><span class="n">id2word</span> <span class="o">=</span> <span class="n">gensim</span><span class="o">.</span><span class="n">corpora</span><span class="o">.</span><span class="n">Dictionary</span><span class="o">.</span><span class="n">load_from_text</span><span class="p">(</span><span class="s">'wiki_en_wordids.txt'</span><span class="p">)</span>
<span class="gp">>>> </span><span class="c"># load corpus iterator</span>
<span class="gp">>>> </span><span class="n">mm</span> <span class="o">=</span> <span class="n">gensim</span><span class="o">.</span><span class="n">corpora</span><span class="o">.</span><span class="n">MmCorpus</span><span class="p">(</span><span class="s">'wiki_en_tfidf.mm'</span><span class="p">)</span>
<span class="gp">>>> </span><span class="c"># mm = gensim.corpora.MmCorpus(bz2.BZ2File('wiki_en_tfidf.mm.bz2')) # use this if you compressed the TFIDF output</span>
<span class="gp">>>> </span><span class="k">print</span> <span class="n">mm</span>
<span class="go">MmCorpus(3199665 documents, 100000 features, 495547400 non-zero entries)</span>
</pre></div>
</div>
<p>Now we’re ready to run distributed LSA on the English Wikipedia:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="gp">>>> </span><span class="c"># extract 400 LSI topics, using a cluster of nodes</span>
<span class="gp">>>> </span><span class="n">lsi</span> <span class="o">=</span> <span class="n">gensim</span><span class="o">.</span><span class="n">models</span><span class="o">.</span><span class="n">lsimodel</span><span class="o">.</span><span class="n">LsiModel</span><span class="p">(</span><span class="n">corpus</span><span class="o">=</span><span class="n">mm</span><span class="p">,</span> <span class="n">id2word</span><span class="o">=</span><span class="n">id2word</span><span class="p">,</span> <span class="n">num_topics</span><span class="o">=</span><span class="mi">400</span><span class="p">,</span> <span class="n">chunksize</span><span class="o">=</span><span class="mi">20000</span><span class="p">,</span> <span class="n">distributed</span><span class="o">=</span><span class="bp">True</span><span class="p">)</span>
<span class="gp">>>> </span><span class="c"># print the most contributing words (both positively and negatively) for each of the first ten topics</span>
<span class="gp">>>> </span><span class="n">lsi</span><span class="o">.</span><span class="n">print_topics</span><span class="p">(</span><span class="mi">10</span><span class="p">)</span>
<span class="go">2010-11-03 16:08:27,602 : INFO : topic #0(200.990): -0.475*"delete" + -0.383*"deletion" + -0.275*"debate" + -0.223*"comments" + -0.220*"edits" + -0.213*"modify" + -0.208*"appropriate" + -0.194*"subsequent" + -0.155*"wp" + -0.117*"notability"</span>
<span class="go">2010-11-03 16:08:27,626 : INFO : topic #1(143.129): -0.320*"diff" + -0.305*"link" + -0.199*"image" + -0.171*"www" + -0.162*"user" + 0.149*"delete" + -0.147*"undo" + -0.144*"contribs" + -0.122*"album" + 0.113*"deletion"</span>
<span class="go">2010-11-03 16:08:27,651 : INFO : topic #2(135.665): -0.437*"diff" + -0.400*"link" + -0.202*"undo" + -0.192*"user" + -0.182*"www" + -0.176*"contribs" + 0.168*"image" + -0.109*"added" + 0.106*"album" + 0.097*"copyright"</span>
<span class="go">2010-11-03 16:08:27,677 : INFO : topic #3(125.027): -0.354*"image" + 0.239*"age" + 0.218*"median" + -0.213*"copyright" + 0.204*"population" + -0.195*"fair" + 0.195*"income" + 0.167*"census" + 0.165*"km" + 0.162*"households"</span>
<span class="go">2010-11-03 16:08:27,701 : INFO : topic #4(116.927): -0.307*"image" + 0.195*"players" + -0.184*"median" + -0.184*"copyright" + -0.181*"age" + -0.167*"fair" + -0.162*"income" + -0.151*"population" + -0.136*"households" + -0.134*"census"</span>
<span class="go">2010-11-03 16:08:27,728 : INFO : topic #5(100.326): 0.501*"players" + 0.318*"football" + 0.284*"league" + 0.193*"footballers" + 0.142*"image" + 0.133*"season" + 0.119*"cup" + 0.113*"club" + 0.110*"baseball" + 0.103*"f"</span>
<span class="go">2010-11-03 16:08:27,754 : INFO : topic #6(92.298): -0.411*"album" + -0.275*"albums" + -0.217*"band" + -0.214*"song" + -0.184*"chart" + -0.163*"songs" + -0.160*"singles" + -0.149*"vocals" + -0.139*"guitar" + -0.129*"track"</span>
<span class="go">2010-11-03 16:08:27,780 : INFO : topic #7(83.811): -0.248*"wikipedia" + -0.182*"keep" + 0.180*"delete" + -0.167*"articles" + -0.152*"your" + -0.150*"my" + 0.144*"film" + -0.130*"we" + -0.123*"think" + -0.120*"user"</span>
<span class="go">2010-11-03 16:08:27,807 : INFO : topic #8(78.981): 0.588*"film" + 0.460*"films" + -0.130*"album" + -0.127*"station" + 0.121*"television" + 0.115*"poster" + 0.112*"directed" + 0.110*"actors" + -0.096*"railway" + 0.086*"movie"</span>
<span class="go">2010-11-03 16:08:27,834 : INFO : topic #9(78.620): 0.502*"kategori" + 0.282*"categoria" + 0.248*"kategorija" + 0.234*"kategorie" + 0.172*"категория" + 0.165*"categoría" + 0.161*"kategoria" + 0.148*"categorie" + 0.126*"kategória" + 0.121*"catégorie"</span>
</pre></div>
</div>
<p>In serial mode, creating the LSI model of Wikipedia with this <strong>one-pass algorithm</strong>
takes about 5.25h on my laptop (OS X, C2D 2.53GHz, 4GB RAM with <cite>libVec</cite>).
In distributed mode with four workers (Linux, dual-core Xeons of 2Ghz, 4GB RAM
with <cite>ATLAS</cite>), the wallclock time taken drops to 1 hour and 41 minutes. You can
read more about various internal settings and experiments in my <a class="reference external" href="http://nlp.fi.muni.cz/~xrehurek/nips/rehurek_nips.pdf">research
paper</a>.</p>
</div>
</div>
</div>
</div>
</div>
</div>
<div class="clearer"></div>
</div>
<div class="related">
<h3>Navigation</h3>
<ul>
<li class="right" style="margin-right: 10px">
<a href="genindex.html" title="General Index"
>index</a></li>
<li class="right" >
<a href="py-modindex.html" title="Python Module Index"
>modules</a> |</li>
<li class="right" >
<a href="dist_lda.html" title="Distributed Latent Dirichlet Allocation"
>next</a> |</li>
<li class="right" >
<a href="distributed.html" title="Distributed Computing"
>previous</a> |</li>
<li><a href="index.html">Gensim home</a>| </li>
<li><a href="tutorial.html">Tutorials</a>| </li>
<li><a href="http://groups.google.com/group/gensim">Support</a>| </li>
<li><a href="https://github.com/piskvorky/gensim/wiki">Contribute</a>| </li>
<li><a href="apiref.html">API reference</a>»</li>
<li><a href="distributed.html" >Distributed Computing</a> »</li>
</ul>
</div>
<div class="footer">
© Copyright 2012, Radim Řehůřek <radimrehurek(at)seznam.cz>.
Last updated on Mar 09, 2012.
</div>
</body>
</html>