This repository has been archived by the owner on Aug 6, 2022. It is now read-only.
-
Notifications
You must be signed in to change notification settings - Fork 0
/
beszamolo.tex
655 lines (501 loc) · 21.9 KB
/
beszamolo.tex
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
\documentclass[12pt,a4paper,oneside]{article} % part, *section, *paragraph
%\documentclass[12pt,a4paper,oneside]{report} % part, *chapter, *section, *paragraph
%\documentclass[12pt,a4paper,oneside]{book} % part, *chapter, *section, *paragraph
\usepackage[english]{babel}
\usepackage[latin2]{inputenc}
\usepackage{setspace}
\usepackage{float}
\usepackage[T1]{fontenc} %\showhyphens{integetnem} \showhyphens{meggyültetvény}
%\usepackage{ucs}\usepackage[utf8x]{inputenc}
%\usepackage{fullpage} %\usepackage[margin=2cm]{geometry}
%\usepackage{amsmath,amssymb}
\usepackage{url} %VAGY %\usepackage[unicode]{hyperref}
\usepackage{listings}
%\usepackage{color}
\usepackage{graphicx}
\usepackage{times}
\usepackage{fullpage}
%\usepackage{showkeys} % \label{} kiírása \ref{}hez
%\let\stdsection\section
%\renewcommand\section{\clearpage\stdsection}
\begin{document}
\onehalfspacing
\begin{center}
\vspace{2em}
%\large{\bf{}BUDAPESTI M\H{U}SZAKI \'ES GAZDAS\'AGTUDOM\'ANYI EGYETEM}
\vspace{.5em}
\includegraphics[width=60mm]{10000000000004C00000014A5D3CE278.eps}
Budapest University of Technology and Economics
Department of Measurement and Information Systems
%\vspace{.5em}
%\large{\bf{}VILLAMOSM\'ERN\"OKI \'ES INFORMATIKAI KAR\\
%M\'ERN\"OK INFORMATIKUS SZAK}
\vspace{2em}
%\Large{Informatikai Technol\'ogi\'ak szakir\'any\\
%Rendszerfejleszt\'es \'agazat}
%\vspace{1.5em}
%\Large{\bf{}\"On\'all\'o labor (BMEVIIIA337)}
%\vspace{1.5em}
\LARGE{\bf{}Fault tolerance and performance of dm-mirror}
\vspace{2em}
\begin{tabular}{ l c}
%hopefully i can avoid the photo this time
%\parbox{30mm}{\includegraphics[width=20mm]{20090926.eps}} &
\parbox{120mm}{
\begin{center}
\normalsize{Vajna Miklós (AYU9RZ),
2nd semester, (MSc) computer engineering student
Consultants: Dr. Szentiványi Gábor, Dr. Szántó Iván, ULX
Internal consultant: Horváth Ákos, MIT
Specialization in Dependable System Design
Project Laboratory 2 Interim Report
2010/11. 1st semester}
\end{center}
} \\
\end{tabular}
\end{center}
\thispagestyle{empty}
\newpage
\tableofcontents
\newpage
\section{Introduction}
The dm-mirror Linux kernel module is a software mirror implementation in the
Linux kernel, which is part of the device-mapper framework, most commonly used
for logical volume management (LVM\cite{lvm}). The de facto common standard in industry
for software RAID in Linux environments is md-mirror nowadays, which is
independent from LVM. My target is to improve dm-mirror in a way so that it's
feature set will be comparable (and then even better) to the already available
dm-mirror, while keeping it based on LVM.
My initial goal is to achieve fault tolerance and optimal performance in a
distributed storage setting with storage and network components having
different performance characteristics.
All my steps aim to converge the design and development dm-mirror in that
direction.
\section{Background}
Postponed: comparison of dm-mirror vs. md-mirror.
\section{The problems of dm-mirror}
My initial problem is that in a distributed environment the local and the
remote nodes have different characteristics and this should be supported when
accessing device nodes. For example, when reading, it's important to use the
fastest node available, or - a sightly fine-grained method - use a weighted
approach to read more from faster nodes. It's also important to tolerate and
handle failures in a proper way, since those happen a lot in distributed
systems.
Having a look at the current dm-mirror\cite{lvmmirror}, I saw:
\begin{itemize}
\item The current read method is a Round-Robin algorithm, which is not capable
of proper handling of device nodes with different read speed.
\item Fault tolerance is in its initial stage: when a device node fails and
later recovers, manual action is needed, which takes time.
\end{itemize}
\section{Implementation}
The LVM mirror consists of multiple legs: all but one serves as a data storage,
and one does logging. This way the storage part of the mirror can be
represented as an array of logical device nodes - in case of two data and one
mirror leg, there will be two elements in the array. We index it from zero, so
the first element will have an index 0, then the second data leg will have an
index 1. The default mirror is the leg which is currently currently used for
reading.
When I read the source code of dm-mirror, I found two problems which seemed to
be relatively easy to fix:
\begin{itemize}
\item In case the two legs of the mirror have different read access, it's a
good idea to completely disable the Round-Robin algorithm and read from the
default mirror only.
\item In case the default mirror is not the fastest one, the user will want to
set the default mirror to a non-zero index value.
\end{itemize}
The current behaviour is that a number of read operations are done on one
device, then a switch occurs. The switch is done exactly when a limit is
reached - this is called \emph{rr\_ios\_set} in the source code. This number has
to be greater or equal to one at the moment.
What I did is the following: I extended it to support zero as a value as well, and in that case a switch never occurs.
The kernel part of the support code can be found in the appendix (first patch).
And then the user-space command to disable the switches is the following:
\begin{verbatim}
# dmsetup message myvg-mymirror 0 \
io_balance round_robin ios 0
\end{verbatim}
The first zero is the identifier of the device, the second one is the number
introducing the real change.
The second problem is about: once we disabled automatic switches, we want to set the default mirror manually. To allow this, I created the second patch, which can be found in the appendix as well.
The user-space setting then can be used as follows:
\begin{itemize}
\item Use \emph{lvs -a -o +devices} to list devices, the \emph{mimage} nodes will be the physical nodes of a mirror.
\item Once you have the right mimage, use \emph{dmsetup ls} to get the real device number, for example 253:3.
\item Finally use \emph{dmsetup message} to send the wished device number to the kernel
\end{itemize}
A possible commandline is:
\begin{verbatim}
# dmsetup message myvg-mymirror 0 io_balance default 253:3
\end{verbatim}
\section{Measurements}
I used the following setup for the tests:
\begin{figure}[H]
\centering
\includegraphics[width=150mm,keepaspectratio]{testarch.eps}
\caption{Architecture of the test environment}
\end{figure}
The two machines were virtual ones, projlab2 hosted the LVM mirror, I ran all
timers there, finally that was the iSCSI initiator. The other box served as the
iSCSI target. Additionally I limited the bandwidth there, and I added the
network delay there as well.
I had two scripts to limit bandwidth and to add network delay to the traffic of
the iSCSI target.
The network delay part is more simple. To add some delay to each outgoing or
incoming package on the \emph{eth0} interface, the \emph{tc} command can be
used:
\begin{verbatim}
tc qdisc add dev eth0 root netem delay 100ms
\end{verbatim}
In case the delay is no longer necessary, a single command can remove the above
rule:
\begin{verbatim}
tc qdisc del dev eth0 root
\end{verbatim}
The Linux Advanced Routing and Traffic Control HOWTO\cite{lartc} explains the
\emph{tc} command in more detail.
There are multiple ways to limit bandwidth. And older technique is Class Based
Queueing, a newer one is Hierarchy Token Bucket. In short, HTB is a successor
to CBQ, see its documentation\cite{htb} fore more details.
The LARC HOWTO even provides a ready to use script to limit bandwidth, I used
that without any major modifications.
\subsection{Read speed}
First I checked the read time of a 2MB-sized file without network latency and
with 200ms delay over an iSCSI device node. I following table and chart
demonstrate the results (all numbers are in seconds, unless otherwise stated):
\begin{center}
\begin{table}[H]
\centering
\begin{tabular}{| l | l |}
\hline
Type & Time \\ \hline
Local read & 0.03 \\
Remote read & 17.642 \\
\hline
\end{tabular}
\caption{Local and remote reads with network latency}
\end{table}
\end{center}
\begin{figure}[H]
\centering
\includegraphics[width=75mm,keepaspectratio]{charts/1_63947f00.eps}
\caption{Comparing local and remote reads with network latency}
\end{figure}
The reason for such a huge difference with 200ms latency is that this delay is
meant for each package, while the transferred data is 2MB.
Then I measured read speed of 2MB data on a mirror (every result is the average
of 3 individual measures):
\begin{center}
\begin{table}[H]
\centering
\begin{tabular}{| l | l | l | l | l |}
\hline
Type & m1 (s) & m2 (s) & m3(s) & avg (s) \\ \hline
Default (RR, n=128) with delay & 9.385 & 0.274 & 1.135 & 3.598 \\
Local only & 0.282 & 0.259 & 0.224 & 0.255 \\
Remote without delay & 1.262 & 2.004 & 1.644 & 1.636 \\
Remote with delay & 22.826 & 20.225 & 13.339 & 18.796 \\
\hline
\end{tabular}
\caption{Mirror read speed of 2MB data}
\end{table}
\end{center}
\begin{figure}[H]
\centering
\includegraphics[width=75mm,keepaspectratio]{charts/7_69a4ab34.eps}
\caption{Comparing mirror read speed of 2MB data}
\end{figure}
As expected, the fastest method is doing all reads on the local node exclusively.
By remote "with" or "without" delay, I mean that in the first situation I used
the \emph{tc} tool to add some simulated delay, while in the second case I
didn't use such a tool, so basically there were no delay. (Because the two
virtual machines were on the same physical host.)
It makes sense to calculate if the results are approximately valid before we
continue. The total delay of packages from the above table is 17.160s. If the
delay for each logical package is 200ms, that should mean about 86 logical
packages. By logical, I mean I don't count physical fragmentation of the IP
packages. When I recorded the network traffic with \emph{tcpdump}, then I got
70 packages having the TCP PUSH flag (last physical IP package of a logical
write). The difference is measure error, we already saw above that the result
varies a bit each time we repeat the test. As a result, sadly it's not entirely
possible to exactly repeat the same test with and without network delay.
Now that we understand the difference between remote reads with and without
network delay, let's compare the other results. Reading using Round Robin (with
delay) is slower than a remote read without delay, we expected that. Also,
reading from remote with delay is slower than reading from local and remote
(what RR does) as well, so the results show what we would predict.
\subsection{Write speed}
First I measured write speed with different network bandwidth available. In this case no LVM technique was involved, I used plain scp to copy data projlab2 to iscsitarget:
\begin{center}
\begin{table}[H]
\centering
\begin{tabular}{| l | l |}
\hline
Type & Time \\ \hline
10kbit & 22.951 \\
100mbit & 1.705 \\
unlimited & 1.767 \\
\hline
\end{tabular}
\caption{Network write speed with 2MB of data}
\end{table}
\end{center}
\begin{figure}[H]
\centering
\includegraphics[width=75mm,keepaspectratio]{charts/2_5fabe87e.eps}
\caption{Comparing network write speed with 2MB of data}
\end{figure}
From now, I measured mirror write speed of 2MB data, with various parameters.
\subsubsection{Singe log, local log, local disks}
Setup: I created myvg from sdb, sdc and sdd, then mymirror on top of myvg.
\begin{figure}[H]
\centering
\includegraphics[width=75mm,keepaspectratio]{setup1.eps}
\caption{Architecture of single log, local log, local disks}
\end{figure}
That is a single measurement (3 times, and then counting the average in seconds): 0.591, 0.603, 0.588 - average: 0.594.
Tear down: I removed myvg.
\subsubsection{Single log, remote log, local and remote disk, different ratios}
By different ratios, I mean limiting network bandwidth different way. I want to
clarify the connection between logical ratios and exact network speeds I used:
\begin{center}
\begin{table}[H]
\centering
\begin{tabular}{| l | l |}
\hline
Logical ratio & physical speed (kilobits) \\ \hline
1:2 & 50000 \\
1:5 & 20000 \\
1:10 & 10000 \\
1:50 & 2000 \\
1:100 & 1000 \\
\hline
\end{tabular}
\caption{Comparison of different network bandwidth ratios and physical speeds}
\end{table}
\end{center}
Setup: I created sdb1 and sdb1 on iscsitarget and started the daemon there.
Then I configured the client, the new devices nodes here are sdf1 and sdf2.
Finally I configured the mirror named mymirror from sdb, sdf1 and sdf2.
\begin{figure}[H]
\centering
\includegraphics[width=150mm,keepaspectratio]{setup2.eps}
\caption{Architecture of single log, remote log, local and remote disks}
\end{figure}
\begin{center}
\begin{table}[H]
\centering
\begin{tabular}{| l | l | l | l | l |}
\hline
Type & m1 (s) & m2 (s) & m3 (s) & avg (s)\\ \hline
1:2 & 1.705 & 1.841 & 1.939 & 1.828 \\
1:5 & 21.716 & 23.399 & 23.503 & 22.872 \\
1:10 & 38.133 & 38.124 & 36.606 & 37.621 \\
1:50 & 47.035 & 48.205 & 47.688 & 47.642 \\
1:100 & 49.633 & 48.240 & 48.057 & 48.643 \\
\hline
\end{tabular}
\caption{Network write speed with single log, remote log, local and remote disk, different ratios}
\end{table}
\end{center}
\begin{figure}[H]
\centering
\includegraphics[width=75mm,keepaspectratio]{charts/3_5e6daa6.eps}
\caption{Comparing network write speed with single log, remote log, local and remote disk, different ratios}
\end{figure}
Tear down: I disabled bandwidth limitation.
\subsubsection{Single log, remote log, local and remote disk, different delays}
No setup, the mirror was already there.
\begin{center}
\begin{table}[H]
\centering
\begin{tabular}{| l | l | l | l | l |}
\hline
Type & m1 (s) & m2 (s) & m3 (s) & avg (s)\\ \hline
5ms & 0.577 & 0.653 & 1.239 & 0.823 \\
10ms & 1.086 & 0.564 & 0.606 & 0.752 \\
50ms & 2.649 & 2.622 & 2.759 & 2.676 \\
100ms & 6.067 & 3.517 & 3.441 & 4.341 \\
200ms & 16.352 & 13.930 & 10.392 & 13.558 \\
\hline
\end{tabular}
\caption{Network write speed with single log, remote log, local and remote disk, different delays}
\end{table}
\end{center}
\begin{figure}[H]
\centering
\includegraphics[width=75mm,keepaspectratio]{charts/4_m724ff87b.eps}
\caption{Comparing network write speed with single log, remote log, local and remote disk, different delays}
\end{figure}
Tear down: I disabled the simulation of network latency, then removed myvg.
\subsubsection{Single log, local log, local and remote disk, different ratios}
Setup: I created mymirror from sdb, sdf1 and sdc.
\begin{figure}[H]
\centering
\includegraphics[width=150mm,keepaspectratio]{setup3.eps}
\caption{Architecture of single log, local log, local and remote disks}
\end{figure}
\begin{center}
\begin{table}[H]
\centering
\begin{tabular}{| l | l | l | l | l |}
\hline
Type & m1 (s) & m2 (s) & m3 (s) & avg (s) \\ \hline
1:2 & 2.460 & 5.277 & 2.415 & 3.384 \\
1:5 & 5.028 & 6.440 & 9.629 & 7.032 \\
1:10 & 4.528 & 11.906 & 4.195 & 6.876 \\
1:50 & 45.302 & 48.226 & 44.406 & 45.978 \\
1:100 & 48.311 & 46.820 & 48.314 & 47.815 \\
\hline
\end{tabular}
\caption{Network write speed with single log, local log, local and remote disk, different ratios}
\end{table}
\end{center}
\begin{figure}[H]
\centering
\includegraphics[width=75mm,keepaspectratio]{charts/5_5d601c11.eps}
\caption{Comparing network write speed with single log, local log, local and remote disk, different ratios}
\end{figure}
Tear down: I disabled bandwidth limitation.
\subsubsection{Single log, local log, local and remote disk, different delays}
No setup, the mirror is the same as with the previous test.
\begin{center}
\begin{table}[H]
\centering
\begin{tabular}{| l | l | l | l | l |}
\hline
Type & m1 (s) & m2 (s) & m3 (s) & avg (s) \\ \hline
5ms & 3.736 & 3.489 & 3.166 & 3.463 \\
10ms & 3.267 & 3.872 & 2.846 & 3.328 \\
50ms & 2.844 & 3.929 & 2.306 & 3.026 \\
100ms & 5.306 & 4.578 & 4.402 & 4.762 \\
200ms & 7.114 & 5.978 & 8.920 & 7.337 \\
\hline
\end{tabular}
\caption{Network write speed with single log, local log, local and remote disk, different delays}
\end{table}
\end{center}
\begin{figure}[H]
\centering
\includegraphics[width=75mm,keepaspectratio]{charts/6_273f2c73.eps}
\caption{Comparing network write speed with single log, local log, local and remote disk, different delays}
\end{figure}
Tear down: I disabled network latency, then removed myvg.
\subsubsection{Mirrored log, local disks}
For now, I tested this on my RHEL5 machine where I already had the backported
lvm2 from Fedora rawhide. This has the benefit that other components of the system
is not changed. The downside is that this way the measurement is less official,
since then this is a custom system.
Setup: I created myvg from sdb, sdc, sdd and sde.
\begin{figure}[H]
\centering
\includegraphics[width=75mm,keepaspectratio]{setup4.eps}
\caption{Architecture of mirrored log, local disks}
\end{figure}
That is a single measurement (3 times, and then counting the average in seconds):
3.183, 3.181, 3.195, average: 3.186.
Tear down: I removed myvg.
\subsubsection{Mirrored log, local and remote disk, different ratios}
Setup: I created myvg from sdb, sdf1, sdc and sdf2.
\begin{figure}[H]
\centering
\includegraphics[width=150mm,keepaspectratio]{setup5.eps}
\caption{Architecture of mirrored log, local and remote disks}
\end{figure}
\begin{center}
\begin{table}[H]
\centering
\begin{tabular}{| l | l | l | l | l |}
\hline
Type & m1 (s) & m2 (s) & m3 (s) & avg (s)\\ \hline
1:2 & 1.756 & 2.026 & 2.316 & 2.032 \\
1:5 & 21.773 & 23.324 & 22.887 & 22.661 \\
1:10 & 38.318 & 38.181 & 38.240 & 38.246 \\
1:50 & 47.423 & 45.946 & 47.901 & 47.09 \\
1:100 & 50.239 & 48.765 & 48.075 & 49.026 \\
\hline
\end{tabular}
\caption{Network write speed with mirrored log, local and remote disk, different ratios}
\end{table}
\end{center}
\begin{figure}[H]
\centering
\includegraphics[width=75mm,keepaspectratio]{charts/8_m1d51a4a2.eps}
\caption{Comparing network write speed with mirrored log, local and remote disk, different ratios}
\end{figure}
Tear down: I stopped the bandwidth limit.
\subsubsection{Mirrored log, local and remote disk, different delays}
No setup, I re-used the mirror from the previous test.
\begin{center}
\begin{table}[H]
\centering
\begin{tabular}{| l | l | l | l | l |}
\hline
Type & m1 (s) & m2 (s) & m3 (s) & avg (s) \\ \hline
5ms & 1.380 & 0.824 & 0.636 & 0.946 \\
10ms & 1.300 & 0.741 & 0.776 & 0.939 \\
50ms & 3.430 & 3.009 & 3.228 & 3.222 \\
100ms & 7.673 & 4.269 & 3.583 & 5.175 \\
200ms & 14.233 & 11.855 & 11.066 & 12.384 \\
\hline
\end{tabular}
\caption{Network write speed with mirrored log, local and remote disk, different delays}
\end{table}
\end{center}
\begin{figure}[H]
\centering
\includegraphics[width=75mm,keepaspectratio]{charts/9.eps}
\caption{Comparing network write speed with mirrored log, local and remote disk, different delays}
\end{figure}
Tear down: I disabled network latency, then removed myvg.
\subsection{Write speed in a real-life environment}
I did the same test in a real environment. The test is the same: I wrote 2MB of
data to the mirror. The results are still in seconds.
\subsubsection*{Single log, remote log, local and remote disks}
I could not change the parameters here, so I simply did the test 3 times: 12.106, 12.586, 12.784; average: 12.49.
\subsubsection*{Single log, local log, local and remote disks}
Same here (not possible to change parameters), the results: 11.706, 12.996, 12.445; average: 12.38.
\subsubsection*{Mirrored log, local disks}
The results: 2.987, 3.146, 3.383; average: 3.17.
\subsubsection*{Mirrored log, local and remote disks}
The results: 12.337, 12.476, 12.896; average: 12.56.
\subsection{Conclusions}
\begin{itemize}
\item The measured numbers are not always the ones we expected, though generally they are OK, I think.
\item One false effect is that \emph{sync} does not do empty the write cache on the \emph{iscsitarget} (remote) node.
\item In general, I would say the mirrored log is a nice feature, as that way
the two leg of the mirror can be really identical.
On the other hand, it seems to be a huge overhead. This means that in case
dm-mirror is running on one machine only, it is not a major benefit. This is
because in case dm-mirror has a single local log, that's already enough.
Note: dm-mirror is running on multiple boxes in cluster environments, so it
still makes sense to have mirrored logs in those cases.
\end{itemize}
\section{What to work on next}
I basically have three ideas on where could I continue:
\begin{itemize}
\item Even though I already did some measurement, more could be necessary. Maybe a more automated setup would provide better numbers. The current measures were executed manually, then I had to copy the numbers to this document, finally I manually created the charts as well. This may worth more automatism.
\item I could do further research on how to improve performance. For example it would be possible to let the dm-mirror kernel module handle do weighted RR, in case a simple RR is suboptimal, but reading from a single disk only isn't the best, either.
\item Finally, sooner or later it will be important to add support for storing the RR number and the default mirror device in the LVM metadata.
\end{itemize}
If it's up to me, I would say the last one is the more interesting, since
currently the dmsetup commands have to be put to some init script which is
executed on startup any time, which is a bit ugly.
\clearpage
\begin{thebibliography}{4}
\addcontentsline{toc}{section}{References}
\bibitem{lvm} Logical Volume Manager 2, http://sources.redhat.com/lvm2/
\bibitem{lvmmirror} Mirror Volumes Using Logical Volume Manager, RedHat Enterprise Linux 5 \\ Documentation, \url{http://bit.ly/bkcn6d}
\bibitem{lartc} Linux Advanced Routing and Traffic Control HOWTO, http://lartc.org/howto/index.html
\bibitem{htb} Hierarchy Token Bucket qdisc in Linux, http://lartc.org/manpages/tc-htb.html
\end{thebibliography}
\clearpage
\appendix
\section{Patches}
\lstset{numbers=left, numberstyle=\tiny, basicstyle=\ttfamily, breaklines=true, frame=single, tabsize=2}
\subsection{Allow setting ios count to 0}
\lstinputlisting{../../patches/0001-dm-mirror-allow-setting-ios-count-to-0.patch}
\subsection{Allow setting the default mirror}
\lstinputlisting{../../patches/0002-dm-mirror-allow-setting-the-default-mirror.patch}
\end{document}