Skip to content
Newer
Older
100644 265 lines (185 sloc) 8.56 KB
bb7b0fb @jthornber [documentation] update thin-provisioning.txt
authored Jul 8, 2011
1 Introduction
2 ============
8383bec @jthornber add multisnap.txt
authored May 24, 2011
3
a72cd8f @snitm dm thin: documentation updates
snitm authored Jul 7, 2011
4 This document descibes a collection of device-mapper targets that
497cad3 @snitm dm thinp: agk's latest tweaks
snitm authored Jul 25, 2011
5 between them implement thin-provisioning and snapshots.
8383bec @jthornber add multisnap.txt
authored May 24, 2011
6
7 The main highlight of this implementation, compared to the previous
497cad3 @snitm dm thinp: agk's latest tweaks
snitm authored Jul 25, 2011
8 implementation of snapshots, is that it allows many virtual devices to
9 be stored on the same data volume. This simplifies administration and
10 allows the sharing of data between volumes, thus reducing disk usage.
8383bec @jthornber add multisnap.txt
authored May 24, 2011
11
497cad3 @snitm dm thinp: agk's latest tweaks
snitm authored Jul 25, 2011
12 Another significant feature is support for an arbitrary depth of
13 recursive snapshots (snapshots of snapshots of snapshots ...). The
14 previous implementation of snapshots did this by chaining together
15 lookup tables, and so performance was O(depth). This new
16 implementation uses a single data structure to avoid this degradation
17 with depth. Fragmentation may still be an issue, however, in some
18 scenarios.
8383bec @jthornber add multisnap.txt
authored May 24, 2011
19
497cad3 @snitm dm thinp: agk's latest tweaks
snitm authored Jul 25, 2011
20 Metadata is stored on a separate device from data, giving the
21 administrator some freedom, for example to:
8383bec @jthornber add multisnap.txt
authored May 24, 2011
22
23 - Improve metadata resilience by storing metadata on a mirrored volume
24 but data on a non-mirrored one.
25
497cad3 @snitm dm thinp: agk's latest tweaks
snitm authored Jul 25, 2011
26 - Improve performance by storing the metadata on SSD.
bb7b0fb @jthornber [documentation] update thin-provisioning.txt
authored Jul 8, 2011
27
28 Status
29 ======
8383bec @jthornber add multisnap.txt
authored May 24, 2011
30
497cad3 @snitm dm thinp: agk's latest tweaks
snitm authored Jul 25, 2011
31 These targets are very much still in the EXPERIMENTAL state. Please
32 do not yet rely on them in production. But do experiment and offer us
33 feedback. Different use cases will have different performance
34 characteristics, for example due to fragmentation of the data volume.
8383bec @jthornber add multisnap.txt
authored May 24, 2011
35
497cad3 @snitm dm thinp: agk's latest tweaks
snitm authored Jul 25, 2011
36 If you find this software is not performing as expected please mail
37 dm-devel@redhat.com with details and we'll try our best to improve
38 things for you.
8383bec @jthornber add multisnap.txt
authored May 24, 2011
39
497cad3 @snitm dm thinp: agk's latest tweaks
snitm authored Jul 25, 2011
40 Userspace tools for checking and repairing the metadata are under
41 development.
bb7b0fb @jthornber [documentation] update thin-provisioning.txt
authored Jul 8, 2011
42
43 Cookbook
44 ========
8383bec @jthornber add multisnap.txt
authored May 24, 2011
45
497cad3 @snitm dm thinp: agk's latest tweaks
snitm authored Jul 25, 2011
46 This section describes some quick recipes for using thin provisioning.
47 They use the dmsetup program to control the device-mapper driver
48 directly. End users will be advised to use a higher-level volume
49 manager such as LVM2 once support has been added.
bb7b0fb @jthornber [documentation] update thin-provisioning.txt
authored Jul 8, 2011
50
51 Pool device
52 -----------
8383bec @jthornber add multisnap.txt
authored May 24, 2011
53
54 The pool device ties together the metadata volume and the data volume.
497cad3 @snitm dm thinp: agk's latest tweaks
snitm authored Jul 25, 2011
55 It maps I/O linearly to the data volume and updates the metadata via
56 two mechanisms:
8383bec @jthornber add multisnap.txt
authored May 24, 2011
57
6913333 @jthornber [multisnap, thin] Rebrand multisnap to thin provisioning with a snaps…
authored Jun 30, 2011
58 - Function calls from the thin targets
8383bec @jthornber add multisnap.txt
authored May 24, 2011
59
497cad3 @snitm dm thinp: agk's latest tweaks
snitm authored Jul 25, 2011
60 - Device-mapper 'messages' from userspace which control the creation of new
61 virtual devices amongst other things.
bb7b0fb @jthornber [documentation] update thin-provisioning.txt
authored Jul 8, 2011
62
63 Setting up a fresh pool device
64 ------------------------------
8383bec @jthornber add multisnap.txt
authored May 24, 2011
65
497cad3 @snitm dm thinp: agk's latest tweaks
snitm authored Jul 25, 2011
66 Setting up a pool device requires a valid metadata device, and a
8383bec @jthornber add multisnap.txt
authored May 24, 2011
67 data device. If you do not have an existing metadata device you can
68 make one by zeroing the first 4k to indicate empty metadata.
69
70 dd if=/dev/zero of=$metadata_dev bs=4096 count=1
71
497cad3 @snitm dm thinp: agk's latest tweaks
snitm authored Jul 25, 2011
72 FIXME How big must $metadata_dev be?
73
bb7b0fb @jthornber [documentation] update thin-provisioning.txt
authored Jul 8, 2011
74 Reloading a pool table
75 ----------------------
abd0ec5 @jthornber [multisnap] get pool table reloading working nicely.
authored Jun 29, 2011
76
bb7b0fb @jthornber [documentation] update thin-provisioning.txt
authored Jul 8, 2011
77 You may reload a pool's table, indeed this is how the pool is resized
497cad3 @snitm dm thinp: agk's latest tweaks
snitm authored Jul 25, 2011
78 if it runs out of space. (N.B. While specifying a different metadata
79 device when reloading is not forbidden at the moment, things will go
80 wrong if it does not route I/O to exactly the same on-disk location as
81 previously.)
8383bec @jthornber add multisnap.txt
authored May 24, 2011
82
bb7b0fb @jthornber [documentation] update thin-provisioning.txt
authored Jul 8, 2011
83 Using an existing pool device
84 -----------------------------
8383bec @jthornber add multisnap.txt
authored May 24, 2011
85
497cad3 @snitm dm thinp: agk's latest tweaks
snitm authored Jul 25, 2011
86 dmsetup create pool \
87 --table "0 20971520 thin-pool $metadata_dev $data_dev \
88 $data_block_size $low_water_mark"
8383bec @jthornber add multisnap.txt
authored May 24, 2011
89
497cad3 @snitm dm thinp: agk's latest tweaks
snitm authored Jul 25, 2011
90 $data_block_size gives the smallest unit of disk space that can be
72c4041 @jthornber [thin] switch to using blocks, rather than sectors
authored Oct 14, 2011
91 allocated at a time expressed in units of 512-byte sectors. People
92 primarily interested in thin provisioning may want to use a value such
93 as 1024. People doing lots of snapshotting may want a smaller value
94 such as 128. $data_block_size must be the same for the lifetime of the
497cad3 @snitm dm thinp: agk's latest tweaks
snitm authored Jul 25, 2011
95 metadata device.
8383bec @jthornber add multisnap.txt
authored May 24, 2011
96
72c4041 @jthornber [thin] switch to using blocks, rather than sectors
authored Oct 14, 2011
97 $low_water_mark is expressed in blocks of size $data_block_size. If
98 free space on the data device drops below this level then a dm event
99 will be triggered which a userspace daemon should catch allowing it to
497cad3 @snitm dm thinp: agk's latest tweaks
snitm authored Jul 25, 2011
100 extend the pool device. Only one such event will be sent.
8383bec @jthornber add multisnap.txt
authored May 24, 2011
101
497cad3 @snitm dm thinp: agk's latest tweaks
snitm authored Jul 25, 2011
102 FIXME - Do we get a second event after a table reload when you're
103 already over the threshold?
8383bec @jthornber add multisnap.txt
authored May 24, 2011
104
bb7b0fb @jthornber [documentation] update thin-provisioning.txt
authored Jul 8, 2011
105 Thin provisioning
106 -----------------
8383bec @jthornber add multisnap.txt
authored May 24, 2011
107
497cad3 @snitm dm thinp: agk's latest tweaks
snitm authored Jul 25, 2011
108 i) Creating a new thinly-provisioned volume.
8383bec @jthornber add multisnap.txt
authored May 24, 2011
109
497cad3 @snitm dm thinp: agk's latest tweaks
snitm authored Jul 25, 2011
110 To create a new thinly- provisioned volume you must send a message to an
111 active pool device, /dev/mapper/pool in this example.
8383bec @jthornber add multisnap.txt
authored May 24, 2011
112
9bc3620 @kergon dm thin: cleanup pool_message and change some message names
kergon authored Jul 20, 2011
113 dmsetup message /dev/mapper/pool 0 "create_thin 0"
8383bec @jthornber add multisnap.txt
authored May 24, 2011
114
497cad3 @snitm dm thinp: agk's latest tweaks
snitm authored Jul 25, 2011
115 Here '0' is an identifier for the volume, a 24-bit number. It's up
116 to the caller to allocate and manage these identifiers. If the
117 identifier is already in use, the message will fail.
118 FIXME With what error?
8383bec @jthornber add multisnap.txt
authored May 24, 2011
119
497cad3 @snitm dm thinp: agk's latest tweaks
snitm authored Jul 25, 2011
120 ii) Using a thinly-provisioned volume.
8383bec @jthornber add multisnap.txt
authored May 24, 2011
121
497cad3 @snitm dm thinp: agk's latest tweaks
snitm authored Jul 25, 2011
122 Thinly-provisioned volumes are activated using the 'thin' target:
8383bec @jthornber add multisnap.txt
authored May 24, 2011
123
6913333 @jthornber [multisnap, thin] Rebrand multisnap to thin provisioning with a snaps…
authored Jun 30, 2011
124 dmsetup create thin --table "0 2097152 thin /dev/mapper/pool 0"
8383bec @jthornber add multisnap.txt
authored May 24, 2011
125
497cad3 @snitm dm thinp: agk's latest tweaks
snitm authored Jul 25, 2011
126 The last parameter is the identifier for the thinp device.
8383bec @jthornber add multisnap.txt
authored May 24, 2011
127
bb7b0fb @jthornber [documentation] update thin-provisioning.txt
authored Jul 8, 2011
128 Internal snapshots
129 ------------------
8383bec @jthornber add multisnap.txt
authored May 24, 2011
130
497cad3 @snitm dm thinp: agk's latest tweaks
snitm authored Jul 25, 2011
131 i) Creating an internal snapshot.
8383bec @jthornber add multisnap.txt
authored May 24, 2011
132
bb7b0fb @jthornber [documentation] update thin-provisioning.txt
authored Jul 8, 2011
133 Snapshots are created with another message to the pool.
134
497cad3 @snitm dm thinp: agk's latest tweaks
snitm authored Jul 25, 2011
135 If the origin device that you wish to snapshot is active, you must
136 suspend it before creating the snapshot.
137 FIXME What happens if you don't?
8383bec @jthornber add multisnap.txt
authored May 24, 2011
138
139 dmsetup suspend /dev/mapper/thin
9bc3620 @kergon dm thin: cleanup pool_message and change some message names
kergon authored Jul 20, 2011
140 dmsetup message /dev/mapper/pool 0 "create_snap 1 0"
8383bec @jthornber add multisnap.txt
authored May 24, 2011
141 dmsetup resume /dev/mapper/thin
142
497cad3 @snitm dm thinp: agk's latest tweaks
snitm authored Jul 25, 2011
143 Here '1' is the identifier for the volume, a 24-bit number. '0' is the
bb7b0fb @jthornber [documentation] update thin-provisioning.txt
authored Jul 8, 2011
144 identifier for the origin device.
ec4dc24 @snitm multisnap: small fixes for Documentation
snitm authored May 24, 2011
145
497cad3 @snitm dm thinp: agk's latest tweaks
snitm authored Jul 25, 2011
146 ii) Using an internal snapshot.
8383bec @jthornber add multisnap.txt
authored May 24, 2011
147
bb7b0fb @jthornber [documentation] update thin-provisioning.txt
authored Jul 8, 2011
148 Once created, the user doesn't have to worry about any connection
149 between the origin and the snapshot. Indeed the snapshot is no
497cad3 @snitm dm thinp: agk's latest tweaks
snitm authored Jul 25, 2011
150 different from any other thinly-provisioned device and can be
bb7b0fb @jthornber [documentation] update thin-provisioning.txt
authored Jul 8, 2011
151 snapshotted itself via the same method. It's perfectly legal to
152 have only one of them active, and there's no ordering requirement on
497cad3 @snitm dm thinp: agk's latest tweaks
snitm authored Jul 25, 2011
153 activating or removing them both. (This differs from conventional
154 device-mapper snapshots.)
8383bec @jthornber add multisnap.txt
authored May 24, 2011
155
497cad3 @snitm dm thinp: agk's latest tweaks
snitm authored Jul 25, 2011
156 Activate it exactly the same way as any other thinly-provisioned volume:
8383bec @jthornber add multisnap.txt
authored May 24, 2011
157
6913333 @jthornber [multisnap, thin] Rebrand multisnap to thin provisioning with a snaps…
authored Jun 30, 2011
158 dmsetup create snap --table "0 2097152 thin /dev/mapper/pool 1"
8383bec @jthornber add multisnap.txt
authored May 24, 2011
159
497cad3 @snitm dm thinp: agk's latest tweaks
snitm authored Jul 25, 2011
160 Deactivation
161 ------------
8383bec @jthornber add multisnap.txt
authored May 24, 2011
162
497cad3 @snitm dm thinp: agk's latest tweaks
snitm authored Jul 25, 2011
163 All devices using a pool must be deactivated before the pool itself
164 can be.
8383bec @jthornber add multisnap.txt
authored May 24, 2011
165
166 dmsetup remove thin
167 dmsetup remove snap
168 dmsetup remove pool
169
bb7b0fb @jthornber [documentation] update thin-provisioning.txt
authored Jul 8, 2011
170 Reference
171 =========
172
173 'thin-pool' target
174 ------------------
8383bec @jthornber add multisnap.txt
authored May 24, 2011
175
bb7b0fb @jthornber [documentation] update thin-provisioning.txt
authored Jul 8, 2011
176 i) Constructor
8383bec @jthornber add multisnap.txt
authored May 24, 2011
177
497cad3 @snitm dm thinp: agk's latest tweaks
snitm authored Jul 25, 2011
178 thin-pool <metadata dev> <data dev> <data block size (sectors)> \
72c4041 @jthornber [thin] switch to using blocks, rather than sectors
authored Oct 14, 2011
179 <low water mark (blocks)> [<number of feature args> [<arg>]*]
72a3e4d @snitm dm multisnap: add variable pool feature arg processing
snitm authored Jun 30, 2011
180
497cad3 @snitm dm thinp: agk's latest tweaks
snitm authored Jul 25, 2011
181 Optional feature arguments:
182 - 'skip_block_zeroing': skips the zeroing of newly-provisioned blocks.
8383bec @jthornber add multisnap.txt
authored May 24, 2011
183
497cad3 @snitm dm thinp: agk's latest tweaks
snitm authored Jul 25, 2011
184 Data block size must be between 64KB (128 sectors) and 1GB
185 (2097152 sectors) inclusive.
57cb522 @snitm dm thin: impose and document constraints on data dev's block_size
snitm authored Jul 21, 2011
186
bb7b0fb @jthornber [documentation] update thin-provisioning.txt
authored Jul 8, 2011
187 ii) Status
94eb34b @jthornber add some info about the multisnap status lines
authored Jun 13, 2011
188
72c4041 @jthornber [thin] switch to using blocks, rather than sectors
authored Oct 14, 2011
189 <transaction id> <used metadata blocks>/<total metadata blocks>
190 <used data blocks>/<total data blocks> <held metadata root>
7672596 @jthornber [dm-thin] change the status line again
authored Oct 14, 2011
191
94eb34b @jthornber add some info about the multisnap status lines
authored Jun 13, 2011
192
497cad3 @snitm dm thinp: agk's latest tweaks
snitm authored Jul 25, 2011
193 transaction id:
194 A 64-bit number used by userspace to help synchronise with metadata
195 from volume managers.
94eb34b @jthornber add some info about the multisnap status lines
authored Jun 13, 2011
196
72c4041 @jthornber [thin] switch to using blocks, rather than sectors
authored Oct 14, 2011
197 used data blocks / total data blocks
198 If the number of free blocks drops below the pool's low water mark a
7672596 @jthornber [dm-thin] change the status line again
authored Oct 14, 2011
199 dm event will be sent to userspace. This event is edge-triggered and
200 it will occur at most once after each resume so volume manager writers
74d187b @snitm agk's latest changes
snitm authored Jul 29, 2011
201 should register for the event and then check the target's status.
a72cd8f @snitm dm thin: documentation updates
snitm authored Jul 8, 2011
202
497cad3 @snitm dm thinp: agk's latest tweaks
snitm authored Jul 25, 2011
203 held metadata root:
204 The location, in sectors, of the metadata root that has been
205 'held' for userspace read access. '-' indicates there is no
206 held root. This feature is not yet implemented so '-' is
207 always returned.
94eb34b @jthornber add some info about the multisnap status lines
authored Jun 13, 2011
208
bb7b0fb @jthornber [documentation] update thin-provisioning.txt
authored Jul 8, 2011
209 iii) Messages
8383bec @jthornber add multisnap.txt
authored May 24, 2011
210
497cad3 @snitm dm thinp: agk's latest tweaks
snitm authored Jul 25, 2011
211 create_thin <dev id>
bb7b0fb @jthornber [documentation] update thin-provisioning.txt
authored Jul 8, 2011
212
497cad3 @snitm dm thinp: agk's latest tweaks
snitm authored Jul 25, 2011
213 Create a new thinly-provisioned device.
214 <dev id> is an arbitrary unique 24-bit identifier chosen by
215 the caller.
bb7b0fb @jthornber [documentation] update thin-provisioning.txt
authored Jul 8, 2011
216
497cad3 @snitm dm thinp: agk's latest tweaks
snitm authored Jul 25, 2011
217 create_snap <dev id> <origin id>
bb7b0fb @jthornber [documentation] update thin-provisioning.txt
authored Jul 8, 2011
218
497cad3 @snitm dm thinp: agk's latest tweaks
snitm authored Jul 25, 2011
219 Create a new snapshot of another thinly-provisioned device.
220 <dev id> is an arbitrary unique 24-bit identifier chosen by
221 the caller.
222 <origin id> is the identifier of the thinly-provisioned device
223 of which the new device will be a snapshot.
bb7b0fb @jthornber [documentation] update thin-provisioning.txt
authored Jul 8, 2011
224
497cad3 @snitm dm thinp: agk's latest tweaks
snitm authored Jul 25, 2011
225 delete <dev id>
bb7b0fb @jthornber [documentation] update thin-provisioning.txt
authored Jul 8, 2011
226
497cad3 @snitm dm thinp: agk's latest tweaks
snitm authored Jul 25, 2011
227 Deletes a thin device. Irreversible.
bb7b0fb @jthornber [documentation] update thin-provisioning.txt
authored Jul 8, 2011
228
497cad3 @snitm dm thinp: agk's latest tweaks
snitm authored Jul 25, 2011
229 set_transaction_id <current id> <new id>
8383bec @jthornber add multisnap.txt
authored May 24, 2011
230
497cad3 @snitm dm thinp: agk's latest tweaks
snitm authored Jul 25, 2011
231 Userland volume managers, such as LVM, need a way to
232 synchronise their external metadata with the internal metadata of the
233 pool target. The thin-pool target offers to store an
234 arbitrary 64-bit transaction id and return it on the target's
235 status line. To avoid races you must provide what you think
236 the current transaction id is when you change it with this
237 compare-and-swap message.
94eb34b @jthornber add some info about the multisnap status lines
authored Jun 13, 2011
238
bb7b0fb @jthornber [documentation] update thin-provisioning.txt
authored Jul 8, 2011
239 'thin' target
240 -------------
8383bec @jthornber add multisnap.txt
authored May 24, 2011
241
497cad3 @snitm dm thinp: agk's latest tweaks
snitm authored Jul 25, 2011
242 i) Constructor
243
244 thin <pool dev> <dev id>
245
246 pool dev:
247 the thin-pool device, e.g. /dev/mapper/my_pool or 253:0
8383bec @jthornber add multisnap.txt
authored May 24, 2011
248
497cad3 @snitm dm thinp: agk's latest tweaks
snitm authored Jul 25, 2011
249 dev id:
250 the internal device identifier of the device to be
251 activated.
94eb34b @jthornber add some info about the multisnap status lines
authored Jun 13, 2011
252
497cad3 @snitm dm thinp: agk's latest tweaks
snitm authored Jul 25, 2011
253 The pool doesn't store any size against the thin devices. If you
254 load a thin target that is smaller than you've been using previously,
255 then you'll have no access to blocks mapped beyond the end. If you
256 load a target that is bigger than before, then extra blocks will be
257 provisioned as and when needed.
bb7b0fb @jthornber [documentation] update thin-provisioning.txt
authored Jul 8, 2011
258
497cad3 @snitm dm thinp: agk's latest tweaks
snitm authored Jul 25, 2011
259 If you wish to reduce the size of your thin device and potentially
260 regain some space then send the 'trim' message to the pool.
0b37722 @jthornber - new-thin message no longer takes a device size.
authored Jun 20, 2011
261
497cad3 @snitm dm thinp: agk's latest tweaks
snitm authored Jul 25, 2011
262 ii) Status
94eb34b @jthornber add some info about the multisnap status lines
authored Jun 13, 2011
263
497cad3 @snitm dm thinp: agk's latest tweaks
snitm authored Jul 25, 2011
264 <nr mapped sectors> <highest mapped sector>
Something went wrong with that request. Please try again.