forked from amadvance/snapraid
-
Notifications
You must be signed in to change notification settings - Fork 0
/
snapraid.d
1174 lines (892 loc) · 41 KB
/
snapraid.d
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
Name{number}
snapraid - SnapRAID Backup For Disk Arrays
Synopsis
:snapraid [-c, --conf CONFIG]
: [-f, --filter PATTERN] [-d, --filter-disk NAME]
: [-m, --filter-missing] [-e, --filter-error]
: [-a, --audit-only] [-h, --pre-hash] [-i, --import DIR]
: [-p, --percentage PERC] [-o, --older-than DAYS]
: [-l, --log FILE]
: [-Z, --force-zero] [-E, --force-empty]
: [-U, --force-uuid] [-D, --force-device]
: [-N, --force-nocopy] [-F, --force-full]
: [-S, --start BLKSTART] [-B, --count BLKCOUNT]
: [-L, --error-limit NUMBER]
: [-v, --verbose] [-q, --quiet]
: status|smart|up|down|diff|sync|scrub|fix|check|list|dup|up|down|pool|devices|rehash
:snapraid [-V, --version] [-H, --help] [-C, --gen-conf CONTENT]
Description
SnapRAID is a backup program for disk arrays. It stores parity
information of your data and it recovers from up to six disk
failures.
SnapRAID is mainly targeted for a home media center, with a lot of
big files that rarely change.
Beside the ability to recover from disk failures, other
features of SnapRAID are:
* All your data is hashed to ensure data integrity and to avoid
silent corruption.
* If the failed disks are too many to allow a recovery,
you lose the data only on the failed disks.
All the data in the other disks is safe.
* If you accidentally delete some files in a disk, you can
recover them.
* You can start with already filled disks.
* The disks can have different sizes.
* You can add disks at any time.
* It doesn't lock-in your data. You can stop using SnapRAID at any
time without the need to reformat or move data.
The official site of SnapRAID is:
:http://snapraid.sourceforge.net
Limitations
SnapRAID is in between a RAID and a Backup program trying to get the best
benefits of them. Although it also has some limitations that you should
consider before using it.
The main one is that if a disk fails, and you haven't recently synced,
you may not able to do a complete recover.
More specifically, you may be unable to recover up to the size of the
amount of the changed or deleted files from the last sync operation.
This happens even if the files changed or deleted are not in the
failed disk.
New added files don't prevent the recovering of the already existing
files. You may only lose the just added files, if they are on the failed
disk.
This is the reason because SnapRAID is better suited for data that
rarely change.
Other limitations are:
* You have different file-systems for each disk.
Using a RAID you have only a big file-system.
* It doesn't stripe data.
With RAID you get a speed boost with striping.
* It doesn't support real-time recovery.
With RAID you do not have to stop working when a disk fails.
* It's able to recover damages only from a limited number of disks.
With a Backup you are able to recover from a complete
failure of the whole disk array.
* Only file, timestamps, symlinks and hardlinks are saved.
Permissions, ownership and extended attributes are not saved.
Getting Started
To use SnapRAID you need to first select one disk of your disk array
to dedicate at the "parity" information. With one disk for parity you
will be able to recover from a single disk failure, like RAID5.
If you want to be able to recover from more disk failures, like RAID6,
you must reserve additional disks for parity. Any additional parity
disk allow to recover from one more disk failure.
As parity disks, you have to pick the biggest disks in the array,
as the parity information may grow in size as the biggest data
disk in the array.
These disks will be dedicated to store the "parity" files.
You should not store your data in them.
The list of files is saved in the "content" files, usually
stored in the data, parity or boot disks.
These files contain the details of your backup, with all the
checksums to verify its integrity.
The "content" file is stored in multiple copies, and each one must
be in a different disk, to ensure that in even in case of multiple
disk failures at least one copy is available.
For example, suppose that you are interested only at one parity level
of protection, and that your disks are present in:
:/mnt/diskp <- selected disk for parity
:/mnt/disk1 <- first disk to backup
:/mnt/disk2 <- second disk to backup
:/mnt/disk3 <- third disk to backup
you have to create the configuration file /etc/snapraid.conf with
the following options:
:parity /mnt/diskp/snapraid.parity
:content /var/snapraid/snapraid.content
:content /mnt/disk1/snapraid.content
:content /mnt/disk2/snapraid.content
:disk d1 /mnt/disk1/
:disk d2 /mnt/disk2/
:disk d3 /mnt/disk3/
If you are in Windows, you should use drive letters and backslashes
instead of slashes, and if you like, also file extensions.
:parity E:\snapraid.parity
:content C:\snapraid\snapraid.content
:content F:\array\snapraid.content
:content G:\array\snapraid.content
:disk d1 F:\array\
:disk d2 G:\array\
:disk d3 H:\array\
At this point you are ready to start the "sync" command to build the
parity information.
:snapraid sync
This process may take some hours the first time, depending on the size
of the data already present in the disks. If the disks are empty
the process is immediate.
You can stop it at any time pressing Ctrl+C, and at the next run it
will start where interrupted.
When this command completes, your data is SAFE.
Now you can start using your array as you like, and periodically
update the parity information running the "sync" command.
Scrubbing
To periodically check the data and parity for errors, you can
run the "scrub" command.
:snapraid scrub
This command verifies the data in your array comparing it with
the hash computed in the "sync" command.
Every run of the command checks about 12% of the array, but not data newer
than 10 days.
You can use the -p, --percentage option to specify a different amount,
and the -o, --older-than option to specify a different age in days.
For example, to check 5% of the array older than 20 days use:
:snapraid -p 5 -o 20 scrub
If during the process, silent or input/output errors are found,
the corresponding blocks are marked as bad in the "content" file,
and listed in the "status" command.
:snapraid status
To fix them, you can use the "fix" command filtering for bad blocks with
the -e, --filter-error options:
:snapraid -e fix
At the next "scrub" the errors will disappear from the "status" report
if really fixed. To make it fast, you can use -p 0 to scrub only blocks
marked as bad.
:snapraid -p 0 scrub
Take care that running "scrub" on a not synced array may result in
errors caused by removed or modified files. These errors are reported
in the "scrub" result, but related blocks are not marked as bad.
Pooling
To have all the files in your array shown in the same directory tree,
you can enable "pooling", that consists in creating a virtual view of all
the files in your array using symbolic links.
You can configure the "pooling" directory in the configuration file with:
:pool /pool
or, if you are in Windows, with:
:pool C:\pool
and then run the "pool" command to create or update the virtual view.
:snapraid pool
If you are using a Unix platform and you want to share such directory
in the network to either Windows or Unix machines, you should add
to your /etc/samba/smb.conf the following options:
:# In the global section of smb.conf
:unix extensions = no
:# In the share section of smb.conf
:[pool]
:comment = Pool
:path = /pool
:read only = yes
:guest ok = yes
:wide links = yes
:follow symlinks = yes
In Windows the same sharing operation is not so straightforward,
because Windows shares the symbolic links as they are, and that
requires the network clients to resolve them remotely.
To make it working, besides sharing in the network the pool directory,
you must also share all the disks independently, using as share points
the disk names as defined in the config file. You must also specify in
the "share" option of the configure file, the Windows UNC path that remote
clients needs to use to access such shared disks.
For example, operating from a server named "darkstar", you can use
the options:
:disk d1 F:\array\
:disk d2 G:\array\
:disk d3 H:\array\
:pool C:\pool
:share \\darkstar
and share the following dirs in the network:
:\\darkstar\pool -> C:\pool
:\\darkstar\d1 -> F:\array
:\\darkstar\d2 -> G:\array
:\\darkstar\d3 -> H:\array
to allow remote clients to access all the files at \\darkstar\\pool.
You may also need to configure remote clients, to enable the access at
remote symlinks with the command:
:fsutil behavior set SymlinkEvaluation L2L:1 R2R:1 L2R:1 R2L:1
Undeleting
SnapRAID is more like a backup program than a RAID system, and it
can be used to restore or undelete files to their previous state using
the -f, --filter option :
:snapraid fix -f FILE
or for a directory:
:snapraid fix -f DIR/
You can also use it to recover only accidentally deleted files inside
a directory using the -m, --filter-missing option, that restores
only missing files, leaving untouched all the others.
:snapraid fix -m -f DIR/
Or to recover all the deleted files in all the drives with:
:snapraid fix -m
Recovering
The worst happened, and you lost a disk!
DO NOT PANIC! You will be able to recover it!
The first thing you have to do is to avoid further changes at you disk array.
Disable any remote connection to it, any scheduled process, including any
scheduled SnapRAID nightly sync or scrub.
Then proceed with the following steps.
STEP 1 -> Reconfigure
You need some space to recover, even better if you already have an additional
disk, but in case, also an external USB or remote disk is enough.
Change the SnapRAID configuration file to make the "disk" option
of the failed disk to point to the place where you have enough empty
space to recover the files.
For example, if you have that disk "d1" failed, you can change from:
:disk d1 /mnt/disk1/
to:
:disk d1 /mnt/new_spare_disk/
STEP 2 -> Fix
Run the fix command, storing the log in an external file with:
:snapraid -d NAME -l fix.log fix
Where NAME is the name of the disk, like "d1" as in our previous example.
This command will take a long time.
Take care that you need also few gigabytes free to store the fix.log file.
Run it from a disk with some free space.
Now you have recovered all the recoverable. If some file is partially or totally
unrecoverable, it will be renamed adding the ".unrecoverable" extension.
You can get a detailed list of all the unrecoverable blocks in the fix.log file
checking all the lines starting with "unrecoverable:"
If you are not satisfied of the recovering, you can retry it as many
time you wish.
For example, if you have removed files from the array after the last
"sync", this may result in some other files not recovered.
In this case, you can retry the "fix" using the -i, --import option,
specifing where these files are now, to include them again in the
recovering process.
If you are satisfied of the recovering, you can now proceed further,
but take care that after syncing you cannot retry the "fix" command
anymore!
STEP 3 -> Check
As paranoid check, you can now run a "check" command to ensure that
everything is OK on the recovered disk.
:snapraid -d NAME -a check
Where NAME is the name of the disk, like "d1" as in our previous example.
The options -d and -a tell SnapRAID to check only the specified disk,
and ignore all the parity data.
This command will take a long time, but if you are not paranoid,
you can skip it.
STEP 4 -> Sync
Run the "sync" command to resynchronize the array with the new disk.
:snapraid sync
If everything is recovered, this command is immediate.
Commands
SnapRAID provides a few simple commands that allow to:
* Prints the status of the array -> "status"
* Controls the disks -> "smart", "up", "down"
* Makes a backup/snapshot -> "sync"
* Periodically checks data -> "scrub"
* Restore the last backup/snapshot -> "fix".
Take care that the commands have to be written in lower case.
status
Prints a summary of the state of the disk array.
It includes information about the parity fragmentation, how old
are the blocks without checking, and all the recorded silent
errors encountered while scrubbing.
Note that the information presented refers at the latest time you
run "sync". Later modifications are not taken into account.
Nothing is modified.
smart
Prints a SMART report of all the disks of the array.
It includes an estimation of the probability of failure in the next
year of each disk allowing to plan maintenance replacements of the
disks that show suspicious attributes.
If a SMART attribute reports that a disk is failing, "FAIL" or "PREFAIL"
is printed for that disk, and SnapRAID returns with an error.
In this case an immediate replacement of the disk is highly recommended.
If the -v, --verbose option is specified a deeper statistical analysis
is provided. This analysis can help you to decide if you need more
or less parity.
This command uses the "smartctl" tool, and it's equivalent to run
"smartctl -a" on all the devices.
If your devices are not autodetected correctly, you can configure
a custom command using the "smartctl" option in the configuration
file.
Nothing is modified.
up
Spins up all the disks of the array.
Nothing is modified.
down
Spins down all the disks of the array.
This command uses the "smartctl" tool, and it's equivalent to run
"smartctl -s standby,now" on all the devices.
Nothing is modified.
diff
Lists all the files modified from the last "sync" that need to have
their parity data recomputed.
This command doesn't check the file data, but only the file timestamp
size and inode.
If a "sync" is not required, the return error code is 0. Otherwise,
it's 1.
Nothing is modified.
sync
Updates the parity information. All the modified files
in the disk array are read, and the corresponding parity
data is updated.
You can stop this process at any time pressing Ctrl+C,
without losing the work already done.
At the next run the "sync" process will start where
interrupted.
If during the process, silent or input/output errors are found,
the corresponding blocks are marked as bad.
Files are identified by path and/or inode and checked by
size and timestamp.
If the file size or timestamp are different, the parity data
is recomputed for the whole file.
If the file is moved or renamed in the same disk, keeping the
same inode, the parity is no recomputed.
If the file is moved to another disk, the parity is recomputed,
but the previously compute hash information is kept.
The "content" and "parity" files are modified if necessary.
The files in the array are NOT modified.
scrub
Scrubs the array, checking for silent or input/output errors in data
and parity disks.
For each command invocation, the 12% of the array is checked, but
nothing that it's more recent than 10 days.
This means that scrubbing once a week, every bit of data is checked
at least one time every two months.
You can use the -p, --percentage option to specify a different amount,
and the -o, --older-than option to specify a different age in days.
You can have a full scrub with "-p 100 -o 0".
The oldest blocks are scrubbed first ensuring an optimal check.
For any silent or input/output error found the corresponding blocks
are marked as bad in the "content" file.
These bad blocks are listed in "status", and can be fixed with "fix -e".
After the fix, at the next scrub they will be rechecked, and if found
corrected, the bad mark will be removed.
It's recommended to run "scrub" on a synced array, to avoid to have
reported error caused by unsynced data. These errors are recognized
as not being silent errors, and the blocks are not marked as bad,
but such errors are reported in the output of the command.
Files are identified only by path, and not by inode.
The "content" file is modified to update the time of the last check
of each block, and to mark bad blocks.
The "parity" files are NOT modified.
The files in the array are NOT modified.
fix
Fix all the files and the parity data.
All the files and the parity data are compared with the snapshot
state saved in the last "sync".
If a difference is found, it's reverted to the stored snapshot.
Note that "fix" doesn't differentiate between errors and intentional
modifications. It inconditionally reverts the file state at the last "sync".
If no other option is specified the full array is processed.
Use the filter options to select a subset of files or disks to operate on.
To only fix the blocks marked bad during "sync" and "scrub",
use the -e, --filter-error option.
As difference from other filter options, with this one fixes are
applied only to files that are not modified from the the latest "sync".
All the files that cannot be fixed are renamed adding
the ".unrecoverable" extension.
Files are identified only by path, and not by inode.
The "content" file is NOT modified.
The "parity" files are modified if necessary.
The files in the array are modified if necessary.
check
Verify all the files and the parity data.
It works like "fix", but it only simulates a recovery and no change
is written in the array.
This command is mostly intended for manual verifications,
like after a recovery process or in other special conditions.
For periodic and scheduled checks uses "scrub".
If you use the -a, --audit-only option, only the file
data is checked, and the parity data is ignored for a
faster run.
Files are identified only by path, and not by inode.
Nothing is modified.
list
Lists all the files contained in the array at the time of the
last "sync".
Nothing is modified.
dup
Lists all the duplicate files. Two files are assumed equal if their
hashes are matching. The file data is not read, but only the
precomputed hashes are used.
Nothing is modified.
up
Spins up all the disks of the array.
Nothing is modified.
down
Spins down all the disks of the array.
Nothing is modified.
pool
Creates or updates in the "pooling" directory a virtual view of all
the files of your disk array.
The files are not really copied here, but just linked using
symbolic links.
When updating, all the present symbolic links and empty
subdirectories are deleted and replaced with the new
view of the array. Any other regular file is left in place.
Nothing is modified outside the pool directory.
devices
Prints the low level devices used by the array.
This command prints the devices associations in place in the array,
and it's mainly intended as a script interface.
The first two columns are the low level device id and path.
The next two columns are the high level device id and path.
The latest column if the disk name in the array.
In most cases you have one low level device for each disk in the
array, but in some more complex configurations, you may have multple
low level devices used by a single disk in the array.
Nothing is modified.
rehash
Schedules a rehash of the whole array.
This command changes the hash kind used, typically when upgrading
from a 32 bits system to a 64 bits one, to switch from
MurmurHash3 to the faster SpookyHash.
If you are already using the optimal hash, this command
does nothing and tells you that nothing has to be done.
The rehash isn't done immediately, but it takes place
progressively during "sync" and "scrub".
You can get the rehash state using "status".
During the rehash, SnapRAID maintains full functionality,
with the only exception of "dup" not able to detect duplicated
files using a different hash.
Options
SnapRAID provides the following options:
-c, --conf CONFIG
Selects the configuration file. If not specified it's assumed
the file "/etc/snapraid.conf" in Unix, and "snapraid.conf" in
the current directory in Windows.
-f, --filter PATTERN
Filters the files to process in "check" and "fix".
Only the files matching the entered pattern are processed.
This option can be used many times.
See the PATTERN section for more details in the
pattern specifications.
In Unix, ensure to quote globbing chars if used.
This option can be used only with "check" and "fix".
Note that it cannot be used with "sync" and "scrub", because they always
process the whole array.
-d, --filter-disk NAME
Filters the files to process in "check" and "fix".
Only the files present in the specified disk are processed.
You must specify a disk name as named in the configuration
file.
In "check", you can make it faster, specifying also -a, --audit-only
option, to avoid to access other disks to check parity data.
If you combine more --filter, --filter-disk and --filter-missing options,
only files matching all the set of filters are selected.
This option can be used many times.
This option can be used only with "check" and "fix".
Note that it cannot be used with "sync" and "scrub", because they always
process the whole array.
-m, --filter-missing
Filters the files to process in "check" and "fix".
Only the files missing/deleted from the array are processed.
When used with "fix", this is a kind of "undelete" command.
If you combine more --filter, --filter-disk and --filter-missing options,
only files matching all the set of filters are selected.
This option can be used only with "check" and "fix".
Note that it cannot be used with "sync" and "scrub", because they always
process the whole array.
-e, --filter-error
Filters the blocks to process in "check" and "fix".
It processes only the blocks marked with silent or input/output
errors during "sync" and "scrub", and listed in "status".
This option can be used only with "check" and "fix".
-p, --percentage PERC
Selects the part of the array to process in "scrub".
PERC is a numeric value from 0 to 100, default is 12.
When specifying 0, only the blocks marked as bad are scrubbed.
This option can be used only with "scrub".
-o, --older-than DAYS
Selects the older the part of the array to process in "scrub".
DAYS is the minimum age in days for a block to be scrubbed,
default is 10.
Blocks marked as bad are always scrubbed despite this option.
This option can be used only with "scrub".
-a, --audit-only
In "check" verifies the hash of the files without
doing any kind of check on the parity data.
If you are interested in checking only the file data this
option can speedup a lot the checking process.
This option can be used only with "check".
-h, --pre-hash
In "sync" runs a preliminary hashing phase of all
the new data to verify the data used in the parity computation.
Usually in "sync" no preliminary hashing is done, and the new
data is hashed just before the parity computation when it's read
for the first time,
Unfortunately, this process happens when the system is under
heavy load, with all disks spinning and a busy CPU.
This is an extreme condition for your machine, and if it has a
latent hardware problem, it's possible to encounter silent errors
what cannot be detected because the data is not yet hashed.
To avoid this risk, you can enable the "pre-hash" mode and have
all the data hashed two times to ensure its integrity.
This option can be used only with "sync".
-i, --import DIR
Imports from the specified directory any file that you deleted
from the array after the last "sync".
If you still have such files, they could be used by "check"
and "fix" to improve the recover process.
The files are read also in subdirectories and they are
identified regardless of their name.
This option can be used only with "check" and "fix".
-Z, --force-zero
Forces the insecure operation of syncing a file with zero
size that before was not.
If SnapRAID detects a such condition, it stops proceeding
unless you specify this option.
This allows to easily detect when after a system crash,
some accessed files were truncated.
This is a possible condition in Linux with the ext3/ext4
filesystems.
This option can be used only with "sync".
-E, --force-empty
Forces the insecure operation of syncing a disk with all
the original files missing.
If SnapRAID detects that all the files originally present
in the disk are missing or rewritten, it stops proceeding
unless you specify this option.
This allows to easily detect when a data file-system is not
mounted.
This option can be used only with "sync".
-U, --force-uuid
Forces the insecure operation of syncing, checking and fixing
with disks that have changed their UUID.
If SnapRAID detects that some disks have changed UUID,
it stops proceeding unless you specify this option.
This allows to detect when your disks are mounted in the
wrong mount points.
It's anyway allowed to have a single UUID change with
single parity, and more with multiple parity, because it's
the normal case of replacing disks after a recovery.
This option can be used only with "sync", "check" or
"fix".
-D, --force-device
Forces the insecure operation of fixing with disks on the same
physical device.
If SnapRAID detects that some disks have the same device ID,
it stops proceeding, because it's not a supported configuration.
But it could happen that you want to temporarily restore a lost
disk in the free space left in an already used disk. and this
option allows you to continue anyway.
This option can be used only with "fix".
-N, --force-nocopy
In "sync", "check and "fix", disables the copy detection heuristic.
Without this option SnapRAID assumes that files with same
attributes, like name, size and timestamp are copies with the
same data.
This allows to identify copied or moved files from one disk
to another, and to reuse the already computed hash information
to detect silent errors or to recover missing files.
This behavior, in some rare cases, may result in false positives,
or in a slow process due the many hash verifications, and this
option allows to resolve them.
This option can be used only with "sync", "check" and "fix".
-F, --force-full
In "sync" forces a full rebuild of the parity.
This option can be used when you reverted back to an old content
file, but using a more recent parity data.
Instead of recomputing the parity from scratch, this allows
to reuse the hashes present in the content file to validate data,
and to maintain data protection during the "sync" process using
the old content file and the parity data you have.
This option can be used only with "sync".
-l, --log FILE
Write a detailed log in the specified file.
If this option is not specified, the warnings and not fatal
errors are printed on the screen, likely resulting in too much
output in case of many errors.
If the path starts with '>>' the file is opened
in append mode. Occurrences of '%D' and '%T' in the name are
replaced with the date and time in the format YYYYMMDD and
HHMMSS. Note that in Windows batch files, you'll have to double
the '%' char, like result-%%D.log. And to use '>>' you'll have
to enclose the name in ", like ">>result.log".
To output the log to standard output or standard error,
you can use respectively ">&1" and ">&2".
-L, --error-limit
Sets a new error limit before stopping execution.
By default SnapRAID stops if it encouters more than 100
Input/Output errors, meaning that likely a disk is going to
die.
This options affects "sync" and "scrub", that are allowed
to continue after the first bunch of disk errors, to try
to complete at most their operations.
Instead, "check" and "fix" always stop at the first error.
-S, --start BLKSTART
Starts the processing from the specified
block number. It could be useful to retry to check
or fix some specific block, in case of a damaged disk.
It's present mainly for advanced manual recovering.
-B, --count BLKCOUNT
Processes only the specified number of blocks.
It's present mainly for advanced manual recovering.
-C, --gen-conf CONTENT_FILE
Generates a dummy configuration file from an existing
content file.
The configuration file is written in the standard output,
and it doesn't overwrite an existing one.
This configuration file also contains the information
needed to reconstruct the disk mount points, in case you
lose the entire system.
-v, --verbose
Prints more information on the screen.
-q, --quiet
Prints less information on the screen.
If specified one time, removes the progress bar, if two
times, the running operations, three times, the info
messages, four times the status messages.
Fatal errors are always printed.
-H, --help
Prints a short help screen.
-V, --version
Prints the program version.
Configuration
SnapRAID requires a configuration file to know where your disk array
is located, and where storing the parity information.
This configuration file is located in /etc/snapraid.conf in Unix or
in the execution directory in Windows.
It should contain the following options (case sensitive):
parity FILE
Defines the file to use to store the parity information.
The parity enables the protection from a single disk
failure, like RAID5.
It must be placed in a disk dedicated for this purpose with
as much free space as the biggest disk in the array.
Leaving the parity disk reserved for only this file ensures that
it doesn't get fragmented, improving the performance.
This option is mandatory and it can be used only one time.
[2,3,4,5,6]-parity FILE
Defines the files to use to store extra parity information.
For each parity file specified, one additional level of protection
is enabled:
* 2-parity enables RAID6 double parity.
* 3-parity enables triple parity
* 4-parity enables quad parity
* 5-parity enables penta (five) parity
* 6-parity enables hexa (six) parity
Each parity level requires also all the files of the previous levels.
Each file must be placed in a disk dedicated for this purpose with
as much free space as the biggest disk in the array.
Leaving the parity disks reserved for only these files ensures that
they doesn't get fragmented, improving the performance.
These options are optional and they can be used only one time.
z-parity FILE
Defines an alternate file and format to store the triple parity.
This option is an alternative at '3-parity' mainly intended for
low-end CPUs like ARM or AMD Phenom, Athlon and Opteron that don't
support the SSSE3 instructions set, and in such case it provides
a better performance.
This format is similar, but faster, at the one used by the ZFS RAIDZ3,
but it doesn't work beyond triple parity.
When using '3-parity' you will be warned if it's recommended to use
the 'z-parity' format for a performance improvement.
It's possible to convert from one format to another, adjusting
the configuration file with the wanted z-parity or 3-parity file,
and using 'fix' to recreate it.
content FILE
Defines the file to use to store the list and checksums of all the
files present in your disk array.
It can be placed in the disk used to store data, parity, or
any other disk available.
If you use a data disk, this file is automatically excluded
from the "sync" process.
This option is mandatory and it can be used more times to save
more copies of the same files.
You have to store at least one copy for each parity disk used
plus one. Using some more doesn't hurt.
disk NAME DIR
Defines the name and the mount point of the disks of the array.
NAME is used to identify the disk, and it must be unique.
DIR is the mount point of the disk in the filesystem.
You can change the mount point as you like, as long you
keep the NAME fixed.
You should use one option for each disk of the array.
nohidden
Excludes all the hidden files and directory.
In Unix hidden files are the ones starting with ".".
In Windows they are the ones with the hidden attribute.
exclude/include PATTERN
Defines the file or directory patterns to exclude and include
in the sync process.
All the patterns are processed in the specified order.
If the first pattern that matches is an "exclude" one, the file
is excluded. If it's an "include" one, the file is included.
If no pattern matches, the file is excluded if the last pattern
specified is an "include", or included if the last pattern
specified is an "exclude".
See the PATTERN section for more details in the pattern
specifications.
This option can be used many times.
blocksize SIZE_IN_KIBIBYTES
Defines the basic block size in kibi bytes of the parity.
One kibi bytes is 1024 bytes. The default blocksize is 256
and it should work for most cases.
A reason to use a different blocksize is if your system has less
than 4 GiB of memory. As a rule of thumb, with 4 GiB or more memory
use the default 256, with 2 GiB use 512, and with 1 GiB use 1024.
In more details SnapRAID requires about TS*28/BS bytes
of RAM memory to run in the 32 bits version, and TS*36/BS
in the 64 bits one. Where TS is the total size in bytes of
your disk array, and BS is the block size in bytes.
For example with 8 disk of 4 TB and a block size of 256 KiB
(1 KiB = 1024 bytes) you have:
:RAM = (8 * 4 * 10^12) * 28 / (256 * 2^10) = 3.2 GiB
Another reason to use a different blocksize is if you have a lot of
small files. In the order of many millions.
For each file, even of few bytes, a whole block of parity is allocated,
and with many files this may result in a lot of unused parity space.
And when you completely fill the parity disk, you are not
allowed to add more files in the data disks.
Anyway, the wasted parity doesn't sum between data disks. Wasted space
resulting from a high number of files in a data disk, limits only
the amount of data in such data disk and not in others.
As approximation, you can assume that half of the block size is
wasted for each file. For example, with 100000 files and a 256 KiB
block size, you are going to waste 13 GB of parity, that may result
in 13 GB less space available in the data disk.
You can get the amount of wasted space in each disk using "status".
This is the amount of space that you must leave free in the data
disks, or use for files not included in the array.
If this value is negative, it means that your are near to fill
the parity, and it represents the space you can still waste.
To avoid the problem, you can use a bigger partition for parity.
For example, if you have the parity partition bigger than 13 GB
than data disks, you have enough extra space to handle up to 100000
files in each data disk.
A trick to get a bigger parity partition in Linux, is to format it
with the command:
:mkfs.ext4 -m 0 -T largefile4 DEVICE
This results in about 1.5% of extra space. Meaning about 60 GB for
a 4 TB disk, that allows about 460000 files in each data disk without
any wasted space.
autosave SIZE_IN_GIGABYTES
Automatically save the state when syncing after the specified amount
of GB processed.
This option is useful to avoid to restart from scratch long "sync"
commands interrupted by a machine crash, or any other event that
may interrupt SnapRAID.
pool DIR
Defines the pooling directory where the virtual view of the disk
array is created using the "pool" command.
The directory must already exist.
share UNC_DIR
Defines the Windows UNC path required to access the disks remotely.
If this option is specified, the symbolic links created in the pool
directory use this UNC path to access the disks.
Without this option the symbolic links generated use only local paths,
not allowing to share the pool directory in the network.