Join GitHub today
How Greyhole Works
- The user defines it's storage pool by listing all the paths of the different disks he wants to use.
- When files are added on Greyhole shares' Landing Zones, the Greyhole daemon moves those files into 1+ of the paths defined as the storage pool. It then replaces the original file (on the share) by a symbolic link pointing to one of the copy created in the pool.
- Samba is configured to use opaque symbolic links, i.e. what are in fact symbolic links in the shares appear as normal files to clients.
- Samba logs all writes, renames & deletes happening in Greyhole shares, using the Greyhole VFS module. /var/spool/greyhole is where the logged operations are kept: the VFS module creates a new file in that folder for each logged operation.
- The Greyhole daemon consumes those logs, and inserts tasks in a custom database (MySQL).
- The Greyhole daemon acts on each tasks in sequential order:
- For writes, it checks the (user-specified) number of copies to keep, and will create as many copies of the file on the different disk included in the storage pool. Once it created the first copy, it will replace the original file by a symbolic link pointing to the new copy created.
- For renames & deletes, it will replicate the operation on all directories part of the storage pool.
What's better than a good example to understand a system!
- /mnt/hdd0: 1TB empty hard-drive
- /mnt/hdd1: 2TB empty hard-drive
[Backups] path = /shares/Backups dfree command = /usr/bin/greyhole-dfree vfs objects = greyhole [RecordedTV] path = /shares/RecordedTV dfree command = /usr/bin/greyhole-dfree vfs objects = greyhole
# Storage pool directories storage_pool_directory = /mnt/hdd0/gh, min_free: 50gb storage_pool_directory = /mnt/hdd1/gh, min_free: 10gb # Shares num_copies[Backups] = 2 num_copies[RecordedTV] = 1
When a remote computer (client) create a file named file1 on the Backups share, for example:
mount_smb //server/Backups /mnt/Backups echo 1 > /mnt/Backups/file1
Then the server running Greyhole will do this:
- (Samba) writes the file into /shares/Backups/file1 (as defined in smb.conf)
- (Samba VFS module) writes a log file in /var/spool/greyhole/; the file name will be a serie of numbers
- (Greyhole daemon) reads the /var/spool/greyhole/ directories, and import each file it finds into it's MySQL database (tasks table); it will then remove the files it has processed from /var/spool/greyhole/
- (Greyhole daemon) read the tasks MySQL table, and executes each task in order. For the above Backups/file1 example, that would mean:
cp /shares/Backups/file1 /mnt/hdd1/gh/Backups/file1 rm /shares/Backups/file1 ln -s /mnt/hdd1/gh/Backups/file1 /shares/Backups/file1 cp /mnt/hdd1/gh/Backups/file1 /mnt/hdd0/gh/Backups/file1
- (Greyhole daemon) writes metadata in /mnt/hdd[0-1]/gh/.gh_metastore/Backups/file1
Here's more examples of how the Greyhole daemon would execute rename or delete tasks:
mv /mnt/Backups/file1 /mnt/Backups/file2
mv /mnt/hdd1/gh/Backups/file1 /mnt/hdd1/gh/Backups/file2 mv /mnt/hdd0/gh/Backups/file1 /mnt/hdd0/gh/Backups/file2 # Update metadata in /mnt/hdd[0-1]/gh/.gh_metastore/Backups/file2
rm /mnt/hdd1/gh/Backups/file2 rm /mnt/hdd0/gh/Backups/file2 rm /mnt/hdd[0-1]/gh/.gh_metastore/Backups/file2