-
Notifications
You must be signed in to change notification settings - Fork 0
taintedkernel/metasync
Folders and files
Name | Name | Last commit message | Last commit date | |
---|---|---|---|---|
Repository files navigation
************ * MetaSync * ************ Metasync is a file-metadata synchronization tool. It allows you to automate propagation of filename and/or directory changes to different remote locations. It's particularly helpful in an rsync (or-similar)-based workflow in combination with relatively large amounts of data. The first use-case was to propagate updates via Lightroom from my photo collection to several rsync-based backups. Any sort of semi-frequent file path updates will benefit from avoiding extra data transfer when syncing to an external location. In terms of implementation, metasync keeps an internal database of files and attemps to determine changes over time through periodic invocation. If too many changes happen simultaneously (file contents updated, rename and move) it may be difficult for metasync to directly detect the correct solution, in which case it will skip, prompt, or abort - especially if lots of other similar changes were occuring in the same "snapshot". Functionality/modes-of-operation: 1. Periodic sync on local source-of-truth filesystem data This will detect and interally update any changes to files on disk. No changes are made to on-disk bytes, except for the metasync database. 2. Reconcilation with external backup / mirror sync When pointed to an external mirror, it will determine what changes have not been propagated over and can generate a script which will re-create these, or optionally perform the operation itself (if direct write access is possible) 3. Bitrot detection Performs a forced read-and-recompute hashes for all files. If any data is detected as changed (without corresponding mtime update), we can prompt user for action (maybe recompute hash a second time to verify not a temporary read error). If we have data stored within DB on external mirrors, we can prompt user or attempt to recover data.
About
A metadata synchronization tool
Resources
Stars
Watchers
Forks
Releases
No releases published
Packages 0
No packages published