Join GitHub today
Swiss army knife for directory comparison and synchronization http://github.com/sebastien/sink
Fetching latest commit…
Cannot retrieve the latest commit at this time.
|Type||Name||Latest commit message||Commit time|
|Failed to load latest commit information.|
== Sink == Swiss army knife for directory comparison and synchronization -- Author: Sébastien Pierre <email@example.com> -- Revision: 1.0.0 (29-Sep-2009) Sink is a tool that allows you to compare multiple directories, take snapshots of their state, and compare the states. It also provides commands to create logical links between files to do micro changes management. Use cases ========= Sink can be very handy for the following uses cases: *) Compare multiple versions of a source tree (replacing 'diff -r') *) Manually merge directories outside of a revision control system *) Synchronize files belonging to different directories *) Track changes in a specific directory *) Backup and deploy directory-wide changes *) Keep shared files up-to-date Comparing directories --------------------- Imagine you have 3 different versions of a specific source tree. In this example, we'll simply checkout three different revisions of the 'sink' development tree: > $ git clone git://github.com/sebastien/sink.git sink-r1 > $ git clone git://github.com/sebastien/sink.git sink-r2 > $ git clone git://github.com/sebastien/sink.git sink-r3 and we compare the three directories: > $ sink diff sink-r1 sink-r2 sink-r3 > No differences as the three directories are the same, we'll put them to different revisions: > $ cd sink-r1 ; git checkout -b older a17609eaabbf8feb3b480d46cb972df0599755fe ; cd .. > $ cd sink-r2 ; git checkout -b old 9e0e6a476f41f0de685d2c92e142b206b46810e4 ; cd .. and we can now compare the directories: > sink diff sink-r1 sink-r2 sink-r3 > 00 ! [+][+] .hgtags > 01 [=][-][-] DESIGN > 02 [=][<][<] Makefile > 03 [=][-][-] NOTES > 04 -!--!-[+] README > 05 [=][-][-] ROADMAP > 06 [=][>][<] TODO > 07 -!-[+][+] setup.py > 08 -!-[+]-!- Documentation/DESIGN.txt > 09 -!-[+]-!- Documentation/MANUAL.txt > 10 [=][=][-] Resources/epydoc.css > 11 -!-[+][+] Scripts/sink > 12 [=][-][-] Sources/sink/Sink.py > 13 [=][-][-] Sources/sink/Tracking.py > 14 -!-[+][+] Sources/sink/linking.py > 15 -!-[+][+] Sources/sink/main.py > 16 -!--!-[+] Sources/sink/snapshot.py > 17 -!-[+][+] Sources/sink/tracking.py sink has found differences and indicates them in the form of a table, where the legend can be found in the 'sink --help diff' command: > [=] no changes [+] file added [>] changed/newer > [-] file removed [<] changed/older > -!- file missing we can have a list of all the files that were added in 'sink-r2' and 'sink-r3' by showing only the added files: > $ sink diff +a sink-r1 sink-r2 sink-r3 > 00 -!-[+][+] .hgtags > 01 -!--!-[+] README > 02 -!-[+][+] setup.py > 03 -!-[+]-!- Documentation/DESIGN.txt > 04 -!-[+]-!- Documentation/MANUAL.txt > 05 -!-[+][+] Scripts/sink > 06 -!-[+][+] Sources/sink/linking.py > 07 -!-[+][+] Sources/sink/main.py > 08 -!--!-[+] Sources/sink/snapshot.py > 09 -!-[+][+] Sources/sink/tracking.py and the output is 'cut' friendly: > $ sink diff +a sink-r1 sink-r2 sink-r3 | cut -d' ' -f3- > .hgtags > README > setup.py > Documentation/DESIGN.txt > Documentation/MANUAL.txt > Scripts/sink > Sources/sink/linking.py > Sources/sink/main.py > Sources/sink/snapshot.py > Sources/sink/tracking.py you can also compare individual changes between all the versions. Let's see the changes made to the 'Sources/sink/tracking.py' file between 'sink-r2' and 'sink-r3': > $ sink diff sink-r2 sink-r3 > 0 -!-[+] README > 1 [=][<] TODO > 2 [=][-] Documentation/DESIGN.txt > 3 [=][-] Documentation/MANUAL.txt > 4 [=][-] Resources/epydoc.css > 5 [=][<] Sources/sink/linking.py > 6 [=][<] Sources/sink/main.py > 7 -!-[+] Sources/sink/snapshot.py > 8 [=][<] Sources/sink/tracking.py the number of the file in the table is 5, so we pass it to sink '-d' option: > $ sink diff -d5 sink-r2 sink-r3 > >> gvimdiff sink-r2/Sources/sink/linking.py sink-r3/Sources/sink/linking.py this starts up 'gvimdiff', showing the differences between the files. This is a great way to do a merge outside of a revision control system. Synchronizing directories ------------------------- So taking our previous example, let's imaging we'd like to synchronize 'sink-r1' so that it is exactly the same as 'sink-r3'. We start by showing the added or modified files: > $ sink +a +m sink-r1 sink-r3 > 00 -!-[+] .hgtags > 01 [=][<] Makefile > 02 -!-[+] README > 03 [=][<] TODO > 04 -!-[+] setup.py > 05 -!-[+] Scripts/sink > 06 -!-[+] Sources/sink/linking.py > 07 -!-[+] Sources/sink/main.py > 08 -!-[+] Sources/sink/snapshot.py > 09 -!-[+] Sources/sink/tracking.py we then pipe the result to 'cut' and 'xargs', first to create the directories that may not exist, and then to copy the files > $ sink +a +m sink-r1 sink-r3 | cut -d' ' -f3- | xargs dirname | sort | uniq| xargs mkdir -p > $ sink +a +m sink-r1 sink-r3 | cut -d' ' -f3- | xargs -I FILE cp sink-r3/FILE sink-r1/FILE and we make sure the directories are the same: > $ sink sink-r1 sink-r3 > 0 [=][-] DESIGN > 1 [=][-] NOTES > 2 [=][-] ROADMAP > 3 [=][-] Resources/epydoc.css > 4 [=][-] Sources/sink/Sink.py > 5 [=][-] Sources/sink/Tracking.py we see that we have files to remove from 'sink-r1', so let's do it: > $ sink +r sink-r1 sink-r3 | cut -d' ' -f3 | xargs -I FILE rm sink-r1/FILE > $ sink sink-r1 sink-r3 > No changes found. and we've done the synchronization right. This may look a little bit verbose, but it's Unix's design philosophy -- have simple tools that do thing, and then integrate with other tools to do higher-level operations. Tracking changes made to a directory ------------------------------------ Let's say you've just downloaded 'fltk', a cross-platform C++ widget library and you'd like to install it in /usr/local -- but you want to keep a receipt of the installed files. First, get fltk, and compile it > $ wget 'http://ftp.easysw.com/pub/fltk/1.1.9/fltk-1.1.9-source.tar.bz2' > $ tar fvxj fltk-1.1.9-source.tar.bz2 > $ cd fltk-1.1.9 ; ./configure --prefix=/usr/local ; make now we take a snapshot of '/usr/local' > $ sink snap /usr/local > usr-local.snap you can now install fltk and see the changes: > $ sudo make install > $ sink usr-local.snap /usr/local > 000 -!-[+] bin/fltk-config > 001 -!-[+] lib/libfltk.a > 002 -!-[+] lib/libfltk_forms.a > 003 -!-[+] lib/libfltk_gl.a > 004 -!-[+] lib/libfltk_images.a > 005 -!-[+] include/FL/Enumerations.H > 006 -!-[+] include/FL/Fl.H > 007 -!-[+] include/FL/Fl_Adjuster.H > 008 -!-[+] include/FL/Fl_BMP_Image.H > 009 -!-[+] include/FL/Fl_Bitmap.H > ... > 434 -!-[+] share/doc/fltk/examples/pixmaps/whiteking_3.xbm > 435 -!-[+] share/doc/fltk/examples/pixmaps/whiteking_4.xbm > 436 -!-[+] share/doc/fltk/examples/pixmaps/yellow.xpm > 437 -!-[+] share/doc/fltk/examples/pixmaps/yellow_bomb.xpm now you could create a receipt for the installation: > $ sink +a +m usr-local.snap /usr/local | cut -d' ' -f3- | xargs -IFILE echo /usr/local/FILE > fltk.receipt or create a tarball with the installed files: > $ tar cvfj fltk-1.1.9-i386.tar.bz2 `sink +a +m usr-local.snap /usr/local | cut -d' ' -f3- | xargs -IFILE echo /usr/local/FILE` Backuping your data ------------------- Let's imagine you just got a new slice on Linode, and you'd like to start doing its configuration. You'd start by creating a snapshot of '/etc': $ sudo sink -s /etc > etc-`date +'%Y%m%d'`.snap you'd then do your modifications and list the changes you made: > $ sudo sink etc-20090929.snaphsot /etc > 000 [=][>] apt/sources.list > 001 [=][>] passwd > 002 [=][>] group > ... and make a tarball out of your changes: > $ tar cvfj node-configured.tar.bz2 `sink +a +m etc-20090929.snap /etc | cut -d' ' -f3- | xargs -IFILE echo etc/FILE` you'll then be able to simply apply the same configuration to a new node by doing this: > $ cd / ; sudo tar fvxj ~/node-configured.tar.bz2 # EOF - vim: ts=2 sw=2 textwidth=80 et syn=kiwi