Join GitHub today
GitHub is home to over 20 million developers working together to host and review code, manage projects, and build software together.
Fetching latest commit…
Cannot retrieve the latest commit at this time.
|Failed to load latest commit information.|
== Sink == Swiss army knife for directory comparison and synchronization -- Author: Sébastien Pierre <email@example.com> -- Revision: 1.0.0 (29-Sep-2009) Sink is a tool that allows you to compare multiple directories, take snapshots of their state, and compare the states. It also provides commands to create logical links between files to do micro changes management. Use cases ========= Sink can be very handy for the following uses cases: *) Compare multiple versions of a source tree (replacing 'diff -r') *) Manually merge directories outside of a revision control system *) Synchronize files belonging to different directories *) Track changes in a specific directory *) Backup and deploy directory-wide changes *) Keep shared files up-to-date Comparing directories --------------------- Imagine you have 3 different versions of a specific source tree. In this example, we'll simply checkout three different revisions of the 'sink' development tree: > $ git clone git://github.com/sebastien/sink.git sink-r1 > $ git clone git://github.com/sebastien/sink.git sink-r2 > $ git clone git://github.com/sebastien/sink.git sink-r3 and we compare the three directories: > $ sink diff sink-r1 sink-r2 sink-r3 > No differences as the three directories are the same, we'll put them to different revisions: > $ cd sink-r1 ; git checkout -b older a17609eaabbf8feb3b480d46cb972df0599755fe ; cd .. > $ cd sink-r2 ; git checkout -b old 9e0e6a476f41f0de685d2c92e142b206b46810e4 ; cd .. and we can now compare the directories: > sink diff sink-r1 sink-r2 sink-r3 > 00 ! [+][+] .hgtags > 01 [=][-][-] DESIGN > 02 [=][<][<] Makefile > 03 [=][-][-] NOTES > 04 -!--!-[+] README > 05 [=][-][-] ROADMAP > 06 [=][>][<] TODO > 07 -!-[+][+] setup.py > 08 -!-[+]-!- Documentation/DESIGN.txt > 09 -!-[+]-!- Documentation/MANUAL.txt > 10 [=][=][-] Resources/epydoc.css > 11 -!-[+][+] Scripts/sink > 12 [=][-][-] Sources/sink/Sink.py > 13 [=][-][-] Sources/sink/Tracking.py > 14 -!-[+][+] Sources/sink/linking.py > 15 -!-[+][+] Sources/sink/main.py > 16 -!--!-[+] Sources/sink/snapshot.py > 17 -!-[+][+] Sources/sink/tracking.py sink has found differences and indicates them in the form of a table, where the legend can be found in the 'sink --help diff' command: > [=] no changes [+] file added [>] changed/newer > [-] file removed [<] changed/older > -!- file missing we can have a list of all the files that were added in 'sink-r2' and 'sink-r3' by showing only the added files: > $ sink diff +a sink-r1 sink-r2 sink-r3 > 00 -!-[+][+] .hgtags > 01 -!--!-[+] README > 02 -!-[+][+] setup.py > 03 -!-[+]-!- Documentation/DESIGN.txt > 04 -!-[+]-!- Documentation/MANUAL.txt > 05 -!-[+][+] Scripts/sink > 06 -!-[+][+] Sources/sink/linking.py > 07 -!-[+][+] Sources/sink/main.py > 08 -!--!-[+] Sources/sink/snapshot.py > 09 -!-[+][+] Sources/sink/tracking.py and the output is 'cut' friendly: > $ sink diff +a sink-r1 sink-r2 sink-r3 | cut -d' ' -f3- > .hgtags > README > setup.py > Documentation/DESIGN.txt > Documentation/MANUAL.txt > Scripts/sink > Sources/sink/linking.py > Sources/sink/main.py > Sources/sink/snapshot.py > Sources/sink/tracking.py you can also compare individual changes between all the versions. Let's see the changes made to the 'Sources/sink/tracking.py' file between 'sink-r2' and 'sink-r3': > $ sink diff sink-r2 sink-r3 > 0 -!-[+] README > 1 [=][<] TODO > 2 [=][-] Documentation/DESIGN.txt > 3 [=][-] Documentation/MANUAL.txt > 4 [=][-] Resources/epydoc.css > 5 [=][<] Sources/sink/linking.py > 6 [=][<] Sources/sink/main.py > 7 -!-[+] Sources/sink/snapshot.py > 8 [=][<] Sources/sink/tracking.py the number of the file in the table is 5, so we pass it to sink '-d' option: > $ sink diff -d5 sink-r2 sink-r3 > >> gvimdiff sink-r2/Sources/sink/linking.py sink-r3/Sources/sink/linking.py this starts up 'gvimdiff', showing the differences between the files. This is a great way to do a merge outside of a revision control system. Synchronizing directories ------------------------- So taking our previous example, let's imaging we'd like to synchronize 'sink-r1' so that it is exactly the same as 'sink-r3'. We start by showing the added or modified files: > $ sink +a +m sink-r1 sink-r3 > 00 -!-[+] .hgtags > 01 [=][<] Makefile > 02 -!-[+] README > 03 [=][<] TODO > 04 -!-[+] setup.py > 05 -!-[+] Scripts/sink > 06 -!-[+] Sources/sink/linking.py > 07 -!-[+] Sources/sink/main.py > 08 -!-[+] Sources/sink/snapshot.py > 09 -!-[+] Sources/sink/tracking.py we then pipe the result to 'cut' and 'xargs', first to create the directories that may not exist, and then to copy the files > $ sink +a +m sink-r1 sink-r3 | cut -d' ' -f3- | xargs dirname | sort | uniq| xargs mkdir -p > $ sink +a +m sink-r1 sink-r3 | cut -d' ' -f3- | xargs -I FILE cp sink-r3/FILE sink-r1/FILE and we make sure the directories are the same: > $ sink sink-r1 sink-r3 > 0 [=][-] DESIGN > 1 [=][-] NOTES > 2 [=][-] ROADMAP > 3 [=][-] Resources/epydoc.css > 4 [=][-] Sources/sink/Sink.py > 5 [=][-] Sources/sink/Tracking.py we see that we have files to remove from 'sink-r1', so let's do it: > $ sink +r sink-r1 sink-r3 | cut -d' ' -f3 | xargs -I FILE rm sink-r1/FILE > $ sink sink-r1 sink-r3 > No changes found. and we've done the synchronization right. This may look a little bit verbose, but it's Unix's design philosophy -- have simple tools that do thing, and then integrate with other tools to do higher-level operations. Tracking changes made to a directory ------------------------------------ Let's say you've just downloaded 'fltk', a cross-platform C++ widget library and you'd like to install it in /usr/local -- but you want to keep a receipt of the installed files. First, get fltk, and compile it > $ wget 'http://ftp.easysw.com/pub/fltk/1.1.9/fltk-1.1.9-source.tar.bz2' > $ tar fvxj fltk-1.1.9-source.tar.bz2 > $ cd fltk-1.1.9 ; ./configure --prefix=/usr/local ; make now we take a snapshot of '/usr/local' > $ sink snap /usr/local > usr-local.snap you can now install fltk and see the changes: > $ sudo make install > $ sink usr-local.snap /usr/local > 000 -!-[+] bin/fltk-config > 001 -!-[+] lib/libfltk.a > 002 -!-[+] lib/libfltk_forms.a > 003 -!-[+] lib/libfltk_gl.a > 004 -!-[+] lib/libfltk_images.a > 005 -!-[+] include/FL/Enumerations.H > 006 -!-[+] include/FL/Fl.H > 007 -!-[+] include/FL/Fl_Adjuster.H > 008 -!-[+] include/FL/Fl_BMP_Image.H > 009 -!-[+] include/FL/Fl_Bitmap.H > ... > 434 -!-[+] share/doc/fltk/examples/pixmaps/whiteking_3.xbm > 435 -!-[+] share/doc/fltk/examples/pixmaps/whiteking_4.xbm > 436 -!-[+] share/doc/fltk/examples/pixmaps/yellow.xpm > 437 -!-[+] share/doc/fltk/examples/pixmaps/yellow_bomb.xpm now you could create a receipt for the installation: > $ sink +a +m usr-local.snap /usr/local | cut -d' ' -f3- | xargs -IFILE echo /usr/local/FILE > fltk.receipt or create a tarball with the installed files: > $ tar cvfj fltk-1.1.9-i386.tar.bz2 `sink +a +m usr-local.snap /usr/local | cut -d' ' -f3- | xargs -IFILE echo /usr/local/FILE` Backuping your data ------------------- Let's imagine you just got a new slice on Linode, and you'd like to start doing its configuration. You'd start by creating a snapshot of '/etc': $ sudo sink -s /etc > etc-`date +'%Y%m%d'`.snap you'd then do your modifications and list the changes you made: > $ sudo sink etc-20090929.snaphsot /etc > 000 [=][>] apt/sources.list > 001 [=][>] passwd > 002 [=][>] group > ... and make a tarball out of your changes: > $ tar cvfj node-configured.tar.bz2 `sink +a +m etc-20090929.snap /etc | cut -d' ' -f3- | xargs -IFILE echo etc/FILE` you'll then be able to simply apply the same configuration to a new node by doing this: > $ cd / ; sudo tar fvxj ~/node-configured.tar.bz2 Keeping shared files up-to-date ------------------------------- Sink's '-l' mode comes in handy when you'd like to use symlinks but you can't (because you put your files under revision control, etc). Let's put this in context: let's say you're just began working on a new Web application ('mywebapp') and that you start by using a couple of files ('reset.css', 'base.css', 'sidebarlayout.css') from your 'mycsslibs' project. You'll probably simply copy the files to 'mywebapp', but if you update them in 'mywebapp' you'll have to propagate back the changes to 'mycsslib', and you might forget to do so. You might also just create symlinks instead of copying the files -- but your SCM system is likely to consider the file as a symlink, so your 'mywebapp' will depend on having 'mycsslibs' installed at the right location. Using Sink '-l' mode you can have most of the advantages of symlinks, while making it easy to propagate and merge back changes. Let's take a tour of the options, assuming 'mywebapp' and 'mycsslibs' are located in the '~/Projects' directory: First we'll create a Sink link database: > cd ~/Projects/mywebapp ; sink -l init we then create the links to the css files we want to borrow from 'mycsslibs': > sink -l add ~/Projects/mycsslib/base.css . > sink -l add ~/Projects/mycsslib/mycsslibs.css . > sink -l add ~/Projects/mycsslib/sidebarlayout.css . now you can check the status of your links (did you do local changes, did the original file changed ?) > sink -l status you can pull changes from the source (meaning updating the 'mywebapp' CSS files according to the 'mycsslibs' files) > sink -l pull or push your local changes to the source (meaning updating the 'mycsslibs' CSS files according to the 'mywebapp' local version) > sink -l push API === I'll write it if you ask for it :) # TODO: Use case for comparing directories across multiple servers # EOF - vim: ts=2 sw=2 textwidth=80 et syn=kiwi