A tool to help manage evolving code dependencies.
I'm working simultaneously on a number of C repos that
depend on each other in nontrivial ways, and all of
which are evolving together. At first, I was manually
copying file changes between repos. I built
syncer as a way to help automate that process.
syncer, you can edit files in either the main
repo, or in a secondary location that uses a copy of
the original file. These edits become easy to notice
and copy over, or merge by hand with the help of a diff.
syncer is designed to be easy to use and invisible to
The primary task of
syncer is to track git repos that
depend on each other; that is, they both have copies of
the same file. If you edit one of the copies of
such a file, then you'll need to re-synchronize
Here's a brief summary of the available actions:
syncer track <repo-name> # Track the current dir with the given repo name # (a name may be, e.g., a github url). syncer track <file1> <file2> # Track the given file pair. syncer check # Check current repo for any incoming out outgoing # file changes. syncer check --all # Check all known repo-name/dir and file/file pairs # for differences. syncer remind # Print all paths affected by last run of # "syncer check"; useful for testing. syncer list # Print all file pairs checked for equality.
Let's see an example. We need a way to name repos. Many people use github, so I'll use github URLs as the names. In reality, any uniquely identifying space-free string is ok as a repo name.
Suppose you're github user
my_png_reader.h is shared between the
https://github.com/bob/photoapp.git. Begin by
syncer where these repos live in your
cd /path/to/pnglib syncer track https://github.com/bob/pnglib.git cd /path/to/photoapp syncer track https://github.com/bob/photoapp.git
You also need to give
syncer a way to know which files in a repo are
actually copies of originals from another. Do this by making the 3rd
line of the file end in the name of the home repo. For example, the
top of the file
my_png_reader.h might look like this:
// my_png_reader.h // // Home repo: https://github.com/bob/pnglib.git
Let's say you edit
my_png_reader.h in your local
Once you're done with your current
photoapp work, you can run
a check with
syncer to identify any copied files that are no longer
identical, like so:
$ syncer check
This begins an interactive process which will notice that the
my_png_reader.h are not identical, and notice which one
is newer. It will show you the diff and give you the option of copying
over the file, or of saving the diff to a file in case you need to
manually make nuanced changes.
syncer check works quickly by only looking for changes
created by or affecting the directory it's run from. If you want to
simultaneously synchronize across all your repos, you can instead run:
$ syncer check --all
Let's say you change many files at once, you run
syncer check to
perform the appropriate copies, but now you've forgotten what testing
needs to be done to verify that the changes are good. You can run
the following command to remind you of every file that was changed
by the last run of
$ syncer remind
This action will list all file pairs compared by
syncer, providing two
absolute paths per line.
The original file path (the version located in that file's primary repo)
is listed first, followed by the copy's path. Custom file pairs, described
below, don't have a designation between original/copy, so theyr'e listed
in the order they were given to
syncer. Sample uses of this action:
$ syncer list $ syncer list | wc # Get a count of file comparisons. $ syncer list | sort | less # Inspect file pairs.
-- custom file pairs
syncer can track repo-agnostic file pairs. For example,
let's say you maintain a cross-platform library in which every platform's
code lives in its own directory. All the header files are thus duplicated,
but should always be identical.
This is a tricky case because both files are in the same repo, so the
above workflow doesn't apply.
Such file pairs can be tracked like so:
syncer track /my/xpltfrm/win/audio.h /my/xpltfrm/lib/mac/audio.h
syncer check will notice differences between these two files in
addition to the above-mentioned repo-based checks. One subtle point here
is that file copies between such file/file pairs will preserve the 3rd
line of the copied-over file in order to make the two modes of operation
compatible. That is, file/file pairs may have different 3rd lines and
still be considered equivalent by
syncer, and such differences are
syncer is completely independent of
github. It works
with subtrees of your directory structure that it calls "repos," but
they could be any subtree. This is partially a result of the design
goal being an ultra-lightweight, highly transparent tool.
All metadata is kept in human-friendly files in the
~/.syncer directory, which
you are free to hand edit.
syncer is a Python 3 script. It assumes Python is
/usr/local/bin/python3. If this isn't
true, edit the first line of
$ git clone https://github.com/tylerneylon/syncer.git $ sudo ln -s $(cd syncer; pwd)/syncer.py /usr/local/bin/syncer
One alternative to this system is
which is painful to use. Another alternative would
be a package manager like
npm, but I have opted to
aim for a lighter-weight solution that I feel makes
life easier for potential users of the libraries I'm
building; C users may not be familiar with
npm is more focused on node.js use cases.
syncer is a choice that maximizes transparency.
It depends on a single comment line -- specifically, the 3rd line -- in the source
files you want to be tracked, but otherwise keeps all
of its metadata in a small human-friendly text
files in the
~/.syncer directory, which you are free to hand edit.