Skip to content
An unremarkable but fairly robust network synchronization tool [Public domain]
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
bin
src/transmat
.gitignore
.hgignore
README.markdown
UNLICENSE

README.markdown

transmat

transmat is a network synchronization tool written in Python.

It is an evolution of the tool that I've been using internally to publish the catseye.tc website for years now. So, it is most suited for publishing a website, although it may have other uses.

It is not particularly remarkable, but since it has been used in hostile environments at points (e.g. unreliable dial-up connections,) it does go to some lengths to be robust, to support lousy protocols (namely FTP), and to minimize transmission time during synchronization.

It would not be difficult to extend it to support scp (mostly just write a ScpSession class compatible with the existing FTPSession class.)

Installation and Requirements

Installation is a non-issue; just put the bin directory from this file distribution on your executable search path. (Or use toolshelf to check out transmat and even this step will be managed for you.)

transmat officially requires Python 2.6.8, but might work with Python 2.5. No particular external Python libraries are required.

Workspace

All of the transmat functions operate on a local directory tree which we'll call the workspace. Further, they assume that the current directory is the workspace on which to operate.

A workspace must contain at least one file, .transmatrc. If this file is not present, the transmat commands will not treat it as a valid workspace, and will exit more-or-less immediately with an error message.

.transmatrc is in Python ConfigFile format (which is roughly the same as Microsoft's .ini format). Here is a sample .transmatrc file to get you started.

[destination]

protocol=ftp
host=example.com
username=ftpuser
password=MyDog'sBirthday

[ignore]

dirs=.svn .git .hg
files=.hg_archival.txt filehash.txt externals.cfg desktop.ini
paths=stats php_uploads projects/contrib

Of course, a workspace typically contains many more files than just this configuration file. In fact, it would typically contain all the files you want to publish to the remote server.

File Index

transmat generally attempts to transfer only files which it thinks are different from the corresponding files on the remote host, or that have no corresponding file on the remote host.

To determine this, a file called the file index is stored on the remote host. The file index contains, for every file in the directory tree on the remote host, the file's filename and an MD5 hash of its contents.

If the file index does not exist on the remote host, it must be created first, using transmat index, and uploaded.

Commands

All of the transmat functions are subcommands of the transmat command:

% transmat <command> [arguments...]

All commands take a --dry-run or -D flag which causes the command to go through the motions, but not actually apply any changes to anything.

transmat upload

Aliases: push, publish, deploy

transmat upload copies files from the workspace to the remote host.

It first downloads the file index from the remote host. It then goes through every file locally, checking the file index to see if it differs from what is on the remote host. If it does differ, it is uploaded, and the (local copy of the) file index is immediately updated and written back to disk. When all files have finished uploading, the new file index is uploaded back to the remote server as well.

transmat download

Aliases: pull, seed

transmat download copies all files from the remote host that do not exist in the workspace, into the workspace.

It does not use the file index. Instead it traverses the entire directory structure of the remote host.

This is useful if you have a set of remote files that you wish to start transferring with transmat, or if you have added more files to the remote server by some non-transmat method.

transmat index

Aliases: rebuild, rehash

transmat index creates a new file index file from all local files in the workspace.

This is useful if you have a set of local files that you wish to start transferring with transmat, even if they are not on a remote server yet.

Once the file index file has been built, it needs to be uploaded to the remote server.

transmat clean

Aliases: purge

transmat clean deletes all files from the remote host that do not exist in the workspace.

It does not use the file index. Instead it traverses the entire directory structure of the remote host.

This is useful because transmat will not otherwise delete files from the remote host that no longer exist in the workspace.

It is, of course, a good idea to run this with --dry-run first, to see what will get deleted.

Use Cases

Say you have a set of files on a remote server (for example, a website you have been manually updating with an FTP client) and you wish to start transferring them with transmat.

  • Choose a workspace and create your .transmatrc.
  • transmat download.
  • transmat index.
  • Upload the file index.
  • transmat upload --dry-run to see that transmat would not upload anything (the file index tells it that your files are no different from what is on the remote server.)
You can’t perform that action at this time.
You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session.