What is vcstool?
Vcstool is a version control system (VCS) tool, designed to make working with multiple repositories easier.
This tool should not be confused with vcstools (with a trailing
s) which provides a Python API for interacting with different version control systems. The biggest differences between the two are:
vcstooldoesn't use any state beside the repository working copies available in the filesystem.
- The file format of
vcstool exportuses the relative paths of the repositories as keys in YAML which avoids collisions by design.
vcstoolhas significantly fewer lines of code than
vcstoolsincluding the command line tools built on top.
Python 2.7 / <= 3.4 support
The latest version supporting Python 2.7 and Python <= 3.4 is 0.2.x from the 0.2.x branch.
How does it work?
Vcstool operates on any folder from where it recursively searches for supported repositories. On these repositories vcstool invokes the native VCS client with the requested command (i.e. diff).
Which VCS types are supported?
How to use vcstool?
vcs can be used similarly to the VCS clients
help command provides a list of available commands with an additional description:
By default vcstool searches for repositories under the current folder. Optionally one path (or multiple paths) can be passed to search for repositories at different locations:
vcs status /path/to/several/repos /path/to/other/repos /path/to/single/repo
Exporting and importing sets of repositories
Vcstool can export and import all the information required to reproduce the versions of a set of repositories.
Vcstool uses a simple YAML format to encode this information.
This format includes a root key
repositories under which each local repository is described by a dictionary keyed by its relative path.
Each of these dictionaries contains keys
version key is omitted the default branch is being used.
repositories: vcstool: type: git url: firstname.lastname@example.org:dirk-thomas/vcstool.git version: master old_tools/rosinstall: type: svn url: https://github.com/vcstools/rosinstall/trunk version: 748
Export set of repositories
vcs export command outputs the path, vcs type, URL and version information for all repositories in YAML format.
The output is usually piped to a file:
vcs export > my.repos
If the repository is currently on the tip of a branch the branch is followed. This implies that a later import might fetch a newer revision if the branch has evolved in the meantime. Furthermore if the local branch has evolved from the remote repository an import might not result in the exact same state.
To make sure to store the exact revision in the exported data use the command line argument
Since a specific revision is not tied to neither a branch nor a remote (for Git and Mercurial) the tool will check if the current hash exists in any of the remotes.
If it exists in multiple the remotes
upstream are considered before any other in alphabetical order.
Import set of repositories
vcs import command clones all repositories which are passed in via
stdin in YAML format.
Usually the data of a previously exported file is piped in:
vcs import < my.repos
import command also supports input in the rosinstall file format.
Beside passing a file path the command also supports passing a URL.
Only for this command vcstool supports the pseudo clients
zip which fetch a tarball / zipfile from a URL and unpack its content.
For those two types the
version key is optional.
If specified only entries from the archive which are in the subfolder specified by the version value are being extracted.
Validate repositories file
vcs validate command takes a YAML file which is passed in via
stdin and validates its contents and format.
The data of a previously-exported file or hand-generated file are piped in:
vcs validate < my.repos
validate command also supports input in the rosinstall file format.
Show log since last tag
vcs log command supports the argument
--limit-untagged which will output the log for all commits since the last tag.
Parallelization and stdin
vcs parallelizes the work across multiple repositories based on the number of CPU cores.
In the case that the invoked commands require input from
stdin that parallelization is a problem.
In order to be able to provide input to each command separately these commands must run sequentially.
When needing to e.g. interactively provide credentials all commands should be executed sequentially by passing:
In the case repositories are using SSH
git@ URLs but the host is not known yet
vcs import automatically falls back to a single worker.
Run arbitrary commands
vcs custom command enables to pass arbitrary user-specified arguments to the vcs invocation.
The set of repositories to operate on can optionally be restricted by the type:
vcs custom --git --args log --oneline -n 10
If the command should work on multiple repositories make sure to pass only generic arguments which work for all of these repository types.
How to install vcstool?
On Debian-based platforms the recommended method is to install the package python3-vcstool. On Ubuntu this is done using apt-get:
If you are using ROS you can get the package directly from the ROS repository:
sudo sh -c 'echo "deb http://packages.ros.org/ros/ubuntu $(lsb_release -sc) main" > /etc/apt/sources.list.d/ros-latest.list' sudo apt install curl # if you haven't already installed curl curl -s https://raw.githubusercontent.com/ros/rosdistro/master/ros.asc | sudo apt-key add - sudo apt-get update sudo apt-get install python3-vcstool
curl -s https://packagecloud.io/install/repositories/dirk-thomas/vcstool/script.deb.sh | sudo bash sudo apt-get update sudo apt-get install python3-vcstool
On other systems, use the PyPI package:
sudo pip install vcstool
For the shells bash, tcsh and zsh vcstool can provide auto-completion of the various VCS commands. In order to enable that feature the shell specific completion file must be sourced.
For bash append the following line to the
For tcsh append the following line to the
For zsh append the following line to the
For fish append the following line to the
How to contribute?
How to report problems?
Before reporting a problem please make sure to use the latest version. Issues can be filled on GitHub after making sure that this problem has not yet been reported.
Please make sure to include as much information, i.e. version numbers from vcstool, operating system, Python and a reproducible example of the commands which expose the problem.
How to try the latest changes?
setup.sh file prepends the
src folder to the
PYTHONPATH and the
scripts folder to the
Then vcstool can be used with the commands
vcs-COMMAND (note the hyphen between
command instead of a space).
-e/--editable flag of
pip can be used:
# from the top level of this repo pip3 install --user -e .