Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with
or
.
Download ZIP
A tool for fetching binary artifacts for builds and tests
Python Shell Makefile
Branch: master
#23 Compare This branch is 233 commits ahead of jhford:master.

Bug 1188571 - Add a setup script field in manifests

There may be a need for a specific tooltool package to e.g run registration
commands of some sort. There used to be a global way to do this with a
setup.sh script handled by the tooltool wrapper shell script, but it quickly
became cumbersome to handle combinations of multiple tooltool packages
requiring each their own setup. Back then, however, the setup was usually
limited to unpacking, which is now handled by tooltool itself.

Anyways, learning from this past experience, it appears better to have each
package handle its own setup separately, without having to deal with the
various combinations that may be requires if multiple packages need their
own setup.
latest commit 568945fbee
@glandium glandium authored djmitche committed
Failed to load latest commit information.
.gitignore merge existing (non-relengapi) repo
.travis.yml Bug 1171004: remove relengapi bits
LICENSE.txt move stuff around a bit
MANIFEST.in Bug 1171004: remove relengapi bits
Makefile
README.md Bug 1171004: remove relengapi bits
coveragerc import validate stuff from relengapi
pep8rc import validate stuff from relengapi
setup.py Bug 1171004: remove relengapi bits
test.sh fix whitespace
test_file.ogg
test_tooltool.py Bug 1188571 - Add a setup script field in manifests
tooltool.py Bug 1188571 - Add a setup script field in manifests
validate.sh Bug 1171004: remove relengapi bits

README.md

Tooltool

This is tooltool. Tooltool is a program that helps make downloading large binaries easier in a CI environment. The program creates a json based manifest that is small compared to the binaries. That manifest is transmitted to the machine that needs the binary somehow (checked in, included in tarball, etc) where the machine will run tooltool to download.

When using the fetch mode, the program will check to see if the file exists locally. If this file does not exist locally the program will try to fetch from one of the base URLs provided. The API that tooltool uses to fetch files is exceedingly simple. the API is that each file request will look for an http resource that is a combination of an arbitrary base url, a directory that is named as the hashing algorithm used and the hashing results of each file stored.

Example, using base url of "http://localhost:8080/tooltool", algorithm of "sha512" and a file that hashes to "abcedf0123456789", tooltool would look for the file at "http://localhost:8080/tooltool/sha512/abcdef0123456789". If there is a local file that has the filename specified in the manifest already, tooltool will not overwrite by default. In this case, tooltool will exit with a non-0 exit value. If overwrite mode is enabled, tooltool will overwrite the local file with the file specified in the manifest.

Server

This repository contains only the tooltool client -- tooltool.py. The tooltool server component is a part of RelengAPI.

If you want to use the client, just copy out tooltool.py -- it has no dependencies.

Development

To hack on the tooltool client, install into a virtualenv with

pip install -e .[test]

Send pull requests through GitHub.

Both the client and the server components are covered by Travis, via the validate.sh script which you can run yourself.

Something went wrong with that request. Please try again.