dtuf   

This module is now DEPRECATED! The Update Framework has moved to version 2 and I don't have time to update dtuf to work with it.

Docker registry bindings for The Update Framework in Python. Uses dxf to store TUF metadata and target files in a Docker registry.

• Easy-to-use Python module and command-line tool.
• Distribute your data with the security and trust features of The Update Framework.
• Backed by the scalability and flexbility of a Docker registry.
• No extra moving parts: just Python client-side and Docker registry server-side.
  • Docker client and daemon not required.
  • Notary not required. See this issue for discussion.
• Supports Docker registry schema v1 and v2.
• Works on Python 2.7 and 3.4.

Examples

Command-line example

This assumes at least DTUF_HOST has been set to the hostname of a Docker registry (see Usage below). You may need to set DTUF_USERNAME and DTUF_PASSWORD too depending on the registry you're using.

You can run your own registry or use a hosted one such as Docker Hub (registry-1.docker.io).

If you want to run your own, see this example script or the unit tests. There are also full instructions available here.

# On the master machine
$ echo 'Hello World!' > demo.txt
$ dtuf create-root-key fred/demo
$ dtuf create-metadata-keys fred/demo
$ dtuf create-metadata fred/demo
$ dtuf push-target fred/demo demo.txt demo.txt
$ dtuf push-metadata fred/demo
# pub key is in dtuf_repos/fred/demo/master/keys/root_key.pub
# distribute it out-of-band

# On some other machine
$ dtuf pull-metadata fred/demo root_key.pub
demo.txt
$ dtuf pull-target fred/demo demo.txt
Hello World!

# Update on the master machine
$ echo 'Update World!' > demo.txt
$ echo 'Another World!' > demo2.txt
$ dtuf push-target fred/demo demo.txt demo.txt
$ dtuf push-target fred/demo demo2.txt demo2.txt
$ dtuf push-metadata fred/demo

# On the other machine
$ dtuf pull-metadata fred/demo
demo.txt
demo2.txt
$ dtuf pull-target fred/demo demo.txt
Update World!
$ dtuf pull-target fred/demo demo2.txt
Another World!

Module example

This example uses the Docker Hub. Change the username, password and repository name to suit.

Publish on the master machine:

from dtuf import DTufMaster

def auth(dtuf, response):
    dtuf.authenticate('fred', 'somepassword', response=response)

dtuf = DTufMaster('registry-1.docker.io', 'fred/demo', auth=auth)

with open('demo.txt', 'w') as f:
    f.write('Hello World!\n')

dtuf.create_root_key()
dtuf.create_metadata_keys()
dtuf.create_metadata()
dtuf.push_target('demo.txt', 'demo.txt')
dtuf.push_metadata()
# pub key is in dtuf_repos/fred/demo/master/keys/root_key.pub
# distribute it out-of-band

Retrieve on some other machine:

from dtuf import DTufCopy

def auth(dtuf, response):
    dtuf.authenticate('barney', 'otherpassword', response=response)

dtuf = DTufCopy('registry-1.docker.io', 'fred/demo', auth=auth)

with open('root_key.pub', 'r') as f:
    assert dtuf.pull_metadata(f.read()) == ['demo.txt']

s = ''
for download in dtuf.pull_target('demo.txt'):
    for chunk in download:
        s += chunk
assert s == 'Hello World!\n'

Usage

The module API is described here.

Environment variables

The dtuf command-line tool uses the following environment variables. Only DTUF_HOST is strictly required but you may need to set others depending on your set up.

• DTUF_HOST - Host where Docker registry is running
• DTUF_INSECURE - Set this to 1 if you want to connect to the registry using http rather than https (which is the default).
• DTUF_USERNAME - Name of user to authenticate as.
• DTUF_PASSWORD - User's password.
• DTUF_REPOSITORIES_ROOT - Directory under which TUF metadata should be stored. Note that the repository name is appended to this before storing the metadata. Defaults to dtuf_repos in the current working directory.
• DTUF_AUTH_HOST - If set, always perform token authentication to this host, overriding the value returned by the registry.
• DTUF_PROGRESS - If this is set to 1, a progress bar is displayed (on standard error) during dtuf push-target, dtuf push-metadata, dtuf pull-metadata and dtuf pull-target. If this is set to 0, a progress bar is not displayed. If this is set to any other value, a progress bar is only displayed if standard error is a terminal.
• DTUF_BLOB_INFO - Set this to 1 if you want dtuf pull-target to prepend each blob with its digest and size (printed in plain text, separated by a space and followed by a newline).
• DTUF_ROOT_KEY_PASSWORD - Password to use for encrypting the TUF root private key. Used by dtuf create-root-key, dtuf create-metadata and dtuf reset-keys. If unset then you'll be prompted for the password.
• DTUF_TARGETS_KEY_PASSWORD - Password to use for encrypting the TUF targets private key. Used by dtuf create-metadata-keys, dtuf create-metadata, dtuf reset-keys and dtuf push-metadata. If unset then you'll be prompted for the password.
• DTUF_SNAPSHOT_KEY_PASSWORD - Password to use for encrypting the TUF snapshot private key. Used by dtuf create-metadata-keys, dtuf create-metadata, dtuf reset-keys and dtuf push-metadata. If unset then you'll be prompted for the password.
• DTUF_TIMESTAMP_KEY_PASSWORD - Password to use for enrypting the TUF timestamp private key. Used by dtuf create-metadata-keys, dtuf create-metadata, dtuf reset-keys and dtuf push-metadata. If unset then you'll be prompted for the password.
• DTUF_ROOT_LIFETIME - Lifetime of the TUF root metadata. After this time expires, you'll need to use dtuf reset-keys and dtuf push-metadata to re-sign the metadata. Defaults to 1 year.
• DTUF_TARGETS_LIFETIME - Lifetime of the TUF targets metadata. After this time expires, you'll need to use dtuf push-metadata to re-sign the metadata. Defaults to 3 months.
• DTUF_SNAPSHOT_LIFETIME - Lifetime of the TUF snapshot metadata. After this time expires, you'll need to use dtuf push-metadata to re-sign the metadata. Defaults to 1 week.
• DTUF_TIMESTAMP_LIFETIME - Lifetime of the TUF timestamp metadata. After this time expires, you'll need to use dtuf push-metadata to re-sign the metadata. Defaults to 1 day.
• DTUF_LOG_FILE - Name of file to write log messages into. Defaults to dtuf.log in the current working directory. Set it to an empty string to disable logging to a file.
• DTUF_FILE_LOG_LEVEL - Name of the Python logging level to use when deciding which messages to write to the log file. Defaults to WARNING.
• DTUF_CONSOLE_LOG_LEVEL - Name of the Python logging level to use when deciding which messages to write to the console. Defaults to WARNING.

Command line options

You can use the following options with dtuf. In each case, supply the name of the repository on the registry you wish to work with as the second argument.

Creating, updating and uploading data

• dtuf create-root-key <repo>

  Create TUF root keypair for the repository.

  The private key is written to $DTUF_REPOSITORIES_ROOT/<repo>/master/keys/root_key and can be moved offline once you've used dtuf create-metadata. You'll need it again if you use dtuf reset-keys when the root metadata expires.

  The public key is written to $DTUF_REPOSITORIES_ROOT/<repo>/master/keys/root_key.pub and can be given to others for use when retrieving a copy of the repository metadata with dtuf pull-metadata.

• dtuf create-metadata-keys <repo>

  Create TUF metadata keypairs for the repository.

  The keys are written to the $DTUF_REPOSITORIES_ROOT/<repo>/master/keys directory. The public keys have a .pub extension.

  You can move the private keys offline once you've used dtuf push-metadata to publish the repository.

  You don't need to give out the metadata public keys since they're published on the repository.

• dtuf create-metadata <repo>

  Create and sign the TUF metadata for the repository.

  You only need to do this once for each repository, and the repository's root and metadata private keys must be available.

• dtuf reset-keys <repo>

  Re-sign the TUF metadata for the repository.

  Call this if you've generated new root or metadata keys (because one of the keys has been compromised, for example) but you don't want to delete the repository and start again.

• dtuf push-target <repo> <target> <file|@target>...

  Upload data to the repository and update the local TUF metadata

  The metadata isn't uploaded until you use dtuf push-metadata.

  The data is given a name (known as the target) and can come from a list of files or existing target names. Existing target names should be prepended with @ in order to distinguish them from filenames.

• dtuf del-target <repo> <target>...

  Delete targets (data) from the repository and update the local TUF metadata.

  The metadata isn't updated on the registry until you use dtuf push-metadata.

  Note that the registry doesn't support deletes yet so expect an error.

• dtuf push-metadata <repo>

  Upload local TUF metadata to the repository.

  The TUF metadata consists of a list of targets (which were uploaded by dtuf push-target), a snapshot of the state of the metadata (list of hashes), a timestamp and a list of public keys.

  The metadata except for the list of public keys will be signed here. The list of public keys was signed (along with the rest of the metadata) when you used dtuf create-metadata (or dtuf reset-keys).

• dtuf list-master-targets <repo>

  Print the names of all the targets defined in the local TUF metadata.

• dtuf get-master-expirations <repo>

  Print the expiration dates of the TUF metadata.

Downloading data

• dtuf pull-metadata <repo> [<root-pubkey-file>|-]

  Download TUF metadata from the repository.

  The metadata is checked for expiry and verified against the root public key for the repository.

  You only need to supply the root public key once, and you should obtain it from the person who uploaded the metadata. If you specify - then the key is read from standard input instead of a file.

  Target data is not downloaded - use dtuf pull-target for that.

  A list of targets which have been updated since you last downloaded them will be printed to standard output, one per line.

• dtuf pull-target <repo> <target>...

  Download targets (data) from the repository to standard output.

  Each target's data consists of one of more separate blobs (depending on how many > were uploaded). All of them will be downloaded.

• dtuf blob-sizes <repo> <target>...

  Print the sizes of all the blobs which make up a list of targets.

• dtuf check-target <repo> <target> <file>...

  Check whether the hashes of a target's blobs match the hashes of list of files. An error message will be displayed if not and the exit code won't be 0.

• dtuf list-copy-targets <repo>

  Print the names of all the targets defined in the local copy of the TUF metadata.

• dtuf get-copy-expirations <repo>

  Print the expiration dates of the local copy of the TUF metadata.

• dtuf list-repos

  Print the names of all the repositories in the registry.

Authentication tokens

dtuf automatically obtains Docker registry authentication tokens using your DTUF_USERNAME and DTUF_PASSWORD environment variables as necessary.

However, if you wish to override this then you can use the following command:

• dtuf auth <repo> <action>...

  Authenticate to the registry using DTUF_USERNAME and DTUF_PASSWORD, and print the resulting token.

  action can be pull, push or *.

If you assign the token to the DTUF_TOKEN environment variable, for example:

DTUF_TOKEN=$(dtuf auth fred/demo pull)

then subsequent dtuf commands will use the token without needing DTUF_USERNAME and DTUF_PASSWORD to be set.

Note however that the token expires after a few minutes, after which dtuf will exit with EACCES.

Installation

pip install python-dtuf

Licence

MIT

Tests

make test

Lint

make lint

Code Coverage

make coverage

coverage.py results are available here.

Coveralls page is here.