Merge 21ff492 into e242a75

.gitignore

@@ -3,6 +3,9 @@ dist/*
 build/*
 env/*
 
+# docs build directory
+docs/build/*
+
 # global file patterns
 *.log
 *.pyc

.readthedocs.yaml

@@ -0,0 +1,19 @@
+# .readthedocs.yaml
+# Read the Docs configuration file
+# See https://docs.readthedocs.io/en/stable/config-file/v2.html for details
+
+# Required
+version: 2
+
+# Build documentation with Sphinx
+sphinx:
+   builder: html
+   configuration: docs/conf.py
+
+# Optionally build your docs in additional formats such as PDF
+formats: []
+
+# Optionally set the version of Python and requirements required to build your docs
+python:
+   install:
+   - requirements: requirements-docs.txt

docs/INSTALLATION.rst

@@ -12,19 +12,13 @@ is available on our `website
 and on the `maintainers page
 <https://github.com/theupdateframework/tuf/blob/develop/docs/MAINTAINERS.txt>`_.
 
-The latest release and its packaging information, such as who signed the
-release and their PGP fingerprint, can also be found on our 1-year `roadmap
-<ROADMAP.md>`_.
-
-
 
 Release Verification
 --------------------
 
-Assuming you trust `the maintainer's PGP key <MAINTAINERS.txt>`_, the detached
-ASC signature can be downloaded and verified.  For example:
-
-::
+Assuming you trust `the maintainer's PGP key
+<https://github.com/theupdateframework/tuf/blob/develop/docs/MAINTAINERS.txt>`_,
+the detached ASC signature can be downloaded and verified.  For example::
 
    $ gpg --verify securesystemslib-0.10.8.tar.gz.asc
    gpg: assuming signed data in 'securesystemslib-0.10.8.tar.gz'
@@ -42,20 +36,20 @@ installation, done simply with one of the following commands:
 
 Installing from Python Package Index (https://pypi.python.org/pypi).
 (Note: Please use "python3 -m pip install --no-use-wheel tuf" if your version
-of pip <= 1.5.6)
-::
+of pip <= 1.5.6)::
+
     $ python3 -m pip install tuf
 
 
 **Alternatively**, if you wish to install from a GitHub release you've already
 downloaded, or a package you obtained in another way, you can instead:
 
-Install from a local source archive:
-::
+Install from a local source archive::
+
     $ python3 -m pip install <path to archive>
 
-Or install from the root directory of the unpacked archive:
-::
+Or install from the root directory of the unpacked archive::
+
     $ python3 -m pip install .
 
 
@@ -68,8 +62,8 @@ be verified, in pure Python.  To fully support RSA, Ed25519, ECDSA, and
 other crypto, you must install the extra dependencies declared by
 securesystemslib.  **Note**: that may require non-Python dependencies, so if
 you encounter an error attempting this pip command, see
-`more instructions below <#non-python-dependencies>`_).
-::
+`more instructions below <#non-python-dependencies>`_). ::
+
     $ python3 -m pip install securesystemslib[crypto,pynacl] tuf
 
 
@@ -84,17 +78,16 @@ For example, PyNaCl and Cryptography -- two libraries used in the full
 installation to support certain cryptographic functions -- may require FFI
 (Foreign Function Interface) development header files.
 
-Debian-based distributions can install the necessary header libraries with apt
-(Advanced Package Tool.)
-::
+Debian-based distributions can install the necessary header libraries with apt::
+
     $ apt-get install build-essential libssl-dev libffi-dev python-dev
 
-Fedora-based distributions can instead install these libraries with dnf.
-::
+Fedora-based distributions can instead install these libraries with dnf::
+
     $ dnf install libffi-devel redhat-rpm-config openssl-devel
 
 OS X users can install these header libraries with the `Homebrew <https://brew.sh/>`_
-package manager, among other options.
-::
+package manager, among other options::
+
     $ brew install python3
     $ brew install libffi

docs/api/api-reference.rst

@@ -0,0 +1,36 @@
+API Reference
+=====================
+
+
+TUF provides multiple APIs:
+
+
+* The low-level :doc:`tuf.api` provides access to a Metadata file abstraction
+  that closely follows the TUF specification's `document formats`_.
+  This API handles de/serialization to and from files and makes it easier to access
+  and modify metadata content safely. It is purely focused on individual
+  pieces of Metadata and provides no concepts like "repository" or "update
+  workflow".
+
+* The `client update workflow`_ is implemented in the :doc:`tuf.ngclient` module:
+  It is a higher-level API that provides ways to query and download target files
+  securely, while handling the TUF update workflow behind the scenes. ngclient
+  is implemented on top of the Metadata API and can be used to implement
+  various TUF clients with relatively little effort.
+
+.. note:: Major API changes are unlikely but these APIs are not yet
+   considered stable, and a higher-level repository operations API is not yet
+   included.
+
+   There is a legacy implementation in the source code (not covered by this
+   documentation): it is in maintenance mode and receives no feature work.
+
+.. toctree::
+   :maxdepth: 2
+   :caption: Contents:
+
+   tuf.api
+   tuf.ngclient
+
+.. _client update workflow: https://theupdateframework.github.io/specification/latest/#detailed-client-workflow
+.. _document formats: https://theupdateframework.github.io/specification/latest/#document-formats

docs/api/tuf.api.metadata.rst

@@ -0,0 +1,7 @@
+Metadata
+---------------------------------
+
+.. automodule:: tuf.api.metadata
+   :members:
+   :undoc-members:
+   :show-inheritance:

docs/api/tuf.api.rst

@@ -0,0 +1,18 @@
+Metadata API
+===============
+
+The low-level Metadata API contains two modules:
+
+* :doc:`tuf.api.metadata` contains the actual Metadata abstraction
+  that higher level libraries and application code should use to interact
+  with TUF metadata. This abstraction provides safe reading and writing to
+  supported file formats and helper functions for accessing and modifying
+  the metadata contents.
+* :doc:`tuf.api.serialization` covers serializing the metadata into
+  specific wire formats (like json).
+
+.. toctree::
+   :hidden:
+
+   tuf.api.metadata
+   tuf.api.serialization

docs/api/tuf.api.serialization.rst

@@ -0,0 +1,10 @@
+Serialization
+=============================
+
+JSON serialization
+-----------------------------
+
+.. automodule:: tuf.api.serialization.json
+   :members:
+   :undoc-members:
+   :show-inheritance:

docs/api/tuf.ngclient.config.rst

@@ -0,0 +1,7 @@
+Configuration
+=============
+
+.. automodule:: tuf.ngclient.config
+   :members:
+   :undoc-members:
+   :show-inheritance:

docs/api/tuf.ngclient.fetcher.rst

@@ -0,0 +1,7 @@
+Fetcher
+============
+
+.. automodule:: tuf.ngclient.fetcher
+   :members:
+   :undoc-members:
+   :show-inheritance:

docs/api/tuf.ngclient.rst

@@ -0,0 +1,19 @@
+ngclient
+========
+
+The ngclient module contains a complete TUF client library implementation.
+
+* :doc:`tuf.ngclient.updater` implements the `detailed client workflow`_
+* :doc:`tuf.ngclient.config` provides optional configuration for the updater
+* :doc:`tuf.ngclient.fetcher` can be used for optional low-level network I/O control
+
+
+.. toctree::
+   :hidden:
+
+   tuf.ngclient.updater
+   tuf.ngclient.config
+   tuf.ngclient.fetcher
+
+.. _detailed client workflow: https://theupdateframework.github.io/specification/latest/#detailed-client-workflow
+

docs/api/tuf.ngclient.updater.rst

@@ -0,0 +1,6 @@
+Updater
+=========
+
+.. automodule:: tuf.ngclient.updater
+   :members:
+   :special-members: __init__

docs/conf.py

@@ -0,0 +1,56 @@
+# Configuration file for the Sphinx documentation builder.
+#
+# This file only contains a selection of the most common options. For a full
+# list see the documentation:
+# https://www.sphinx-doc.org/en/master/usage/configuration.html
+
+# -- Path setup --------------------------------------------------------------
+
+# If extensions (or modules to document with autodoc) are in another directory,
+# add these directories to sys.path here. If the directory is relative to the
+# documentation root, use os.path.abspath to make it absolute, like shown here.
+#
+import os
+import sys
+sys.path.insert(0, os.path.abspath(os.path.join('..')))
+
+import tuf
+
+# -- Project information -----------------------------------------------------
+
+project = 'TUF'
+copyright = '2021, New York University and the TUF contributors'
+author = 'New York University and the TUF contributors'
+
+
+# -- General configuration ---------------------------------------------------
+
+master_doc = 'index'
+
+# Add any Sphinx extension module names here, as strings. They can be
+# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
+# ones.
+extensions = ['sphinx.ext.napoleon']
+
+# Add any paths that contain templates here, relative to this directory.
+templates_path = ['_templates']
+
+# List of patterns, relative to source directory, that match files and
+# directories to ignore when looking for source files.
+# This pattern also affects html_static_path and html_extra_path.
+exclude_patterns = ['GETTING_STARTED.rst', 'OVERVIEW.rst', 'TAP.rst']
+
+# -- Options for HTML output -------------------------------------------------
+
+# The theme to use for HTML and HTML Help pages.  See the documentation for
+# a list of builtin themes.
+#
+html_theme = 'sphinx_rtd_theme'
+
+# Add any paths that contain custom static files (such as style sheets) here,
+# relative to this directory. They are copied after the builtin static files,
+# so a file named "default.css" will overwrite the builtin "default.css".
+#html_static_path = ['_static']
+
+autodoc_mock_imports = ['securesystemslib']
+

docs/index.rst

@@ -0,0 +1,19 @@
+TUF Developer Documentation
+===========================
+
+This documentation provides essential information for those developing software
+with the `Python reference implementation of The Update Framework (TUF)
+<https://github.com/theupdateframework/tuf>`_.
+
+The reference implementation provides easy-to-use components for Python
+developers but also aims to be a readable guide and demonstration for those
+working on implementing TUF in their own languages, environments, or update
+systems.
+
+.. toctree::
+   :maxdepth: 1
+   :caption: Contents:
+
+   api/api-reference
+   CONTRIBUTORS
+   INSTALLATION

requirements-docs.txt

@@ -0,0 +1,14 @@
+# Install requirements needed in for the documentation build
+
+# pinned tuf runtime dependencies (should auto-update and -trigger ci/cd)
+-r requirements-pinned.txt
+
+# install sphinx and its extensions
+sphinx
+sphinx-rtd-theme
+
+# Docutils versions >=0.17.0 have incompatibilites with
+# sphinx-rtd-theme and fail to render some features.
+# Pin the version until readthedocs release their fix
+# (readthedocs/sphinx_rtd_theme#1113).
+docutils<0.17.0

tox.ini

@@ -54,3 +54,11 @@ commands =
     mypy
 
     bandit -r tuf
+
+[testenv:docs]
+deps =
+    -r{toxinidir}/requirements-docs.txt
+
+changedir = {toxinidir}
+commands =
+    sphinx-build -b html docs docs/build/html

tuf/api/metadata.py

@@ -987,6 +987,7 @@ class DelegatedRole(Role):
         - paths is set: delegates targets matching any path pattern in paths
         - path_hash_prefixes is set: delegates targets whose target path hash
             starts with any of the prefixes in path_hash_prefixes
+
     paths and path_hash_prefixes are mutually exclusive: both cannot be set,
         at least one of them must be set.
 

tuf/ngclient/updater.py

@@ -1,9 +1,9 @@
 # Copyright 2020, New York University and the TUF contributors
 # SPDX-License-Identifier: MIT OR Apache-2.0
 
-"""TUF module that implements the client update workflow.
+"""Client update workflow implementation
 
-This module contains the Updater class that provides an implementation of the
+The Updater class provides an implementation of the
 `TUF client workflow
 <https://theupdateframework.github.io/specification/latest/#detailed-client-workflow>`_.
 Updater provides an API to query available targets and to download them in a
@@ -20,6 +20,7 @@
   * When metadata is up-to-date, targets can be dowloaded. The repository
     snapshot is consistent so multiple targets can be downloaded without
     fear of repository content changing. For each target:
+
       * :func:`~tuf.ngclient.updater.Updater.get_one_valid_targetinfo()` is
         used to find information about a specific target. This will load new
         targets metadata as needed (from local cache or remote repository).
@@ -31,11 +32,12 @@
 Below is a simple example of using the Updater to download and verify
 "file.txt" from a remote repository. The required environment for this example
 is:
+
     * A webserver running on http://localhost:8000, serving TUF repository
       metadata at "/tuf-repo/" and targets at "/targets/"
     * Local metadata directory "~/tufclient/metadata/" is writable and contains
       a root metadata version for the remote repository
-    * Download directory "~/target-downloads/" is writable
+    * Download directory "~/tufclient/downloads/" is writable
 
 Example::