Add metadata module with container classes for TUF role metadata, including
methods to read/serialize/write from and to JSON, perform TUF-compliant
metadata updates, and create and verify signatures.

The 'Metadata' class provides a container for inner TUF metadata objects (Root,
Timestamp, Snapshot, Targets) (i.e. OOP composition)

The 'Signed' class provides a base class to aggregate common attributes (i.e.
version, expires, spec_version) of the inner metadata classes. (i.e. OOP
inheritance). The name of the class also aligns with the 'signed' field of
the outer metadata container.

Based on prior observations in TUF's sister project in-toto, this architecture
seems to well represent the metadata model as it is defined in the
specification (see in-toto/in-toto#98 and in-toto/in-toto#142 for related
discussions).

This commits also adds tests.

**TODO: See doc header TODO list**

**Additional design considerations**
(also in regards to prior sketches of this module)

 - Aims at simplicity, brevity and recognizability of the wireline metadata
   format.

 - All attributes that correspond to fields in TUF JSON metadata are public.
   There doesn't seem to be a good reason to protect them with leading
   underscores and use setters/getters instead, it just adds more code, and
   impedes recognizability of the wireline metadata format.

 - Although, it might be convenient to have short-cuts on the Metadata class
   that point to methods and attributes that are common to all subclasses of
   the contained Signed class (e.g. Metadata.version instead of
   Metadata.signed.version, etc.), this also conflicts with goal of
   recognizability of the wireline metadata. Thus we won't add such short-cuts
   for now. See:
   theupdateframework#1060 (comment)

 - Signing keys and a 'consistent_snapshot' boolean are not on the targets
   metadata class. They are a better fit for management code. See:
   theupdateframework#1060 (comment),
   and theupdateframework#660.

 - Does not use sslib schema checks (see TODO notes about validation in
   doc header)

 - Does not use existing tuf utils, such as make_metadata_fileinfo,
   build_dict_conforming_to_schema, if it is easy and more explicit to
   just re-implement the desired behavior on the metadata classes.

 - All datetime's are treated as UTC. Since timezone info is not captured in
   the wireline metadata format it should not be captured in the internal
   representation either.

 - Does not use 3rd-party dateutil package, in order to minimize dependency
   footprint, which is especially important for update clients which often have
   to vendor their dependencies.
   However, compatibility between the more advanced dateutil.relativedelta (e.g
   handles leap years automatically) and timedelta is tested.

 - Uses PEP8 indentation (4 space) and Google-style doc string instead of
   sslab-style. See
   secure-systems-lab/code-style-guidelines#20

 - Does not support Python =< 3.5

Co-authored-by: Trishank Karthik Kuppusamy <trishank.kuppusamy@datadoghq.com>
Co-authored-by: Joshua Lock <jlock@vmware.com>
Co-authored-by: Teodora Sechkova <tsechkova@vmware.com>
Signed-off-by: Lukas Puehringer <lukas.puehringer@nyu.edu>

The new metadata module uses constructs that are only available
on Python >= 3.6 (typing, f-format strings, etc.).

Signed-off-by: Lukas Puehringer <lukas.puehringer@nyu.edu>

Add generic read from json class method that returns a Metadata
object with a signed field that contains the appropriate Signed
subclass, based on the signed._type field of the read metadata.

Signed-off-by: Lukas Puehringer <lukas.puehringer@nyu.edu>

Add simple as_json Metadata method and use it instead of repository
lib's internal _get_written_metadata function in write_to_json.

This commit further adds code documentation and the possibility to
write compact json by excluding whitespace to write_to_json, and
also removes a call to the sign method from write_to_json.

The commit also adds tests.

Signed-off-by: Lukas Puehringer <lukas.puehringer@nyu.edu>

Signed-off-by: Lukas Puehringer <lukas.puehringer@nyu.edu>

Signed-off-by: Lukas Puehringer <lukas.puehringer@nyu.edu>

Add simple methods to create or verify signatures of the
canonical_signed property of a Metadata object.

See corresponding docstrings for behavior and design
considerations.

The commit also adds tests and updates the test setup to load
some test keys into memory.

Signed-off-by: Lukas Puehringer <lukas.puehringer@nyu.edu>

Signed-off-by: Lukas Puehringer <lukas.puehringer@nyu.edu>

Remove metadata factory on Signed class, for the sake of API
simplicity/non-ambiguity, i.e. it's enough to have one
way of loading any Metadata, that is:
Metadata.read_from_json

Signed-off-by: Lukas Puehringer <lukas.puehringer@nyu.edu>

Consistenly rename de/serialization interface methods, using
a 'from_' and 'to_' prefix.

read_from_json -> from_json_file
write_to_json  -> to_json_file
as_json        -> to_json
as_dict        -> to_dict
signed_bytes   -> to_canonical_bytes

The latter is also changed from a property to a method for
consistency with the other serialization methods.

Signed-off-by: Lukas Puehringer <lukas.puehringer@nyu.edu>

This commit better separates the Metadata class model from the
Metadata wireline format, by tailoring the constructors
towards class-based parameters and adding an additional
factory classmethod that creates Metadata objects based on the
wireline json/dictionary metadata representation. (pythonic
way of constructor overloading).

This 'from_dict' factory method recurses into the 'from_dict'
methods of each contained complex field/attribute that is also
represented by a class. Currently 'signed' is the only such
attribute.

This commit further:
- Changes optional constructor keyword arguments to mandatory
positional arguments: Reduces code and simplifies usage by
restricting it. For now, users are unlikely to call
constructor directly anyway, but the 'from_dict' factory (or
its 'from_json_file' wrapper) instead.

- Removes Signed.__expiration (datetime) vs. Signed.expires
(datestring) dichotomy: Keeping only one representation of the
same attribute in memory makes the interface simpler and less
ambiguous. We choose the datetime object, because it is more
convenient to modify. Transformation from and to the string
format required by the tuf wireline format is performed in the
corresponding metadata de/serialization methods, i.e.
('to_dict' and 'from_dict').

Signed-off-by: Lukas Puehringer <lukas.puehringer@nyu.edu>

Add convenience wrapper that takes a json string and passes it
to from_dict to create a Metadata object.

Signed-off-by: Lukas Puehringer <lukas.puehringer@nyu.edu>

Signed-off-by: Lukas Puehringer <lukas.puehringer@nyu.edu>

Change Metadata.verify(key) behavior to raise an exception if
none or multiple signatures for the passed key are found on the
Metadata object.

Signed-off-by: Lukas Puehringer <lukas.puehringer@nyu.edu>

See:
Add root metadata class to new TUF metadata model theupdateframework#1137
Add classes for complex metadata fields theupdateframework#1139
Add input validation to simple metadata api theupdateframework#1140

Signed-off-by: Lukas Puehringer <lukas.puehringer@nyu.edu>

Use builtin datetime instead of external iso6801 for simple
datetime string parsing. Also see
theupdateframework#1065

Signed-off-by: Lukas Puehringer <lukas.puehringer@nyu.edu>