Added TAP for TUF Version Management #107
Conversation
tap12.md
Outdated
|
|
||
| If the server spec-version is lower than the client spec-version, the client shall terminate the update and report an error. | ||
|
|
||
| If the major version (the first digit) of the spec-version has been incremented, the client must update before proceeding. This could be an automatic process or an error could be reported, requesting a manual client update. |
There was a problem hiding this comment.
Thanks for the TAP, Marina!
Not clear what this means, though. The client must update what from where?
mnm678
changed the title
…andled by a repository
Specifically: allow for delegated targets using an old spec version add a next_spec_version field to root require multiple repositories used by a client (as in tap 4) to use the same spec version. some minor grammar and organizational edits
added a description of use cases to motivation changed the specification to use a directory structure for major version changes
tap12.md
Outdated
|
|
||
| If the major version (the first digit) of the spec-version has been incremented, the client must update before proceeding. This could be an automatic process or an error could be reported, requesting a manual client update. | ||
|
|
||
| If a minor version or patch number of the spec-version has been incremented, the client should report this and may update, but can chose to proceed without further action. |
There was a problem hiding this comment.
| If a minor version or patch number of the spec-version has been incremented, the client should report this and may update, but can chose to proceed without further action. | |
| If a minor version or patch number of the spec-version has been incremented, the client should report this and may update, but can choose to proceed without further action. |
tap12.md
Outdated
|
|
||
| # Security Analysis | ||
|
|
||
| There should be minimal security impact. Ensuring that the client is up to date should improve security in the event that a security vulnerability is patched in a release of the spec. |
There was a problem hiding this comment.
What is the impact of letting the client be one or more minor versions behind? Also, how does the method by which the client gets an updated client impact security?
tap12.md
Outdated
| If the major version (the first digit) of the spec-version has been incremented, the client must update before proceeding. This could be an automatic process or an error could be reported, requesting a manual client update. | ||
|
|
||
| If a minor version or patch number of the spec-version has been incremented, the client should report this and may update, but can chose to proceed without further action. | ||
|
|
There was a problem hiding this comment.
Does the repo ever need to keep multiple versions of metadata? How does this work?
tap12.md
Outdated
|
|
||
| # Specification | ||
|
|
||
| The root metadata already contains the TUF spec-version. The client shall compare the spec-version in the root metadata (server spec-version) with the spec-version of the local client (client spec-version). The client shall then proceed as described below. |
There was a problem hiding this comment.
What if the root file's format changes?
candidate-tuf-versions.md
Outdated
|
|
||
| # Abstract | ||
|
|
||
| This TAP clarifies how to manage updates to the TUF spec that include non-backwards compatible (breaking) changes. Breaking changes mean that a client and server must implement the changes at the same time in order to continue functioning as expected. This TAP will define a procedure for TUF clients to ensure that their version of the TUF spec is compatible with the metadata they download. |
There was a problem hiding this comment.
| This TAP clarifies how to manage updates to the TUF spec that include non-backwards compatible (breaking) changes. Breaking changes mean that a client and server must implement the changes at the same time in order to continue functioning as expected. This TAP will define a procedure for TUF clients to ensure that their version of the TUF spec is compatible with the metadata they download. | |
| This TAP clarifies how to manage updates to the TUF spec that include non-backwards compatible (breaking) changes. Breaking changes mean that a client and server must implement the changes at the same time in order to continue functioning as expected. This TAP defines a procedure for TUF clients to ensure that their version of the TUF spec is compatible with the metadata they download. |
There was a problem hiding this comment.
Is server the right term or is it repo?
candidate-tuf-versions.md
Outdated
|
|
||
| When a TUF client contains a map file for multiple repositories, an additional step must be added to ensure a consensus is possible across repositories that are using different spec versions. In order for this procedure to work, all repositories should maintain metadata according to old spec versions as explained in [How a repository updates](#how-a-repository-updates). | ||
|
|
||
| When parsing a mapping, the client should first look for a threshold as described in [TAP 4](https://github.com/theupdateframework/taps/blob/master/tap4.md) using the client's TUF spec version. If a threshold is not met with this spec version, the client should try with the previous major spec version, and so on until a threshold is reached or the version becomes negative. |
There was a problem hiding this comment.
Why do all files need to have the same spec number? Also, how can a version become negative?
candidate-tuf-versions.md
Outdated
|
|
||
| ## Changes to delegations | ||
|
|
||
| Delegated targets should be stored in directories with their major spec version just like top level metadata on the repository. For each targets metadata file, a TUF client should download the highest supported version. This highest supported version will be found using the same procedure as described in [Changes to the update process](#changes-to-the-update-process). |
There was a problem hiding this comment.
So clients will go into multiple directories to look for targets metadata? What if the version number on the target file that is of the higher spec version is lower than that of another target file? (Notice there is confusion here with the version number of a signed file vs a version number of the spec.)
candidate-tuf-versions.md
Outdated
|
|
||
| Delegated targets should be stored in directories with their major spec version just like top level metadata on the repository. For each targets metadata file, a TUF client should download the highest supported version. This highest supported version will be found using the same procedure as described in [Changes to the update process](#changes-to-the-update-process). | ||
|
|
||
| In order to support this behavior, TUF clients should maintain the ability to parse targets metadata files from old spec versions. This can be done through functions that call sections of the old client version and translate these to be used by the new client. The client may make an old version obsolete if they choose, but will risk being unable to download new targets from delegations that are not updated. |
There was a problem hiding this comment.
Who decides how old of files to accept?
candidate-tuf-versions.md
Outdated
|
|
||
| If the repository is updated to a new minor or patch spec version, this may be done by uploading new metadata files in the new format to the proper directory. So if a repository updates from 2.0.0 to 2.1.0, the 2.1.0 metadata would go in the directory named 2.0.0. Minor version changes are backwards compatible, so clients using version 2.0.0 will still be able to parse metadata written using version 2.1.0. | ||
|
|
||
| A repository may continue to support old major TUF spec versions by creating metadata both in the old location and in the new directory. The repository may maintain as many versions as the repository manager wishes. If there are security concerns with an old spec version, that version should be phased out as soon as possible. The version can be phased out by removing the directory containing that version from the repository. |
There was a problem hiding this comment.
If this happens, how does one follow the chain of root files forward?
candidate-tuf-versions.md
Outdated
|
|
||
| ## How a repository updates | ||
|
|
||
| When a repository manager chooses to update the repository to a new major TUF spec version, they create a new directory on the repository named for the major version (for example 2.0.0). This directory will contain all metadata files with the new spec version. After creating the directory, the repository creates and signs root, snapshot, timestamp, and top level targets metadata using the new TUF spec version and places these metadata files in the directory. Clients will now be able to use these metadata files once their TUF spec versions are also updated. |
There was a problem hiding this comment.
I understand what you propose, but understanding why this is the right approach is also important.
candidate-tuf-versions.md
Outdated
|
|
||
| # Abstract | ||
|
|
||
| This TAP clarifies how to manage updates to the TUF specification that include non-backwards compatible (breaking) changes. A breaking change means that a client and repository must implement the change at the same time in order to continue functioning as expected. This includes any feature that affects the parsing or validation of metadata, or any other changes that mean a client can no longer safely and reliable perform an update if the repository is updated out of sync with the client. To facilitate finding breaking changes, this TAP defines a procedure for TUF clients to ensure that their version of the TUF specification is compatible with the specification version of the metadata they download. This TAP allows clients and repositories to update from their current version of the TUF specification to a version that is not backwards compatible. |
There was a problem hiding this comment.
| This TAP clarifies how to manage updates to the TUF specification that include non-backwards compatible (breaking) changes. A breaking change means that a client and repository must implement the change at the same time in order to continue functioning as expected. This includes any feature that affects the parsing or validation of metadata, or any other changes that mean a client can no longer safely and reliable perform an update if the repository is updated out of sync with the client. To facilitate finding breaking changes, this TAP defines a procedure for TUF clients to ensure that their version of the TUF specification is compatible with the specification version of the metadata they download. This TAP allows clients and repositories to update from their current version of the TUF specification to a version that is not backwards compatible. | |
| This TAP clarifies how to manage updates to the TUF specification that include non-backwards compatible (breaking) changes. A breaking change means that a client and repository must implement the change at the same time in order to continue functioning as expected. This includes any feature that affects the parsing or validation of metadata, or any other changes that mean a client can no longer safely and reliably perform an update if the repository is updated out of sync with the client. To facilitate finding breaking changes, this TAP defines a procedure for TUF clients to ensure that their version of the TUF specification is compatible with the specification version of the metadata they download. This TAP allows clients and repositories to update from their current version of the TUF specification to a version that is not backwards compatible. |
Is the TUF style to line wrap TAPs? I'm finding it slightly painful to suggest edits in this format...
There was a problem hiding this comment.
This TAP allows clients and repositories to update from their current version of the TUF specification to a version that is not backwards compatible.
This phrasing is really odd. It may be easier to explain what we are intending to support. I assume you'll explain how this TAP mandates that repos generate metadata for these cases and how clients can update to a version that supports the right metadata version. You will need to explain also what assumptions you are making (the server will always generate duplicate copies of old metadata, the client updater will always be able to update to support the latest version of TUF, are you handling all of these cases, etc.)
candidate-tuf-versions.md
Outdated
|
|
||
| As they do not affect compatibility, non-breaking changes may happen independently on clients and repositories. Any specification versions that do not contain breaking changes may be updated to at any time without worrying about compatibility. This TAP clarifies how TUF version numbers are formatted so that breaking changes are distinct from non-breaking changes. | ||
|
|
||
| In order to allow clients to update independently of repositories, the repositories need to continue support for old TUF versions for some period of time after updating. In this TAP, repositories support multiple versions by separating versions with breaking changes into different directories. These directories allow a client to choose the most recent metadata they support while allowing for flexibility in how long a client will take to update to a new spec version. Similarly, allowing the client to maintain multiple versions allows the client to use metadata from multiple directories that are not in communication and so do not coordinate an update to the most recent TUF specification. |
There was a problem hiding this comment.
So this is when a client is older than a repo. What about the opposite?
candidate-tuf-versions.md
Outdated
|
|
||
| In order to find compatible updates on a repository, a client must keep track of its version of the TUF specification. To do so, a global variable or other local storage option should contain the client spec version. For simplicity, this field should be formatted according to Semantic Versioning so that it can be directly compared to the spec version in root metadata. | ||
|
|
||
| For compatibility, TUF clients may maintain old versions of the client that support previous spec versions. These old versions can be used if the client downloads metadata from a repository or delegated role that does not support the current TUF specification. To allow for this behavior, when a new version of the TUF client is implemented it may contain the ability to call certain functions from the old TUF client for parsing and validating metadata. |
There was a problem hiding this comment.
For compatibility, TUF clients may maintain old versions of the client
What does "of the client" refer to here?
wrapped all text at 80 chars to enable easier diff viewing for future versions.
|
Hi, this was opened at the end of 2018. Is there any plan to move it to Draft anytime soon? :) |
* Define breaking changes once at the beginning, then use the term consistently * Describe how this TAP relates to TAP 5 * Various minor edits
There was a problem hiding this comment.
Some observations;
- The specification already seems to have moved to semantic versioning
- It would be good to describe how updates should occur on the repository in v1.0.x of the specification, so that the changes within this TAP are more apparent. See Describe how updates should occur on the repository side specification#105
- The specification has been moving away from prescribing POUFs. See Replace strict JSON requirement with a flexible requirement specification#102 and Remove file system requirements specification#103 and tap11
I have mixed feelings about this TAP. On one hand, it seems like a well thought out and logical approach to the problem – kudos for that Marina. On the other, do we really want/need this to be part of the specification? As the specification moves away from recommending wire formats and even repository representations, should it seek to encode concepts of version management? Of course, with my reference implementation maintainer hat on – this would be a complexity and maintenance burden increase in the implementation.
Perhaps a route forward would be to make an advisory TAP for version management and ensure there's sufficient metadata in the specification, and sufficient discipline around version management in the specification, to support it?
candidate-tuf-versions.md
Outdated
| repositories can still coordinate after a non-breaking change occurs. One common | ||
| framework used to separate versions by type is Semantic Versioning. Semantic | ||
| Versioning is a versioning scheme popular across open source projects that | ||
| categorizes specification versions by the scope and criticality of their |
There was a problem hiding this comment.
| categorizes specification versions by the scope and criticality of their | |
| categorizes specification by the scope and criticality of their |
candidate-tuf-versions.md
Outdated
| TAP. | ||
|
|
||
|
|
||
| ## Version Number Format |
There was a problem hiding this comment.
This recommendation of the TAP has already been implemented and documented for the specification https://github.com/theupdateframework/specification#versioning
There was a problem hiding this comment.
Ok, let's link to that and save space if we can.
There was a problem hiding this comment.
I added a link to the Reference Implementation section of the TAP. Do you think I should remove the description here as well?
candidate-tuf-versions.md
Outdated
| For existing TUF clients to continue operation after this TAP is implemented, | ||
| repositories may store metadata from before TUF 1.0.0 in the top level | ||
| repository (with no directory named 0.0.0). This allows existing clients to | ||
| continue downloading metadata from the repository. So a TUF repository that | ||
| upgrades from version 0.12.0 to version 1.0.0 may look like: |
There was a problem hiding this comment.
Agreed. Though the text (and the example below) needs updating as the specification is already at 1.0.x
candidate-tuf-versions.md
Outdated
| recommended that clients maintain functions that can be used to validate | ||
| metadata from previous TUF specification versions. These functions allow a | ||
| client to maintain old versions of the specification while still supporting the | ||
| most recent version. |
There was a problem hiding this comment.
As I'm reading it, only delegated targets files can be a different version. All top-level metadata must be the same specification version.
candidate-tuf-versions.md
Outdated
| period of time after upgrading. This grace period gives existing clients that | ||
| implement old versions of the TUF specification time to implement support for a | ||
| new specification version. Repositories achieve this using a directory structure |
There was a problem hiding this comment.
I wonder whether this TAP should recommend a mechanism for identifying when a repositories metadata is no longer supported or on its way to being deprecated.
If I have a client which only supports specification version 1.0.0, but the repository intends to stop updating 1.0.0 metadata, it might be useful to be able to detect that.
Should repository metadata be able to signal its deprecation timeframe?
There was a problem hiding this comment.
My initial thought was that this communication would happen out of band (a blog post/announcement/etc). But looking at it again, it might be nice to add an optional 'deprecated on' field that the repository can use to indicate that a specification version will no longer be supported after a certain date.
There was a problem hiding this comment.
Agreed, I think it makes sense for the repository to be able to indicate the difference between metadata which has expired and metadata which is intentionally no longer updated.
That might be a field for a deprecation timestamp for a given metadata version, or a "supported versions" metadata field that is replicated across metadata versions?
|
Interesting perspective. I'd like for those who are interested to have a
chat and see if we can understand your perspective better and perhaps
reconcile the differences. Our thought when working on this TAP was we
were strictly making it better for implementers and maintainers, but if
there is some potential we could be doing the opposite, we should better
understand how...
…On Thu, Jul 9, 2020 at 11:07 AM Joshua Lock ***@***.***> wrote:
***@***.**** commented on this pull request.
Some observations;
- The specification already seems to have moved to semantic versioning
<https://github.com/theupdateframework/specification#versioning>
- It would be good to describe how updates should occur on the
repository in v1.0.x of the specification, so that the changes within this
TAP are more apparent. See theupdateframework/specification#105
<theupdateframework/specification#105>
- The specification has been moving away from prescribing POUFs. See
theupdateframework/specification#102
<theupdateframework/specification#102> and
theupdateframework/specification#103
<theupdateframework/specification#103> and
tap11 <https://github.com/theupdateframework/taps/blob/master/tap11.md>
I have mixed feelings about this TAP. On one hand, it seems like a well
thought out and logical approach to the problem – kudos for that Marina. On
the other, do we really want/need this to be part of the specification? As
the specification moves away from recommending wire formats and even
repository representations, should it seek to encode concepts of version
management? Of course, with my reference implementation maintainer hat on –
this would be a complexity and maintenance burden increase in the
implementation.
Perhaps a route forward would be to make an advisory TAP for version
management and ensure there's sufficient metadata in the specification, and
sufficient discipline around version management in the specification, to
support it?
------------------------------
In candidate-tuf-versions.md
<#107 (comment)>
:
> +
+## Use case 5: Backwards compatibility for existing clients
+
+Existing TUF clients (written before this TAP is implemented) will still expect
+to download and parse metadata. Before this TAP there was no method for determining compatibility
+between a client and a repository, so existing TUF repositories should continue
+to distribute metadata using their existing method to support existing clients. This
+means that this mechanism for allowing backwards compatible updates must itself
+be backwards compatible.
+
+# Rationale
+
+We propose this TAP because as TUF continues to evolve, the need for TUF clients
+and repositories to upgrade their TUF metadata format will grow. This sets up the potential for clients
+that are no longer able to perform updates because they cannot parse the
+metadata generated by repositories, or for clients to install the wrong images
⬇️ Suggested change
-metadata generated by repositories, or for clients to install the wrong images
+metadata generated by repositories, or for clients to install the wrong targets
------------------------------
In candidate-tuf-versions.md
<#107 (comment)>
:
> +
+Existing TUF clients (written before this TAP is implemented) will still expect
+to download and parse metadata. Before this TAP there was no method for determining compatibility
+between a client and a repository, so existing TUF repositories should continue
+to distribute metadata using their existing method to support existing clients. This
+means that this mechanism for allowing backwards compatible updates must itself
+be backwards compatible.
+
+# Rationale
+
+We propose this TAP because as TUF continues to evolve, the need for TUF clients
+and repositories to upgrade their TUF metadata format will grow. This sets up the potential for clients
+that are no longer able to perform updates because they cannot parse the
+metadata generated by repositories, or for clients to install the wrong images
+due to changes in TUF. Both of these issues prevent clients from installing the
+images intended by repository managers, and could lead to critical security or
⬇️ Suggested change
-images intended by repository managers, and could lead to critical security or
+targets intended by repository managers, and could lead to critical security or
------------------------------
In candidate-tuf-versions.md
<#107 (comment)>
:
> +whether the TUF specification version they implement is compatible with the TUF
+version of the metadata they receive. This ensures that clients parse
+metadata according to the correct version to prevent errors from any changes to
+the metadata in the new version. Second, clients need a way to use this
+information to determine how to access metadata that is compatible with the
+version of the TUF specification they implement.
+
+To address the first issue, this TAP proposes standardizing the specification
+version field to separate breaking changes from non-breaking changes and
+ensure that the field can be compared across clients and repositories.
+Separating out non-breaking changes allows these backwards compatible changes to
+happen at any time without affecting TUF's operation. Therefore, clients and
+repositories can still coordinate after a non-breaking change occurs. One common
+framework used to separate versions by type is Semantic Versioning. Semantic
+Versioning is a versioning scheme popular across open source projects that
+categorizes specification versions by the scope and criticality of their
⬇️ Suggested change
-categorizes specification versions by the scope and criticality of their
+categorizes specification by the scope and criticality of their
------------------------------
In candidate-tuf-versions.md
<#107 (comment)>
:
> +version of the metadata they receive. This ensures that clients parse
+metadata according to the correct version to prevent errors from any changes to
+the metadata in the new version. Second, clients need a way to use this
+information to determine how to access metadata that is compatible with the
+version of the TUF specification they implement.
+
+To address the first issue, this TAP proposes standardizing the specification
+version field to separate breaking changes from non-breaking changes and
+ensure that the field can be compared across clients and repositories.
+Separating out non-breaking changes allows these backwards compatible changes to
+happen at any time without affecting TUF's operation. Therefore, clients and
+repositories can still coordinate after a non-breaking change occurs. One common
+framework used to separate versions by type is Semantic Versioning. Semantic
+Versioning is a versioning scheme popular across open source projects that
+categorizes specification versions by the scope and criticality of their
+changes. Breaking changes in a specification would warrant a MAJOR version
⬇️ Suggested change
-changes. Breaking changes in a specification would warrant a MAJOR version
+changes. Breaking changes in the specification would warrant a MAJOR version
------------------------------
In candidate-tuf-versions.md
<#107 (comment)>
:
> +# Specification
+
+As stated above, this TAP defines a variety of procedures to be added to the
+update process on both clients and repos. These changes include the way in which
+version numbers are determined, the addition of new directories on repositories,
+new metadata parsing functions on clients, and a new process for finding
+compatible metadata.
+
+For clarity, throughout this TAP *upgrading* refers to the process of moving
+from one TUF specification version to another while *updating* refers to the
+process of downloading and validating packages as specified by TUF. The rest of
+this section details the changes to the TUF specification recommended by this
+TAP.
+
+
+## Version Number Format
This recommendation of the TAP has already been implemented and documented
for the specification
https://github.com/theupdateframework/specification#versioning
------------------------------
In candidate-tuf-versions.md
<#107 (comment)>
:
> +still be able to securely and reliably perform updates from repositories that
+support the TAPs. There may be minor differences as to which updates a client
+installs across minor versions. If a repository maintainer is worried about the
+impact of a minor change on clients, it may choose to wait to implement the
+change with a major specification version (for example, they could remain on
+version 2.4.6 after the release of 2.5). Patches are used to fix typos and make
+small changes to existing features. More details about what constitutes a major,
+minor, or patch change can be found at https://semver.org/.
+
+### Changes to TAPs
+In order to manage changes to TUF, TAPs shall be tied to a version of the TUF
+specification.
+
+Once a TAP is accepted, a TUF Version field in the header should list the first
+TUF version that will include the TAP. The Preamble Header description in TAP 1
+shall be updated to include the TUF Version field.
⬇️ Suggested change
-shall be updated to include the TUF Version field.
+has been updated to include the TUF Version field.
------------------------------
In candidate-tuf-versions.md
<#107 (comment)>
:
> +For existing TUF clients to continue operation after this TAP is implemented,
+repositories may store metadata from before TUF 1.0.0 in the top level
+repository (with no directory named 0.0.0). This allows existing clients to
+continue downloading metadata from the repository. So a TUF repository that
+upgrades from version 0.12.0 to version 1.0.0 may look like:
Agreed. Though the text (and the example below) needs updating as the
specification is already at 1.0.x
------------------------------
In candidate-tuf-versions.md
<#107 (comment)>
:
> +
+The specification that follows adopts the third option, and supports version
+changes on both clients and repositories. This requires both clients and
+repositories to maintain multiple versions of the TUF spec, as well as changes
+to the way repositories store metadata and clients parse this metadata.
+
+For the repositories this means continuing support for old TUF versions for some
+period of time after upgrading. This grace period gives the client time to
+upgrade to a new specification version. Versions with breaking changes are kept
+in separate directories that allow a client to choose the most recent metadata
+they support. To save space, target files should remain in the parent directory
+on the repository. The metadata files (in directories according to their spec
+ version) can point to target files relative to the parent directory.
+
+On the client side, this TAP also requires maintenance of multiple specification
+versions to allow for communication with various repositories. To do so, it is
That's how I'm reading it.
------------------------------
In candidate-tuf-versions.md
<#107 (comment)>
:
> +For repositories, this means continuing support for old TUF versions for some
+period of time after upgrading. This grace period gives existing clients that
+implement old versions of the TUF specification time to implement support for a
+new specification version. Repositories achieve this using a directory structure
+with a directory for each supported TUF specification version. These directories
+contain metadata that supports the given TUF specification version. Using these
+directories, a client is able to choose the most recent metadata they support.
+More details about this directory structure are contained in the [specification](#how-a-repository-updates).
+
+On the client side, this TAP also requires maintenance of multiple versions that
+support different TUF specification
+versions to allow for communication with various repositories. To do so, it is
+recommended that clients maintain functions that can be used to validate
+metadata from previous TUF specification versions. These functions allow a
+client to maintain old versions of the specification while still supporting the
+most recent version.
As I'm reading it, only delegated targets files can be a different
version. All top-level metadata must be the same specification version.
------------------------------
In candidate-tuf-versions.md
<#107 (comment)>
:
> +period of time after upgrading. This grace period gives existing clients that
+implement old versions of the TUF specification time to implement support for a
+new specification version. Repositories achieve this using a directory structure
I wonder whether this TAP should recommend a mechanism for identifying
when a repositories metadata is no longer supported or on its way to being
deprecated.
If I have a client which only supports specification version 1.0.0, but
the repository intends to stop updating 1.0.0 metadata, it might be useful
to be able to detect that.
Should repository metadata be able to signal its deprecation timeframe?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#107 (review)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAGRODYZLRA3AHGMOFAWDLDR2XMLXANCNFSM4GLTCGFA>
.
|
Let's talk about this more in the TAP meeting. Specification version management and backwards compatibility was a bit of a challenge for the reference implementation, so I think it would be nice to at least have some formal guidance about managing spec versions (especially as we move forward with TAPs that include breaking changes). Because a few of these recommendations require changes to TAPs and metadata storage, I think it will be easier to include those changes in the specification. However, if implementations have their own method for version management I don't think these recommendations should supersede those preferences. |
Co-authored-by: Joshua Lock <jlock@vmware.com>
|
I've been thinking about this quite a bit whilst offline and now disagree vehemently with myself from yesterday (re-learning a lesson about reviewing). This does need to be in the specification. I completely agree that this makes things easier for integrators. The reference implementation would want to implement something along these lines, so that we can support our adopters in updating to newer versions of the specification as we improve the standard. I didn't have any doubts about that. Whilst there is a complexity increase for implementers, it is manageable. In the case of the reference implementation this further necessitates the refactor we have been thinking about and validates some of the early design proposals there. Thinking about some of our major adopters that have multiple repositories and/or multiple clients interacting with the repositories (i.e. Docker/Notary and PyPI) I realised that if we don't include this in the specification, we make things potentially trickier for implementers who have to be aware of the advisory TAP, and for implementers who might not recognise the need to upgrade specification compatibility throughout the life of their repository. In short: the complexity increase seems more than worthwhile. Implementers and integrators, particularly those with more controlled update systems where there's only one repository and only one client, can always choose not to handle multiple versions of the specification. Apologies for the noise, I hope you all don't mind me doing so much of my thinking "out loud" here. |
|
Apologies for the noise, I hope you all don't mind me doing so much of my
thinking "out loud" here.
No, this is really great! We want to have this sort of rationale /
thinking captured somewhere. I'm sure that others may have had similar
thoughts and the fact that your thought process is written down clearly may
help others come to the same conclusion.
… |
candidate-tuf-versions.md
Outdated
| In order to allow clients to parse the root metadata chain, root metadata files | ||
| shall not be deleted even once a version is deprecated. | ||
|
|
||
| A repository may indicate the planned phase out of a major version using an |
There was a problem hiding this comment.
@joshuagl I added a deprecation field as discussed. A couple of notes:
- I decided to add the field to targets metadata. I considered root, but changes to root metadata are usually harder to manage (with several offline keys). Also, the targets metadata can be used to indicate deprecation both for top-level metadata and for delegated targets metadata.
- I'm not a big fan of the name
deprecation_timestamp, so am open to any clearer or shorter name suggestions
There was a problem hiding this comment.
Not putting this in root seems sensible. Is Targets the best role to include this in? Would the Snapshot make more sense? Or are you trying to avoid having this in metadata which will often be signed by online keys?
Regarding the name, would becomes_obsolete work better? It's not shorter, but may be clearer?
There was a problem hiding this comment.
I went with targets instead of snapshot/timestamp mostly to allow the field to differ in delegated targets. The only attack that I can think of on this field would be a denial of service (or convincing a client to upgrade early, but it's hard to imagine that being a successful attack), so the online keys shouldn't be too much of a problem. However in this design delegated targets may use a different TUF specification version than the top-level metadata, so having this field for each targets metadata file seems useful. For example an individual delegated targets file could stop supporting version 1.x before the top-level metadata, so this field would allow them to indicate this change to the client.
I like becomes_obsolete, I'll update that in the text.
There was a problem hiding this comment.
Ah, yes. I forgot about the desire to support different metadata versions in delegated targets.
I was thinking it might make more sense in an earlier role in the client workflow. If we have root, timestamp or snapshot metadata which has expired a client can't know whether that's because the metadata is obsolete, or whether something else happened (because we check the expiration on root, timestamp and snapshot and abort if the expiration has passed long before we ever download targets).
My hope was that this metadata field would
- inform clients when a version of the metadata which hasn't expired is due to be deprecated – this use-case appears to be supported by having
becomes_obsoletein targets metadata - allow clients to figure out whether metadata that has expired is due to deprecation or something more worrisome – this use-case does not seem to be supported by having
becomes_obsoletein targets metadata
There was a problem hiding this comment.
We could also add this field to the timestamp metadata to address case 2 (with the caveat that this would not help if root metadata expires). This would also ensure that the client checks this field even if no target files have been updated (when they might not download targets metadata).
There was a problem hiding this comment.
I still have reservations about this not being in the root metadata – how do clients detect unintentionally expired root metadata (or an attack) vs. obsolete root metadata? Or, phrased differently, how does a client know when updates may be available, just at a different end-point, versus updates being unavailable?
I acknowledge your concerns about including becomes_obsolete in the root metadata being harder to manage. Updating metadata specification versions is a big thing, but should it require a quorum of the root signers and their offline keys? 🤷
The reason I keep belabouring this point is because, to my mind, this is an aspect of TUF's freshness security design principle. The TUF website states that:
a client should be able to recognize that updates may exist that they haven't been able to obtain.
I'd argue that's not possible if the client can't detect the difference between root metadata expiring and root metadata being marked as obsolete.
There was a problem hiding this comment.
I've read and thought about this and I can see why we're having an extended discussion on this point. I think this is a hard decision.
If you believe the repository is the arbiter of truth and trust, then keeping it in the root role makes sense. This does effectively let the repository's root role deprecate large swaths of targets files though. This can be good or bad.
If you think that the developer keys (targets metadata files) are what should be trusted, then having a field in targets makes sense. (I would argue for version number instead of timestamp since there isn't a global clock / sync requirement for most of TUF except perhaps to print a warning about outdated timestamp metadata.) This allows parties that delegate make the choice about what versions of metadata are alright. I do feel like this option means that no one will ever use this and that old metadata will be kept around indefinitely, because I just can't see developers worrying about using this flag...
Anyways, this is just my brain dump on the subject...
There was a problem hiding this comment.
how do clients detect unintentionally expired root metadata (or an attack) vs. obsolete root metadata?
Root metadata is designed to expire fairly infrequently, so clients would have a long time (potentially a year or more) to notice the becomes_obsolete field in timestamp metadata. However, I agree that updating the specification should be done with the knowledge of the root signers, so maybe including this field in root might be reasonable as it is rarely updated.
If you believe the repository is the arbiter of truth and trust, then keeping it in the root role makes sense. This does effectively let the repository's root role deprecate large swaths of targets files though. This can be good or bad.
I think regardless of whether this field goes in timestamp or root, it also needs to exist in targets metadata for this reason. In this design delegated targets metadata can use a different specification version than the top-level roles, so they need their own way to indicate deprecation. This is especially important when targets metadata is replicated across multiple repositories.
I would argue for version number instead of timestamp since there isn't a global clock / sync requirement for most of TUF except perhaps to print a warning about outdated timestamp metadata.
This may be another argument in favor of putting this field in root. Root metadata is always versioned whereas timestamp metadata is only versioned if consistent snapshots are used.
There was a problem hiding this comment.
(I would argue for version number instead of timestamp since there isn't a global clock / sync requirement for most of TUF except perhaps to print a warning about outdated timestamp metadata.)
Is that true? The detailed client workflow in the latest spec states:
1.8. Check for a freeze attack. The latest known time should be lower than the expiration timestamp in the trusted root metadata file (version N). If the trusted root metadata file has expired, abort the update cycle, report the potential freeze attack. On the next update cycle, begin at step 0 and version N of the root metadata file.
I'm not sure how checking for a becomes_obsolete date in the metadata is any different from the requirement here to abandon an update if the root metadata file appears to have expired?
There was a problem hiding this comment.
Thinking about this more, I think that a timestamp might be more useful to clients than a version number as it allows them to make a plan for upgrading, especially in airgapped environments or anywhere else when upgrading might take some planning.
So, per this discussion I will update the TAP to include the becomes_obsolete field in root and targets, and leave its contents as the timestamp for now. This field is optional so if updating root or using a timestamp is an issue, it could be left out.
candidate-tuf-versions.md
Outdated
| In order to allow clients to parse the root metadata chain, root metadata files | ||
| shall not be deleted even once a version is deprecated. | ||
|
|
||
| A repository may indicate the planned phase out of a major version using an |
There was a problem hiding this comment.
Not putting this in root seems sensible. Is Targets the best role to include this in? Would the Snapshot make more sense? Or are you trying to avoid having this in metadata which will often be signed by online keys?
Regarding the name, would becomes_obsolete work better? It's not shorter, but may be clearer?
Co-authored-by: Joshua Lock <jlock@vmware.com>
Signed-off-by: marinamoore <mmoore32@calpoly.edu>
candidate-tuf-versions.md
Outdated
| recommended that clients maintain functions that can be used to validate | ||
| metadata from previous TUF specification versions. These functions allow a | ||
| client to maintain old versions of the specification while still supporting the | ||
| most recent version. |
There was a problem hiding this comment.
What do you mean by "clients"? I don't think you mean the TUF reference implementation itself, but clients that use them in turn, right?
candidate-tuf-versions.md
Outdated
| TAP. | ||
|
|
||
|
|
||
| ## Version Number Format |
There was a problem hiding this comment.
Ok, let's link to that and save space if we can.
candidate-tuf-versions.md
Outdated
| and so clients and repositories need to use code that supports the same major | ||
| version when performing an update in order to maintain security and functionality. | ||
|
|
||
| If the change adds a new feature that is backwards compatible, for example in |
There was a problem hiding this comment.
Could you list an example backwards-compatible new feature that qualifiers for a minor version? I'm hard-pressed to think of one right now.
There was a problem hiding this comment.
TAPs 13 and 4 are both backwards-compatible new features. TAP 13 is an especially good example because it is a change that only affects the client.
candidate-tuf-versions.md
Outdated
|
|
||
| ``` | ||
| - Targets files | ||
| - 1.0.0 metadata files |
There was a problem hiding this comment.
| - 1.0.0 metadata files | |
| - 1.0.0 | |
| |- 1.0.0 metadata files |
There was a problem hiding this comment.
The metadata files actually go in the top level directory to allow for backwards compatibility with existing 1.0.0 implementations that do not know about this directory structure.
candidate-tuf-versions.md
Outdated
| TUF clients must store the TUF specification versions they support and may add | ||
| functions to maintain old versions. In order to find compatible updates on a | ||
| repository, a client must keep track of the TUF specification versions it | ||
| supports. To do so, a global variable or other local storage option should |
There was a problem hiding this comment.
Let's not recommend global variables, please.
candidate-tuf-versions.md
Outdated
| * Otherwise, the client should warn the user that the TUF specification version | ||
| they are using will be deprecated at the time indicated by `becomes_obsolete` | ||
| and proceed with the update. | ||
|
|
candidate-tuf-versions.md
Outdated
| client from updating. However, this denial of service attack is always possible | ||
| for an attacker on the network with the ability to alter network traffic. | ||
|
|
||
| # Backwards Compatibility |
There was a problem hiding this comment.
I think you should mention how current repositories and clients should continue to keep their current metadata and targets directories, for clients that predate this TAP.
candidate-tuf-versions.md
Outdated
| update. In this way, updates to the repository's TUF version do not impact the | ||
| security of an update. | ||
|
|
||
| ## How a client upgrades |
There was a problem hiding this comment.
That is a new level of meta. Love it.
|
Thanks for the review @trishankatdatadog! I responded to a couple of comments inline. @JustinCappos You're still labeled as requesting a change, is there anything else I should address before this gets merged as a draft? |
Signed-off-by: marinamoore <mnm678@gmail.com>
Done! I think the number 14 avoids any conflicts. |
This TAP clarifies how breaking changes can be implemented in TUF and ensures that clients are aware of these changes.