From 0b4354505c6722521dcfefd6211e32f96ee349e3 Mon Sep 17 00:00:00 2001 From: William Woodruff Date: Tue, 9 Jan 2024 17:18:28 -0500 Subject: [PATCH 01/17] peps: add draft PEP for index digital attestations Signed-off-by: William Woodruff --- peps/pep-9999.rst | 259 ++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 259 insertions(+) create mode 100644 peps/pep-9999.rst diff --git a/peps/pep-9999.rst b/peps/pep-9999.rst new file mode 100644 index 00000000000..a20908a14da --- /dev/null +++ b/peps/pep-9999.rst @@ -0,0 +1,259 @@ +PEP: 9999 +Title: Index support for digital attestations +Author: William Woodruff +Sponsor: +PEP-Delegate: +Discussions-To: +Status: Draft +Type: Informational +Topic: Packaging +Content-Type: text/x-rst +Created: 08-Jan-2024 + +Abstract +======== + +This PEP proposes a collection of changes related to the upload and distribution +of digitally signed attestations on a Python package repository, such as PyPI. + +These changes have two subcomponents: + +* Changes to the currently unstandardized PyPI upload API; +* Changes to the :pep:`503` and :pep:`691` "simple" APIs. + +This PEP does not recommend a specific digital attestation format, nor does +it make a policy recommendation around mandatory digital attestations on +release uploads or their subsequent verification by installing clients like +``pip``. + +Rationale +========= + +Desire for digital signatures on Python packages has been repeatedly +expressed by both package maintainers and downstream users: + +* Maintainers wish to demonstrate the integrity and authenticity of their + package uploads; +* Individual downstream users wish to verify package integrity and authenticity + without placing additional trust in their index's honesty; +* "Bulk" downstream users (such as Operating System distributions) wish to + perform similar verifications and potentially re-expose or countersign + for their own downstream packaging ecosystems. + +This proposal seeks to accommodate each of the above use cases. + +Design Considerations +--------------------- + +This PEP identifies the following design considerations when evaluating +both its own proposed changes and previous work in the same or adjacent +areas of Python packaging: + +1. Index accessibility: digital attestations for Python packages + are ideally retrievable directly from the index itself, as "detached" + resources. + + This both simplifies some compatibility concerns (by avoiding + the need to modify the distribution formats themselves) and also simplifies + the behavior of potential installing clients (by allowing them to + retrieve each attestation before its corresponding package without needing + to do streaming decompression). + +2. Verification by the index itself: in addition to enabling verification + by installing clients, each digital attestation is *ideally* verifiable + in some form by the index itself. + + This both increases the overall quality + of attestations uploaded to the index (preventing, for example, users + from accidentally uploading incorrect or invalid attestations) and also + enables UI and UX refinements on the index itself (such as a "provenance" + view for each uploaded package). + +3. General applicability: digital attestations should be applicable to + *any and every* package uploaded to the index, regardless of its format + (sdist or wheel) or interior contents. + +4. Metadata support: this PEP refers to "digital attestations" rather than + just "digital signatures" to emphasize the ideal presence of additional + metadata within the cryptographic envelope. + + For example, to prevent domain separation between a distribution's name and + its contents, the digital attestation could be performed over + ``HASH(name || HASH(contents))`` rather than just ``HASH(contents)``. + +Previous Work +------------- + +PGP signatures +^^^^^^^^^^^^^^ + +PyPI and other indices have historically supported PGP signatures on uploaded +distributions. These could be supplied during upload, and could be retrieved +by installing clients via the ``data-gpg-sig`` attribute in the :pep:`503` +API, the ``gpg-sig`` key on the :pep:`691` API, or via an adjacent +``.asc``-suffixed URL. + +PGP signature uploads have been disabled on PyPI since +`May 2023 `_, after +`an investigation `_ +determined that the majority of signatures (which, themselves, constituted a +tiny percentage of overall uploads) could not be associated with a public key or +otherwise meaningfully verified. + +In their previously supported form on PyPI, PGP signatures satisfied +considerations (1) and (3) above but not (2) (owing to the need for external +keyservers and key distribution) or (4) (due to PGP signatures typically being +constructed over just an input file, without any associated signed metadata). + +:pep:`427` wheel signatures +^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +:pep:`427` (and its :ref:`living counterpart `) +specify the :term:`wheel format `. + +This format includes accommodations for digital signatures embedded directly +into the wheel, in either JWS or S/MIME format. These signatures are specified +over a :pep:`376` RECORD, which is modified to include a cryptographic digest +for each recorded file in the wheel. + +While wheel signatures are fully specified, they do not appear to be broadly +used; the official `wheel tooling `_ deprecated +signature generation and verification support +`in 0.32.0 `_, which was +released in 2018. Additionally, wheel signatures do not satisfy any of +the above considerations (due to the "attached" nature of the signatures, +non-verifiability on the index itself, and support for wheels only). + +Specification +============= + +Upload endpoint changes +----------------------- + +The current upload API is not standardized. However, we propose the following +changes to it: + +* In addition to the current top-level ``content`` and ``gpg_signature`` fields, + the index **SHALL** accept ``attestations`` as an additional multipart form + field. +* The new ``attestations`` field **SHALL** be a JSON object. +* The JSON object **SHALL** have one or more keys, each identifying an + attestation format known to the index. If any key does not identify an + attestation format known to the index, the index **MUST** reject the upload. +* The value associated with each well-known key **SHALL** be a JSON object. +* Each attestation value **MUST** be verifiable by the index. If the index fails + to verify any attestation in ``attestations``, it **MUST** reject the upload. + +In addition to the above, the index **SHALL** enforce a consistency +policy for release attestations via the following: + +* If the first file under a new release is supplied with ``attestations``, + then all subsequently uploaded files under the same release **MUST** also + have ``attestations``. Conversely, if the first file under a new release + does not have any ``attestations``, then all subsequent uploads under the + same release **MUST NOT** have ``attestations``. +* All files under the same release **MUST** have the same set of well-known + attestation format keys. + +The index **MUST** reject any file upload that does not satisfy these +consistency properties. + +Index changes +------------- + +:pep:`503` +^^^^^^^^^^ + +* When an uploaded file has an attestation, the index **MAY** include a + ``data-attestations`` attribute on its file link, with a value of either + ``true`` or ``false``. +* When ``data-attestations`` is ``true``, the index **MUST** serve the uploaded + file's attestations at the same URL, but with ``.attestations`` appended to + it. For example, if ``HolyGrail-1.0.tar.gz`` existed and had associated + attestations, those attestations would be located at + ``HolyGrail-1.0.tar.gz.attestations``. +* The attestations served by the index **SHALL** be in JSON format, with the + same well-known keys as accepted by the upload endpoint. + +.. TODO: Represent the trusted publisher metadata here, somehow. + +:pep:`691` +^^^^^^^^^^ + +* When an uploaded file has an attestation, the index **MAY** include an + ``attestations`` key in the ``file`` dictionary for that file. +* The ``attestations`` key, when present, **MUST** be a JSON object with + the same well-known keys and values as accepted by the upload endpoint. + +In addition to the above, when an uploaded file was uploaded via +`Trusted Publishing `_: + +* The ``file`` dictionary **MAY** include a ``publisher`` key. +* The ``publisher`` key, when present, **MUST** be a JSON object with a + representation of the file's Trusted Publisher configuration at the time + of upload. +* The keys within the ``publisher`` object are specific to each + Trusted Publisher but include, at minimum, a ``type`` key uniquely identifying + the type of Trusted Publisher. For example, for a GitHub-based Trusted + Publisher, the ``type`` may be the string ``"GitHub"``. + +Security Implications +===================== + +This PEP is "mechanical" in nature; it provides only the plumbing for future +digital attestations on package indices, without specifying their concrete +cryptographic details. + +As such, we do not identify any positive or negative security implications +for this PEP. + +Appendix 1: Example Uploaded Attestations +========================================= + +This appendix provides a fictional example of the ``attestations`` field +submitted on file upload, with two fictional attestations (``publish`` and +``timestamp``): + +.. code-block:: json + + { + "publish": { + "mediaType": "application/vnd.dev.sigstore.bundle+json;version=0.2", + "verificationMaterial": { /* omitted for brevity */ }, + "messageSignature": { + "messageDigest": { + "algorithm": "some-hash-algo", + "digest": "digest-here" + }, + "signature": "signature-here" + } + }, + "timestamp": { + "cms": "some-long-blob-here" + } + } + +The payloads of these fictional attestations are purely illustrative. + +Appendix 2: Example Trusted Publisher Representation +==================================================== + +This appendix provides a fictional example of a ``publisher`` key within +a :pep:`691` ``project.files[]`` listing: + +.. code-block:: json + + "publisher": { + "type": "GitHub", + "repository_name": "HolyGrail", + "repository_owner": "octocat", + "repository_owner_id": "1", + "workflow_filename": "publish.yml", + "environment": null + } + +Copyright +========= + +This document is placed in the public domain or under the +CC0-1.0-Universal license, whichever is more permissive. From cce1c960b75708e5805d3bf98437d4074788bf97 Mon Sep 17 00:00:00 2001 From: William Woodruff Date: Tue, 9 Jan 2024 17:24:21 -0500 Subject: [PATCH 02/17] peps: remove pep refs in headers Signed-off-by: William Woodruff --- peps/pep-9999.rst | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/peps/pep-9999.rst b/peps/pep-9999.rst index a20908a14da..3730806a025 100644 --- a/peps/pep-9999.rst +++ b/peps/pep-9999.rst @@ -161,8 +161,8 @@ consistency properties. Index changes ------------- -:pep:`503` -^^^^^^^^^^ +Simple Index +^^^^^^^^^^^^ * When an uploaded file has an attestation, the index **MAY** include a ``data-attestations`` attribute on its file link, with a value of either @@ -177,8 +177,8 @@ Index changes .. TODO: Represent the trusted publisher metadata here, somehow. -:pep:`691` -^^^^^^^^^^ +JSON-based Simple API +^^^^^^^^^^^^^^^^^^^^^ * When an uploaded file has an attestation, the index **MAY** include an ``attestations`` key in the ``file`` dictionary for that file. From 8538ad1045f1c4d1f06985f30085321dc7a3057f Mon Sep 17 00:00:00 2001 From: William Woodruff Date: Wed, 10 Jan 2024 10:40:46 -0500 Subject: [PATCH 03/17] temporary discussion link Signed-off-by: William Woodruff --- peps/pep-9999.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/peps/pep-9999.rst b/peps/pep-9999.rst index 3730806a025..dafc76ff2ba 100644 --- a/peps/pep-9999.rst +++ b/peps/pep-9999.rst @@ -3,7 +3,7 @@ Title: Index support for digital attestations Author: William Woodruff Sponsor: PEP-Delegate: -Discussions-To: +Discussions-To: https://discuss.python.org/t/pre-pep-exposing-trusted-publisher-provenance-on-pypi/42337/40 Status: Draft Type: Informational Topic: Packaging From 105ef93cea4db75763cf38a36ee682e896dbe67c Mon Sep 17 00:00:00 2001 From: William Woodruff Date: Wed, 10 Jan 2024 11:46:11 -0500 Subject: [PATCH 04/17] Apply suggestions from code review Co-authored-by: Hugo van Kemenade <1324225+hugovk@users.noreply.github.com> --- peps/pep-9999.rst | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/peps/pep-9999.rst b/peps/pep-9999.rst index dafc76ff2ba..6e770486551 100644 --- a/peps/pep-9999.rst +++ b/peps/pep-9999.rst @@ -3,11 +3,10 @@ Title: Index support for digital attestations Author: William Woodruff Sponsor: PEP-Delegate: -Discussions-To: https://discuss.python.org/t/pre-pep-exposing-trusted-publisher-provenance-on-pypi/42337/40 +Discussions-To: https://discuss.python.org/t/pre-pep-exposing-trusted-publisher-provenance-on-pypi/42337 Status: Draft Type: Informational Topic: Packaging -Content-Type: text/x-rst Created: 08-Jan-2024 Abstract From da927e764d956af4bac9d5e04e64e0ab6f35e394 Mon Sep 17 00:00:00 2001 From: William Woodruff Date: Wed, 10 Jan 2024 17:44:32 -0500 Subject: [PATCH 05/17] remove empty Sponsor and PEP-Delegate for now Signed-off-by: William Woodruff --- peps/pep-9999.rst | 2 -- 1 file changed, 2 deletions(-) diff --git a/peps/pep-9999.rst b/peps/pep-9999.rst index 6e770486551..07ff60e1af8 100644 --- a/peps/pep-9999.rst +++ b/peps/pep-9999.rst @@ -1,8 +1,6 @@ PEP: 9999 Title: Index support for digital attestations Author: William Woodruff -Sponsor: -PEP-Delegate: Discussions-To: https://discuss.python.org/t/pre-pep-exposing-trusted-publisher-provenance-on-pypi/42337 Status: Draft Type: Informational From 8ca38463a55073c89b5b64358974a43bc1b32f8c Mon Sep 17 00:00:00 2001 From: William Woodruff Date: Wed, 10 Jan 2024 17:46:43 -0500 Subject: [PATCH 06/17] remove another :pep: in section title Signed-off-by: William Woodruff --- peps/pep-9999.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/peps/pep-9999.rst b/peps/pep-9999.rst index 07ff60e1af8..c8dc2e5b19e 100644 --- a/peps/pep-9999.rst +++ b/peps/pep-9999.rst @@ -102,8 +102,8 @@ considerations (1) and (3) above but not (2) (owing to the need for external keyservers and key distribution) or (4) (due to PGP signatures typically being constructed over just an input file, without any associated signed metadata). -:pep:`427` wheel signatures -^^^^^^^^^^^^^^^^^^^^^^^^^^^ +Wheel signatures +^^^^^^^^^^^^^^^^ :pep:`427` (and its :ref:`living counterpart `) specify the :term:`wheel format `. From 63169a1e951e2b7645c29ab745029fa4de4c1ead Mon Sep 17 00:00:00 2001 From: William Woodruff Date: Thu, 11 Jan 2024 15:07:35 -0500 Subject: [PATCH 07/17] feedback Signed-off-by: William Woodruff --- peps/pep-9999.rst | 39 ++++++++++++++++++++++++++++++++++----- 1 file changed, 34 insertions(+), 5 deletions(-) diff --git a/peps/pep-9999.rst b/peps/pep-9999.rst index c8dc2e5b19e..55e715e7862 100644 --- a/peps/pep-9999.rst +++ b/peps/pep-9999.rst @@ -11,12 +11,17 @@ Abstract ======== This PEP proposes a collection of changes related to the upload and distribution -of digitally signed attestations on a Python package repository, such as PyPI. +of digitally signed attestations and metadata used to verify them on a Python +package repository, such as PyPI. These changes have two subcomponents: -* Changes to the currently unstandardized PyPI upload API; -* Changes to the :pep:`503` and :pep:`691` "simple" APIs. +* Changes to the currently unstandardized PyPI upload API, allowing clients + to upload digital attestations; +* Changes to the :pep:`503` and :pep:`691` "simple" APIs, allowing clients + to retrieve both digital attestations and + `Trusted Publishing `_ metadata + for individual release files. This PEP does not recommend a specific digital attestation format, nor does it make a policy recommendation around mandatory digital attestations on @@ -39,6 +44,13 @@ expressed by both package maintainers and downstream users: This proposal seeks to accommodate each of the above use cases. +While this PEP does not recommend a specific digital attestation format, +it does recognize the utility of Trusted Publishing as a pre-existing, +"zero-configuration" source of strong provenance for Python packages. +Consequently this PEP includes a proposed scheme for exposing each release +file's Trusted Publisher metadata, with the expectation that a future digital +attestation format will likely require it. + Design Considerations --------------------- @@ -78,6 +90,17 @@ areas of Python packaging: its contents, the digital attestation could be performed over ``HASH(name || HASH(contents))`` rather than just ``HASH(contents)``. +5. Consistent release attestations: if a file belonging to a release has a + set of digital attestations, then all of the other files belonging to that + release should also have the same types of attestations. + + This simplifies the downstream use story for digital attestations, and + prevents potentially vulnerable "swiss cheese" release patterns (where + a verifier checks for a valid attestation on ``HolyGrail-1.0.tar.gz`` + but their installing client actually resolves an attacker-controlled, + platform-specific ``.whl`` instead). + + Previous Work ------------- @@ -101,11 +124,15 @@ In their previously supported form on PyPI, PGP signatures satisfied considerations (1) and (3) above but not (2) (owing to the need for external keyservers and key distribution) or (4) (due to PGP signatures typically being constructed over just an input file, without any associated signed metadata). +Similarly, PyPI's historical implementation of PGP did not satisfy consideration +(5), due to a lack of consistency checks between different release files +(and an inability to perform those checks due to no access to the signer's +public key). Wheel signatures ^^^^^^^^^^^^^^^^ -:pep:`427` (and its :ref:`living counterpart `) +:pep:`427` (and its :ref:`living PyPA counterpart `) specify the :term:`wheel format `. This format includes accommodations for digital signatures embedded directly @@ -117,7 +144,9 @@ While wheel signatures are fully specified, they do not appear to be broadly used; the official `wheel tooling `_ deprecated signature generation and verification support `in 0.32.0 `_, which was -released in 2018. Additionally, wheel signatures do not satisfy any of +released in 2018. + +Additionally, wheel signatures do not satisfy any of the above considerations (due to the "attached" nature of the signatures, non-verifiability on the index itself, and support for wheels only). From af8c6046443d8a02d5921569fc7d5751ea31d036 Mon Sep 17 00:00:00 2001 From: William Woodruff Date: Thu, 11 Jan 2024 15:55:10 -0500 Subject: [PATCH 08/17] add sponsor, PEP-Delegate Signed-off-by: William Woodruff --- peps/pep-9999.rst | 2 ++ 1 file changed, 2 insertions(+) diff --git a/peps/pep-9999.rst b/peps/pep-9999.rst index 55e715e7862..40f56c7747b 100644 --- a/peps/pep-9999.rst +++ b/peps/pep-9999.rst @@ -1,6 +1,8 @@ PEP: 9999 Title: Index support for digital attestations Author: William Woodruff +Sponsor: Donald Stufft +PEP-Delegate: Donald Stufft Discussions-To: https://discuss.python.org/t/pre-pep-exposing-trusted-publisher-provenance-on-pypi/42337 Status: Draft Type: Informational From fddbf6d61c189d371ad5dc561db44b5ba794d94b Mon Sep 17 00:00:00 2001 From: William Woodruff Date: Thu, 11 Jan 2024 15:56:36 -0500 Subject: [PATCH 09/17] reword Signed-off-by: William Woodruff --- peps/pep-9999.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/peps/pep-9999.rst b/peps/pep-9999.rst index 40f56c7747b..fa8d5057f5d 100644 --- a/peps/pep-9999.rst +++ b/peps/pep-9999.rst @@ -51,7 +51,7 @@ it does recognize the utility of Trusted Publishing as a pre-existing, "zero-configuration" source of strong provenance for Python packages. Consequently this PEP includes a proposed scheme for exposing each release file's Trusted Publisher metadata, with the expectation that a future digital -attestation format will likely require it. +attestation format will likely make use of it. Design Considerations --------------------- From 3543a8ecb3c835b8cc499e165b63991bba311a38 Mon Sep 17 00:00:00 2001 From: William Woodruff Date: Fri, 12 Jan 2024 09:55:17 -0500 Subject: [PATCH 10/17] use my ToB email Signed-off-by: William Woodruff --- peps/pep-9999.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/peps/pep-9999.rst b/peps/pep-9999.rst index fa8d5057f5d..b04e8b65c8b 100644 --- a/peps/pep-9999.rst +++ b/peps/pep-9999.rst @@ -1,6 +1,6 @@ PEP: 9999 Title: Index support for digital attestations -Author: William Woodruff +Author: William Woodruff Sponsor: Donald Stufft PEP-Delegate: Donald Stufft Discussions-To: https://discuss.python.org/t/pre-pep-exposing-trusted-publisher-provenance-on-pypi/42337 From f411b48108da962f49bd95845682be911b8dc029 Mon Sep 17 00:00:00 2001 From: Facundo Tuesca Date: Fri, 12 Jan 2024 15:57:38 +0100 Subject: [PATCH 11/17] Add author --- peps/pep-9999.rst | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/peps/pep-9999.rst b/peps/pep-9999.rst index b04e8b65c8b..2d2c518ff0f 100644 --- a/peps/pep-9999.rst +++ b/peps/pep-9999.rst @@ -1,6 +1,7 @@ PEP: 9999 Title: Index support for digital attestations -Author: William Woodruff +Author: William Woodruff , + Facundo Tuesca Sponsor: Donald Stufft PEP-Delegate: Donald Stufft Discussions-To: https://discuss.python.org/t/pre-pep-exposing-trusted-publisher-provenance-on-pypi/42337 From fa2063af61bf135b050f6706665f31e80348a566 Mon Sep 17 00:00:00 2001 From: William Woodruff Date: Fri, 12 Jan 2024 10:09:45 -0500 Subject: [PATCH 12/17] Update peps/pep-9999.rst Co-authored-by: Hugo van Kemenade <1324225+hugovk@users.noreply.github.com> --- peps/pep-9999.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/peps/pep-9999.rst b/peps/pep-9999.rst index 2d2c518ff0f..3c6dc4d2a9c 100644 --- a/peps/pep-9999.rst +++ b/peps/pep-9999.rst @@ -1,4 +1,4 @@ -PEP: 9999 +PEP: 740 Title: Index support for digital attestations Author: William Woodruff , Facundo Tuesca From 9da483517d26975bda356789e11821485fed282f Mon Sep 17 00:00:00 2001 From: William Woodruff Date: Fri, 12 Jan 2024 10:10:42 -0500 Subject: [PATCH 13/17] PEP 740, owners Signed-off-by: William Woodruff --- .github/CODEOWNERS | 1 + peps/{pep-9999.rst => pep-0740.rst} | 0 2 files changed, 1 insertion(+) rename peps/{pep-9999.rst => pep-0740.rst} (100%) diff --git a/.github/CODEOWNERS b/.github/CODEOWNERS index ebd54cd877f..f0146888f8f 100644 --- a/.github/CODEOWNERS +++ b/.github/CODEOWNERS @@ -614,6 +614,7 @@ peps/pep-0733.rst @encukou @vstinner @zooba @iritkatriel peps/pep-0734.rst @ericsnowcurrently peps/pep-0735.rst @brettcannon peps/pep-0737.rst @vstinner +peps/pep-0740.rst @dstufft # ... # peps/pep-0754.rst # ... diff --git a/peps/pep-9999.rst b/peps/pep-0740.rst similarity index 100% rename from peps/pep-9999.rst rename to peps/pep-0740.rst From f8e74f15bcba70c153814ca5b6cada7e0528adc2 Mon Sep 17 00:00:00 2001 From: William Woodruff Date: Fri, 12 Jan 2024 10:13:21 -0500 Subject: [PATCH 14/17] PEP 740: use my personal email (again) Apparently the PEP index can't handle multiple emails for a single human name. Signed-off-by: William Woodruff --- peps/pep-0740.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/peps/pep-0740.rst b/peps/pep-0740.rst index 3c6dc4d2a9c..e65b79edc64 100644 --- a/peps/pep-0740.rst +++ b/peps/pep-0740.rst @@ -1,6 +1,6 @@ PEP: 740 Title: Index support for digital attestations -Author: William Woodruff , +Author: William Woodruff , Facundo Tuesca Sponsor: Donald Stufft PEP-Delegate: Donald Stufft From 1c476a8f3f025abb3043bb1c9a1ea2d4a41772ce Mon Sep 17 00:00:00 2001 From: William Woodruff Date: Tue, 16 Jan 2024 14:54:50 -0500 Subject: [PATCH 15/17] 740: attestation recommendations Signed-off-by: William Woodruff --- peps/pep-0740.rst | 29 +++++++++++++++++++++++++++++ 1 file changed, 29 insertions(+) diff --git a/peps/pep-0740.rst b/peps/pep-0740.rst index e65b79edc64..e9a00cae1e0 100644 --- a/peps/pep-0740.rst +++ b/peps/pep-0740.rst @@ -236,6 +236,35 @@ cryptographic details. As such, we do not identify any positive or negative security implications for this PEP. +Recommendations +=============== + +This PEP does not recommend specific attestation formats. It does, +however, make the following recommendations to package indices seeking +to create new or implement pre-existing attestation formats: + +1. Consult the :ref:`living PyPA specifications ` + first, and determine if any currently defined attestation formats suit + your purpose. +2. If no suitable attestation format is defined under the PyPA specifications, + consider submitting it to the PyPA specifications for longevity and reuse + purposes. + +When designing a new attestation format, we make the following recommendations: + +1. Pick a short, but unique name for your attestation format; this name will + serve as the attestation's identifier in the upload and index APIs. + + When appropriate for an attestation format, we recommend using ``:`` as a + domain separator. For example, an attestation format that provides publish + provenance using `Sigstore `_ might have the + name ``sigstore:publish``. +2. Prefer parsimony in your format: avoid optional fields and functionality, + avoid unnecessary cryptographic agility and message malleability, and ensure + that verifying the attestation communicates something meaningful beyond a + basic integrity check (since the index itself already supplies cryptographic + digests for this purpose). + Appendix 1: Example Uploaded Attestations ========================================= From 388a999fdce285d124ddd1b12bdad8505b91ac4b Mon Sep 17 00:00:00 2001 From: William Woodruff Date: Tue, 16 Jan 2024 15:00:24 -0500 Subject: [PATCH 16/17] 740: index trust subsection Signed-off-by: William Woodruff --- peps/pep-0740.rst | 13 +++++++++++++ 1 file changed, 13 insertions(+) diff --git a/peps/pep-0740.rst b/peps/pep-0740.rst index e9a00cae1e0..5ff7bcefa41 100644 --- a/peps/pep-0740.rst +++ b/peps/pep-0740.rst @@ -236,6 +236,19 @@ cryptographic details. As such, we do not identify any positive or negative security implications for this PEP. +Index trust +----------- + +This PEP does **not** increase (or decrease) trust in the index itself: +the index is still effectively trusted to honestly deliver unmodified package +distributions, since a dishonest index capable of modifying package +contents could also dishonestly modify or omit package attestations. +As a result, this PEP's presumption of index trust is equivalent to the +unstated presumption with earlier mechanisms, like PGP and Wheel signatures. + +This PEP does not preclude or exclude future index trust mechanisms, such +as :pep:`458` and/or :pep:`480`. + Recommendations =============== From 25723eecddf7af3022fa805f5a8c0cd619f78f94 Mon Sep 17 00:00:00 2001 From: William Woodruff Date: Thu, 18 Jan 2024 17:00:01 -0500 Subject: [PATCH 17/17] 740: provenance objects Signed-off-by: William Woodruff --- peps/pep-0740.rst | 96 +++++++++++++++++++++++++++++++++-------------- 1 file changed, 68 insertions(+), 28 deletions(-) diff --git a/peps/pep-0740.rst b/peps/pep-0740.rst index 5ff7bcefa41..edb17b2df47 100644 --- a/peps/pep-0740.rst +++ b/peps/pep-0740.rst @@ -156,6 +156,8 @@ non-verifiability on the index itself, and support for wheels only). Specification ============= +.. _upload-endpoint: + Upload endpoint changes ----------------------- @@ -190,41 +192,71 @@ consistency properties. Index changes ------------- +.. _provenance-object: + +Provenance objects +^^^^^^^^^^^^^^^^^^ + +The index will serve uploaded attestations along with metadata that can assist +in verifying them in the form of JSON serialized objects. + +These "provenance objects" will be available via both the :pep:`503` Simple Index +and :pep:`691` JSON-based Simple API as described below, and will have the +following structure: + +.. code-block:: json + + { + "publisher": { + "type": "important-ci-service", + "claims": {}, + "vendor-property": "foo", + "another-property": 123 + }, + "attestations": { + "some-attestation": {/* ... */}, + "another-attestation": {/* ... */} + } + } + +* ``publisher`` is an **optional** JSON object, containing a + representation of the file's Trusted Publisher configuration at the time + the file was uploaded to the package index. The keys within the ``publisher`` + object are specific to each Trusted Publisher but include, at minimum: + + * A ``type`` key, which **MUST** be a JSON string that uniquely identifies the + kind of Trusted Publisher. + * A ``claims`` key, which **MUST** be a JSON object containing any context-specific + claims retained by the index during Trusted Publisher authentication. + + All other keys in the ``publisher`` object are publisher-specific. A full + illustrative example of a ``publisher`` object is provided in :ref:`appendix-2`. +* ``attestations`` is a **required** JSON object, containing one or + more attestation objects as identified by their keys. This object is + a superset of ``attestations`` object supplied by the uploader through the + ``attestations`` field, as described in :ref:`upload-endpoint`. + + Because ``attestations`` is a superset of the file's original uploaded attestations, + the index **MAY** chose to embed additional attestations of its own. + Simple Index ^^^^^^^^^^^^ -* When an uploaded file has an attestation, the index **MAY** include a - ``data-attestations`` attribute on its file link, with a value of either +* When an uploaded file has one or more attestations, the index **MAY** include a + ``data-provenance`` attribute on its file link, with a value of either ``true`` or ``false``. -* When ``data-attestations`` is ``true``, the index **MUST** serve the uploaded - file's attestations at the same URL, but with ``.attestations`` appended to - it. For example, if ``HolyGrail-1.0.tar.gz`` existed and had associated - attestations, those attestations would be located at - ``HolyGrail-1.0.tar.gz.attestations``. -* The attestations served by the index **SHALL** be in JSON format, with the - same well-known keys as accepted by the upload endpoint. - -.. TODO: Represent the trusted publisher metadata here, somehow. +* When ``data-provenance`` is ``true``, the index **MUST** serve a + :ref:`provenance object ` at the same URL, but with + ``.provenance`` appended to it. For example, if ``HolyGrail-1.0.tar.gz`` + exists and has associated attestations, those attestations would be located + within the provenance object hosted at ``HolyGrail-1.0.tar.gz.provenance``. JSON-based Simple API ^^^^^^^^^^^^^^^^^^^^^ -* When an uploaded file has an attestation, the index **MAY** include an - ``attestations`` key in the ``file`` dictionary for that file. -* The ``attestations`` key, when present, **MUST** be a JSON object with - the same well-known keys and values as accepted by the upload endpoint. - -In addition to the above, when an uploaded file was uploaded via -`Trusted Publishing `_: - -* The ``file`` dictionary **MAY** include a ``publisher`` key. -* The ``publisher`` key, when present, **MUST** be a JSON object with a - representation of the file's Trusted Publisher configuration at the time - of upload. -* The keys within the ``publisher`` object are specific to each - Trusted Publisher but include, at minimum, a ``type`` key uniquely identifying - the type of Trusted Publisher. For example, for a GitHub-based Trusted - Publisher, the ``type`` may be the string ``"GitHub"``. +* When an uploaded file has one or more attestations, the index **MAY** include a + ``provenance`` object in the ``file`` dictionary for that file. +* ``provenance``, when present, **MUST** be a :ref:`provenance object `. Security Implications ===================== @@ -278,6 +310,8 @@ When designing a new attestation format, we make the following recommendations: basic integrity check (since the index itself already supplies cryptographic digests for this purpose). +.. _appendix-1: + Appendix 1: Example Uploaded Attestations ========================================= @@ -306,16 +340,22 @@ submitted on file upload, with two fictional attestations (``publish`` and The payloads of these fictional attestations are purely illustrative. +.. _appendix-2: + Appendix 2: Example Trusted Publisher Representation ==================================================== This appendix provides a fictional example of a ``publisher`` key within -a :pep:`691` ``project.files[]`` listing: +a :pep:`691` ``project.files[].provenance`` listing: .. code-block:: json "publisher": { "type": "GitHub", + "claims": { + "ref": "refs/tags/v1.0.0", + "sha": "da39a3ee5e6b4b0d3255bfef95601890afd80709" + }, "repository_name": "HolyGrail", "repository_owner": "octocat", "repository_owner_id": "1",