[UI/UX support for attestations] PyPI Audit

[nlhkabu]

Attestations on PyPI

Summary

PyPI's attestations are buried in the UI, explained with technical jargon, and are easily confused with weaker security features like "Verified URLs". The documentation also fails to explain their practical value to the average user.

The new attestations UI must be designed within the context of PyPIs other security features to limit user confusion between different and complimentary security concepts/features.

New, user-friendly documentation is also needed to explain attestations to end consumers, including helping users understand attestations within the context of PyPI's other security features.

Attestations discoverability

Attestations are difficult to discover on PyPI. To find an attestation, a user must navigate to an individual package page, click on "Download files", and then select the "View details" link next to the relevant file:

Attestations content

Attestations are displayed under a "Provenance" heading, alongside information about the trusted publisher

1. The Subject

• Artifact: sigstore-3.6.3.tar.gz
  • This is the specific file that the attestation is about.

2. Publisher

• release.yml on sigstore/sigstore-python The trusted publisher who the package maintainer used to publish the file to PyPI. This content is not part of the attestation itself.

3. Attestation Statement

This section contains the specific, verifiable claims about the software artifact.

• Statement type: https://in-toto.io/Statement/v1
  • This indicates this attestation follows the in-toto framework, a standard for supply chain security.
• Predicate type: https://docs.pypi.org/attestations/publish/v1
  • This specifies the purpose of the attestation: it is a claim about a package being published to the Python Package Index (PyPI).
• Subject name: sigstore-3.6.3.tar.gz
  • The name of the file being discussed.
• Subject digest: 9f957ef...afc10
  • This is a unique cryptographic hash (like a digital fingerprint) of the file's content. It guarantees that the attestation applies to this exact version of the file. If even one bit in the file were different, the digest would not match.
• Sigstore transparency entry: 230534740
  • This is the entry ID in Sigstore's public, immutable transparency log (Rekor). Anyone can use this ID to look up and verify that this attestation was recorded and has not been altered.
• Sigstore integration time: Jun 6, 2025, 7:10:29 AM
  • This is the timestamp when the attestation was recorded in the transparency log. It provides a secure and verifiable record of when the artifact was built and signed.

4. Attestation Source Repository (Where the Code Came From)

This section links the final software package directly back to its source code.

• Permalink: sigstore/sigstore-python@0f889402b0c8...b689f20
  • This is the most crucial link. It points to the exact commit hash of the source code that was used for the build. This ensures you know precisely which version of the code the package was created from.
• Branch / Tag: refs/tags/v3.6.3
  • This gives the human-readable Git tag associated with that commit, confirming this was the v3.6.3 release.
• Owner: https://github.com/sigstore
  • The GitHub organisation that owns the code.
• Access: public
  • The source code is publicly available for anyone to inspect.

5. Attestation Publication Detail (How It Was Built)

This section describes the environment where the software was built and published.

• Token Issuer: https://token.actions.githubusercontent.com
  • The build process was authenticated using a short-lived security token from GitHub Actions, not a long-term secret key.
• Runner Environment: github-hosted
  • The build ran on a standard, clean virtual machine provided by GitHub, not on an individual’s personal computer.
• Publication workflow: release.yml@0f889402b0c8...b689f20
  • This re-confirms that the release.yml workflow at the specific source code commit was responsible for the publication.
• Trigger Event: release
  • The workflow was initiated automatically by a "release" event on GitHub, which is a standard practice for creating official releases.

How users are likely to understand this interface

This UI appears to be most useful for users who are already deeply informed about attestations, somewhat useful (and somewhat overwhelming) for users with some understanding of attestations, and overwhelming for users without any knowledge of attestations. [TBC via user interviews]

No help content is available to assist users who do not understand the content. The UI is jargon heavy, with technical terms not explained.

The Security Architect (Concerned / Informed)

• Likely to understand all fields
• Likely to appreciate the link to the Rekor log, and probably click on it
• Likely to notice that the commit hash in the source repository's permalink is the same as the one in the publication workflow

The Pragmatic Developer (Somewhat Concerned / Informed)

• Likely to understand the "Source repository" section , including what is meant by "Owner", "Branch", "Tag", etc.
• May not understand the conceptual difference between the publisher (trusted publisher on PyPI) and the publication detail in the attestation
• Likely to understand that the runner environment and trigger event means that this file was produced by an automated, official release process
• Unlikely to understand many of the terms in the "Statement" section, including "Predicate", or what a "Statement Type" is.
• Unlikely to understand the significance of the Rekor log, nor click on it

The Incidental Consumer (Not Concerned / Not Informed)

• Likely to understand the term "Provenance", and therefore conceptually understand that the content below is about the origin/history of the package
• Unlikely to understand the majority of the interface
• May recognise "GitHub" as a common (and trustworthy) software tool

Other security features on PyPI

PyPI has several other security features, which may cause confusion when presenting attestations:

Trusted publisher

Package maintainers can register a CI pipeline as a trusted publisher, which allows them to bypass the traditional requirement to configure and manage an API key for automating the publishing process. If configured, the trusted publisher will be displayed alongside any attestations in the "Provenance" section of the "File details" UI.

Latest version badge

When viewing the latest version of a package, PyPI prominently displays a "Latest version" badge:

An alternative version of this badge is displayed when viewing an older version of a package, which indicates that the package is outdated and that a newer version is available instead:

Verified / Unverified project details

PyPI can verify URLs in project metadata, if the package uploader uses a trusted publisher. When URLs are verified they are displayed as "Verified details" in the project sidebar:

Or "unverified details" when not verified:

The potential for confusion between "verified by PyPI" and attestions is high, because the concepts are related and the user-facing signals may be misinterpreted.

Malware reporting button

Users can report a project as suspected malware via the project sidebar (button is located at the bottom of the sidebar):

Instructions on reporting a project as malware are contained in the PyPI security policy page

File hashes

Users can configure their project to use hashes to verify that the file they download is the exact same one the author uploaded:

Mental models required to understand security on PyPI

Of PyPI's security features, users are most likely to confuse "Trusted Publishing", "Verified by PyPI" and File hashes with attestations. This is particularly true for users with less understanding of packaging and security concepts (i.e our "Incidental Consumer" persona).

For a user to correctly interpret the strength and scope of each security feature on PyPI, they need to understand:

1. What the security feature tells them
2. How valuable (or "strong") the information is
3. What the security features protects them from
4. Where/How to use the security feature
5. What the risk is if/when the security feature is missing

This is a complex mental model, as described by the table below:

Feature	Trusted Publishing	Verified by PyPI (URLs)	File Attestations	File Hashes
What It Tells Me	The file was uploaded by a secure, automated process from a "trusted" publisher.	The project's name on PyPI is legitimately linked to the source code repository shown on the page.	This exact file was built from a specific commit in the source code, by a specific, verifiable process.	The file I downloaded is a perfect, bit-for-bit copy of the file on PyPI's servers.
Strength of Guarantee	Moderate. It secures the upload process, but says nothing about the file's contents or origin.	Low. A useful identity check, but it's a one-time link verification that doesn't apply to the files at all.	Very High. A cryptographic guarantee connecting the file you are about to install directly to its source code.	High (for its purpose). Guarantees the file wasn't corrupted or changed during download.
What It Protects Me From	A hacker using a maintainer's stolen password or API key to upload a malicious package.	A fake package (tnsorflow) trying to impersonate a real project (tensorflow) by linking to the real source code.	A malicious file that was created separately and uploaded instead of one built from the public source code.	A corrupted download, or an attacker swapping the file with a malicious one as it travels over the network to you.
How I Use It / Where I Look	Check for Uploaded using Trusted Publishing? Yes on the file's detail page.	Look for the green checkmark (✅) on the sidebar on the main project page.	Scrutinise the "Provenance" section on the file's detail page for proof of origin.	Your package manager (like pip) checks this automatically. You can manually verify the hash if needed.
If It's Missing, What's the Risk?	The file could have been uploaded by anyone with the maintainer's credentials. The upload process itself is not inherently secure.	The project's source code link might be misleading or point to an imposter repository. You have to investigate the project's identity yourself.	This is the biggest risk. There is no verifiable proof connecting the file to the claimed source code. You are implicitly trusting the maintainer and the upload process completely.	This should never be missing. If a hash doesn't match, the file is tampered with or corrupted. Do not use it.

Displaying attestations on PyPI - recommendations

To avoid creating user confusion, attestations must be considered within the context of the other security features on PyPI. The design should be guided by these principles:

1. Visual hierarchy

The design must visually communicate the relative importance of each security signal. The strongest guarantee should be the most prominent.

• How will the visual weight (size, colour, placement, iconography) of the new attestation display compare to the existing "Verified by PyPI" checkmark?
• Should the design intentionally de-emphasise weaker signals to prevent them from creating a false sense of security?

2. Conceptual grouping

The layout should tell a coherent story about how PyPI's security features relate to one another, rather than presenting them as an unrelated list.

• Should features that describe the file (Attestations, Hashes, Trusted Publishing) be grouped together, separate from features that describe the project (Verified URLs), as in the current design?
• How can the design show that "Trusted Publishing" is the enabler for an attestation, rather than a separate, competing signal?

3. Audience and information density

The design must serve multiple user personas simultaneously, from the expert who needs raw data to the novice who needs a clear summary.

• How can the UI provide a simple, at-a-glance "trust badge" for the less-informed user, while still offering a clear path to the detailed attestation data for the security architect?
• Should progressive disclosure be used to hide complex technical details by default, preventing overwhelm?

Supporting Documentation

PyPI has two documentation sources:

• PyPI FAQs ("Help")
• PyPI user documentation ("Docs")

In addition, PyPI has a security page, linked to from the footer under "report security issue".

Currently, there is limited security information on PyPI's "Help/FAQ" page. This includes:

1. Account / authentication help (2fa, security tokens, etc.)
2. A basic description of file hashes
3. Information about projects in quarantine

No information about attestations is contained in the FAQs, and trusted publishing is only mentioned from the perspective of project maintainers.

By contrast, the PyPI documentation contains a deep technical reference about both trusted publishers and attestations. It is primarily written by experts, for experts. It excels at documenting the "how" for package maintainers and the "what" for security professionals. However, there is a significant gap between the low-level implementation details provided, and the high-level, practical guidance needed by end-users. Most notably:

1. There is no high-level page that describes all of PyPI's security features, the limitations of each, and how they "fit together" to help users make better security decisions.
2. There is no high-level content that answers the simple question: "What is an attestation, and why should I care?" in plain, non-technical language.
3. The Consuming attestations page describes a manual, expert-level verification process, and does not mention integration with package installers or requirement files, which are the consumption methods that matter to the vast majority of users.

Similarities and differences with npm and RubyGems

TBC