Suggesting changes on Level 2 and Level 3

[kairoaraujo]

Last Thursday I had a great chat with @eddie-knight

My overall feedback is moving from Level 1 to 2 is quite challenging if compared to Level 2 to 3.
I feel that as being in Level 2 you have great maturity and the project probably is running on great stability of contributors, and maintainers. Whereas being in Level 1, mostly of projects are in a phase of trying to organize themselves and showing the value of the project (hard moment I can tell by myself).

As I discussed with @eddie-knight, I suggest moving the following from Level 2 to Level 3:

• OSPS-BR-06.01: All released software assets MUST be signed or accounted for in a signed manifest including each asset's cryptographic hashes.
• OSPS-VM-06.01: The project MUST publicly publish data about discovered vulnerabilities.

IMHO, OSPS-BR-06.01 has the same level of complexity as OSPS-QA-02.02:

The OSPS-VM-06.01 requires great project maturity and a maintainer (or core contributor) who understands how to publish discovered vulnerabilities.

[TheFoxAtWork]

Thanks @kairoaraujo for creating this issue. I wanted to add some additional perspective here because while I understand these can be more complex to implement, particularly given the other controls at level 1.

OSPS-VM-06 - Publishing vulnerabilities is hyper critical for a projects adopters to understand their risk. While some source forges make this very simple not all do, and this control's implementation experience may identify opportunities for the community to assist in making this a "stretch" from level 1 to 2, much easier. I think the key here is whether elements of this control's implementation experience could be broken down further to simply publishing versus more comprehensive information like, determining if they are affected and any mitigations as without a colleague to leverage with this experience, it can be frustrating and time consuming to produce.

OSPS-BR-06.01 - For software signing, I'd like to understand a bit more on the nuances of the complexity here if you could elaborate? With tooling like sigstore, the community has strived to make this very simple and easy, over other heritage signing mechanisms like GPG. The important outcome of this control is to ensure potential adopters or users of the project know the software they plan to use is authentic, with its integrity intact since signing and provides them a mechanism to verify it. For me the value of implementing this is essential for the project in establishing and improving its reputation for adopters as "genuine", and the challenges to implement (as I currently understand them) fall short when compared.

Another thought, as we see this in CNCF, level 1 is often light, far easier to achieve to incentivize projects to pursue it, it's not a lot of work but provides good value. Achieving level 2 is always difficult, it is, effectively, pushing the project to cross the security "chasm" (referring to Geoffrey Moore's Crossing the Chasm). It's like a viability test if you will, not every project will cross it, and that is okay, but the ones that do, have taken the necessary steps to continue to push forward and build security acumen from being a dedicated speciality consideration, to a natural feature and function of daily project operations.

I'm curious as to your thoughts. Thank you again!

[kairoaraujo]

Hi @TheFoxAtWork,

Thank you for sharing your perspective here, I will try to share more of my (personal) experiences here as a maintainer of projects that are in the situation of level 2.

Level 2: for any code project that has at least 2 maintainers and a small number of consistent users

I'm a maintainer of python-tuf, a critical library for TUF in different other projects, including sigstore, we have a limited number of active maintainers and I see a lot of effort those maintainers are doing to keep everything in shape. From engaging in mentorship to growing the contribution and keeping the project healthy, some do it in their spare time.

In python-tuf for example, we sign releases and we have a policy to one maintainer cuts the release and the other verifies the signature and approves it. Definitely, it is Level 3 I would say.

In the RSTUF project (OpenSSF), we are also struggling with the active maintainers' roles.
RSTUF is all about The Update Framework (signing) and we still not signing our releases (ok, it is a shame), but my justification (a.k.a excuse) is that we are still in the beta releases and we want to start signing on the first release.

I can also say about Archivista, we know that exists a couple of users out there, but we are still low in contributions back and maintainers to get the project on the level we would like to.

My opinion here was more from a practical experience from the field while running projects that we want to grow and do all the best practices, but we still struggling.

I know Sigstore is there to make it easier (We support it in RSTUF and python-tuf powers it in sigstore-python).
We have many SBOM tools. But still, there is a process to build the OSS project and adopt it.

My suggestion was to put together the SBOMS, Signing, and Provenance on level 3. It is ok OSPS-VM-06 to be in level 2, as you mentioned very well.

[TheFoxAtWork]

This is great information, thank you for sharing! It highlights (for me at least) the balance you are trying to achieve and the realities of growing projects. You also touched on a key point that I think would really distinguish the control further as a maturity element

we are still in the beta releases and we want to start signing on the first release.

That makes complete sense. I wondering what we can do to strike a balance here. @eddie-knight does it make sense to caveat this control to stable releases?

[eddie-knight]

I added a level 3 VEX requirement in PR #205 so that there are less stringent requirements around the delivery mechanism in OSPS-VM-06. Let me know if that doesn't properly implement the corresponding feedback here.

we are still in the beta releases and we want to start signing on the first release

If you are tagging your github release assets as "pre-release," your intent here should fall within the existing guidelines.

Regarding the overwhelming length of Level 2 requirements, I've proposed some adjustments in 205 and #206 to hopefully spread it out a bit more. @TheFoxAtWork can you take a peek to see whether I was overzealous there? cc/ @puerco @funnelfiasco

[TheFoxAtWork]

Reads well. Overzealous? Not so much. I think this helps things significantly for projects.

[funnelfiasco]

@kairoaraujo it seems like #205 and #206 address this issue, so I'll close it. Thanks for your feedback! If you feel like it's not fully addressed, please let me know