Proposes to add a new utilities buildpack for Syft

[dmikusa]

Summary

A proposal to add a utility buildpack that will install the Syft CLI.

Use Cases

This tool can be used to generate SBoM material in standard formats like SPDX & CycloneDX.

Checklist

• I have viewed, signed, and submitted the Contributor License Agreement.
• I have linked issue(s) that this PR should close using keywords or the Github UI (See docs)
• I have added an integration test, if necessary.
• I have reviewed the styleguide for guidance on my code quality.
• I'm happy with the commit history on this PR (I have rebased/squashed as needed).

[sophiewigmore]

@dmikusa-pivotal I haven't been following along with the latest iterations of the BOM work. What was the rationale for adding this buildpack, rather than following creating the BOM alongside buildpacks like we did in the Node.js language family, and now in Go? Is it mostly because of the format versatility (support for SPDX too)? Would this replace the need for BOM creation solutions like the ones I outlined?

[dmikusa]

@dmikusa-pivotal I haven't been following along with the latest iterations of the BOM work. What was the rationale for adding this buildpack, rather than following creating the BOM alongside buildpacks like we did in the Node.js language family, and now in Go? Is it mostly because of the format versatility (support for SPDX too)? Would this replace the need for BOM creation solutions like the ones I outlined?

The main rationale for this RFC is to make this tool an option for SBoM management. As you said, there are some benefits of using the same tool across multiple buildpacks. For the Java team, we have several build tools to support, Maven and Gradle being the largest two but there's also SBT, Lein, and Clojure Tools, so rather than integrate different plugins for each tool we can use Syft across them all. It'll give us consistent output across the tools. Plus, as you mentioned it supports a variety of output formats so we don't need different plugins for CycloneDX and SPDX or conversion tools.

That said, this RFC is not in any way trying to dictate that language families adopt this particular tool. It's just to make it available and something we officially maintain. Language teams can choose to adopt or not adopt it on their own.

[sophiewigmore]

@dmikusa-pivotal understood, thanks for outlining that.

[sambhav]

I am a bit torn between using syft as a binary vs as a library. Using it as a binary means that each buildpack doesn't need to include the entire syft dependency chain but on the other hand we need some sort of bin wrapper / library. Regardless +1 on the decision to go with syft. It's probably the fastest sbom generation tool that I have come across.

If we do decide to go the syft binary route, it would be really nice to have a reusable go lib to invoke syft bom generation commands instead of relying on pexec/effect directly.

[dmikusa]

I am a bit torn between using syft as a binary vs as a library. Using it as a binary means that each buildpack doesn't need to include the entire syft dependency chain but on the other hand we need some sort of bin wrapper / library.

Yes, that's tricky. There are definitely advantages both ways. I think that since this RFC is primarily for making an official buildpack that would provide syft as a CLI, this unlocks the ability for folks to easily consume the CLI while still reserving the right for individual language teams to not go that route. If they would prefer using a library instead, or even some other tool.

That said, I think the CLI does provide some advantages. It's pretty well encapsulated, including things like reading from standard configuration locations out-of-the-box. We also don't need a lot of customization with the tool, what it does with the standard commands is exactly what we need. I think if we had to customize the operation of it more, then perhaps going the library route could make some sense. Like previously mentioned, this RFC doesn't lock us into anything, we could still pivot and use the library if down the road we end up needing more customization. The nice thing is that CLI or library are both going to give a consistent experience.

Anyway, that is why I'm leaning towards CLI at this point. I'm sure someone could make a compelling argument for the library route too. Fortunately, we don't have to pick exclusively one or the other.

If we do decide to go the syft binary route, it would be really nice to have a reusable go lib to invoke syft bom generation commands instead of relying on pexec/effect directly.

In my initial trials, I have been using libbs for this. It's basically one efffect.Executor call to run the binary and copy the output to a file, and it'll update all of the Paketo "build system" buildpacks (Maven, Gradle, etc..) to have this support. This will essentially give you a SBoM for the application and its dependencies.

I could see moving this into libpak so that it's easy to call from other buildpacks as well. I haven't thought it all through, but we might end up calling that from other CNBs too (like maybe executable-jar or Spring Boot CNB).

It's outside of this RFC, but I'll make sure that happens.

[ForestEckhardt]

I could see moving this into libpak so that it's easy to call from other buildpacks as well. I haven't thought it all through, but we might end up calling that from other CNBs too (like maybe executable-jar or Spring Boot CNB).

Is there a reason why you would want to put this in libpak as opposed to putting it into the buildpack itself. If we would like to make the functionality universal for at least buildpacks written in Go it would make sense to me to attache the functionality to the buildpack as opposed to a implementation library as that would then force you to import two implementation libraries if you are using something that is not libpak or alternatively it would force you to copy and paste code into your implementation.

It would also make sense from a functionality upgrade standpoint (maybe adding common helpers and what not) to have it versioned with the buildpack and its release.

With that being said if we did want a shared interaction library I would argue that it should maybe be at least mentioned in the RFC as future work. I don't think that details need to be completely fleshed out but some sort of mention makes sense in my head.

[dmikusa]

I could see moving this into libpak so that it's easy to call from other buildpacks as well. I haven't thought it all through, but we might end up calling that from other CNBs too (like maybe executable-jar or Spring Boot CNB).

Is there a reason why you would want to put this in libpak as opposed to putting it into the buildpack itself. If we would like to make the functionality universal for at least buildpacks written in Go it would make sense to me to attache the functionality to the buildpack as opposed to a implementation library as that would then force you to import two implementation libraries if you are using something that is not libpak or alternatively it would force you to copy and paste code into your implementation.

It would also make sense from a functionality upgrade standpoint (maybe adding common helpers and what not) to have it versioned with the buildpack and its release.

With that being said if we did want a shared interaction library I would argue that it should maybe be at least mentioned in the RFC as future work. I don't think that details need to be completely fleshed out but some sort of mention makes sense in my head.

@ForestEckhardt I only mentioned libpak because I believe @samj1912 is using it. I wouldn't intend for anyone to use syft through libpak unless they are already using libpak. IMHO, there wouldn't be an advantage to using syft this way unless you're already using libpak.

[dmikusa]

As mentioned in the Paketo WG meeting today, we need to get some additional approvals on this.

@paketo-buildpacks/utilities-maintainers If you could provide your 👍/👎 that would be much appreciated. Thanks