Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

publish go-mod/vendor as release artifact #2622

Open
ctr49 opened this issue Feb 11, 2024 · 2 comments
Open

publish go-mod/vendor as release artifact #2622

ctr49 opened this issue Feb 11, 2024 · 2 comments
Labels
enhancement New feature or request release relating to the release process of syft

Comments

@ctr49
Copy link

ctr49 commented Feb 11, 2024

What would you like to be added: I would like to see the go-mod/cache added as release artifact next to the source and the compiled binaries. Please extend the Release action to also publish a compressed tarball of all used dependencies.

Why is this needed: To have an available and consistent dependency package for distributions that want to distribute syft.

Additional context:
For source packages, some (many/all?) distributions require all artifacts to build the package (including their dependencies) to be present if not relying on distribution specific library/include dependencies (which would be rather unusual for a go package). These dependencies also need to be available for offline usage and may not be fetched at compile time through the language-specific module resolver (i.e. go mod download or npm i). Usually this results in someone creating a tarball of he deps. But this tarball could be much easier created and made available by the source package (here) rather than by each distribution package maintainer. Also, there would be higher consistency across distributions when made available from an authoritative party.

If size is an issue (> 500MB per version!), it could be limited to still supported or last x versions.

Example: Gentoo's syft package has a tarball of all dependencies at https://dev.gentoo.org/~williamh/dist/${VERSION}-deps.tar.xz which makes the whole package update process unnecessarily complicated.

@ctr49 ctr49 added the enhancement New feature or request label Feb 11, 2024
@wagoodman wagoodman added the release relating to the release process of syft label Feb 13, 2024
@wagoodman
Copy link
Contributor

I see how that would be useful to the downstream packagings. Instead of a full go mod download, what about an ad-hoc go mod vendor?

# in repo root...
$ go mod vendor -v
$ tar -cJf syft-vendor-cache.tar.xz  ./vendor
$ ls -alh *.tar.xz
-rw-r--r--@  1 wagoodman  staff    10M Feb 13 10:47 syft-vendor-cache.tar.xz

vs

# in repo root...
$ GOMODCACHE=$(pwd)/cache go mod download
$ tar -cJf syft-mod-cache.tar.xz  ./cache
$ ls -alh *.tar.xz
-rw-r--r--@   1 wagoodman  staff   524M Feb 13 10:46 syft-mod-cache.tar.xz

The nice thing about this is that it's much smaller, doesn't contain unnecessary content such as tests for dependencies, and would be easier to publish.

My question is that would using the vendor cache over the mod cache work for your use case?

I think the only modification to the workflow would be:

  • no need to set GOMODCACHE or GOPATH
  • unarchive the vendor tar into the root of the repo
  • run go build -mod=vendor instead

@ctr49
Copy link
Author

ctr49 commented Feb 13, 2024

Yes, you are perfectly right. Having the vendor-dir should be sufficient.

@ctr49 ctr49 changed the title publish go-mod/cache as release artifact publish go-mod/vendor as release artifact Feb 19, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request release relating to the release process of syft
Projects
Status: Backlog
Development

No branches or pull requests

2 participants