Skip to content

Add SBOM generation and upload to DependencyTrack workflows#1014

Merged
AxelRICHARD merged 1 commit intoeclipse-syson:mainfrom
OtterdogTest:sbom-workflow
Feb 11, 2025
Merged

Add SBOM generation and upload to DependencyTrack workflows#1014
AxelRICHARD merged 1 commit intoeclipse-syson:mainfrom
OtterdogTest:sbom-workflow

Conversation

@iliescuioana
Copy link
Contributor

As previously discussed, this PR aims to bootstrap the EF Security Team initiative of implementing automated SBOM generation for projects and uploading them to our DependencyTrack instance, with the goal of enhancing software supply chain security.

We wanted this to seamlessly integrate with your existing release processes, so we designed and implemented 2 generation workflows:

  • 1 maven workflow: meant to generate an SBOM for the backend product (i.e. syson/backend/application/)
  • 1 npm workflow: meant to generate an SBOM for the frontend product (i.e syson/frontend/)

Currently the workflows are triggered by either pushes to the main branch that modify dependency files or by pushing new tags. Several Cyclonedx ecosystem specific plugins are used to generate the SBOMs, which are then uploaded as artifacts. The “store-sbom-data” reusable workflow stores additional metadata about the project and upon completion, the self service system downloads the SBOM from artifacts and uploads it to our DependencyTrack instance.

We have tested the workflows and everything worked successfully, however feel free to update them as needed. Please let us know if you have any concerns or questions we can help with.

PLEASE READ ALL ITEMS AND CHECK ONLY RELEVANT CHECKBOXES BELOW

Project management

  • Has the pull request been added to the relevant milestone?
  • Have the priority: and pr: labels been added to the pull request? (In case of doubt, start with the labels priority: low and pr: to review later)
  • Have the relevant issues been added to the pull request?
  • Have the relevant labels been added to the issues? (area:, type:)
  • Have the relevant issues been added to the same project milestone as the pull request?

Changelog and release notes

  • Has the CHANGELOG.adoc + doc/content/modules/user-manual/pages/release-notes/YYYY.MM.0.adoc been updated to reference the relevant issues?
  • Have the relevant API breaks been described in the CHANGELOG.adoc + doc/content/modules/user-manual/pages/release-notes/YYYY.MM.0.adoc?
  • In case of a change with a visual impact, are there any screenshots in the doc/content/modules/user-manual/pages/release-notes/YYYY.MM.0.adoc?
  • In case of a key change, has the change been added to Key highlights section in doc/content/modules/user-manual/pages/release-notes/YYYY.MM.0.adoc?
  • Are the new / upgraded dependencies mentioned in the relevant section of the CHANGELOG.adoc + doc/content/modules/user-manual/pages/release-notes/YYYY.MM.0.adoc?

Documentation

  • Have you included an update of the documentation in your pull request? Please ask yourself if an update (installation manual, user manual, developer manual...) is needed and add one accordingly.

Tests

  • Is the code properly tested? Any pull request (fix, enhancement or new feature) should come with a test (or several). It could be unit tests, integration tests or cypress tests depending on the context. Only doc and releng pull request do not need for tests.

@AxelRICHARD
Copy link
Member

Could you also rebase the PR please? I am not able to merge it, the merge button is grayed with the following message: "This branch is out-of-date with the base branch"

@AxelRICHARD
Copy link
Member

Could you please change the commit message by adding a Signed-Off-By (see previous commits on this repo) and also [releng] at the beginning of the message title (Add SBOM generation a... => [releng] Add SBOM generation a...)?

@netomi
Copy link

netomi commented Feb 10, 2025

@iliescuioana you can get information about adding a signoff message here: https://git-scm.com/docs/git-commit#Documentation/git-commit.txt---signoff

To change the commits in this branch you can follow the process as explained here: https://stackoverflow.com/questions/25356810/how-to-squash-all-commits-on-branch

@netomi
Copy link

netomi commented Feb 10, 2025

@AxelRICHARD may I suggest to add such information to your contributing guide?

@AxelRICHARD
Copy link
Member

@AxelRICHARD may I suggest to add such information to your contributing guide?

Yes you can :) We have embryonic contributor and developer guides here: https://doc.mbse-syson.org/syson/v2025.1.0/user-manual/contribute.html & https://doc.mbse-syson.org/syson/v2025.1.0/user-manual/integration/developer-guide.html but we clearly have to improve them.

@AxelRICHARD
Copy link
Member

Could you also please rebase the PR again? We don't merge branches, only rebase them to have a linear historic, and unfortunately I can't do it myself on this PR.

@netomi
Copy link

netomi commented Feb 10, 2025

@iliescuioana could you enable for the PR to allow maintainers to edit?

@netomi netomi force-pushed the sbom-workflow branch 2 times, most recently from 9767123 to 0ce5d53 Compare February 11, 2025 10:07
@iliescuioana
Copy link
Contributor Author

@iliescuioana could you enable for the PR to allow maintainers to edit?

For reference: https://github.com/orgs/community/discussions/5634, unfortunately you cannot allow maintainers to edit on forked repos in an org.

@netomi netomi force-pushed the sbom-workflow branch 2 times, most recently from bf7ce68 to 5af6303 Compare February 11, 2025 10:11
Signed-off-by: Thomas Neidhart <thomas.neidhart@gmail.com>
@netomi
Copy link

netomi commented Feb 11, 2025

commit has been amended as requested and the PR has been rebased

@AxelRICHARD AxelRICHARD merged commit 0f15bb6 into eclipse-syson:main Feb 11, 2025
3 checks passed
@netomi netomi deleted the sbom-workflow branch February 11, 2025 10:42
@AxelRICHARD
Copy link
Member

Hello @netomi, @iliescuioana

Since few days/weeks, the SBOM build on the backend part fails on SysON github CI.
See the following failing build for example:
https://github.com/eclipse-syson/syson/actions/runs/13545587400

I don't know what to do to fix it.

Could you help me on that please?

(Note that the SBOM build for the frontend do not fail)

@iliescuioana
Copy link
Contributor Author

Hi @AxelRICHARD , took a quick look at it and this is what I found: they’re not all failing, for example this recent one completed successfully and it was triggered by a change to the version number of an external dependency (not published by syson). The ones that are failing however, such as this one seem to have been triggered by a release, which also changed version numbers of internal syson products/modules that the parent depends on. The latter ones, considering this build step are then published to maven in the build workflow.

What I suspect happens here is that when a new release is created, both the build and the sbom workflows start at the same time for the new version. The SBOM plugin needs to pull all dependencies of the product with the new versions to compute the dependecy tree, however these “internal” dependencies have not yet been published by the build process, as it takes longer. The SBOM plugin then fails as a result with errors such as:
Could not find artifact org.eclipse.syson:syson-siriusweb-customnodes-metamodel-edit:jar:2025.2.0 in github (https://maven.pkg.github.com/eclipse-syson/syson)

This could be the case for frontend as well if it relies on “internal” dependencies that have just had their versions changed at the same time with the parent, but the publishing is probably quicker than the pulling there. It’s definitely a corner case that we had no way of testing during development, since we are not committers and cannot simulate publishing new versions of deps.

In terms of solutions to fix it, what comes to mind is figuring out a way to trigger the sbom workflows AFTER the build/publish one completes for cases where they’d run in parallel or introduce some sort of delay. That’d mean fiddling with these triggers. On a quick look, found this workflow_run event trigger that could potentially be used in the sbom workflow instead of the events that also trigger a build. This way you could have the sbom one wait for successful completion of the build run. If that works, there’s minor fiddling with this step to remove the no longer necessary conditional statements.

@iliescuioana
Copy link
Contributor Author

Let me know if you need help trying that out. Unfortunately I don't have sufficient permissions to actually test it myself properly e2e, only what I can do on a fork that already works because there's no build/publishing delay.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants