New option to only return outdated components and/or only direct dependencies in the ComponentResource #2568

walterdeboer · 2023-03-06T19:52:31Z

Description

Added onlyOutdated ApiParam and onlyDirect ApiParam to /v1/component/project endpoind to only show outdated components and/or only direct dependencies

Addressed Issue

closes #2567, adresses #1833

Additional Details

Only direct dependencies are reported when onlyOutdated is true. Also added a directDepencies switch for those who want to overwrite them in their project

Checklist

I have read and understand the contributing guidelines
This PR fixes a defect, and I have provided tests to verify that the fix is effective
This PR implements an enhancement, and I have provided tests to verify that it works as intended
This PR introduces changes to the database model, and I have added corresponding update logic
This PR introduces new or alters existing behavior, ~~and I have updated the documentation accordingly~~

src/main/java/org/dependencytrack/persistence/ComponentQueryManager.java

src/main/java/org/dependencytrack/tasks/repositories/MavenMetaAnalyzer.java

UWollesen · 2023-03-29T05:50:32Z

Thank you for this feature. But you can actually upgrade indirect dependencies. In Gradle projects you can force versions of transitive dependencies. Of course you have to test the compatibility to your direct dependencies but sometimes it's the only way to get rid of some known vulnerabilities. So maybe showing of transitive dependencies or another switch to filter them would be a nice idea.

walterdeboer · 2023-03-29T20:46:30Z

When you don’t turn on the switch, you’ll see all dependencies. Isn’t this what you need then? Or do you mean you wan’t to see only transitive, without the direct dependencies? Seems to me that this would add much. Wouldn’t you want to fix all dependencies then? I didn’t know gradle could do that. Good to know 🙂

UWollesen · 2023-03-30T06:00:04Z

No, I meant that I want to see all outdated dependencies, direct and transitive. ;)
And here is the link to the documentation of this Gradle feature: https://docs.gradle.org/current/userguide/dependency_constraints.html

walterdeboer · 2023-03-30T17:57:51Z

Oh sorry, of course. I was under the temporary illusion this functionality already existed.. I'll add an extra switch 👍

UWollesen · 2023-03-31T05:10:49Z

Thank you very much. This will help a lot.

msymons · 2023-04-27T12:38:24Z

@walterdeboer, following the release of v4.8.0, this PR now has conflicts that need to be resolved before it can be merged. Please can you fix things? Then the PR can be reviewed.

…ect dependencies of the project Signed-off-by: Walter de Boer <walterdeboer@dbso.nl>

melba-lopez · 2023-06-20T02:26:58Z

src/main/java/org/dependencytrack/persistence/ComponentQueryManager.java

        final PaginatedResult result;
        String querySring ="SELECT FROM org.dependencytrack.model.Component WHERE project == :project ";
        if (filter != null) {
            querySring += " && (project == :project) && name.toLowerCase().matches(:name)";
        }
        if (onlyOutdated) {
+            // Components are considered outdated when metadata does exists, but the version is different than latestVersion
+            // Different should always mean version < latestVersion


@walterdeboer @nscuro have we given any thought to latest version in a repo may be a BETA vs "prod/GA/GTM/stable" code? if the version we have is the last stable release, but the repo is showing a beta release...do we want to flag that as outdated?

@melba-lopez That's a good point, and I fear it is somewhat subjective. That said, the focus is on flagging risk, so the question becomes "am I at risk of being outdated when there's a new BETA version available?". Unless the latest stable version has risky vulns (which would be flagged separately), relying on BETA software in itself would be an operational risk.

What @walterdeboer is trying to solve here is risk being flagged despite there not being any sane way forward (no stable version to update to).

An extension of this feature could be to capture both latest stable and latest unstable version from remote repositories, and then letting the users decide what they want to consider "latest".

melba-lopez · 2023-06-20T02:31:51Z

src/main/java/org/dependencytrack/persistence/QueryManager.java

-    public PaginatedResult getComponents(final Project project, final boolean includeMetrics, final boolean onlyOutdated) {
-        return getComponentQueryManager().getComponents(project, includeMetrics, onlyOutdated);
+    public PaginatedResult getComponents(final Project project, final boolean includeMetrics, final boolean onlyOutdated, final boolean onlyDirect) {
+        return getComponentQueryManager().getComponents(project, includeMetrics, onlyOutdated, onlyDirect);


@walterdeboer Now seeing this PR. If i understand this right, this is enabling a new toggle switch to show only direct dependencies, hence why the previous PR #2537 has code comments about direct dependencies only?

I've mixed up the code in #2537 I hope i've got it right now...

I've mixed things up in PR #2537 with this one... I've pushed the right code in the other PR now

melba-lopez · 2023-06-20T02:38:57Z

src/test/java/org/dependencytrack/resources/v1/ComponentResourceTest.java

                metaComponent.setLastCheck(new Date());
                qm.persist(metaComponent);
+            } else {
+                // 500 components with no version metadata, all transitive dependencies


@walterdeboer Is the test here trying to state that if there is no metadata provided in the SBOM, or if a manually created component is made without metadata, that we assume it is transitive? Just trying to understand this comment here.

This has nothing to do with the SBOM, when DT detects new versions in the remote repository, it creates a RepositoryMetaComponent. The components here have no RepositoryMetaComponent, so they represent te most current versions

Signed-off-by: Walter de Boer <walterdeboer@dbso.nl>

melba-lopez · 2023-06-27T16:37:02Z

@walterdeboer unfortunately the CI Test is failing 😞

Signed-off-by: Walter de Boer <walterdeboer@dbso.nl>

walterdeboer · 2023-06-27T20:03:06Z

@walterdeboer unfortunately the CI Test is failing 😞

@nscuro Hmm, the PolicyEngineTest fails randomly. I hope I fixed it

melba-lopez

Thanks for the quick turnaround for fixing the test failures!

nscuro

Thanks @walterdeboer, this is going to be super useful! 🚀

And of course thanks @melba-lopez for reviewing!

walterdeboer changed the title ~~Feature/2567~~ New option to only return outdated components in the ComponentResource Mar 6, 2023

sonatype-lift bot reviewed Mar 6, 2023

View reviewed changes

src/main/java/org/dependencytrack/persistence/ComponentQueryManager.java Show resolved Hide resolved

sonatype-lift bot reviewed Mar 6, 2023

View reviewed changes

src/main/java/org/dependencytrack/tasks/repositories/MavenMetaAnalyzer.java Outdated Show resolved Hide resolved

walterdeboer force-pushed the feature/2567 branch 2 times, most recently from 52c0694 to 64e8a24 Compare March 6, 2023 21:21

walterdeboer mentioned this pull request Mar 6, 2023

Add "Outdated only" and "Direct only" switch to only show outdated components DependencyTrack/frontend#437

Merged

2 tasks

walterdeboer force-pushed the feature/2567 branch from 64e8a24 to 8100665 Compare March 6, 2023 22:04

walterdeboer marked this pull request as ready for review March 6, 2023 22:13

walterdeboer force-pushed the feature/2567 branch 3 times, most recently from 29e4b7b to 7a0adc3 Compare March 11, 2023 23:27

walterdeboer force-pushed the feature/2567 branch from 3fd0898 to b220f21 Compare March 31, 2023 07:54

walterdeboer changed the title ~~New option to only return outdated components in the ComponentResource~~ New option to only return outdated components and/or only direct dependenncies in the ComponentResource Mar 31, 2023

nscuro added this to the 4.9 milestone Apr 18, 2023

Add onlyOutdated switch to only show outdated components that are dir…

995c27e

…ect dependencies of the project Signed-off-by: Walter de Boer <walterdeboer@dbso.nl>

walterdeboer force-pushed the feature/2567 branch from b220f21 to b427473 Compare May 3, 2023 20:21

melba-lopez approved these changes Jun 20, 2023

View reviewed changes

walterdeboer mentioned this pull request Jun 23, 2023

New VersionDistance policy to check and report outdated components #2537

Merged

5 tasks

new onlyDirect switch do optionally exclude transitive dependencies

b5330eb

Signed-off-by: Walter de Boer <walterdeboer@dbso.nl>

walterdeboer force-pushed the feature/2567 branch from b427473 to b5330eb Compare June 23, 2023 21:03

FIx random failing PolicyEngineTest

b58d2cc

Signed-off-by: Walter de Boer <walterdeboer@dbso.nl>

walterdeboer requested a review from melba-lopez June 27, 2023 20:03

walterdeboer requested a review from nscuro June 27, 2023 20:03

melba-lopez approved these changes Jun 28, 2023

View reviewed changes

nscuro approved these changes Jun 28, 2023

View reviewed changes

nscuro merged commit a7e5122 into DependencyTrack:master Jun 28, 2023
10 checks passed

nscuro changed the title ~~New option to only return outdated components and/or only direct dependenncies in the ComponentResource~~ New option to only return outdated components and/or only direct dependencies in the ComponentResource Jul 7, 2023

nscuro mentioned this pull request Jul 7, 2023

Display when updated version are available in Dependency Graph DependencyTrack/frontend#538

Open

2 tasks

github-actions bot locked as resolved and limited conversation to collaborators Aug 6, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

New option to only return outdated components and/or only direct dependencies in the ComponentResource #2568

New option to only return outdated components and/or only direct dependencies in the ComponentResource #2568

walterdeboer commented Mar 6, 2023 •

edited

UWollesen commented Mar 29, 2023

walterdeboer commented Mar 29, 2023 via email •

edited

UWollesen commented Mar 30, 2023

walterdeboer commented Mar 30, 2023 •

edited

UWollesen commented Mar 31, 2023

msymons commented Apr 27, 2023

melba-lopez Jun 20, 2023

nscuro Jun 21, 2023

melba-lopez Jun 20, 2023

walterdeboer Jun 23, 2023

walterdeboer Jun 23, 2023

melba-lopez Jun 20, 2023

walterdeboer Jun 23, 2023

melba-lopez commented Jun 27, 2023

walterdeboer commented Jun 27, 2023

melba-lopez left a comment

nscuro left a comment

New option to only return outdated components and/or only direct dependencies in the ComponentResource #2568

New option to only return outdated components and/or only direct dependencies in the ComponentResource #2568

Conversation

walterdeboer commented Mar 6, 2023 • edited

Description

Addressed Issue

Additional Details

Checklist

UWollesen commented Mar 29, 2023

walterdeboer commented Mar 29, 2023 via email • edited

UWollesen commented Mar 30, 2023

walterdeboer commented Mar 30, 2023 • edited

UWollesen commented Mar 31, 2023

msymons commented Apr 27, 2023

melba-lopez Jun 20, 2023

Choose a reason for hiding this comment

nscuro Jun 21, 2023

Choose a reason for hiding this comment

melba-lopez Jun 20, 2023

Choose a reason for hiding this comment

walterdeboer Jun 23, 2023

Choose a reason for hiding this comment

walterdeboer Jun 23, 2023

Choose a reason for hiding this comment

melba-lopez Jun 20, 2023

Choose a reason for hiding this comment

walterdeboer Jun 23, 2023

Choose a reason for hiding this comment

melba-lopez commented Jun 27, 2023

walterdeboer commented Jun 27, 2023

melba-lopez left a comment

Choose a reason for hiding this comment

nscuro left a comment

Choose a reason for hiding this comment

walterdeboer commented Mar 6, 2023 •

edited

walterdeboer commented Mar 29, 2023 via email •

edited

walterdeboer commented Mar 30, 2023 •

edited