Trivy should read Maven repositories from settings.xml

[candrews]

Description

Currently, Trivy reads Maven repositories from the repositories section of pom.xml of the project being scanned and checks settings.xml only for the username/password credentials to access that repository.

Trivy should also read repositories from settings.xml. That would better mimic Maven's behavior, further improving scan accuracy and improving offline support.

For example, given this ~/.m2/settings.xml:

<?xml version="1.0" encoding="UTF-8"?>
<settings>
    <servers>
        <server>
            <id>mycompany-maven-central-mirror</id>
            <username>myusername</username>
            <password>mypassword</password>
        </server>
        <server>
            <id>mycompany-internal-releases</id>
            <username>mypassword</username>
            <password>mypassword</password>
        </server>
        <server>
            <id>mycompany-maven-central-mirror</id>
            <username>mypassword</username>
            <password>mypassword</password>
        </server>
        <server>
            <id>mycompany-internal-snapshots</id>
            <username>mypassword</username>
            <password>mypassword</password>
        </server>
    </servers>
    <profiles>
        <profile>
            <id>mycompany</id>
            <repositories>
                <repository>
                    <id>mycompany-maven-central-mirror</id>
                    <url>https://mycompany.example.com/repository/maven-central-mirror</url>
                    <releases>
                        <checksumPolicy>fail</checksumPolicy>
                    </releases>
                    <snapshots>
                        <checksumPolicy>fail</checksumPolicy>
                    </snapshots>
                </repository>
                <repository>
                    <id>mycompany-internal-releases</id>
                    <url>https://mycompany.example.com/repository/internal-releases</url>
                    <releases>
                        <checksumPolicy>fail</checksumPolicy>
                    </releases>
                    <snapshots>
                        <enabled>false</enabled>
                    </snapshots>
                </repository>
                <repository>
                    <id>mycompany-internal-snapshots</id>
                    <url>https://mycompany.example.com/repository/internal-snapshots</url>
                    <releases>
                        <enabled>false</enabled>
                        <checksumPolicy>fail</checksumPolicy>
                    </releases>
                    <snapshots>
                        <enabled>true</enabled>
                        <checksumPolicy>fail</checksumPolicy>
                    </snapshots>
                </repository>
            </repositories>
        </profile>
    </profiles>
    <activeProfiles>
        <activeProfile>mycompany</activeProfile>
    </activeProfiles>
    <mirrors>
        <mirror>
            <id>mycompany-maven-central-mirror</id>
            <name>mycompany-maven-central-mirror</name>
            <url>https://mycompany.example.com/repository/maven-central-mirror</url>
            <mirrorOf>central</mirrorOf>
        </mirror>
    </mirrors>
</settings>

Trivy would:

• Add mycompany-internal-releases with the URL https://mycompany.example.com/repository/internal-releases and the provided username/password to the repositories to search for releases
• Add mycompany-internal-snapshots with the URL https://mycompany.example.com/repository/internal-snapshots and the provided username/password to the repositories to search for snapshots
• Instead of add https://repo.maven.apache.org/maven2/ to the releases repositories list, add https://mycompany.example.com/repository/maven-central-mirror and use the provided username/password to access it because it is configured as a mirrorOf central

Target

None

Scanner

None

[DmitriyLewen]

Hello @candrews
Thanks for information!

Add mycompany-internal-releases with the URL https://mycompany.example.com/repository/internal-releases and the provided username/password to the repositories to search for releases
Add mycompany-internal-snapshots with the URL https://mycompany.example.com/repository/internal-snapshots and the provided username/password to the repositories to search for snapshots

Can you tell about the order of the repositories in this case?
for example, if settings.xml and pom.xml contain repositories, which repository will maven access first?

Instead of add https://repo.maven.apache.org/maven2/ to the releases repositories list, add https://mycompany.example.com/repository/maven-central-mirror and use the provided username/password to access it because it is configured as a mirrorOf central

In this case we don't need to check https://repo.maven.apache.org/maven2/. Right? We only need to check https://mycompany.example.com/repository/maven-central-mirror

[candrews]

In this case we don't need to check https://repo.maven.apache.org/maven2/. Right? We only need to check https://mycompany.example.com/repository/maven-central-mirror

Correct. <mirrorOf>central</mirrorOf> replace https://repo.maven.apache.org/maven2/ which is great for offline use cases.

[candrews]

Can you tell about the order of the repositories in this case?
for example, if settings.xml and pom.xml contain repositories, which repository will maven access first?

The repository search order is documented at https://maven.apache.org/guides/mini/guide-multiple-repositories.html#repository-order

Remote repository URLs are queried in the following order for artifacts until one returns a valid result:

effective settings:
    Global settings.xml
    User settings.xml
local effective build POM:
    Local pom.xml
    Parent POMs, recursively
    Super POM
effective POMs from dependency path to the artifact.

For each of these locations, the repositories within the profiles are queried first in the order outlined at Introduction to build profiles.

Before downloading from a repository, mirrors configuration is applied.

Effective settings and local build POM, with profile taken into account, can easily be reviewed to see their repositories order with mvn help:effective-settings and mvn help:effective-pom -Dverbose.

[DmitriyLewen]

We have already found a bug in this documentation - aquasecurity/go-dep-parser#289 (comment).

Can you check the order using mvn?

[malmor]

Hey, this feature would be really helpful for us too.

We are running trivy inside our CI/CD pipelines and scan multiple Java-based projects with pom.xml files. We have an internal JFrog artifactory instance running at https://repo.company.local which contains internal packages and is also configured as a proxy for multiple upstream repositories (e.g. https://repo1.maven.org/maven2/, https://packages.atlassian.com/maven-external/).

During our builds we use a settings.xml similar to the example above to run all traffic through our central instance.

I am not that familiar with Go, so I am not sure if I can create a pull request with such a change / refactoring. Is there something else (examples, reproductions, ...) we could provide to help with the implementation?

[DmitriyLewen]

Hello @malmor
Unfortunately, we have a lot of work at the moment. So we don't have time to work on all tasks.

I created #7807 for this task.

Is there something else (examples, reproductions, ...) we could provide to help with the implementation?

You can help with testing how mvn chooses priority for remote repositories in some cases (write your result in #7807 please):

• when there are multiple profiles
• when settings.xml and pom.xml use remote repositories
• does the order of repositories change if maven central is rewritten
• perhaps you can think more controversial cases

[DmitriyLewen]

Created #7807 for this task.