feat: add git submodule analyzer

[nejch]

Description

Provides a git submodule analyzer, currently mainly for SBOM generation. A future use case for vulnerability scanning (if feasible) would be for people who build their dependencies from source rather than packages (very common in C/C++ and the embedded ecosystem).

This is an initial PR to get feedback. We discussed with @DmitriyLewen in aquasecurity/go-dep-parser#144 (which adds a similar feature) it might be better to start here with a draft for these non-packaged dependencies.

The full parser is here in trivy rather than in go-dep-parser because we only use .gitmodules to discover if submodules are present, but the actual parsing needs to be done in the git index (git commands via go-git in this case).

The testing set up is a bit more involved because I couldn't find a good way to mock all submodule-related plumbing. However, it's still quite performant, and it tests against real git behavior:

go test -run ^Test_gitSubmoduleAnalyzer_Analyze$ github.com/aquasecurity/trivy/pkg/fanal/analyzer/git/submodule

ok  	github.com/aquasecurity/trivy/pkg/fanal/analyzer/git/submodule	0.075s

Result for https://github.com/git-fixtures/submodule:

make build
./trivy repo --format cyclonedx https://github.com/git-fixtures/submodule.git
2022-12-27T12:32:45.043+0100    INFO    "--format cyclonedx" disables security checks. Specify "--security-checks vuln" explicitly if you want to include vulnerabilities in the CycloneDX report.
Enumerating objects: 4, done.
Counting objects: 100% (4/4), done.
Compressing objects: 100% (3/3), done.
Total 4 (delta 0), reused 4 (delta 0), pack-reused 0
{
  "bomFormat": "CycloneDX",
  "specVersion": "1.4",
  "serialNumber": "urn:uuid:59c26bf6-a0f0-4668-a6eb-37755c29272a",
  "version": 1,
  "metadata": {
    "timestamp": "2022-12-27T11:32:45+00:00",
    "tools": [
      {
        "vendor": "aquasecurity",
        "name": "trivy",
        "version": "0.35.0-43-g52c1fb4f"
      }
    ],
    "component": {
      "bom-ref": "9f318aaf-e2a4-4a08-9a40-dab75d22d767",
      "type": "application",
      "name": "https://github.com/git-fixtures/submodule.git",
      "properties": [
        {
          "name": "aquasecurity:trivy:SchemaVersion",
          "value": "2"
        }
      ]
    }
  },
  "components": [
    {
      "bom-ref": "pkg:git-submodule/https:%2F%2Fgithub.com%2Fgit-fixtures%2Fbasic.git@6ecf0ef2c2dffb796033e5a02219af86ec6584e5",
      "type": "library",
      "name": "https://github.com/git-fixtures/basic.git",
      "version": "6ecf0ef2c2dffb796033e5a02219af86ec6584e5",
      "purl": "pkg:git-submodule/https:%2F%2Fgithub.com%2Fgit-fixtures%2Fbasic.git@6ecf0ef2c2dffb796033e5a02219af86ec6584e5",
      "properties": [
        {
          "name": "aquasecurity:trivy:PkgType",
          "value": "git-submodule"
        }
      ]
    },
    {
      "bom-ref": "pkg:git-submodule/https:%2F%2Fgithub.com%2Fgit-fixtures%2Fsubmodule.git@47770b26e71b0f69c0ecd494b1066f8d1da4fc03",
      "type": "library",
      "name": "https://github.com/git-fixtures/submodule.git",
      "version": "47770b26e71b0f69c0ecd494b1066f8d1da4fc03",
      "purl": "pkg:git-submodule/https:%2F%2Fgithub.com%2Fgit-fixtures%2Fsubmodule.git@47770b26e71b0f69c0ecd494b1066f8d1da4fc03",
      "properties": [
        {
          "name": "aquasecurity:trivy:PkgType",
          "value": "git-submodule"
        }
      ]
    },
    {
      "bom-ref": "f73aab6d-1d44-4025-bf3a-ec253b31dfcf",
      "type": "application",
      "name": ".gitmodules",
      "properties": [
        {
          "name": "aquasecurity:trivy:Type",
          "value": "git-submodule"
        },
        {
          "name": "aquasecurity:trivy:Class",
          "value": "lang-pkgs"
        }
      ]
    }
  ],
  "dependencies": [
    {
      "ref": "f73aab6d-1d44-4025-bf3a-ec253b31dfcf",
      "dependsOn": [
        "pkg:git-submodule/https:%2F%2Fgithub.com%2Fgit-fixtures%2Fbasic.git@6ecf0ef2c2dffb796033e5a02219af86ec6584e5",
        "pkg:git-submodule/https:%2F%2Fgithub.com%2Fgit-fixtures%2Fsubmodule.git@47770b26e71b0f69c0ecd494b1066f8d1da4fc03"
      ]
    },
    {
      "ref": "9f318aaf-e2a4-4a08-9a40-dab75d22d767",
      "dependsOn": [
        "f73aab6d-1d44-4025-bf3a-ec253b31dfcf"
      ]
    }
  ],
  "vulnerabilities": []
}

Time on the boost library with 161 submodules:

make build
git clone https://github.com/boostorg/boost.git ../boost/
time ./trivy fs --format cyclonedx ../boost/
# ...
real    0m0.332s
user    0m0.447s
sys     0m0.035s

Related issues

• Part 1 of Analyzer support for git-based dependencies #3067

Related PRs

• feat: add kas parser go-dep-parser#143 (another git-based dependency parser)
• feat: add gitlab-ci include parser go-dep-parser#144 (another git-based dependency parser)

Checklist

• I've read the guidelines for contributing to this repository.
• I've followed the conventions in the PR title.
• I've added tests that prove my fix is effective or that my feature works.
• I've updated the documentation with the relevant information (if needed).
• I've added usage information (if the PR introduces new options)
• I've included a "before" and "after" example to the description (if the PR is a user interface change).

[nejch]

@knqyf263 just wanted to check whether I should keep rebasing this to keep it up to date from conflicts, let me know if you'd still be interested in this PR. Thanks a lot 🙇

[github-actions]

This PR is stale because it has been labeled with inactivity.

[nejch]

This PR is stale because it has been labeled with inactivity.

I was rebasing in the past to keep it up-to-date, but there was no activity here. I'll rebase next when the bot complains again.

[knqyf263]

We're sorry to be late. Could you suspend the work until further notice? We hope to discuss it this month.

[github-actions]

This PR is stale because it has been labeled with inactivity.