--vuln-type library returning null in JSON when no vulnerabilites are found

[OwenMatsuda]

Description

Command running: trivy image --list-all-pkgs --vuln-type library -f json debian:10.6
Option --vuln-type library returns null in JSON when no vulnerabilities are found, even if --list-all-pkgs is also present.

What did you expect to happen?

I would expect it to return in JSON the Target, Type, the list of Packages, and an empty list of Vulnerabilities.

What happened instead?

It simply returns null.

Output of run with -debug:

2021-02-01T08:29:44.155-1000	DEBUG	Severities: UNKNOWN,LOW,MEDIUM,HIGH,CRITICAL
2021-02-01T08:29:44.208-1000	DEBUG	cache dir:  /Users/$USER/Library/Caches/trivy
2021-02-01T08:29:44.209-1000	INFO	Need to update DB
2021-02-01T08:29:44.209-1000	INFO	Downloading DB...
2021-02-01T08:29:44.903-1000	DEBUG	release name: v1-2021020112
2021-02-01T08:29:44.904-1000	DEBUG	asset name: trivy-light-offline.db.tgz
2021-02-01T08:29:44.904-1000	DEBUG	file name doesn't match
2021-02-01T08:29:44.904-1000	DEBUG	asset name: trivy-light.db.gz
2021-02-01T08:29:44.904-1000	DEBUG	file name doesn't match
2021-02-01T08:29:44.904-1000	DEBUG	asset name: trivy-offline.db.tgz
2021-02-01T08:29:44.904-1000	DEBUG	file name doesn't match
2021-02-01T08:29:44.904-1000	DEBUG	asset name: trivy.db.gz
2021-02-01T08:29:45.029-1000	DEBUG	asset URL: https://github-releases.githubusercontent.com/216830441/a61a3f80-6488-11eb-8c55-6d691ed757aa?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20210201%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20210201T182807Z&X-Amz-Expires=300&X-Amz-Signature=d514164bb9e900efb7df53ffe94bb28674ad8ccfab9b8b445de19bd8a1a9f4f8&X-Amz-SignedHeaders=host&actor_id=0&key_id=0&repo_id=216830441&response-content-disposition=attachment%3B%20filename%3Dtrivy.db.gz&response-content-type=application%2Foctet-stream
19.93 MiB / 19.93 MiB [--------------------------------] 100.00% 4.32 MiB p/s 5s
2021-02-01T08:29:50.202-1000	DEBUG	Updating database metadata...
2021-02-01T08:29:50.203-1000	DEBUG	DB Schema: 1, Type: 1, UpdatedAt: 2021-02-01 12:22:39.009526881 +0000 UTC, NextUpdate: 2021-02-02 00:22:39.009526481 +0000 UTC, DownloadedAt: 2021-02-01 18:29:50.202728 +0000 UTC
2021-02-01T08:29:53.630-1000	DEBUG	Vulnerability type:  [library]
2021-02-01T08:29:58.979-1000	DEBUG	Artifact ID: sha256:ef05c61d51129e3866d5b71b4f44864919dd2b9e5f2644f0a511703182acf8f9
2021-02-01T08:29:58.979-1000	DEBUG	Blob IDs: [sha256:114ca5b7280f3b49e94a67659890aadde83d58a8bde0d9020b2bc8c902c3b9de]
2021-02-01T08:29:58.980-1000	INFO	Trivy skips scanning programming language libraries because no supported file was detected

Output of trivy -v:

Version: 0.15.0
Vulnerability DB:
  Type: Light
  Version: 1
  UpdatedAt: 2021-02-01 12:22:39.009526881 +0000 UTC
  NextUpdate: 2021-02-02 00:22:39.009526481 +0000 UTC
  DownloadedAt: 2021-02-01 18:29:50.202728 +0000 UTC

Additional details (base image name, container registry info...):

Just tested using Debian:10.6.

[knqyf263]

You specified --vuln-type library and it scans only dependencies of programming languages. debian:10.6 has only OS packages, so it returns an empty. If you want to see the list of OS packages, you need to specify --vuln-type os with --list-all-pkgs.

[OwenMatsuda]

Shouldn't specifying --vuln-type library be specific only to scanning for vulnerabilities? It seems misleading to set an argument called --vuln-type and have it affect the list of packages. Although, the concern isn't that it returns an empty. It returns only null with nothing else. I would expect it to at least have something like:

[
  {
    "Target": "debian:10.6 (debian 10.6)",
    "Type": "debian",
    "Packages": [],
    "Vulnerabilities": []
  }
]

However, it returns just

null

[knqyf263]

I see. It makes sense. Trivy probably should return a list of OS packages if --list-all-pkgs is specified, even though --vuln-type library is also specified.

[mtpereira]

Hi! 👋 I'm looking at this issue in the context of Hacktoberfest 2021.

---

After spending some time looking at the source, it seems to be that this is more of a usability issue: the flag --list-all-pkgs as 2 different meanings, depending on the vulnerability type requested:

• scanOSPkgs (--vuln-type os) will list OS packages when --list-all-pkgs is also passed;
• scanLibrary (--vuln-type lirabry, the case reported initially) will list "application" packages (i.e., Ruby gems, Python packages, etc) when --list-all-pkgs is also passed.

And in fact, if we pass --list-all-pkgs and --vuln-type os it works as expected:

./trivy image --list-all-pkgs --format json --vuln-type os debian:10.6 |
2021-10-18T22:02:49.512+0100	�[34mINFO�[0m	Detected OS: debian
2021-10-18T22:02:49.512+0100	�[34mINFO�[0m	Detecting Debian vulnerabilities...
{
  "SchemaVersion": 2,
  "ArtifactName": "debian:10.6",
  "ArtifactType": "container_image",
  "Metadata": {
    "OS": {
      "Family": "debian",
      "Name": "10.6"
    },
[snip]
  "Results": [
    {
      "Target": "debian:10.6 (debian 10.6)",
      "Class": "os-pkgs",
      "Type": "debian",
      "Packages": [
        {
          "Name": "adduser",
          "Version": "3.118",
          "SrcName": "adduser",
          "SrcVersion": "3.118",
          "Layer": {
            "Digest": "sha256:756975cb9c7e7933d824af9319b512dd72a50894232761d06ef3be59981df838",
            "DiffID": "sha256:114ca5b7280f3b49e94a67659890aadde83d58a8bde0d9020b2bc8c902c3b9de"
          }
        },
[snip]

And if you pass it an image with Application packages, it lists them correctly (but not the OS packages):

./trivy image --list-all-pkgs --format json --vuln-type library mtpereira/serverless:1.32.0 
2021-10-18T22:11:11.167+0100	�[34mINFO�[0m	Number of language-specific files: 2
2021-10-18T22:11:11.167+0100	�[34mINFO�[0m	Detecting python-pkg vulnerabilities...
2021-10-18T22:11:11.168+0100	�[34mINFO�[0m	Detecting node-pkg vulnerabilities...
{
  "SchemaVersion": 2,
  "ArtifactName": "mtpereira/serverless:1.32.0",
  "ArtifactType": "container_image",
  "Metadata": {
    "OS": {
      "Family": "alpine",
      "Name": "3.8.1"
    },
**[snip]**
    
  "Results": [
    {
      "Target": "Node.js",
      "Class": "lang-pkgs",
      "Type": "node-pkg",
      "Packages": [
        {
          "Name": "ignore-walk",
          "Version": "3.0.1",
          "License": "ISC",
          "Layer": {
            "Digest": "sha256:bd603adec670d2232a2b80a57427d9cb87a509795d23f7e5e33ff384cec34f2e",
            "DiffID": "sha256:cb18274b2dd164cdee39e92aef2d59482f2fac0f66edd2a863b5dbcab0048498"
          },
          "FilePath": "usr/local/lib/node_modules/npm/node_modules/ignore-walk/package.json"
        }
**[snip]**

I believe this is somewhat the opposite of what was described here:

Shouldn't specifying --vuln-type library be specific only to scanning for vulnerabilities? It seems misleading to set an argument called --vuln-type and have it affect the list of packages.

It is not --vuln-type library that is affecting --list-all-pkgs, but the other way around.

---

All this being said, I'd like to propose 2 possible solutions:

1. Improve the help description of --list-all-pkgs to clarify what it does in each contexts.
2. Have separate flags for listing OS and Application packages, making it clearer which is which and what they list.

Personally I'd prefer option 1, has it does not require adding yet another entry to ScanOptions, and all the propagation of that in the code base, but I'm happy to implement either!

Thanks for any input on how to proceed, @knqyf263 and @OwenMatsuda ! 🙇

[SimonAlling]

Not even specific to --vuln-type, AFAICT. Let's say I want to check images for the presence of Log4Shell, using these images as examples:

• ghcr.io/christophetd/log4shell-vulnerable-app, which has Log4Shell
• alpine:3.13.6, which doesn't have Log4Shell but does have other vulnerabilities
• alpine:3.13.7, which has no vulnerabilities at all

$ JQ_FILTER='.Results | map(.Vulnerabilities) | add | any(.VulnerabilityID == "CVE-2021-44228")'
$ trivy -q image -f json ghcr.io/christophetd/log4shell-vulnerable-app | jq "${JQ_FILTER}"
true
$ trivy -q image -f json alpine:3.13.6 | jq "${JQ_FILTER}"
false
$ trivy -q image -f json alpine:3.13.7 | jq "${JQ_FILTER}"
jq: error (at <stdin>:60): Cannot iterate over null (null)
$ trivy -q image -f json alpine:3.13.7 | jq .Results
[
  {
    "Target": "alpine:3.13.7 (alpine 3.13.7)",
    "Class": "os-pkgs",
    "Type": "alpine"
  }
]

In my opinion, the only reasonable way to represent the empty set of vulnerabilities in JSON is [], but it is represented by … uhm … omission. And without omitempty, it is represented as null in JSON, i.e. nil in Go unless I'm mistaken.

Unfortunately, fixing this may not be trivial due to Go's garbage type system, where nil is a member of almost every type. In the case of running trivy image alpine:3.13.7, the nil that ends up being ignored in the JSON output kind of seems to originate in Detect in pkg/detector/ospkg/alpine/alpine.go.
However, even with

diff --git a/pkg/detector/ospkg/alpine/alpine.go b/pkg/detector/ospkg/alpine/alpine.go
index d4b39b10..d8c89724 100644
--- a/pkg/detector/ospkg/alpine/alpine.go
+++ b/pkg/detector/ospkg/alpine/alpine.go
@@ -87,7 +87,8 @@ func (s *Scanner) Detect(osVer string, pkgs []ftypes.Package) ([]types.DetectedV
        log.Logger.Debugf("alpine: os version: %s", osVer)
        log.Logger.Debugf("alpine: the number of packages: %d", len(pkgs))
 
-       var vulns []types.DetectedVulnerability
+       var vulns = []types.DetectedVulnerability{}
+
        for _, pkg := range pkgs {
                advisories, err := s.vs.Get(osVer, pkg.SrcName)
                if err != nil {
diff --git a/pkg/report/writer.go b/pkg/report/writer.go
index 0e5769e0..ab55195e 100644
--- a/pkg/report/writer.go
+++ b/pkg/report/writer.go
@@ -58,7 +58,7 @@ type Result struct {
        Class             ResultClass                      `json:"Class,omitempty"`
        Type              string                           `json:"Type,omitempty"`
        Packages          []ftypes.Package                 `json:"Packages,omitempty"`
-       Vulnerabilities   []types.DetectedVulnerability    `json:"Vulnerabilities,omitempty"`
+       Vulnerabilities   []types.DetectedVulnerability    `json:"Vulnerabilities"`
        MisconfSummary    *MisconfSummary                  `json:"MisconfSummary,omitempty"`
        Misconfigurations []types.DetectedMisconfiguration `json:"Misconfigurations,omitempty"`
 }

the value is null in the final JSON, so it seems like the Vulnerabilities field is overwritten with nil somewhere in the process. I tried tracking down where, but tracing data flow in Go is a daunting task, and the type checker certainly isn't very helpful.

[YuviGold]

Hey, I was looking to contribute to the repository and this looks like a nice issue :)
From what I can tell there are 3 different issues raised in this thread.

1. When a result has no Vulnerabilities, it omits the Vulnerabilities field. The same goes for every other member of the Result struct since they all have the omitempty field.
   

   trivy/pkg/report/writer.go

   Lines 59 to 67 in 8da20c8

   	type Result struct {
   	Target string `json:"Target"`
   	Class ResultClass `json:"Class,omitempty"`
   	Type string `json:"Type,omitempty"`
   	Packages []ftypes.Package `json:"Packages,omitempty"`
   	Vulnerabilities []types.DetectedVulnerability `json:"Vulnerabilities,omitempty"`
   	MisconfSummary *MisconfSummary `json:"MisconfSummary,omitempty"`
   	Misconfigurations []types.DetectedMisconfiguration `json:"Misconfigurations,omitempty"`
   	}

2. When specifying a specific vulnerability type using --vuln-type and there are no findings, the entire Results array is empty.
   It makes it harder for scripts to check specifically a certain field.

Shouldn't specifying --vuln-type library be specific only to scanning for vulnerabilities? It seems misleading to set an argument called --vuln-type and have it affect the list of packages. Although, the concern isn't that it returns an empty. It returns only null with nothing else. I would expect it to at least have something like:

[
  {
    "Target": "debian:10.6 (debian 10.6)",
    "Type": "debian",
    "Packages": [],
    "Vulnerabilities": []
  }
]

However, it returns just

null

3. The --list-all-pkgs flag makes no effect when combined with --vuln-type library although it should.
   As @knqyf263 mentioned.

I see. It makes sense. Trivy probably should return a list of OS packages if --list-all-pkgs is specified, even though --vuln-type library is also specified.

I'm going work only on resolving the 3rd one as it was acknowledged.
@knqyf263 Please let me know from product perspective whether the 1st and the 2nd issues are also need solving or are they working like that by design. I was able to solve them locally but I want to make sure if that's necessary before opening a PR :)

[knqyf263]

Hi @YuviGold, thanks for summarizing. After some consideration, I think the name of the option --vuln-type is confusing and the root cause of this issue. As @OwenMatsuda mentioned,

It seems misleading to set an argument called --vuln-type and have it affect the list of packages.

--pkg-type or something like that may be better to clarify it affects the list of packages and vulnerabilities both. We also need to update the help message of the option. Please let me know if you have a better idea.

If --pkg-type library is specified, Trivy should not show any information regarding OS packages. It means it doesn't show a list of OS packages and vulnerabilities even when --list-all-pkgs is specified.

So, I think we should fix #1. To keep simplicity, we should not support #3. I'm sorry to confuse you, @YuviGold.

We have updated the JSON schema, so it no longer returns null.
#1050

If null Results causes an issue, we can replace it with an empty [].

[YuviGold]

I'll start from the bottom - regarding the last point, you are right.
The JSON schema doesn't return null but the Results is empty, so I was wondering whether it would make much sense to explicitly mention there are no results instead. (i.e. for scripts)

Regarding the --pkg-type flag - IMO it makes the same confusion. Since the returned Result report has both Packages and Vulnerabilities. So instead of mentioning the latter, it's the former, while it would still be the same behavior.
Maybe it should be just --scan-type? A scan type is agnostic to the actual results coming up from scanOSPkgs (which probably should be renamed to just scanOS since it returns both packages and vulnerabilities) or scanLibrary functions.

WDYT? @knqyf263

[knqyf263]

--scan-type sounds like how Trivy will scan. But we want to describe what will be scanned here. Like --scan-target...? I believe we can have a better name... Naming is always difficult for me 😔

[YuviGold]

--scan-type sounds like how Trivy will scan. But we want to describe what will be scanned here. Like --scan-target...? I believe we can have a better name... Naming is always difficult for me 😔

Honestly --scan-target sounds great to me!
Then, I'll drop the ongoing PR for this change.

In the spirit of renaming flags, would you like to rename --list-all-pkgs to just --list-pkgs?

[knqyf263]

Trivy supports misconfigurations as well as vulnerabilities. I'm concerned that --scan-target sounds like vulnerabilities or misconfigurations. We already have the option --security-checks for that purpose.

Trivy supports two types of packages now.

• OS packages
• Language-specific packages
  • we used to call it "library", but we changed it to language-specific packages since OS package is also a library and ambiguous.

I have summarized the expected behavior.

	OS package list	OS package vulnerabilities	Langueage-specific package list	Language-specific vulnerabilities
--vuln-type os	❌	✅	❌	❌
--vuln-type library	❌	❌	❌	✅
--vuln-type os,library	❌	✅	❌	✅
--vuln-type os --list-all-pkgs	✅	✅	❌	❌
--vuln-type library --list-all-pkgs	❌	❌	✅	✅
--vuln-type os,library --list-all-pkgs	✅	✅	✅	✅

--pkg-type doesn't mean a package type to be listed, but to be listed and scanned. Does anybody have an idea? --list-and-scan-pkg-type is too long 😢

Also, "all" in --list-all-pkgs means showing all the packages even if they don't have any vulnerabilities. Trivy only shows packages with any vulnerabilities by default. Is it confusing? Is --list-pkgs better?

[YuviGold]

Thank you for the elaborative explanation!

I think that whenever a user wants to check an image its either to scan it or to list it. I don't see why to do both, it would just print a lot of data and could mislead what actually is vulnerable.

In that case, it might be more trivial to have a subcommand for trivy image list and trivy image scan accordingly.

In addition, you mentioned misconfiguration as a security check. I think that these two different levels are a bit confusing...
From what I understand:

• security checks
• • misconfigurations
• • vulnerabilities
• • • OS packages
• • • Language specific packagaes

What about flattening it to be a 1 level security-check/scan-target? I believe misconfiguration is also an exploit/vulnerability and this distinction causes part of the confusion

[knqyf263]

In that case, it might be more trivial to have a subcommand for trivy image list and trivy image scan accordingly.

This will be a breaking change. I want to keep the command as is because Trivy is a vulnerability scanner and the package listing feature should be done in conjunction with vulnerability detection.

I believe misconfiguration is also an exploit/vulnerability and this distinction causes part of the confusion

We've never heard of the confusion due to vulnerabilities/misconfigurations so far. Actually, vulnerabilities are patch-related things. In a general sense, misconfiguration is also a vulnerability as you said, but in the IT industry, I think they are recognized as different things.

[YuviGold]

I see.
In that case

Also, "all" in --list-all-pkgs means showing all the packages even if they don't have any vulnerabilities. Trivy only shows packages with any vulnerabilities by default. Is it confusing? Is --list-pkgs better?

@knqyf263 Yes, I think --list-pkgs is far better.
Lots of discussions for such a small change. Anything else? :)