FATAL error in image scan: failed to analyze image: failed to extract files: Could not extract the archive #139

UnAfraid · 2019-08-28T08:49:49Z

Description

Attempting to scan few public packages like openjdk and it fails to extract the archive, but downloading it locally and using the hash works fine
It looks related to the tags because if i don't specify one it works, even latest causes it to fail with Could not extract archive

What happened?
OpenJDK

root@docker2:~# trivy --exit-code 0 --clear-cache --auto-refresh openjdk:11.0-jre
2019-08-28T11:40:12.337+0300    INFO    Removing image caches...
2019-08-28T11:40:12.344+0300    INFO    Updating vulnerability database...
2019-08-28T11:40:20.931+0300    FATAL   error in image scan: failed to analyze image: failed to extract files: Could not extract the archive

Docker

root@docker2:~# trivy --clear-cache --auto-refresh docker:stable-dind
2019-08-28T11:44:47.411+0300    INFO    Removing image caches...
2019-08-28T11:44:47.474+0300    INFO    Updating vulnerability database...
2019-08-28T11:44:58.257+0300    FATAL   error in image scan: failed to analyze image: failed to extract files: Could not extract the archive

Golang

root@docker2:~# trivy --clear-cache --auto-refresh golang:1.12-alpine
2019-08-28T11:46:45.829+0300    INFO    Removing image caches...
2019-08-28T11:46:45.848+0300    INFO    Updating vulnerability database...
2019-08-28T11:47:02.212+0300    FATAL   error in image scan: failed to analyze image: failed to extract files: Could not extract the archive

Output of run with -debug:

root@docker2:~# trivy -d --clear-cache --auto-refresh openjdk:11.0-jre
2019-08-28T11:42:45.246+0300    DEBUG   cache dir:  /root/.cache/trivy
2019-08-28T11:42:45.246+0300    INFO    Removing image caches...
2019-08-28T11:42:45.256+0300    DEBUG   db path: /root/.cache/trivy/db/trivy.db
2019-08-28T11:42:45.263+0300    INFO    Updating vulnerability database...
2019-08-28T11:42:45.263+0300    DEBUG   git pull
2019-08-28T11:42:46.437+0300    DEBUG   total updated files: 1
2019-08-28T11:42:46.441+0300    DEBUG   Vulnerability type:  [os library]
2019-08-28T11:42:53.927+0300    FATAL   error in image scan:
    github.com/aquasecurity/trivy/pkg.Run
        /root/project/pkg/run.go:164
  - failed to analyze image:
    github.com/aquasecurity/trivy/pkg/scanner.ScanImage
        /root/project/pkg/scanner/scan.go:34
  - failed to extract files:
    github.com/aquasecurity/fanal/analyzer.Analyze
        /go/pkg/mod/github.com/aquasecurity/fanal@v0.0.0-20190819081512-f04452b627c6/analyzer/analyzer.go:127
  - Could not extract the archive
    github.com/aquasecurity/fanal/extractor.init.ializers
        /go/pkg/mod/github.com/aquasecurity/fanal@v0.0.0-20190819081512-f04452b627c6/extractor/extractor.go:12
    runtime.main
        /usr/local/go/src/runtime/proc.go:188
    runtime.goexit
        /usr/local/go/src/runtime/asm_amd64.s:1337
root@docker2:~#

Output of trivy -v:

root@docker2:~# trivy -v
trivy version 0.1.6
root@docker2:~#

Additional details (base image name, container registry info...):
openjdk:11.0-jre
openjdk:latest
docker:stable-dind
docker:latest
golang:1.12-alpine
golang:latest

And more

The text was updated successfully, but these errors were encountered:

masahiro331 · 2019-08-29T04:43:49Z

@UnAfraid
Please check below tasks.

docker pull golang:1.12-alpine
trivy golang:1.12-alpine

If you failed task 1, check your network.

UnAfraid · 2019-08-29T14:01:21Z

@masahiro331
It works if i download the image, i mention that above
I don't have any issues with the network
I only happens when i add tag, if i leave the image without tag it seems to work.

knqyf263 · 2019-08-31T14:26:19Z

@UnAfraid Thank you for reporting. This is due to network timeout. Currently, the timeout value is hardcoded, so we have to implement --timeout option.

masahiro331 · 2019-09-01T23:31:35Z

@UnAfraid

Does the below code solve it?

export TRIVY_TIMEOUT_SEC=120s
trivy golang:latest

UnAfraid · 2019-09-02T07:24:46Z

@masahiro331
with v0.1.6 no, i tried that while i was looking for similar issue, some people say it helps it did not help for me.
I tried it just now again still no.
Result is the same

masahiro331 · 2019-09-02T13:31:48Z

@UnAfraid
Perhaps this problem can be solved.
Please clear the cache and extend the timeout.

export TRIVY_TIMEOUT_SEC=360s
trivy --clear-cache golang:latest

There are two problems.

The first is that the first scan timeout.
Second, the tar created on timeout was cached incompletely.

UnAfraid · 2019-09-02T15:40:29Z

i tried with --clear-cache because there is warning saying to use it whenever the tag is 'latest'

masahiro331 · 2019-09-02T17:14:31Z

I'm sorry. Thank you for cooperating many times.

Does the error occur even if the timeout is actually extended and clear-cache is specified?

Is the cache updated?

ls -la  /root/.cache/fanal
trivy --clear-cache golang:latest
ls -la  /root/.cache/fanal

UnAfraid · 2019-09-03T08:17:17Z

root@docker2:~# ls -la  /root/.cache/fanal
total 49468
drwxr-xr-x 2 root root     4096 Sep  3 11:16 .
drwx------ 7 root root     4096 Sep  3 11:16 ..
-rw-r--r-- 1 root root      478 Sep  3 11:16 sha256:16a3d8aca6cd76595d0fa99f7dcaf6c388c021bc6f5020d794e8292b88e5aac3
-rw-r--r-- 1 root root      479 Sep  3 11:16 sha256:2ea1f7804402db2da64e84a26bd591f41667ad69cd7f2a2c6106d9bb04dde260
-rw-r--r-- 1 root root 50379856 Sep  3 11:16 sha256:4ae16bd4778367b46064f39554128dd2fda2803a5747fddeff74059f353391c9
-rw-r--r-- 1 root root   244866 Sep  3 11:16 sha256:96465440c20877524189ae75d361dd29e5d0df330a8dac9427f972b429fe0159
-rw-r--r-- 1 root root      125 Sep  3 11:16 sha256:96d705baf026261543501205212451776c49ddaa48a0347afab3494275e0fc13
-rw-r--r-- 1 root root      478 Sep  3 11:16 sha256:bbab4ec87ac4f89eaabdf68dddbd1dd930e3ad43bded38d761b89abf9389a893
-rw-r--r-- 1 root root      477 Sep  3 11:16 sha256:e0ec5610455ae43994616afca2caf7da592cb975d7474ed112baa42e5c616d17
root@docker2:~# trivy --clear-cache golang:latest
2019-09-03T11:16:47.393+0300    INFO    Removing image caches...
2019-09-03T11:16:47.412+0300    INFO    Updating vulnerability database...
2019-09-03T11:16:58.489+0300    FATAL   error in image scan: failed to analyze image: failed to extract files: Could not extract the archive
root@docker2:~# ls -la  /root/.cache/fanal
total 57096
drwxr-xr-x 2 root root     4096 Sep  3 11:16 .
drwx------ 7 root root     4096 Sep  3 11:16 ..
-rw-r--r-- 1 root root   253232 Sep  3 11:16 sha256:16a3d8aca6cd76595d0fa99f7dcaf6c388c021bc6f5020d794e8292b88e5aac3
-rw-r--r-- 1 root root      479 Sep  3 11:16 sha256:2ea1f7804402db2da64e84a26bd591f41667ad69cd7f2a2c6106d9bb04dde260
-rw-r--r-- 1 root root 50379856 Sep  3 11:16 sha256:4ae16bd4778367b46064f39554128dd2fda2803a5747fddeff74059f353391c9
-rw-r--r-- 1 root root      475 Sep  3 11:16 sha256:96465440c20877524189ae75d361dd29e5d0df330a8dac9427f972b429fe0159
-rw-r--r-- 1 root root      125 Sep  3 11:16 sha256:96d705baf026261543501205212451776c49ddaa48a0347afab3494275e0fc13
-rw-r--r-- 1 root root  7804467 Sep  3 11:16 sha256:bbab4ec87ac4f89eaabdf68dddbd1dd930e3ad43bded38d761b89abf9389a893
-rw-r--r-- 1 root root     1369 Sep  3 11:16 sha256:e0ec5610455ae43994616afca2caf7da592cb975d7474ed112baa42e5c616d17
root@docker2:~#

masahiro331 · 2019-09-04T04:47:13Z

When I tried it in another environment, happened connection reset by peer.
The timeout setting was not applied in this environment.

Add debug log point.
https://github.com/aquasecurity/fanal/blob/master/extractor/docker/docker.go#L272

$ ./trivy --clear-cache golang:latest
2019-09-04T13:39:51.736+0900	INFO	Removing image caches...
2019-09-04T13:39:51.757+0900	INFO	Updating vulnerability database...
read tcp 10.15.53.23:53957->104.18.121.25:443: read: connection reset by peer # Add log
2019-09-04T13:40:24.344+0900	FATAL	error in image scan: failed to analyze image: failed to extract files: Could not extract the archive

christian-weiss · 2019-12-06T11:06:53Z

@knqyf263 please add more debug lines here:

dump that archive to disk (i guess it sometimes may contain HTML or other responses)
print a warning with a file path to that dumped archive

A debug line, that informs me that "Updating vulnerability database..." has finished would be helpful, to be sure that "read: connection reset by peer" is from image download and not from DB download (less guessing, easyer to read logs).

Please also add messages that informs me if trivy had successful connected to local docker daemon but couldn't find that image locally. Of if it skips the local daemon (could not find the daemon, etc.) and does a direct fallback to remote repo or if it is skipping local docker daemon because of using trivy cache - would be helpful on debugging (more clear situation).

dlemel8 · 2020-10-04T19:56:35Z

Hey @knqyf263 @masahiro331 , is this issue is still relevant?
I've tried to contribute by resolving this issue, but I can't reproduce it.
It also seems the the cli has changed.

knqyf263 · 2020-10-05T07:08:37Z

@christian-weiss I'm sorry to miss your message for a long time.

A debug line, that informs me that "Updating vulnerability database..." has finished would be helpful, to be sure that "read: connection reset by peer" is from image download and not from DB download (less guessing, easyer to read logs).

We have a progress bar now and you can know if the DB has finished downloading.

Please also add messages that informs me if trivy had successful connected to local docker daemon but couldn't find that image locally. Of if it skips the local daemon (could not find the daemon, etc.) and does a direct fallback to remote repo or if it is skipping local docker daemon because of using trivy cache - would be helpful on debugging (more clear situation).

--debug shows the detail now such as Docker Engine doesn't work or the image doesn't exist in your local Docker Engine.

Thanks.

knqyf263 · 2020-10-05T07:12:58Z

@dlemel8 Thank you for the offer. I believe this issue was fixed with --timeout option, so let me close it. If someone is still facing this issue, please feel free to mention it here.

automation555 · 2021-12-09T07:57:16Z

021-12-09T13:21:54.590+0530 FATAL scan error:
github.com/aquasecurity/trivy/pkg/commands/artifact.runWithTimeout
/home/runner/work/trivy/trivy/pkg/commands/artifact/run.go:71

image scan failed:
github.com/aquasecurity/trivy/pkg/commands/artifact.scan
/home/runner/work/trivy/trivy/pkg/commands/artifact/run.go:216
failed analysis:
github.com/aquasecurity/trivy/pkg/scanner.Scanner.ScanArtifact
/home/runner/work/trivy/trivy/pkg/scanner/scan.go:98
analyze error:
github.com/aquasecurity/fanal/artifact/image.Artifact.Inspect
/home/runner/go/pkg/mod/github.com/aquasecurity/fanal@v0.0.0-20211111090223-628ff1de3ee1/artifact/image/image.go:105
timeout:
github.com/aquasecurity/fanal/artifact/image.Artifact.inspect
/home/runner/go/pkg/mod/github.com/aquasecurity/fanal@v0.0.0-20211111090223-628ff1de3ee1/artifact/image/image.go:175
context deadline exceeded

I am getting such errors while scanning my image.
command I have used is "trivy -debug --timeout=32000000s image "

trivy version

trivy --version
Version: 0.21.1
Vulnerability DB:
Type: Full
Version: 1
UpdatedAt: 2021-12-09 06:41:19.012028373 +0000 UTC
NextUpdate: 2021-12-09 12:41:19.012028073 +0000 UTC
DownloadedAt: 2021-12-09 07:44:20.201428587 +0000 UTC

thangamani-arun · 2022-01-21T07:44:50Z

This issue still exist for trivy version 0.17.2. could you please confirm in which version does it implemented and working ?

thangamani-arun · 2022-01-21T07:46:07Z

@knqyf263 any comment please ?

afdesk · 2022-01-21T07:49:28Z

@thangamani-arun thanks for your report!
Could you try the latest trivy version v0.22.0?

https://github.com/aquasecurity/trivy/releases/tag/v0.22.0

thangamani-arun · 2022-01-24T07:02:15Z

@afdesk We have tried with v0.20.1, it is taking the timeout value as configured. Thanks for your support.

* Created analyzer for NuGet. Signed-off-by: Johannes Tegnér <johannes@jitesoft.com> * refactor(nuget): sort imports Co-authored-by: knqyf263 <knqyf263@gmail.com>

…asecurity#139) Co-authored-by: Teppei Fukuda <knqyf263@gmail.com>

UnAfraid added the kind/bug Categorizes issue or PR as related to a bug. label Aug 28, 2019

knqyf263 added the good first issue Denotes an issue ready for a new contributor, according to the "help wanted" guidelines. label Aug 31, 2019

masahiro331 mentioned this issue Sep 1, 2019

Add timeout option #143

Merged

shad0wrunner mentioned this issue Sep 3, 2019

"Failed to analyze image: failed to extract files" error goodwithtech/dockle#51

Closed

christian-weiss mentioned this issue Dec 6, 2019

Scan for image built in a previous stage of a TravisCI build looks instead in the docker registry #198

Closed

morawskim mentioned this issue Apr 8, 2020

CI - trivy błąd stasiu20/ssorder#120

Closed

knqyf263 closed this as completed Oct 5, 2020

liamg pushed a commit that referenced this issue Jun 7, 2022

Feat: Created analyzer for NuGet lockfiles. (#139)

410bee7

* Created analyzer for NuGet. Signed-off-by: Johannes Tegnér <johannes@jitesoft.com> * refactor(nuget): sort imports Co-authored-by: knqyf263 <knqyf263@gmail.com>

liamg pushed a commit that referenced this issue Jun 7, 2022

Feat: Created analyzer for NuGet lockfiles. (#139)

48a25d8

* Created analyzer for NuGet. Signed-off-by: Johannes Tegnér <johannes@jitesoft.com> * refactor(nuget): sort imports Co-authored-by: knqyf263 <knqyf263@gmail.com>

josedonizetti referenced this issue in josedonizetti/trivy Jun 24, 2022

feat: add unit tests for AWS SNS rules (#139)

3335f7a

knqyf263 added a commit to knqyf263/trivy that referenced this issue Feb 9, 2024

refactor(jar): write debug message when wrong inner jar is found (aqu…

a5451f4

…asecurity#139) Co-authored-by: Teppei Fukuda <knqyf263@gmail.com>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

FATAL error in image scan: failed to analyze image: failed to extract files: Could not extract the archive #139

FATAL error in image scan: failed to analyze image: failed to extract files: Could not extract the archive #139

UnAfraid commented Aug 28, 2019 •

edited

Loading

masahiro331 commented Aug 29, 2019

UnAfraid commented Aug 29, 2019

knqyf263 commented Aug 31, 2019

masahiro331 commented Sep 1, 2019

UnAfraid commented Sep 2, 2019

masahiro331 commented Sep 2, 2019

UnAfraid commented Sep 2, 2019

masahiro331 commented Sep 2, 2019 •

edited

Loading

UnAfraid commented Sep 3, 2019

masahiro331 commented Sep 4, 2019 •

edited

Loading

christian-weiss commented Dec 6, 2019

dlemel8 commented Oct 4, 2020

knqyf263 commented Oct 5, 2020 •

edited

Loading

knqyf263 commented Oct 5, 2020

automation555 commented Dec 9, 2021 •

edited

Loading

thangamani-arun commented Jan 21, 2022

thangamani-arun commented Jan 21, 2022

afdesk commented Jan 21, 2022 •

edited

Loading

thangamani-arun commented Jan 24, 2022

FATAL error in image scan: failed to analyze image: failed to extract files: Could not extract the archive #139

FATAL error in image scan: failed to analyze image: failed to extract files: Could not extract the archive #139

Comments

UnAfraid commented Aug 28, 2019 • edited Loading

masahiro331 commented Aug 29, 2019

UnAfraid commented Aug 29, 2019

knqyf263 commented Aug 31, 2019

masahiro331 commented Sep 1, 2019

UnAfraid commented Sep 2, 2019

masahiro331 commented Sep 2, 2019

UnAfraid commented Sep 2, 2019

masahiro331 commented Sep 2, 2019 • edited Loading

UnAfraid commented Sep 3, 2019

masahiro331 commented Sep 4, 2019 • edited Loading

christian-weiss commented Dec 6, 2019

dlemel8 commented Oct 4, 2020

knqyf263 commented Oct 5, 2020 • edited Loading

knqyf263 commented Oct 5, 2020

automation555 commented Dec 9, 2021 • edited Loading

thangamani-arun commented Jan 21, 2022

thangamani-arun commented Jan 21, 2022

afdesk commented Jan 21, 2022 • edited Loading

thangamani-arun commented Jan 24, 2022

UnAfraid commented Aug 28, 2019 •

edited

Loading

masahiro331 commented Sep 2, 2019 •

edited

Loading

masahiro331 commented Sep 4, 2019 •

edited

Loading

knqyf263 commented Oct 5, 2020 •

edited

Loading

automation555 commented Dec 9, 2021 •

edited

Loading

afdesk commented Jan 21, 2022 •

edited

Loading