trivy image scan suddenly timing out #3421
Comments
|
Hello, I am having the same issue there. |
knqyf263
added
triage/support
|
Hello, maybe it is related to search.maven.org: |
|
Looks like GitHub Container Registry distributing Trivy DB is something wrong now. Please try it again later. |
|
Seeing the same issue with Trivy |
|
Is everyone scanning images with JAR files? If so, maven might be relevant like @dusik66 pointed out. |
|
In my case, yes, I am scanning an image for a java proj. |
|
Can you try |
|
I reported the same issue here: aquasecurity/trivy-action#190 |
|
If you run with
https://aquasecurity.github.io/trivy/v0.36/docs/advanced/air-gap/ |
|
so |
Right. |
|
To skip downloading DB, you can pass |
Until current timeout issues are resolved. See: aquasecurity/trivy#3421 See also: aquasecurity/trivy-action#190
|
We are probably facing a similar issue — starting from today (the first fail was at
Setting |
|
What happens if we set |
This comment was marked as duplicate.
This comment was marked as duplicate.
1 similar comment
This comment was marked as duplicate.
This comment was marked as duplicate.
This comment was marked as duplicate.
This comment was marked as duplicate.
|
Still happening in my case and works after retrying 3 times: |
|
Hello guys!! I checked this problem now. Looks like it was temporal down and it works now. @paultung may be your problem has different reason. Can you send your |
|
Hello, the problem still exists. |
|
Hello @Dmitry-Dymnov Also can you say version of Trivy? We fixed case where Trivy would stop scanning after receiving error for internal jars.(#2989) Regards, Dmitriy |
|
@DmitriyLewen, until yesterday, these images were scanned without problems. I'm sorry, but I can't give you the file ((( |
I've just run a trivy execution and it's failing again |
|
seems like search.maven.org is still unstable or has a more aggressive rate limiting. What is search.maven.org used for? Only for checking whether an update fixing a vulnerability of maven dependency is available? |
|
Can someone explain how Trivy checks if a given image is a Java image and takes the decision to invoke the Maven endpoint which seems to be going down this last week? Also...
Why is it not recommended to remove the Maven dependency? (Truly out of curiosity, does it plays a great part into Java vulnerabilities?) |
Correct me if I am wrong. I think we want to keep the maven search functionality as needed as mentioned above for jar identification method sake. Removing the maven search dependency doesn't really help unless there is a better way for jar identification as durcon mentioned above. |
This comment was marked as off-topic.
This comment was marked as off-topic.
This comment was marked as off-topic.
This comment was marked as off-topic.
|
Hey guys, just letting you know I'm still observing the timeout error in my pipeline flow. When running Trivy with --debug option enabled, I can observe that Trivy struggles while trying to call search.maven.org still: 2023-02-01T00:04:35.742Z DEBUG retrying request {"request": "GET https://search.maven.org/solrsearch/select?q=1%3A%22053aae0415da5e4be7a7dfbae6d63880d0765b97%22&rows=1&wt=json (status: 504)", "timeout": "20s", "remaining": 5}
2023-02-01T00:04:36.282Z DEBUG retrying request {"request": "GET https://search.maven.org/solrsearch/select?q=1%3A%228eed85a5337c7b4e84156cc0f3921930fb0d780b%22&rows=1&wt=json (status: 504)", "timeout": "20s", "remaining": 5}
.
.
.
2023-02-01T00:07:12.147Z FATAL image scan error:
github.com/aquasecurity/trivy/pkg/commands/artifact.Run
/home/runner/work/trivy/trivy/pkg/commands/artifact/run.go:397
- scan error:
github.com/aquasecurity/trivy/pkg/commands/artifact.(*runner).scanArtifact
/home/runner/work/trivy/trivy/pkg/commands/artifact/run.go:257
- scan failed:
github.com/aquasecurity/trivy/pkg/commands/artifact.scan
/home/runner/work/trivy/trivy/pkg/commands/artifact/run.go:588
- failed analysis:
github.com/aquasecurity/trivy/pkg/scanner.Scanner.ScanArtifact
/home/runner/work/trivy/trivy/pkg/scanner/scan.go:[146](https://code.ifoodcorp.com.br/ifood/local-commerce/oms/connection/gateway-core/-/jobs/22925032#L146)
- analyze error:
github.com/aquasecurity/trivy/pkg/fanal/artifact/image.Artifact.Inspect
/home/runner/work/trivy/trivy/pkg/fanal/artifact/image/image.go:127
- timeout:
github.com/aquasecurity/trivy/pkg/fanal/artifact/image.Artifact.inspect
/home/runner/work/trivy/trivy/pkg/fanal/artifact/image/image.go:248
- context deadline exceeded
[INFO] - Generating JSON report...
2023-02-01T00:12:12.456Z FATAL image scan error: scan error: scan failed: failed analysis: analyze error: timeout: context deadline exceededMaven status page reported a downtime of 23 minutes earlier today. Although the timeout, I can still make the request locally from my machine using an HTTP client just fine, with the request being answered in 1.5s: |
This comment was marked as off-topic.
This comment was marked as off-topic.
This comment was marked as off-topic.
This comment was marked as off-topic.
|
Thanks all for your patience. We managed to include the improvement in v0.37.0 🎉🎉 We're grateful for everyone who helped make it happen. Trivy no longer relies on the third-party API and instead uses the pre-built database for Java scanning. The Java index database will be downloaded when JAR files are found. The scan results should be almost the same as before. You don't have to use It can be used even in air-gapped environments💯 |
|
It may have a critical issue. We're on it. |
Oh no - I was just about to test it in our pipeline! Thankyou for the quick notification though. |
It's working fine for me this morning. Both locally, and in AWS CodePipeline. |
|
v0.37.0 has a critical issue. Please try v0.37.1 instead. |
|
v0.37.1 works for us Thankyou. |
|
Hi, looks good, but I'm running a central Trivy server to support CI pipelines, but the Java DB seems to get downloaded by each client (taking about 10s) instead of being downloaded and cached by the server. Is that intended (or have I misconfigured something)? I guess ideally only the server would directly download the Java DB every few days, potentially transferring it to the clients if necessary. |
Good question. Yes, we're aware of that and it is intended for now as most users are running Trivy in standalone mode. We needed to get the Java scan working as soon as possible, so we put client/server aside. Improving client/server will be the next task. |
|
FYI: there are two known issues.
|
|
Just a quick question as we updated today to 0.37.1: We run a central trivy server for trivy clients that run in our build pipelines to minimize download traffic. This works perfectly fine for the existing vuln-db(s?) but not for the java-db, it seems that the clients try to download the java-db individually and do not use the server for that. |
Do you use the flags |
|
We cut v0.37.2 and it should improve the performance. Please let us know if you still see the same issue with v0.37.2. Those who are using v0.37.0 and v0.37.1 need to update Trivy to v0.37.2 as it has a breaking change. You may see the following error message. Please clear the cached database with
Also, we added mitigation for #3514. We think it will not happen in most cases, but we will try to fix the root cause in v0.38.0. You can track the topic of the Java DB in client/server mode here. Summary
Thanks for your patience. |
|
@knqyf263 please urgently update the trivy-action as well with this new version. This is a big breaking change in our pipeline since you release trivy v0.37.2 |
I created already a PR for it and wait for approval so that the pipeline is running. 😆 |
|
What about people using "skip-update" in an off-line environment? db:
# We are air-gapped
skip-update: true Edit: Oh.. Apparently we now also need to download a separate "Java index database".... https://aquasecurity.github.io/trivy/v0.37/docs/advanced/air-gap/#download-the-java-index-database1 |
and others
Trivy image scan was taking less than 1 minute as of yesterday. Now suddenly it is timing out.
Settting ``--security-checks vuln` has no effect, as was suggested from output.
Tried this with v0.24.2 and current latest
The text was updated successfully, but these errors were encountered: