Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unable to download meta file: https://nvd.nist.gov/feeds/json/cve/1.0/nvdcve-1.0-modified.meta #2039

Closed
meselfi opened this issue Jul 2, 2019 · 40 comments

Comments

@meselfi
Copy link

commented Jul 2, 2019

Added dependency-check to a fresh maven project and got this error? Can't download the referenced resource from at browser either.

[ERROR] Failed to execute goal org.owasp:dependency-check-maven:5.1.0:check (default) on project x: Fatal exception(s) analyzing X: One or more exceptions occurred during analysis:
[ERROR] Unable to download meta file: https://nvd.nist.gov/feeds/json/cve/1.0/nvdcve-1.0-modified.meta
[ERROR] No documents exist
[ERROR] -> [Help 1]

@meselfi meselfi added the question label Jul 2, 2019
@AILazerka

This comment has been minimized.

Copy link

commented Jul 2, 2019

Same happens for me on Gradle when after some time of usage of the plugin I started failing the issue in logs of the build pipeline. Tried updating to latest version, but issue happens constantly using any of versions (then only urls are different).

Checking for updates and analyzing dependencies for vulnerabilities
Unable to download meta file: https://nvd.nist.gov/feeds/json/cve/1.0/nvdcve-1.0-modified.json.gz
Unable to update 1 or more Cached Web DataSource, using local data instead. Results may not include recent vulnerabilities.
Unable to continue dependency-check analysis.
Generating report for project ...
:dependencyCheckAnalyze FAILED
====== !LONG RUNNING TASK! ======
:dependencyCheckAnalyze took 30292ms

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':dependencyCheckAnalyze'.
> java.lang.NullPointerException (no error message)

...

nvd.nist.gov cannot be even discovered.

@AILazerka

This comment has been minimized.

Copy link

commented Jul 2, 2019

@meselfi, this question relates to the
#2002

@meselfi

This comment has been minimized.

Copy link
Author

commented Jul 2, 2019

Seems nvd.nist.gov is down. https://twitter.com/SorenTPoulsen/status/1145998287322996736

@emansom

This comment has been minimized.

Copy link

commented Jul 2, 2019

Also experiencing this issue. Results in Jenkins jobs failing.

@rjimgal

This comment has been minimized.

Copy link
Contributor

commented Jul 2, 2019

Might be a good opportunity to set up a Nexus OSS raw proxy repository and starting caching ;)

@stepio

This comment has been minimized.

Copy link
Contributor

commented Jul 2, 2019

@rjimgal ,

I had exactly the same idea. But is there a simple way to configure plugin for using internal proxy instead of the official website?

@OrangeDog

This comment has been minimized.

Copy link

commented Jul 2, 2019

@emansom add <failOnError>false</failOnError> to not fail the build.
@rjimgal how are you going to start caching something that's currently broken?

@Nriver

This comment has been minimized.

Copy link

commented Jul 2, 2019

nvd.nist.gov has been down for hours today, but it is up now. It would be nice to have a place for a mirror data server just in case the site is down and you can not get DependencyCheck working on a new environment.

@emansom

This comment has been minimized.

Copy link

commented Jul 2, 2019

@emansom add <failOnError>false</failOnError> to not fail the build.

CVE checking is a policy enforced requirement here. Thanks for the tip though!

nvd.nist.gov is up again! 🎉

@rjimgal

This comment has been minimized.

Copy link
Contributor

commented Jul 2, 2019

@stepio cveUrlModified and cveUrlBase can be configured (https://jeremylong.github.io/DependencyCheck/dependency-check-maven/configuration.html)

@OrangeDog just to prepare yourself for next outage ;-)

@jeremylong

This comment has been minimized.

Copy link
Owner

commented Jul 3, 2019

I highly recommend the usage of the nist-data-mirror.

@ddugovic

This comment has been minimized.

Copy link

commented Jul 4, 2019

I have running a local nist-data-mirror which I can download from, yet in my project build I cannot figure out where to set cveUrlBase or cveUrlModified (everything I am trying has no effect whatsoever on the build, although I can connect to my mirror in a web browser):

  • my project pom.xml ?
                        <plugin>
                                <groupId>org.owasp</groupId>
                                <artifactId>dependency-check-maven</artifactId>
                                <version>4.0.2</version>
                                <configuration>
                                        <cveUrlBase>http://MYHOST/nvdcve-1.0-%d.json.gz</cveUrlBase>
                                        <cveUrlModified>http://MYHOST/nvdcve-1.0-modified.json.gz</cveUrlModified>
                                </configuration>
                                <executions>
                                        <execution>
                                                <goals>
                                                        <goal>check</goal>
                                                </goals>
                                        </execution>
                                </executions>
                        </plugin>
  • on the Maven command line executed by Jenkins (-DcveUrlBase=... etc.) ?
  • some other configuration file ?
@malejpavouk

This comment has been minimized.

Copy link

commented Jul 4, 2019

If you do not have a highly available central DB (or want to work offline), it is also possible to dockerize the NVD database.

For those that are Gradle shops, we have prepared two articles on how to achieve it: https://medium.com/zoom-techblog/dockerized-dependency-check-building-nvd-image-a5af78cc6228

The code is under MIT, feel free to use it.

@albuch

This comment has been minimized.

Copy link
Contributor

commented Jul 4, 2019

@ddugovic JSON Feeds we're implemented with v5.0.0 of dependency-check-maven. You should upgrade to 5.1.0 so that your posted configuration works.
Before that (v4.0.2 and earlier) the XML feeds were used which use different configuration properties.

@ddugovic

This comment has been minimized.

Copy link

commented Jul 4, 2019

@albuch Thanks very much, upgrading to 5.1.0 solves my problem!

@stepio

This comment has been minimized.

Copy link
Contributor

commented Jul 8, 2019

@rjimgal, @jeremylong,

I have a small question about proxying.

Correct me if I'm wrong, but it looks that as of now plugin supports proxying only *.gz files, but not *.meta files. So this a nice tip to improve build time, but it won't help with issues like we had in this thread.

screenshot

Or do I miss anything?

@stepio

This comment has been minimized.

Copy link
Contributor

commented Jul 10, 2019

I was wrong. Got next results:

[DEBUG] Attempting retrieval of https://my.example.com/artifactory/nvd-nist-gov/nvdcve-1.0-modified.meta
...
[DEBUG] Attempting retrieval of https://my.example.com/artifactory/nvd-nist-gov/nvdcve-1.0-2002.meta
[DEBUG] Attempting retrieval of https://my.example.com/artifactory/nvd-nist-gov/nvdcve-1.0-2003.meta
[DEBUG] Attempting retrieval of https://my.example.com/artifactory/nvd-nist-gov/nvdcve-1.0-2004.meta
[DEBUG] Attempting retrieval of https://my.example.com/artifactory/nvd-nist-gov/nvdcve-1.0-2005.meta
...
[DEBUG] Attempting retrieval of https://my.example.com/artifactory/nvd-nist-gov/nvdcve-1.0-2004.json.gz

With next configuration properties:

<cveUrlBase>https://my.example.com/artifactory/nvd-nist-gov/nvdcve-1.0-%d.json.gz</cveUrlBase>
<cveUrlModified>https://my.example.com/artifactory/nvd-nist-gov/nvdcve-1.0-modified.json.gz</cveUrlModified>
@denniseffing

This comment has been minimized.

Copy link

commented Aug 1, 2019

@emansom add <failOnError>false</failOnError> to not fail the build.

Adding this does not work. The build fails regardless.
Is there any way to ignore nvd.nist.gov downtimes and let the build pass without introducing a cache? This would allow us to work on caching at a later date.

@jeremylong

This comment has been minimized.

Copy link
Owner

commented Aug 1, 2019

Consider using a local NVD cache like the (nist-data-mirror](https://github.com/stevespringett/nist-data-mirror)

@denniseffing

This comment has been minimized.

Copy link

commented Aug 1, 2019

This requires setting up a nightly job which pulls down the latest NVD files from NIST as stated here. This also requires setting up an infrastructure component that makes these files available internally.

We would like to postpone infrastructure configuration for NVD file caching but use the dependency-check-maven plugin without failing the build if NIST is not available. However, this does not seem to be possible.

@Nriver

This comment has been minimized.

Copy link

commented Aug 2, 2019

Is there a way to disable update? When NIST is not avaliable, I can not do the scan. But I have scanned before and a previous database is downloaded, should it be nice to run the check with the stroed database by disable the update when NIST is not avaliable.

@jeremylong

This comment has been minimized.

Copy link
Owner

commented Aug 3, 2019

Disable the autoUpdate property. This varies depending on if you are using the CLI, maven or gradle plugin, etc. See the documentation

@denniseffing

This comment has been minimized.

Copy link

commented Aug 3, 2019

Let me rephrase my issue:
The Maven dependency check plugin fails the build if the website is not available, even if the property failOnError is set to false. We want the database to automatically update but don't want to fail our build if NIST is not available. This does not seem to be possible at the moment.

@jeremylong

This comment has been minimized.

Copy link
Owner

commented Aug 3, 2019

Yes - as the database must exist.

@2Fey

This comment has been minimized.

Copy link

commented Aug 12, 2019

I have same problem, cause Jenkins proxy have bug. This bug fixed in notrealesed version.
https://issues.jenkins-ci.org/browse/JENKINS-57383?page=com.atlassian.streams.streams-jira-plugin%3Aactivity-stream-issue-tab .
So I decide to install nist-data-mirror.
After I fix issue with install nist-data-mirror, I find new problem. My jenkins can't download https://raw.githubusercontent.com/Retirejs/retire.js/master/repository/jsrepository.json , cause proxy is broke in 2.176.2 version. @jeremylong have you got any information about mirrors to Retirejs?

[DependencyCheck] [ERROR] Failed to initialize the RetireJS repo
[DependencyCheck] org.owasp.dependencycheck.data.update.exception.UpdateException: Failed to initialize the RetireJS repo
[DependencyCheck] 	at org.owasp.dependencycheck.data.update.RetireJSDataSource.initializeRetireJsRepo(RetireJSDataSource.java:151)
[DependencyCheck] 	at org.owasp.dependencycheck.data.update.RetireJSDataSource.update(RetireJSDataSource.java:97)
[DependencyCheck] 	at org.owasp.dependencycheck.Engine.doUpdates(Engine.java:922)
[DependencyCheck] 	at org.owasp.dependencycheck.Engine.initializeAndUpdateDatabase(Engine.java:723)
[DependencyCheck] 	at org.owasp.dependencycheck.Engine.analyzeDependencies(Engine.java:653)
[DependencyCheck] 	at org.owasp.dependencycheck.App.runScan(App.java:251)
[DependencyCheck] 	at org.owasp.dependencycheck.App.run(App.java:183)
[DependencyCheck] 	at org.owasp.dependencycheck.App.main(App.java:80)
[DependencyCheck] Caused by: org.owasp.dependencycheck.utils.DownloadFailedException: Download failed, unable to copy 'https://raw.githubusercontent.com/Retirejs/retire.js/master/repository/jsrepository.json' to '/var/jenkins_home/tools/org.jenkinsci.plugins.DependencyCheck.tools.DependencyCheckInstallation/dependency-check-5.2.1/data/jsrepository.json'
[DependencyCheck] 	at org.owasp.dependencycheck.utils.Downloader.fetchFile(Downloader.java:91)
[DependencyCheck] 	at org.owasp.dependencycheck.utils.Downloader.fetchFile(Downloader.java:68)
[DependencyCheck] 	at org.owasp.dependencycheck.data.update.RetireJSDataSource.initializeRetireJsRepo(RetireJSDataSource.java:149)
[DependencyCheck] 	... 7 common frames omitted
[DependencyCheck] Caused by: org.owasp.dependencycheck.utils.DownloadFailedException: Error downloading file https://raw.githubusercontent.com/Retirejs/retire.js/master/repository/jsrepository.json; unable to connect.
[DependencyCheck] 	at org.owasp.dependencycheck.utils.HttpResourceConnection.obtainConnection(HttpResourceConnection.java:238)
[DependencyCheck] 	at org.owasp.dependencycheck.utils.HttpResourceConnection.fetch(HttpResourceConnection.java:138)
[DependencyCheck] 	at org.owasp.dependencycheck.utils.Downloader.fetchFile(Downloader.java:87)
[DependencyCheck] 	... 9 common frames omitted
[DependencyCheck] Caused by: java.net.SocketTimeoutException: connect timed out
[DependencyCheck] 	at java.net.PlainSocketImpl.socketConnect(Native Method)
[DependencyCheck] 	at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
[DependencyCheck] 	at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
[DependencyCheck] 	at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
[DependencyCheck] 	at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
[DependencyCheck] 	at java.net.Socket.connect(Socket.java:589)
[DependencyCheck] 	at sun.net.NetworkClient.doConnect(NetworkClient.java:175)
[DependencyCheck] 	at sun.net.www.http.HttpClient.openServer(HttpClient.java:463)
[DependencyCheck] 	at sun.net.www.http.HttpClient.openServer(HttpClient.java:558)
[DependencyCheck] 	at sun.net.www.protocol.https.HttpsClient.<init>(HttpsClient.java:264)
[DependencyCheck] 	at sun.net.www.protocol.https.HttpsClient.New(HttpsClient.java:367)
[DependencyCheck] 	at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.getNewHttpClient(AbstractDelegateHttpsURLConnection.java:191)
[DependencyCheck] 	at sun.net.www.protocol.http.HttpURLConnection.plainConnect0(HttpURLConnection.java:1156)
[DependencyCheck] 	at sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:1050)
[DependencyCheck] 	at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:177)
[DependencyCheck] 	at sun.net.www.protocol.https.HttpsURLConnectionImpl.connect(HttpsURLConnectionImpl.java:162)
[DependencyCheck] 	at org.owasp.dependencycheck.utils.HttpResourceConnection.obtainConnection(HttpResourceConnection.java:178)
[DependencyCheck] 	... 11 common frames omitted
@2Fey

This comment has been minimized.

Copy link

commented Aug 12, 2019

I find command --retireJsUrl so i can just create my mirror with httpd. So I think that just adding jsrepository.json page is enough, is it?
https://jeremylong.github.io/DependencyCheck/dependency-check-cli/arguments.html

@jeremylong

This comment has been minimized.

Copy link
Owner

commented Aug 13, 2019

Correct - you just need to mirror the additional file.

@ST-DDT

This comment has been minimized.

Copy link

commented Sep 24, 2019

Server is down again/broken. Test-URL: https://nvd.nist.gov/feeds/json/cve/1.0/nvdcve-1.0-modified.meta

Stacktrace (Click to expand)
[INFO] --- dependency-check-maven:5.2.1:check (cve-check) @ test-service ---
[INFO] Checking for updates
[ERROR] Unable to download meta file: https://nvd.nist.gov/feeds/json/cve/1.0/nvdcve-1.0-modified.meta
org.owasp.dependencycheck.data.update.exception.UpdateException: Unable to download meta file: https://nvd.nist.gov/feeds/json/cve/1.0/nvdcve-1.0-modified.meta
    at org.owasp.dependencycheck.data.update.NvdCveUpdater.getMetaFile (NvdCveUpdater.java:347)
    at org.owasp.dependencycheck.data.update.NvdCveUpdater.getUpdatesNeeded (NvdCveUpdater.java:385)
    at org.owasp.dependencycheck.data.update.NvdCveUpdater.update (NvdCveUpdater.java:122)
    at org.owasp.dependencycheck.Engine.doUpdates (Engine.java:922)
    at org.owasp.dependencycheck.Engine.initializeAndUpdateDatabase (Engine.java:723)
    at org.owasp.dependencycheck.Engine.analyzeDependencies (Engine.java:653)
    at org.owasp.dependencycheck.maven.BaseDependencyCheckMojo.runCheck (BaseDependencyCheckMojo.java:1403)
    at org.owasp.dependencycheck.maven.BaseDependencyCheckMojo.execute (BaseDependencyCheckMojo.java:802)
    at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo (DefaultBuildPluginManager.java:137)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:210)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:156)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:148)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:117)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:81)
    at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build (SingleThreadedBuilder.java:56)
    at org.apache.maven.lifecycle.internal.LifecycleStarter.execute (LifecycleStarter.java:128)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:305)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:192)
    at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:105)
    at org.apache.maven.cli.MavenCli.execute (MavenCli.java:956)
    at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:288)
    at org.apache.maven.cli.MavenCli.main (MavenCli.java:192)
    at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
    at jdk.internal.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62)
    at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke (Method.java:566)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (Launcher.java:282)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launch (Launcher.java:225)
    at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (Launcher.java:406)
    at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:347)
Caused by: org.owasp.dependencycheck.utils.DownloadFailedException: Download failed, unable to retrieve 'https://nvd.nist.gov/feeds/json/cve/1.0/nvdcve-1.0-modified.meta'
    at org.owasp.dependencycheck.utils.Downloader.fetchContent (Downloader.java:115)
    at org.owasp.dependencycheck.data.update.NvdCveUpdater.getMetaFile (NvdCveUpdater.java:340)
    at org.owasp.dependencycheck.data.update.NvdCveUpdater.getUpdatesNeeded (NvdCveUpdater.java:385)
    at org.owasp.dependencycheck.data.update.NvdCveUpdater.update (NvdCveUpdater.java:122)
    at org.owasp.dependencycheck.Engine.doUpdates (Engine.java:922)
    at org.owasp.dependencycheck.Engine.initializeAndUpdateDatabase (Engine.java:723)
    at org.owasp.dependencycheck.Engine.analyzeDependencies (Engine.java:653)
    at org.owasp.dependencycheck.maven.BaseDependencyCheckMojo.runCheck (BaseDependencyCheckMojo.java:1403)
    at org.owasp.dependencycheck.maven.BaseDependencyCheckMojo.execute (BaseDependencyCheckMojo.java:802)
    at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo (DefaultBuildPluginManager.java:137)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:210)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:156)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:148)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:117)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:81)
    at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build (SingleThreadedBuilder.java:56)
    at org.apache.maven.lifecycle.internal.LifecycleStarter.execute (LifecycleStarter.java:128)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:305)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:192)
    at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:105)
    at org.apache.maven.cli.MavenCli.execute (MavenCli.java:956)
    at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:288)
    at org.apache.maven.cli.MavenCli.main (MavenCli.java:192)
    at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
    at jdk.internal.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62)
    at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke (Method.java:566)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (Launcher.java:282)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launch (Launcher.java:225)
    at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (Launcher.java:406)
    at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:347)
Caused by: org.owasp.dependencycheck.utils.DownloadFailedException: Error downloading file https://nvd.nist.gov/feeds/json/cve/1.0/nvdcve-1.0-modified.meta; unable to connect.
    at org.owasp.dependencycheck.utils.HttpResourceConnection.obtainConnection (HttpResourceConnection.java:238)
    at org.owasp.dependencycheck.utils.HttpResourceConnection.fetch (HttpResourceConnection.java:138)
    at org.owasp.dependencycheck.utils.Downloader.fetchContent (Downloader.java:110)
    at org.owasp.dependencycheck.data.update.NvdCveUpdater.getMetaFile (NvdCveUpdater.java:340)
    at org.owasp.dependencycheck.data.update.NvdCveUpdater.getUpdatesNeeded (NvdCveUpdater.java:385)
    at org.owasp.dependencycheck.data.update.NvdCveUpdater.update (NvdCveUpdater.java:122)
    at org.owasp.dependencycheck.Engine.doUpdates (Engine.java:922)
    at org.owasp.dependencycheck.Engine.initializeAndUpdateDatabase (Engine.java:723)
    at org.owasp.dependencycheck.Engine.analyzeDependencies (Engine.java:653)
    at org.owasp.dependencycheck.maven.BaseDependencyCheckMojo.runCheck (BaseDependencyCheckMojo.java:1403)
    at org.owasp.dependencycheck.maven.BaseDependencyCheckMojo.execute (BaseDependencyCheckMojo.java:802)
    at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo (DefaultBuildPluginManager.java:137)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:210)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:156)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:148)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:117)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:81)
    at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build (SingleThreadedBuilder.java:56)
    at org.apache.maven.lifecycle.internal.LifecycleStarter.execute (LifecycleStarter.java:128)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:305)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:192)
    at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:105)
    at org.apache.maven.cli.MavenCli.execute (MavenCli.java:956)
    at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:288)
    at org.apache.maven.cli.MavenCli.main (MavenCli.java:192)
    at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
    at jdk.internal.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62)
    at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke (Method.java:566)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (Launcher.java:282)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launch (Launcher.java:225)
    at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (Launcher.java:406)
    at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:347)
Caused by: javax.net.ssl.SSLHandshakeException: Remote host terminated the handshake
    at sun.security.ssl.SSLSocketImpl.handleEOF (SSLSocketImpl.java:1321)
    at sun.security.ssl.SSLSocketImpl.decode (SSLSocketImpl.java:1160)
    at sun.security.ssl.SSLSocketImpl.readHandshakeRecord (SSLSocketImpl.java:1063)
    at sun.security.ssl.SSLSocketImpl.startHandshake (SSLSocketImpl.java:402)
    at sun.net.www.protocol.https.HttpsClient.afterConnect (HttpsClient.java:567)
    at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect (AbstractDelegateHttpsURLConnection.java:185)
    at sun.net.www.protocol.https.HttpsURLConnectionImpl.connect (HttpsURLConnectionImpl.java:163)
    at org.owasp.dependencycheck.utils.HttpResourceConnection.obtainConnection (HttpResourceConnection.java:178)
    at org.owasp.dependencycheck.utils.HttpResourceConnection.fetch (HttpResourceConnection.java:138)
    at org.owasp.dependencycheck.utils.Downloader.fetchContent (Downloader.java:110)
    at org.owasp.dependencycheck.data.update.NvdCveUpdater.getMetaFile (NvdCveUpdater.java:340)
    at org.owasp.dependencycheck.data.update.NvdCveUpdater.getUpdatesNeeded (NvdCveUpdater.java:385)
    at org.owasp.dependencycheck.data.update.NvdCveUpdater.update (NvdCveUpdater.java:122)
    at org.owasp.dependencycheck.Engine.doUpdates (Engine.java:922)
    at org.owasp.dependencycheck.Engine.initializeAndUpdateDatabase (Engine.java:723)
    at org.owasp.dependencycheck.Engine.analyzeDependencies (Engine.java:653)
    at org.owasp.dependencycheck.maven.BaseDependencyCheckMojo.runCheck (BaseDependencyCheckMojo.java:1403)
    at org.owasp.dependencycheck.maven.BaseDependencyCheckMojo.execute (BaseDependencyCheckMojo.java:802)
    at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo (DefaultBuildPluginManager.java:137)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:210)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:156)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:148)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:117)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:81)
    at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build (SingleThreadedBuilder.java:56)
    at org.apache.maven.lifecycle.internal.LifecycleStarter.execute (LifecycleStarter.java:128)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:305)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:192)
    at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:105)
    at org.apache.maven.cli.MavenCli.execute (MavenCli.java:956)
    at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:288)
    at org.apache.maven.cli.MavenCli.main (MavenCli.java:192)
    at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
    at jdk.internal.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62)
    at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke (Method.java:566)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (Launcher.java:282)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launch (Launcher.java:225)
    at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (Launcher.java:406)
    at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:347)
Caused by: java.io.EOFException: SSL peer shut down incorrectly
    at sun.security.ssl.SSLSocketInputRecord.decode (SSLSocketInputRecord.java:167)
    at sun.security.ssl.SSLTransport.decode (SSLTransport.java:108)
    at sun.security.ssl.SSLSocketImpl.decode (SSLSocketImpl.java:1152)
    at sun.security.ssl.SSLSocketImpl.readHandshakeRecord (SSLSocketImpl.java:1063)
    at sun.security.ssl.SSLSocketImpl.startHandshake (SSLSocketImpl.java:402)
    at sun.net.www.protocol.https.HttpsClient.afterConnect (HttpsClient.java:567)
    at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect (AbstractDelegateHttpsURLConnection.java:185)
    at sun.net.www.protocol.https.HttpsURLConnectionImpl.connect (HttpsURLConnectionImpl.java:163)
    at org.owasp.dependencycheck.utils.HttpResourceConnection.obtainConnection (HttpResourceConnection.java:178)
    at org.owasp.dependencycheck.utils.HttpResourceConnection.fetch (HttpResourceConnection.java:138)
    at org.owasp.dependencycheck.utils.Downloader.fetchContent (Downloader.java:110)
    at org.owasp.dependencycheck.data.update.NvdCveUpdater.getMetaFile (NvdCveUpdater.java:340)
    at org.owasp.dependencycheck.data.update.NvdCveUpdater.getUpdatesNeeded (NvdCveUpdater.java:385)
    at org.owasp.dependencycheck.data.update.NvdCveUpdater.update (NvdCveUpdater.java:122)
    at org.owasp.dependencycheck.Engine.doUpdates (Engine.java:922)
    at org.owasp.dependencycheck.Engine.initializeAndUpdateDatabase (Engine.java:723)
    at org.owasp.dependencycheck.Engine.analyzeDependencies (Engine.java:653)
    at org.owasp.dependencycheck.maven.BaseDependencyCheckMojo.runCheck (BaseDependencyCheckMojo.java:1403)
    at org.owasp.dependencycheck.maven.BaseDependencyCheckMojo.execute (BaseDependencyCheckMojo.java:802)
    at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo (DefaultBuildPluginManager.java:137)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:210)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:156)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:148)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:117)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:81)
    at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build (SingleThreadedBuilder.java:56)
    at org.apache.maven.lifecycle.internal.LifecycleStarter.execute (LifecycleStarter.java:128)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:305)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:192)
    at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:105)
    at org.apache.maven.cli.MavenCli.execute (MavenCli.java:956)
    at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:288)
    at org.apache.maven.cli.MavenCli.main (MavenCli.java:192)
    at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
    at jdk.internal.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62)
    at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke (Method.java:566)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (Launcher.java:282)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launch (Launcher.java:225)
    at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (Launcher.java:406)
    at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:347)
@lutzhorn

This comment has been minimized.

Copy link

commented Sep 24, 2019

Disable the autoUpdate property. This varies depending on if you are using the CLI, maven or gradle plugin, etc. See the documentation

Of course, autoUpdate can be set to false like this:

<plugin>
  <!-- ... -->
  <configuration>
    <autoUpdate>false</autoUpdate>
  </configuration>
</plugin>

But this is not a reasonable advice. We can't enable and disable autoUpdate just because NIST is down. How are we expected to deal with this behaviour in CI?

@catap

This comment has been minimized.

Copy link

commented Sep 24, 2019

@lutzhorn but if someone using CI that is runs on fresh instance somewhere in the clouds... well... it brokes CI workflow :)

@jeremylong

This comment has been minimized.

Copy link
Owner

commented Sep 24, 2019

@lutzhorn to ensure availability - I would highly recommend running the nist-data-mirror:

java -jar nist-data-mirror.jar <mirror-directory> json

In the above command you could also mirror the XML - but those datafeeds are going away on October 9th, 2019 - I hope everyone has upgraded to ODC 5.x. Also, there is a docker container for the nist-data-mirror - but we may need to modify it as it currently downloads the JSON and XML data feeds by default.

@christian-weiss

This comment has been minimized.

Copy link

commented Sep 24, 2019

nvd.nist.gov should use a high-available CDN (e.g. Amazon) or provide some mirrors for a fallback.
I know you always recommend to have a local mirror, but working on that topic at both ends is better i think.

@christian-weiss

This comment has been minimized.

Copy link

commented Sep 24, 2019

Can someone upload his cache to a github.com repo - to let new vuls users get started (for experiments, not for production).
(else we had to wait for nvd.nist.gov to come up again)

@michha

This comment has been minimized.

Copy link

commented Sep 24, 2019

its online again

@Nriver

This comment has been minimized.

Copy link

commented Sep 29, 2019

Now I use --noupdate during scan tasks, and add a cronjob to execute --updateonly once a day in the midnight.

@kevvvvyp

This comment has been minimized.

Copy link

commented Sep 30, 2019

Is this down again?
[INFO] --- dependency-check-maven:5.2.1:check (default-cli) @ common --- [INFO] Checking for updates [ERROR] Unable to download meta file: https://nvd.nist.gov/feeds/json/cve/1.0/nvdcve-1.0-modified.meta org.owasp.dependencycheck.data.update.exception.UpdateException: Unable to download meta file: https://nvd.nist.gov/feeds/json/cve/1.0/nvdcve-1.0-modified.meta at org.owasp.dependencycheck.data.update.NvdCveUpdater.getMetaFile(NvdCveUpdater.java:347) at org.owasp.dependencycheck.data.update.NvdCveUpdater.getUpdatesNeeded(NvdCveUpdater.java:385) at org.owasp.dependencycheck.data.update.NvdCveUpdater.update(NvdCveUpdater.java:122) at org.owasp.dependencycheck.Engine.doUpdates(Engine.java:922) at org.owasp.dependencycheck.Engine.initializeAndUpdateDatabase(Engine.java:723) at org.owasp.dependencycheck.Engine.analyzeDependencies(Engine.java:653) at org.owasp.dependencycheck.maven.BaseDependencyCheckMojo.runCheck(BaseDependencyCheckMojo.java:1403) at org.owasp.dependencycheck.maven.BaseDependencyCheckMojo.execute(BaseDependencyCheckMojo.java:802) at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:134) at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:208) at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:154) at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:146) at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:117) at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:81) at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build(SingleThreadedBuilder.java:51) at org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:128) at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:309) at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:194) at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:107) at org.apache.maven.cli.MavenCli.execute(MavenCli.java:993) at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:345) at org.apache.maven.cli.MavenCli.main(MavenCli.java:191) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:289) at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:229) at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:415) at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:356) Caused by: org.owasp.dependencycheck.utils.DownloadFailedException: Download failed, unable to retrieve 'https://nvd.nist.gov/feeds/json/cve/1.0/nvdcve-1.0-modified.meta' at org.owasp.dependencycheck.utils.Downloader.fetchContent(Downloader.java:115) at org.owasp.dependencycheck.data.update.NvdCveUpdater.getMetaFile(NvdCveUpdater.java:340) ... 29 more Caused by: org.owasp.dependencycheck.utils.DownloadFailedException: Error downloading file https://nvd.nist.gov/feeds/json/cve/1.0/nvdcve-1.0-modified.meta; unable to connect. at org.owasp.dependencycheck.utils.HttpResourceConnection.obtainConnection(HttpResourceConnection.java:238) at org.owasp.dependencycheck.utils.HttpResourceConnection.fetch(HttpResourceConnection.java:138) at org.owasp.dependencycheck.utils.Downloader.fetchContent(Downloader.java:110) ... 30 more Caused by: javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target at sun.security.ssl.Alerts.getSSLException(Alerts.java:192) at sun.security.ssl.SSLSocketImpl.fatal(SSLSocketImpl.java:1959) at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:302) at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:296) at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1514) at sun.security.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:216) at sun.security.ssl.Handshaker.processLoop(Handshaker.java:1026) at sun.security.ssl.Handshaker.process_record(Handshaker.java:961) at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:1072) at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1385) at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1413) at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1397) at sun.net.www.protocol.https.HttpsClient.afterConnect(HttpsClient.java:559) at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:185) at sun.net.www.protocol.https.HttpsURLConnectionImpl.connect(HttpsURLConnectionImpl.java:153) at org.owasp.dependencycheck.utils.HttpResourceConnection.obtainConnection(HttpResourceConnection.java:178) ... 32 more Caused by: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:397) at sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:302) at sun.security.validator.Validator.validate(Validator.java:260) at sun.security.ssl.X509TrustManagerImpl.validate(X509TrustManagerImpl.java:324) at sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:229) at sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:124) at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1496) ... 43 more Caused by: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target at sun.security.provider.certpath.SunCertPathBuilder.build(SunCertPathBuilder.java:141) at sun.security.provider.certpath.SunCertPathBuilder.engineBuild(SunCertPathBuilder.java:126) at java.security.cert.CertPathBuilder.build(CertPathBuilder.java:280) at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:392) ... 49 more

@lutzhorn

This comment has been minimized.

Copy link

commented Sep 30, 2019

Is this down again?

Yes, see #2222.

@ST-DDT

This comment has been minimized.

Copy link

commented Sep 30, 2019

I wouldn't call it down (It works with normal browser). It just doesn't work with default Java.

@lutzhorn

This comment has been minimized.

Copy link

commented Sep 30, 2019

I wouldn't call it down (It works with normal browser). It just doesn't work with default Java.

Which in the context of this Maven plugin is as good as down :)

@jeremylong

This comment has been minimized.

Copy link
Owner

commented Sep 30, 2019

Closing as this is a duplicate of #2222. A work around is documented in #2222.

@jeremylong jeremylong closed this Sep 30, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
You can’t perform that action at this time.