Docker run failure #51

Sab0tag3d · 2018-11-15T10:01:22Z

Describe the bug
Can't tun docker because of

Failed tests: 
  IT01_ArtifactControllerTest.getLatestVersionMavenTest:282 null

To Reproduce
Steps to reproduce the behavior: Run command from the guide:
sudo docker run -it --rm -v ${PWD}/docker:/exporter --env-file ./docker/.env -e mvn_flags=-DexcludedGroups=com.sap.psr.vulas.shared.categories.Slow vulas-build-img

OS Description

Docker version 18.09.0, build 4d60db4  

Linux 4.15.0-39-generic #42-Ubuntu SMP Tue Oct 23 15:48:01 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux

Log

[INFO] REST service to analyze and compare libraries ...... FAILURE [ 24.601 s]```


[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  03:26 min
[INFO] Finished at: 2018-11-15T09:53:56Z
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.18.1:test (default-test) on project rest-lib-utils: There are test failures.
[ERROR] 
[ERROR] Please refer to /vulas/rest-lib-utils/target/surefire-reports for the individual test results.
[ERROR] -> [Help 1]
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.18.1:test (default-test) on project rest-lib-utils: There are test failures.

Please refer to /vulas/rest-lib-utils/target/surefire-reports for the individual test results.
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:215)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:156)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:148)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:117)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:81)
    at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build (SingleThreadedBuilder.java:56)
    at org.apache.maven.lifecycle.internal.LifecycleStarter.execute (LifecycleStarter.java:128)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:305)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:192)
    at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:105)
    at org.apache.maven.cli.MavenCli.execute (MavenCli.java:956)
    at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:288)
    at org.apache.maven.cli.MavenCli.main (MavenCli.java:192)
    at sun.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke (Method.java:498)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (Launcher.java:289)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launch (Launcher.java:229)
    at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (Launcher.java:415)
    at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:356)
Caused by: org.apache.maven.plugin.MojoFailureException: There are test failures.

Please refer to /vulas/rest-lib-utils/target/surefire-reports for the individual test results.
    at org.apache.maven.plugin.surefire.SurefireHelper.reportExecution (SurefireHelper.java:82)
    at org.apache.maven.plugin.surefire.SurefirePlugin.handleSummary (SurefirePlugin.java:254)
    at org.apache.maven.plugin.surefire.AbstractSurefireMojo.executeAfterPreconditionsChecked (AbstractSurefireMojo.java:854)
    at org.apache.maven.plugin.surefire.AbstractSurefireMojo.execute (AbstractSurefireMojo.java:722)
    at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo (DefaultBuildPluginManager.java:137)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:210)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:156)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:148)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:117)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:81)
    at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build (SingleThreadedBuilder.java:56)
    at org.apache.maven.lifecycle.internal.LifecycleStarter.execute (LifecycleStarter.java:128)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:305)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:192)
    at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:105)
    at org.apache.maven.cli.MavenCli.execute (MavenCli.java:956)
    at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:288)
    at org.apache.maven.cli.MavenCli.main (MavenCli.java:192)
    at sun.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke (Method.java:498)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (Launcher.java:289)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launch (Launcher.java:229)
    at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (Launcher.java:415)
    at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:356)
[ERROR] 
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
[ERROR] 
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR]   mvn <goals> -rf :rest-lib-utils
Cleaning old archives
Copying new archives
cp: can't stat 'rest-backend/target/*.?ar': No such file or directory
cp: can't stat 'rest-lib-utils/target/*.?ar': No such file or directory

The text was updated successfully, but these errors were encountered:

serenaponta · 2018-11-15T10:11:39Z

This failure should be fixed by PR #50 now merged.

Please try again and close the issue if solved.

Sab0tag3d · 2018-11-15T10:42:46Z

I still have an errors but now another test failed:

Results :

Failed tests: 
  BugControllerTest.testGetBug:117 Status expected:<200> but was:<500>
  BugControllerTest.testPost:155 Status expected:<200> but was:<500>
  BugControllerTest.testPostCVE20140050:187 Status expected:<200> but was:<500>
  LibraryControllerTest.testPostCommonsFileUpload:117 JSON path "$.wellknownDigest"
Expected: is <true>
     but: was null
  LibraryControllerTest.testPostJinja2:93 JSON path "$.wellknownDigest"
Expected: is <true>
     but: was null
Tests in error: 
  CveControllerTest.testPostCommonsFileUpload:71 » NumberFormat For input string...

Tests run: 33, Failures: 5, Errors: 1, Skipped: 0

[INFO] REST service to manage scan results and bugs ....... FAILURE [ 29.318 s]

[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  03:34 min
[INFO] Finished at: 2018-11-15T10:38:24Z
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.18.1:test (default-test) on project rest-backend: There are test failures.
[ERROR] 
[ERROR] Please refer to /vulas/rest-backend/target/surefire-reports for the individual test results.
[ERROR] -> [Help 1]
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.18.1:test (default-test) on project rest-backend: There are test failures.

Please refer to /vulas/rest-backend/target/surefire-reports for the individual test results.
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:215)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:156)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:148)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:117)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:81)
    at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build (SingleThreadedBuilder.java:56)
    at org.apache.maven.lifecycle.internal.LifecycleStarter.execute (LifecycleStarter.java:128)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:305)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:192)
    at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:105)
    at org.apache.maven.cli.MavenCli.execute (MavenCli.java:956)
    at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:288)
    at org.apache.maven.cli.MavenCli.main (MavenCli.java:192)
    at sun.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke (Method.java:498)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (Launcher.java:289)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launch (Launcher.java:229)
    at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (Launcher.java:415)
    at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:356)
Caused by: org.apache.maven.plugin.MojoFailureException: There are test failures.

Please refer to /vulas/rest-backend/target/surefire-reports for the individual test results.
    at org.apache.maven.plugin.surefire.SurefireHelper.reportExecution (SurefireHelper.java:82)
    at org.apache.maven.plugin.surefire.SurefirePlugin.handleSummary (SurefirePlugin.java:254)
    at org.apache.maven.plugin.surefire.AbstractSurefireMojo.executeAfterPreconditionsChecked (AbstractSurefireMojo.java:854)
    at org.apache.maven.plugin.surefire.AbstractSurefireMojo.execute (AbstractSurefireMojo.java:722)
    at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo (DefaultBuildPluginManager.java:137)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:210)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:156)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:148)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:117)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:81)
    at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build (SingleThreadedBuilder.java:56)
    at org.apache.maven.lifecycle.internal.LifecycleStarter.execute (LifecycleStarter.java:128)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:305)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:192)
    at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:105)
    at org.apache.maven.cli.MavenCli.execute (MavenCli.java:956)
    at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:288)
    at org.apache.maven.cli.MavenCli.main (MavenCli.java:192)
    at sun.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke (Method.java:498)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (Launcher.java:289)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launch (Launcher.java:229)
    at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (Launcher.java:415)
    at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:356)
[ERROR] 
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
[ERROR] 
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR]   mvn <goals> -rf :rest-backend
Cleaning old archives
rm: can't remove '/exporter/**/*.?ar': No such file or directory
Copying new archives
cp: can't stat 'rest-backend/target/*.?ar': No such file or directory

Naramsim · 2018-11-15T12:27:28Z

Hi, A test that requires network connectivity is failing. I see that a server responds with a 500 instead of a 200.

Until we understand which server does block you, could you try to skip tests that rely on network? By using this Maven flag -DexcludedGroups="com.sap.psr.vulas.shared.categories.RequiresNetwork"

docker run -it --rm -v ${PWD}/docker:/exporter --env-file ./docker/.env -e mvn_flags=-DexcludedGroups=com.sap.psr.vulas.shared.categories.RequiresNetwork vulas-build-img

Sab0tag3d · 2018-11-15T12:50:28Z

I have ran docker with this flags before and got this errors:

sudo docker run -it --rm -v ${PWD}/docker:/exporter --env-file ./docker/.env -e mvn_flags=-DexcludedGroups=com.sap.psr.vulas.shared.categories.Slow vulas-build-img

But you sent me new command:

docker run -it --rm -v ${PWD}/docker:/exporter --env-file ./docker/.env -e mvn_flags=-DexcludedGroups=com.sap.psr.vulas.shared.categories.RequiresNetwork vulas-build-img

And it also crashed:

[ERROR] Failures: 
[ERROR]   WarAnalyzerTest.testInstrument:64
[INFO] 
[ERROR] Tests run: 53, Failures: 1, Errors: 0, Skipped: 4
[INFO] 
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary for Vulas 3.0.11-SNAPSHOT:
[INFO] 
[INFO] Vulas .............................................. SUCCESS [ 12.858 s]
[INFO] Shared classes ..................................... SUCCESS [ 19.501 s]
[INFO] Framework and base classes for language analysis ... SUCCESS [ 11.830 s]
[INFO] Java - Init ........................................ SUCCESS [  0.092 s]
[INFO] Java - Core classes for Java analysis .............. FAILURE [ 24.142 s]
[INFO] Java - Core classes and SPI for Java reachability analysis SKIPPED
[INFO] Java - Wala call graph constructor service ......... SKIPPED
[INFO] Python - Core classes for Python analysis .......... SKIPPED
[INFO] Interface for Git and SVN repos .................... SKIPPED
[INFO] Analysis of security patches ....................... SKIPPED
[INFO] Analysis of libraries in regards to security patches SKIPPED
[INFO] Command Line Interface ............................. SKIPPED
[INFO] Plugin for Maven ................................... SKIPPED
[INFO] Frontend for analysis results ...................... SKIPPED
[INFO] Frontend for bug administration .................... SKIPPED
[INFO] Frontend for patch analyzer ........................ SKIPPED
[INFO] REST service to analyze and compare libraries (init) SKIPPED
[INFO] REST service to analyze and compare libraries ...... SKIPPED
[INFO] REST service to manage scan results and bugs ....... SKIPPED

serenaponta · 2018-11-15T13:15:01Z

Hello,

most of the tests failing when executing
sudo docker run -it --rm -v ${PWD}/docker:/exporter --env-file ./docker/.env -e mvn_flags=-DexcludedGroups=com.sap.psr.vulas.shared.categories.Slow vulas-build-img

involve network connection. In particular, these are the endpoints used:

http://search.maven.org/solrsearch/select?q=1:<SHA1>&rows=20&wt=json
https://pypi.python.org/pypi/<name>/<version>/json
http://www.cvedetails.com/cve-details.php?t=1&cve_id=CVE-2014-0050

Can you check whether they are reachable from the system where docker is running?
Could you run again
sudo docker run -it --rm -v ${PWD}/docker:/exporter --env-file ./docker/.env -e mvn_flags=-DexcludedGroups=com.sap.psr.vulas.shared.categories.Slow vulas-build-img
and attached the complete console output to the issue?
Could you run again
docker run -it --rm -v ${PWD}/docker:/exporter --env-file ./docker/.env -e mvn_flags=-DexcludedGroups=com.sap.psr.vulas.shared.categories.RequiresNetwork vulas-build-img
and attached the complete console output to the issue?

Sab0tag3d · 2018-11-15T13:43:06Z

Maven's site returns stack trace and 500 error when you added " in the end of url:
http://search.maven.org/solrsearch/select?q=1:&rows=20&wt=json%22
But all sites are avaliable.

I've attached output of commands:

sudo docker run -it --rm -v ${PWD}/docker:/exporter --env-file ./docker/.env -e mvn_flags=-DexcludedGroups=com.sap.psr.vulas.shared.categories.Slow vulas-build-img >> 1.txt

1.txt

sudo docker run -it --rm -v ${PWD}/docker:/exporter --env-file ./docker/.env -e mvn_flags=-DexcludedGroups=com.sap.psr.vulas.shared.categories.RequiresNetwork vulas-build-img >> 2.txt

2.txt

serenaponta · 2018-11-15T14:16:23Z

The issue is currently not reproducible from our side.

Could you also share the .env configuration file you are using (obfuscating sensitive information-if any)?

serenaponta · 2018-11-15T15:04:56Z

We were able to reproduce the issue commenting out the proxy configurations in .env. In case you do not need any proxy, still the properties needs to stay as follows

# Proxy
HTTP_PROXY_HOST=
HTTPS_PROXY_HOST=
HTTP_PROXY_PORT=
HTTPS_PROXY_PORT=
NON_PROXY_HOSTS=

# Proxy (you should not need to modify this)
http_proxy=
https_proxy=
no_proxy=

We will improve the proxy handling to cover such case.

Thanks for reporting the issue!

Sab0tag3d · 2018-11-15T15:14:47Z

It is strange, I have the same .env file but it fails

VULAS_RELEASE=3.0.11-SNAPSHOT
VULAS_ENV=prod

# PostgreSQL
POSTGRES_USER=
POSTGRES_PASSWORD=

# Jira (optional)
JIRA_USER=
JIRA_PASSWORD=

# Proxy
HTTP_PROXY_HOST=
HTTPS_PROXY_HOST=
HTTP_PROXY_PORT=
HTTPS_PROXY_PORT=
NON_PROXY_HOSTS=
http_proxy=
https_proxy=
no_proxy=

# Spring (should be equal to the values in PostgreSQL section)
spring.datasource.username=
spring.datasource.password=

serenaponta · 2018-11-15T15:17:45Z

The fields

# PostgreSQL
POSTGRES_USER=
POSTGRES_PASSWORD=

# Spring (should be equal to the values in PostgreSQL section)
spring.datasource.username=
spring.datasource.password=

must always be provided (with matching values), e.g.,

# PostgreSQL
POSTGRES_USER=abc
POSTGRES_PASSWORD=abcd

# Spring (should be equal to the values in PostgreSQL section)
spring.datasource.username=abc
spring.datasource.password=abcd

Sab0tag3d · 2018-11-15T15:35:50Z

I've changed this configuration but it wasn't help.

copernico · 2018-11-16T09:06:47Z

@Sab0tag3d Could you check if the fix we just merged addresses you problem?

serenaponta · 2018-11-16T12:06:36Z

The build is still failing, we are currenty working on it.

Sab0tag3d · 2018-11-16T12:07:42Z

This issue fixed, thanks.
But now I have problems with rest backend, It wont's start.
http://localhost:8033/backend return 503

and in haproxy I see:
Layer4 connection problem: Connection refused

Do you have any ideas how I can fix that?

serenaponta · 2018-11-16T13:32:01Z

With 5224c31 we fixed the build error and the "connection refused" issue you reported. In our test the setup of the docker images with any postgres user and no proxy settings is now successfull (Note that the backend takes >20sec to start).

Thank you for reporting all the issues you encountered and helping us improving the vulnerability assessment tool!

Naramsim added the docker Docker related label Nov 15, 2018

copernico closed this as completed in f5f3a46 Nov 16, 2018

copernico reopened this Nov 16, 2018

serenaponta closed this as completed in 5224c31 Nov 16, 2018

serenaponta reopened this Nov 16, 2018

Naramsim closed this as completed Nov 29, 2018

This issue was closed.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Docker run failure #51

Docker run failure #51

Sab0tag3d commented Nov 15, 2018 •

edited

Loading

serenaponta commented Nov 15, 2018 •

edited

Loading

Sab0tag3d commented Nov 15, 2018

Naramsim commented Nov 15, 2018 •

edited

Loading

Sab0tag3d commented Nov 15, 2018

serenaponta commented Nov 15, 2018 •

edited by Naramsim

Loading

Sab0tag3d commented Nov 15, 2018

serenaponta commented Nov 15, 2018

serenaponta commented Nov 15, 2018 •

edited

Loading

Sab0tag3d commented Nov 15, 2018

serenaponta commented Nov 15, 2018

Sab0tag3d commented Nov 15, 2018

copernico commented Nov 16, 2018

serenaponta commented Nov 16, 2018

Sab0tag3d commented Nov 16, 2018

serenaponta commented Nov 16, 2018

Docker run failure #51

Docker run failure #51

Comments

Sab0tag3d commented Nov 15, 2018 • edited Loading

serenaponta commented Nov 15, 2018 • edited Loading

Sab0tag3d commented Nov 15, 2018

Naramsim commented Nov 15, 2018 • edited Loading

Sab0tag3d commented Nov 15, 2018

serenaponta commented Nov 15, 2018 • edited by Naramsim Loading

Sab0tag3d commented Nov 15, 2018

serenaponta commented Nov 15, 2018

serenaponta commented Nov 15, 2018 • edited Loading

Sab0tag3d commented Nov 15, 2018

serenaponta commented Nov 15, 2018

Sab0tag3d commented Nov 15, 2018

copernico commented Nov 16, 2018

serenaponta commented Nov 16, 2018

Sab0tag3d commented Nov 16, 2018

serenaponta commented Nov 16, 2018

Sab0tag3d commented Nov 15, 2018 •

edited

Loading

serenaponta commented Nov 15, 2018 •

edited

Loading

Naramsim commented Nov 15, 2018 •

edited

Loading

serenaponta commented Nov 15, 2018 •

edited by Naramsim

Loading

serenaponta commented Nov 15, 2018 •

edited

Loading