Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Failure of a test run should cause build failure #55

Merged
merged 1 commit into from Aug 29, 2011

Conversation

rtyley
Copy link
Contributor

@rtyley rtyley commented Aug 29, 2011

The ITestRunListener interface has a testRunFailed() method which is called
if the test run itself failed - a failure in the test infrastructure, not a
test failure. This patch fixes the AbstractInstrumentationMojo so that the
build will be failed if testRunFailed() was called.

(I should have caught this in d285892, gah)

An example of build messages before and after this update follows - the
test run failure here was caused by mounting my SDCard while the integration
test app was supposed to be running on it.

BEFORE:

[INFO] --- maven-android-plugin:3.0.0-alpha-3:internal-integration-test (default-internal-integration-test) @ agit-integration-tests ---
[INFO] Found 1 devices connected with the Android Debug Bridge
[INFO] android.device parameter not set, using all attached devices
[INFO] Running instrumentation tests in com.madgag.agit.tests on 0123456789ABCDEF (avdName=null)
[INFO] Run started: com.madgag.agit.tests, 0 tests:
[INFO] Run failed: Unable to find instrumentation info for: ComponentInfo{com.madgag.agit.tests/android.test.InstrumentationTestRunner}
[INFO] Run ended: 0 ms
[INFO] Tests run: 0, Failures: 0, Errors: 0
[INFO] Report file written to /home/roberto/development/agit-parent/agit-integration-tests/target/surefire-reports/TEST-0123456789ABCDEF_HTC_HTCDesire.xml
[INFO]
[INFO] --- cargo-maven2-plugin:1.1.1:stop (stop-container) @ agit-integration-tests ---
[INFO] [beddedLocalContainer] Jetty 7.x Embedded is stopping...
2011-08-29 17:27:07.938:INFO::stopped o.e.j.w.WebAppContext{/cargocpc,file:/tmp/jetty-0.0.0.0-18181-cargocpc.war-_cargocpc-any-/webapp/},/home/roberto/development/agit-parent/agit-integration-tests/target/cargo/configurations/jetty7x/cargocpc.war
WAI contextDestroyed - javax.servlet.ServletContextEvent[source=ServletContext@o.e.j.w.WebAppContext{/mini-git-server-war-0.4,file:/tmp/jetty-0.0.0.0-18181-mini-git-server-war-0.4.war-_mini-git-server-war-0.4-any-/webapp/},/home/roberto/.m2/repository/com/madgag/mini-git-server-war/0.4/mini-git-server-war-0.4.war]
[2011-08-29 17:27:07,950] INFO com.google.gerrit.sshd.SshDaemon : Stopped Gerrit SSHD
2011-08-29 17:27:07.952:INFO::stopped o.e.j.w.WebAppContext{/mini-git-server-war-0.4,file:/tmp/jetty-0.0.0.0-18181-mini-git-server-war-0.4.war-_mini-git-server-war-0.4-any-/webapp/},/home/roberto/.m2/repository/com/madgag/mini-git-server-war/0.4/mini-git-server-war-0.4.war
[INFO] [beddedLocalContainer] Jetty 7.x Embedded is stopped
[INFO]
[INFO] --- maven-install-plugin:2.3.1:install (default-install) @ agit-integration-tests ---
[INFO] Installing /home/roberto/development/agit-parent/agit-integration-tests/target/agit-integration-tests-1.25-SNAPSHOT.apk to /home/roberto/.m2/repository/com/madgag/agit-integration-tests/1.25-SNAPSHOT/agit-integration-tests-1.25-SNAPSHOT.apk
[INFO] Installing /home/roberto/development/agit-parent/agit-integration-tests/pom.xml to /home/roberto/.m2/repository/com/madgag/agit-integration-tests/1.25-SNAPSHOT/agit-integration-tests-1.25-SNAPSHOT.pom
[INFO] Installing /home/roberto/development/agit-parent/agit-integration-tests/target/agit-integration-tests-1.25-SNAPSHOT.jar to /home/roberto/.m2/repository/com/madgag/agit-integration-tests/1.25-SNAPSHOT/agit-integration-tests-1.25-SNAPSHOT.jar
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] Agit parent POM project ........................... SUCCESS [0.270s]
[INFO] agit test utils ................................... SUCCESS [1.591s]
[INFO] Agit - git client for Android ..................... SUCCESS [27.079s]
[INFO] Agit integration tests ............................ SUCCESS [1:18.589s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 1:48.219s
[INFO] Finished at: Mon Aug 29 17:27:14 BST 2011
[INFO] Final Memory: 35M/322M
[INFO] ------------------------------------------------------------------------

AFTER:

[INFO] Running instrumentation tests in com.madgag.agit.tests on 0123456789ABCDEF (avdName=null)
[INFO] Run started: com.madgag.agit.tests, 0 tests:
[INFO] Run failed: Unable to find instrumentation info for: ComponentInfo{com.madgag.agit.tests/android.test.InstrumentationTestRunner}
[INFO] Run ended: 0 ms
[INFO] Tests run: 0, Failures: 0, Errors: 0
[INFO] Report file written to /home/roberto/development/agit-parent/agit-integration-tests/target/surefire-reports/TEST-0123456789ABCDEF_HTC_HTCDesire.xml
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] Agit parent POM project ........................... SUCCESS [0.222s]
[INFO] agit test utils ................................... SUCCESS [1.650s]
[INFO] Agit - git client for Android ..................... SUCCESS [26.669s]
[INFO] Agit integration tests ............................ FAILURE [49.740s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 1:18.982s
[INFO] Finished at: Mon Aug 29 19:08:03 BST 2011
[INFO] Final Memory: 39M/173M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal com.jayway.maven.plugins.android.generation2:maven-android-plugin:3.0.0-SNAPSHOT:internal-integration-test (default-internal-integration-test) on project agit-integration-tests: Test run failed to complete: Unable to find instrumentation info for: ComponentInfo{com.madgag.agit.tests/android.test.InstrumentationTestRunner} -> [Help 1]

The ITestRunListener interface has a testRunFailed() method which is called
if the test run itself failed - a failure in the test infrastructure, not a
test failure. This patch fixes the AbstractInstrumentationMojo so that the
build will be failed if testRunFailed() was called.

(I should have caught this in d285892, gah)

An example of build messages before and after this update follows - the
test run failure here was caused by mounting my SDCard while the integration
test app was supposed to be running on it.

BEFORE:

[INFO] --- maven-android-plugin:3.0.0-alpha-3:internal-integration-test (default-internal-integration-test) @ agit-integration-tests ---
[INFO] Found 1 devices connected with the Android Debug Bridge
[INFO] android.device parameter not set, using all attached devices
[INFO] Running instrumentation tests in com.madgag.agit.tests on 0123456789ABCDEF (avdName=null)
[INFO]   Run started: com.madgag.agit.tests, 0 tests:
[INFO]   Run failed: Unable to find instrumentation info for: ComponentInfo{com.madgag.agit.tests/android.test.InstrumentationTestRunner}
[INFO]   Run ended: 0 ms
[INFO]   Tests run: 0,  Failures: 0,  Errors: 0
[INFO] Report file written to /home/roberto/development/agit-parent/agit-integration-tests/target/surefire-reports/TEST-0123456789ABCDEF_HTC_HTCDesire.xml
[INFO]
[INFO] --- cargo-maven2-plugin:1.1.1:stop (stop-container) @ agit-integration-tests ---
[INFO] [beddedLocalContainer] Jetty 7.x Embedded is stopping...
2011-08-29 17:27:07.938:INFO::stopped o.e.j.w.WebAppContext{/cargocpc,file:/tmp/jetty-0.0.0.0-18181-cargocpc.war-_cargocpc-any-/webapp/},/home/roberto/development/agit-parent/agit-integration-tests/target/cargo/configurations/jetty7x/cargocpc.war
WAI contextDestroyed - javax.servlet.ServletContextEvent[source=ServletContext@o.e.j.w.WebAppContext{/mini-git-server-war-0.4,file:/tmp/jetty-0.0.0.0-18181-mini-git-server-war-0.4.war-_mini-git-server-war-0.4-any-/webapp/},/home/roberto/.m2/repository/com/madgag/mini-git-server-war/0.4/mini-git-server-war-0.4.war]
[2011-08-29 17:27:07,950] INFO  com.google.gerrit.sshd.SshDaemon : Stopped Gerrit SSHD
2011-08-29 17:27:07.952:INFO::stopped o.e.j.w.WebAppContext{/mini-git-server-war-0.4,file:/tmp/jetty-0.0.0.0-18181-mini-git-server-war-0.4.war-_mini-git-server-war-0.4-any-/webapp/},/home/roberto/.m2/repository/com/madgag/mini-git-server-war/0.4/mini-git-server-war-0.4.war
[INFO] [beddedLocalContainer] Jetty 7.x Embedded is stopped
[INFO]
[INFO] --- maven-install-plugin:2.3.1:install (default-install) @ agit-integration-tests ---
[INFO] Installing /home/roberto/development/agit-parent/agit-integration-tests/target/agit-integration-tests-1.25-SNAPSHOT.apk to /home/roberto/.m2/repository/com/madgag/agit-integration-tests/1.25-SNAPSHOT/agit-integration-tests-1.25-SNAPSHOT.apk
[INFO] Installing /home/roberto/development/agit-parent/agit-integration-tests/pom.xml to /home/roberto/.m2/repository/com/madgag/agit-integration-tests/1.25-SNAPSHOT/agit-integration-tests-1.25-SNAPSHOT.pom
[INFO] Installing /home/roberto/development/agit-parent/agit-integration-tests/target/agit-integration-tests-1.25-SNAPSHOT.jar to /home/roberto/.m2/repository/com/madgag/agit-integration-tests/1.25-SNAPSHOT/agit-integration-tests-1.25-SNAPSHOT.jar
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] Agit parent POM project ........................... SUCCESS [0.270s]
[INFO] agit test utils ................................... SUCCESS [1.591s]
[INFO] Agit - git client for Android ..................... SUCCESS [27.079s]
[INFO] Agit integration tests ............................ SUCCESS [1:18.589s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 1:48.219s
[INFO] Finished at: Mon Aug 29 17:27:14 BST 2011
[INFO] Final Memory: 35M/322M
[INFO] ------------------------------------------------------------------------


AFTER:

[INFO] Running instrumentation tests in com.madgag.agit.tests on 0123456789ABCDEF (avdName=null)
[INFO]   Run started: com.madgag.agit.tests, 0 tests:
[INFO]   Run failed: Unable to find instrumentation info for: ComponentInfo{com.madgag.agit.tests/android.test.InstrumentationTestRunner}
[INFO]   Run ended: 0 ms
[INFO]   Tests run: 0,  Failures: 0,  Errors: 0
[INFO] Report file written to /home/roberto/development/agit-parent/agit-integration-tests/target/surefire-reports/TEST-0123456789ABCDEF_HTC_HTCDesire.xml
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] Agit parent POM project ........................... SUCCESS [0.222s]
[INFO] agit test utils ................................... SUCCESS [1.650s]
[INFO] Agit - git client for Android ..................... SUCCESS [26.669s]
[INFO] Agit integration tests ............................ FAILURE [49.740s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 1:18.982s
[INFO] Finished at: Mon Aug 29 19:08:03 BST 2011
[INFO] Final Memory: 39M/173M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal com.jayway.maven.plugins.android.generation2:maven-android-plugin:3.0.0-SNAPSHOT:internal-integration-test (default-internal-integration-test) on project agit-integration-tests: Test run failed to complete: Unable to find instrumentation info for: ComponentInfo{com.madgag.agit.tests/android.test.InstrumentationTestRunner} -> [Help 1]
@mosabua
Copy link
Member

mosabua commented Aug 29, 2011

You just missed the cut off for r4 ..

mosabua added a commit that referenced this pull request Aug 29, 2011
Failure of a test run should cause build failure
@mosabua mosabua merged commit 3cfc683 into simpligility:master Aug 29, 2011
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants