Builds failing on jenkins.dataverse.org

[djbrooke]

A bit of a follow on to #6528, we're still seeing the test suite not complete on jenkins.dataverse.org.

https://jenkins.dataverse.org/job/IQSS-dataverse-develop/310/
https://jenkins.dataverse.org/job/IQSS-dataverse-develop/311/

etc.

[donsizemore]

So, to fix the Maven http/https switch in dataverse-ansible I kept the "original" SP1 version at https://repo1.maven.org/maven2/org/jboss/weld/weld-osgi-bundle/2.2.10.SP1/weld-osgi-bundle-2.2.10.SP1.jar rather than the "Final" link Phil and Pete had been using for docker-aio, et al.

Glassfish is angry about weld. Perhaps we want Final:

[2020-01-16T21:22:18.596+0000] [glassfish 4.1] [SEVERE] [] [] [tid: _ThreadID=12 _ThreadName=Thread-9] [timeMillis: 1579209738596] [levelValue: 1000] [[
  org.osgi.framework.BundleException: Unresolved constraint in bundle org.glassfish.fighterfish.osgi-http [296]: Unable to resolve 296.0: missing requirement [296.0] osgi.wiring.package; (&(osgi.wiring.package=com.sun.enterprise.web)(version>=3.1.1)) [caused by
: Unable to resolve 271.0: missing requirement [271.0] osgi.wiring.package; (&(osgi.wiring.package=com.sun.enterprise.web.connector.grizzly)(version>=4.1.0)(!(version>=5.0.0))) [caused by: Unable to resolve 269.0: missing requirement [269.0] osgi.wiring.package
; (&(osgi.wiring.package=org.glassfish.web.loader)(version>=4.1.0)(!(version>=5.0.0))) [caused by: Unable to resolve 267.0: missing requirement [267.0] osgi.wiring.package; (osgi.wiring.package=org.jboss.weld.interceptor.proxy) [caused by: Unable to resolve 281
.0: missing requirement [281.0] osgi.wiring.package; (osgi.wiring.package=org.glassfish.weld) [caused by: Unable to resolve 282.0: missing requirement [282.0] osgi.wiring.package; (&(osgi.wiring.package=org.jboss.weld)(version>=1.0.0)(!(version>=2.0.0)))]]]]]
        at org.apache.felix.framework.Felix.resolveBundleRevision(Felix.java:3974)
        at org.apache.felix.framework.Felix.startBundle(Felix.java:2037)
        at org.apache.felix.framework.Felix.setActiveStartLevel(Felix.java:1291)
        at org.apache.felix.framework.FrameworkStartLevelImpl.run(FrameworkStartLevelImpl.java:304)
        at java.lang.Thread.run(Thread.java:748)]]

[2020-01-16T21:22:18.595+0000] [glassfish 4.1] [INFO] [] [] [tid: _ThreadID=12 _ThreadName=Thread-8] [timeMillis: 1579209738595] [levelValue: 800] [[
  ERROR: Bundle org.glassfish.fighterfish.osgi-http [296] Error starting file:/usr/local/glassfish4/glassfish/modules/autostart/osgi-http.jar (org.osgi.framework.BundleException: Unresolved constraint in bundle org.glassfish.fighterfish.osgi-http [296]: Unable to resolve 296.0: missing requirement [296.0] osgi.wiring.package; (&(osgi.wiring.package=com.sun.enterprise.web)(version>=3.1.1)) [caused by: Unable to resolve 271.0: missing requirement [271.0] osgi.wiring.package; (&(osgi.wiring.package=com.sun.enterprise.web.connector.grizzly)(version>=4.1.0)(!(version>=5.0.0))) [caused by: Unable to resolve 269.0: missing requirement [269.0] osgi.wiring.package; (&(osgi.wiring.package=org.glassfish.web.loader)(version>=4.1.0)(!(version>=5.0.0))) [caused by: Unable to resolve 267.0: missing requirement [267.0] osgi.wiring.package; (osgi.wiring.package=org.jboss.weld.interceptor.proxy) [caused by: Unable to resolve 281.0: missing requirement [281.0] osgi.wiring.package; (osgi.wiring.package=org.glassfish.weld) [caused by: Unable to resolve 282.0: missing requirement [282.0] osgi.wiring.package; (&(osgi.wiring.package=org.jboss.weld)(version>=1.0.0)(!(version>=2.0.0)))]]]]])]]

[2020-01-16T21:22:18.744+0000] [glassfish 4.1] [SEVERE] [] [] [tid: _ThreadID=12 _ThreadName=Thread-9] [timeMillis: 1579209738744] [levelValue: 1000] [[
  org.osgi.framework.BundleException: Unresolved constraint in bundle org.glassfish.fighterfish.osgi-web-container [301]: Unable to resolve 301.0: missing requirement [301.0] osgi.wiring.package; (&(osgi.wiring.package=com.sun.enterprise.web)(version>=4.0.0)) [caused by: Unable to resolve 271.0: missing requirement [271.0] osgi.wiring.package; (&(osgi.wiring.package=com.sun.enterprise.web.connector.grizzly)(version>=4.1.0)(!(version>=5.0.0))) [caused by: Unable to resolve 269.0: missing requirement [269.0] osgi.wiring.package; (&(osgi.wiring.package=org.glassfish.web.loader)(version>=4.1.0)(!(version>=5.0.0))) [caused by: Unable to resolve 267.0: missing requirement [267.0] osgi.wiring.package; (osgi.wiring.package=org.jboss.weld.interceptor.proxy) [caused by: Unable to resolve 281.0: missing requirement [281.0] osgi.wiring.package; (osgi.wiring.package=org.glassfish.weld) [caused by: Unable to resolve 282.0: missing requirement [282.0] osgi.wiring.package; (&(osgi.wiring.package=org.jboss.weld)(version>=1.0.0)(!(version>=2.0.0)))]]]]]
        at org.apache.felix.framework.Felix.resolveBundleRevision(Felix.java:3974)
        at org.apache.felix.framework.Felix.startBundle(Felix.java:2037)
        at org.apache.felix.framework.Felix.setActiveStartLevel(Felix.java:1291)
        at org.apache.felix.framework.FrameworkStartLevelImpl.run(FrameworkStartLevelImpl.java:304)
        at java.lang.Thread.run(Thread.java:748)]]

[2020-01-16T21:22:18.744+0000] [glassfish 4.1] [INFO] [] [] [tid: _ThreadID=12 _ThreadName=Thread-8] [timeMillis: 1579209738744] [levelValue: 800] [[
  ERROR: Bundle org.glassfish.fighterfish.osgi-web-container [301] Error starting file:/usr/local/glassfish4/glassfish/modules/autostart/osgi-web-container.jar (org.osgi.framework.BundleException: Unresolved constraint in bundle org.glassfish.fighterfish.osgi-web-container [301]: Unable to resolve 301.0: missing requirement [301.0] osgi.wiring.package; (&(osgi.wiring.package=com.sun.enterprise.web)(version>=4.0.0)) [caused by: Unable to resolve 271.0: missing requirement [271.0] osgi.wiring.package; (&(osgi.wiring.package=com.sun.enterprise.web.connector.grizzly)(version>=4.1.0)(!(version>=5.0.0))) [caused by: Unable to resolve 269.0: missing requirement [269.0] osgi.wiring.package; (&(osgi.wiring.package=org.glassfish.web.loader)(version>=4.1.0)(!(version>=5.0.0))) [caused by: Unable to resolve 267.0: missing requirement [267.0] osgi.wiring.package; (osgi.wiring.package=org.jboss.weld.interceptor.proxy) [caused by: Unable to resolve 281.0: missing requirement [281.0] osgi.wiring.package; (osgi.wiring.package=org.glassfish.weld) [caused by: Unable to resolve 282.0: missing requirement [282.0] osgi.wiring.package; (&(osgi.wiring.package=org.jboss.weld)(version>=1.0.0)(!(version>=2.0.0)))]]]]])]]

Let me update the link in dataverse-ansible and try a manual run.

[donsizemore]

I think we're good:

Tests run: 115, Failures: 0, Errors: 0, Skipped: 4
<snip>
Finished: SUCCESS

via https://jenkins.dataverse.org/job/IQSS-dataverse-develop/313/console

[poikilotherm]

Beware. I ran into the same issue today, because I used the JBoss JAR, not the one with the glassfish4 suffix. I see the same wrong URL in your post above.

I tested both Final and SP1, it didn't seem to make a difference.

There even is a 2.2.16, but with Payara at the horizon maybe we should just stick with good ol' versions.

[donsizemore]

@poikilotherm correct — I swapped it out in favor of https://search.maven.org/remotecontent?filepath=org/jboss/weld/weld-osgi-bundle/2.2.10.Final/weld-osgi-bundle-2.2.10.Final-glassfish4.jar at
https://github.com/IQSS/dataverse-ansible/blob/master/tasks/dataverse-glassfish.yml#L48

[djbrooke]

I see builds of blue, passing tests too. What a wonderful world. Let me know if we can close this or if there needs to be further discussion. Thank you again for the work here!

[pdurbin]

I'm very happy to see that https://jenkins.dataverse.org/job/IQSS-dataverse-develop/313/testReport/ shows edu.harvard.iq.dataverse.api taking over 3 minutes:

This means the API tests actually ran. On the build prior to this the API tests took only 0.27 seconds which means they didn't actually run.

So "develop" is fine. I'll grab this and take a look at our jobs for pull requests: https://jenkins.dataverse.org/job/IQSS-Dataverse-Develop-PR/

[pdurbin]

Hmm, in my pull request from this morning, I'm seeing a new failure at https://jenkins.dataverse.org//job/IQSS-Dataverse-Develop-PR/view/change-requests/job/PR-6538/1//testReport/junit/edu.harvard.iq.dataverse.api/SwordIT/testCreateDatasetPublishDestroy/

I'll dig in a little more.

[pdurbin]

Expected status code <200> doesn't match actual status code <400>.

at edu.harvard.iq.dataverse.api.SwordIT.testCreateDatasetPublishDestroy(SwordIT.java:558)

Huh. Oddly, publishing via SWORD failed.

[donsizemore]

SwordIT was one of the usual failure points in the API test suite on slower storage (hence our moving the API test suite to EC2), though in that case Glassfish would return a 500 response from whatever error forward through each test.

[pdurbin]

@donsizemore thanks, I'm trying to match up

https://jenkins.dataverse.org/job/IQSS-Dataverse-Develop-PR/view/change-requests/job/PR-6538/1/consoleFull

with the two server.log files (one rotated) at

https://jenkins.dataverse.org/job/IQSS-Dataverse-Develop-PR/view/change-requests/job/PR-6538/1/execution/node/3/ws/target/

but I'm not sure why that 400 error occured.

I'm still on that branch (pull request #6538) on my laptop and I just ran SwordIT and it passed so I'm feeling pretty good that it's some sort of flake in the API test suite, that nothing is actually broken.

[pdurbin]

Ok, I'm seeing successful builds on both develop and pull requests:

• https://jenkins.dataverse.org/job/IQSS-dataverse-develop/315/
• https://jenkins.dataverse.org/job/IQSS-Dataverse-Develop-PR/view/change-requests/job/PR-6539/4/

That's good enough for me. @donsizemore thanks for all your help!! 🎉 🎉

Closing.