Invalid OSGi manifest in 'activiti-engine-5.13.jar'

[beattidp]

Apparently the MANIFEST.MF in the 5.13 release version of 'activiti-engine' is technically invalid; incorrect, and should not pass even the initial parsing stage. The present mechanism for build validation likely contains a less strict facility for manifest parsing -- as the manifest contains a wildcard-like entry, "org.activiti.osgi*" which should not be present.

Incidentally, I discovered this as Eclipse Virgo choked on activiti-engine-5.13.jar when I dropped it into 'repository/usr'.
The activiti-engine-5.13.jar file was just one of 42 total dependency files I was deploying; all others imported correctly.
Some snippets of the log are shown below, along with a shell command which identifies the offending line number within the manifest.

org.eclipse.virgo.util.osgi.manifest.parse.BundleManifestParseException: Error parsing bundle manifest header 
HP014E:[col 4,175]: Expected a semicolon or comma but found '*'

 .. org.activiti.engine.test.mock;version="[5.13,5.14)",org.activiti.osgi*;resolution:=optional,org.apache.commons.lang;version="[2.4,3)" ..

$ unzip -c activiti-engine-5.13.jar META-INF/MANIFEST.MF | grep -n "org.activiti.osgi"
376: .test.mock;version="[5.13,5.14)",org.activiti.osgi*;resolution:=optio

• Notice the entry, "org.activiti.osgi*"

The jar file was pulled via Maven, from http://maven.alfresco.com/nexus/content/repositories/activiti

        <dependency>
            <groupId>org.activiti</groupId>
            <artifactId>activiti-engine</artifactId>
            <version>5.13</version>
        </dependency>

Also found a verify similar issue (which was since fixed) for Jersey; https://java.net/jira/browse/JERSEY-1426 .

Meanwhile, can anyone suggest a workaround? I know I could manually fix the manifest, but not sure just what to replace this entry with.

[beattidp]

I pulled the head from Github today, and built with Maven ('mvn verify'). Notice the warnings reported by the Maven bundle plugin.

[INFO] --- maven-compiler-plugin:2.3.2:compile (default-compile) @ activiti-bpmn-converter ---
[INFO] Compiling 74 source files to /usr/home/beattidp/Activiti/modules/activiti-bpmn-converter/target/classes
[INFO] 
[INFO] --- maven-bundle-plugin:2.1.0:manifest (bundle-manifest) @ activiti-bpmn-converter ---
[WARNING] Warning in manifest for org.activiti:activiti-bpmn-converter:jar:5.14-SNAPSHOT : Superfluous export-package instructions: [org.activiti.bpmn.util]
[WARNING] Warning in manifest for org.activiti:activiti-bpmn-converter:jar:5.14-SNAPSHOT : Did not find matching referal for org.activiti.osgi*

[INFO] --- maven-compiler-plugin:2.3.2:compile (default-compile) @ activiti-bpmn-model ---
[INFO] Compiling 78 source files to /usr/home/beattidp/Activiti/modules/activiti-bpmn-model/target/classes
[INFO] 
[INFO] --- maven-bundle-plugin:2.1.0:manifest (bundle-manifest) @ activiti-bpmn-model ---
[WARNING] Warning in manifest for org.activiti:activiti-bpmn-model:jar:5.14-SNAPSHOT : Did not find matching referal for org.activiti.osgi*

Simply changing the version of maven-bundle-plugin to a more recent version produces a slightly more insightful clue, as well as a fatal build error.

[INFO] --- maven-bundle-plugin:2.4.0:manifest (bundle-manifest) @ activiti-bpmn-model ---
in default pack? 
[WARNING] Manifest org.activiti:activiti-bpmn-model:jar:5.14-SNAPSHOT : Unused Import-Package instructions: [org.activiti.*, org.activiti.osgi*] 
[ERROR] Manifest org.activiti:activiti-bpmn-model:jar:5.14-SNAPSHOT : The default package '.' is not permitted by the Import-Package syntax.

[tijsrademakers]

should be okay now

[franck102]

I am seeing another issue with the activiti-explorer 5.14 war with Virgo, duplicate import packages it seems, although I haven't been able to determine which jar is causing the problem:

[[13-12-20 17:15:51] INFO  [start-signalling-8] System.out                            05:15:51,732 [start-signalling-8] ERROR org.springframework.web.context.ContextLoader  - Context initialization failed 

[[13-12-20 17:15:51] INFO  [start-signalling-8] System.out                            org.eclipse.virgo.nano.serviceability.Assert$FatalAssertionException: input packageImports must not contain duplicate items 

[beattidp]

I have seen this previously in my Maven projects, and traced it to a duplicate declaration
which I entered manually in addition to the import inferred by the Maven bundle plugin.
That is, if I specifically declare a dependency in a module POM, in most cases I need not
also explicitly declare it in the 'Import-Package' section.

[franck102]

The culprit is in C:\activiti-5.14\wars\activiti-explorer\WEB-INF\lib\activiti-engine-5.14.jar\META-INF\MANIFEST.MF, which has a duplicate import for org.activiti.osgi:

org.activiti.osgi;resolution:=optional,org.activiti.osgi;resolution:=optional

Any chance this will be addressed? This prevents deploying activiti in Eclipse Virgo.

Thanks!

[tijsrademakers]

This is addressed in the current Github master. We also released Activiti 5.14 to Maven central a couple of weeks ago and this also includes the OSGi fix. So if you retrieve the activiti engine 5.14 JAR from Maven central it should be ok.

[franck102]

The exact same problem still exists in the 5.14 distribution. I can confirm that simply removing the duplicate entry allows Activiti to deploy fine as a war in Virgo 3.6.1.
I haven't checked the Github master.

[franck102]

Other OSGI issues with the 5.14 distribution:

• The activiti-simple-workflow-5.14.jar MANIFEST doesn't export any packages; those packages are required (imported) by activiti-rest-5.14.jar
• The activiti-bpmn-layout-5.14.jar probably contains a bogus MANIFEST file, here is the exception from Virgo:

Caused by: org.eclipse.virgo.util.osgi.manifest.parse.BundleManifestParseException: Error parsing bundle manifest header []
HP009E:Unexpectedly ran out of data
at org.eclipse.virgo.util.osgi.manifest.parse.standard.StandardHeaderParser.parseBundleSymbolicName(StandardHeaderParser.java:96)
at org.eclipse.virgo.util.osgi.manifest.internal.StandardBundleSymbolicName.parse(StandardBundleSymbolicName.java:33)
at org.eclipse.virgo.util.osgi.manifest.internal.BaseParameterised.resetFromParseString(BaseParameterised.java:42)
at org.eclipse.virgo.util.osgi.manifest.internal.StandardBundleManifest.initializeHeaders(StandardBundleManifest.java:124)
at org.eclipse.virgo.util.osgi.manifest.internal.StandardBundleManifest.(StandardBundleManifest.java:79)
at org.eclipse.virgo.util.osgi.manifest.internal.StandardBundleManifest.(StandardBundleManifest.java:101)
at org.eclipse.virgo.util.osgi.manifest.internal.StandardBundleManifest.(StandardBundleManifest.java:97)
at org.eclipse.virgo.util.osgi.manifest.BundleManifestFactory.createBundleManifest(BundleManifestFactory.java:91)
at org.eclipse.virgo.util.osgi.manifest.BundleManifestFactory.createBundleManifest(BundleManifestFactory.java:102)
at org.eclipse.virgo.kernel.artifact.internal.BundleManifestUtils.readBundleManifest(BundleManifestUtils.java:74)
at org.eclipse.virgo.kernel.artifact.bundle.BundleBridge.generateArtifactDescriptor(BundleBridge.java:78)

[franck102]

activiti-json-converter doesn't have any export directives either in its MANIFEST... I wonder how those MANIFESTs are being generated?

I am having to repackage all the jars using basic default BND instructions and that seems to work just fine... this is what I am using:

########### pom.xml

<plugin>
                <groupId>org.ops4j</groupId>
                <artifactId>maven-pax-plugin</artifactId>
                <version>1.5</version>
                <extensions>true</extensions>
            </plugin>
            <plugin>
                <groupId>org.apache.felix</groupId>
                <artifactId>maven-bundle-plugin</artifactId>
                <version>2.3.7</version>
                <configuration>
                    <dumpInstructions>${project.build.directory}/${project.build.finalName}.bnd</dumpInstructions>
                    <instructions>
                        <Bundle-SymbolicName>${bundle.symbolicName}</Bundle-SymbolicName>
                        <Bundle-Version>${bundle.version}</Bundle-Version>
                        <!-- export wrapped contents: don't use Export-Package:* as this bundles everything on the classpath -->
                        <_exportcontents>${exported-content}</_exportcontents>
                        <Private-Package>!*</Private-Package>
                        <!-- each module can override these defaults in their osgi.bnd file -->
                        <_include>-osgi.bnd</_include>
                    </instructions>
                </configuration>
            </plugin>

======== osgi.bnd:

Embed-Dependency: *;scope=compile|runtime;type=!pom;inline=true

[tijsrademakers]

@franck102 are you sure that you looked at the Activiti 5.14 Engine JAR from Maven central?
The Activiti JSON converter and simple worklfow and bpmn layout modules were not OSGi ready in 5.14. This is fixed for most modules in the current Github master.

[franck102]

I had looked at activiti-engine-5.14.jar found in the Activiti download from the web site. You are right that the version on maven central doesn't have the duplicate entry.
So there are different flavors of the 5.14 bundles out there, that is a bit worrying... minor version numbers would be nice.

That being said both jars have hard dependencies on secondary packages of 3rd party bundles, i.e. it seems like only the first package is marked as optional:

org.drools;resolution:=optional;version="[5.5,6)",org.drools.builder;version="[5.5,6)",<more drools packages>

This creates unwanted hard dependencies on drools, junit, org.junit, and possibly other bundles. Here are the bnd instructions I used to fix this when repackaging:

Embed-Dependency: *;scope=compile|runtime;type=!pom;inline=true
Import-Package: junit.*;resolution=optional,org.junit.*;resolution=optional,org.drools.*;resolution=optional,*

I have managed to deploy activiti-rest on Virgo, here are the bundles I had to repackage if anyone is trying to do the same:

activiti-engine
activiti-simple-workflow
activiti-bpmn-layout
activiti-json-converter
antlr:antlr:2.7.7
math.geom2d:javaGeom:0.11.1
org.antlr:stringtemplate:3.2.1
org.tinyjee.jgraphx:jgraphx:1.10.4.1

Deploying drools would require some additional repackaging...

[MPriess]

Optional dependencies to JUnit in a OSGI manifest are a little bit ugly. Actually the activiti-engine artifact provide classes which inherite from JUnit and help you to test the engine. This classes are also used to test the engine itself. A solution could be to extract the JUnit helper classes into a own artifact for example activiti-test similiar like in the Apache Camel Project and also move the integration testcases which are testing the engine into a seperate module to avoid cycle dependencies between the maven artifacts.

If I look into this ticket there are a lot of peope out there which use Activiti in a OSGi container. I think it would be nice to implement an OSGi integration test in the future. BTW. I already opened a ticket for that ;) @franck102 feel free to contribute.

[franck102]

A test would be fantastic... here are some thoughts and a first contribution. As far as I know there are no straightforward tools to statically validate a set of OSGI bundles.
The most reliable approach I can think of would be to actually load the bundles in Equinox, and this is something that should be automatable in the build. The steps would be:

• Create a test directory that contains the bundles listed in config.ini below + org.eclipse.osgi_3.8.1.v20120830-144521.jar
• In that directory create a configuration subdir with the config.ini file below
• Run java -jar org.eclipse.osgi_3.8.1.v20120830-144521.jar -console ( -consoleLog to get errors on stdout)
• Check the log file in configuration for errors

There is probably a smarter way to start equinox command line to simply have exit with a status of 0 / non-zero depending on whether initial resolution triggered any errors...

The proposed configuration is somewhat arbitrary, and involves quite a bit of repackaging as described earlier in this thread.

config.ini:

osgi.bundles=./org.apache.felix.gogo.runtime_0.10.0.v201209301036.jar@start,\
    ./org.apache.felix.gogo.command_0.10.0.v201209301215.jar@start,\
    ./org.apache.felix.gogo.shell_0.10.0.v201211091412.jar@start,\
    ./org.eclipse.equinox.console_1.0.100.v20121001-124408.jar@start,\
    ./javax.servlet_3.0.0.v201112011016.jar@start,\
    ./javax.mail_1.4.0.v201005080615.jar@start,\
    ./javax.persistence_2.0.4.v201112161009.jar@start,\
    ./jcl-over-slf4j-1.5.11.jar@start,\
    ./org.springframework.core_3.1.0.RELEASE.jar@start,\
    ./org.springframework.beans_3.1.0.RELEASE.jar@start,\
    ./commons-io-2.4.jar@start,\
    ./jackson-core-asl-1.9.9.jar@start,\
    ./jackson-mapper-asl-1.9.9.jar@start,\
    ./java-uuid-generator-3.1.3.jar@start,\
    ./commons-email-1.2.jar@start,\
    ./commons-lang3-3.1.jar@start,\
    ./joda-time-2.1.jar@start,\
    ./mybatis-3.2.2.jar@start,\
    ./commons-fileupload-1.2.2.jar@start,\
    ./org.restlet.ext.fileupload-2.0.15.jar@start,\
    ./org.restlet.ext.jackson-2.0.15.jar@start,\
    ./org.restlet.ext.servlet-2.0.15.jar@start,\
    ./org.restlet-2.0.15.jar@start,\
    ./antlr-2.7.7.jar@start,\
    ./stringtemplate-3.2.1.jar@start,\
    ./geom2d-javaGeom-0.11.1.jar@start,\
    ./tinyjee-jgraphx-1.10.4.1.jar@start,\
    ./org.slf4j.api_1.7.2.v20121108-1250.jar@start,\
    ./activiti-engine-5.14.jar@start,\
    ./activiti-bpmn-converter-5.14.jar@start,\
    ./activiti-bpmn-layout-5.14.jar@start,\
    ./activiti-bpmn-model-5.14.jar@start,\
    ./activiti-common-rest-5.14.jar@start,\
    ./activiti-simple-workflow-5.14.jar@start,\
    ./activiti-rest-5.14.jar@start,\
    ./activiti-json-converter-5.14.jar@start
osgi.console.enable.builtin=false
#osgi.console=<port>
eclipse.ignoreApp=true
org.osgi.framework.system.packages = \
 javax.accessibility,\
 javax.activation,\
 javax.activation;version="1.1.1",\
 javax.activity,\
 javax.annotation,\
 javax.annotation;version="1.0.0",\
 javax.annotation;version="1.1.0",\
 javax.annotation.processing,\
 javax.crypto,\
 javax.crypto.interfaces,\
 javax.crypto.spec,\
 javax.imageio,\
 javax.imageio.event,\
 javax.imageio.metadata,\
 javax.imageio.plugins.bmp,\
 javax.imageio.plugins.jpeg,\
 javax.imageio.spi,\
 javax.imageio.stream,\
 javax.inject,\
 javax.jws,\
 javax.jws;version="2.0",\
 javax.jws.soap,\
 javax.jws.soap;version="2.0",\
 javax.lang.model,\
 javax.lang.model.element,\
 javax.lang.model.type,\
 javax.lang.model.util,\
 javax.management,\
 javax.management.loading,\
 javax.management.modelmbean,\
 javax.management.monitor,\
 javax.management.openmbean,\
 javax.management.relation,\
 javax.management.remote,\
 javax.management.remote.rmi,\
 javax.management.timer,\
 javax.naming,\
 javax.naming.directory,\
 javax.naming.event,\
 javax.naming.ldap,\
 javax.naming.spi,\
 javax.net,\
 javax.net.ssl,\
 javax.print,\
 javax.print.attribute,\
 javax.print.attribute.standard,\
 javax.print.event,\
 javax.rmi,\
 javax.rmi.CORBA,\
 javax.rmi.ssl,\
 javax.script,\
 javax.script;version="1.1",\
 javax.security.auth,\
 javax.security.auth.callback,\
 javax.security.auth.kerberos,\
 javax.security.auth.login,\
 javax.security.auth.spi,\
 javax.security.auth.x500,\
 javax.security.cert,\
 javax.security.sasl,\
 javax.sound.midi,\
 javax.sound.midi.spi,\
 javax.sound.sampled,\
 javax.sound.sampled.spi,\
 javax.sql,\
 javax.sql.rowset,\
 javax.sql.rowset.serial,\
 javax.sql.rowset.spi,\
 javax.swing,\
 javax.swing.border,\
 javax.swing.colorchooser,\
 javax.swing.event,\
 javax.swing.filechooser,\
 javax.swing.plaf,\
 javax.swing.plaf.basic,\
 javax.swing.plaf.metal,\
 javax.swing.plaf.multi,\
 javax.swing.plaf.synth,\
 javax.swing.table,\
 javax.swing.text,\
 javax.swing.text.html,\
 javax.swing.text.html.parser,\
 javax.swing.text.rtf,\
 javax.swing.tree,\
 javax.swing.undo,\
 javax.tools,\
 javax.transaction,\
 javax.transaction;version="1.0.1",\
 javax.transaction;version="1.1.0",\
 javax.transaction.xa,\
 javax.transaction.xa;version="1.0.1",\
 javax.transaction.xa;version="1.1.0",\
 javax.xml,\
 javax.xml;version="1.0.1",\
 javax.xml.bind,\
 javax.xml.bind;version="2.0",\
 javax.xml.bind.annotation,\
 javax.xml.bind.annotation;version="2.0",\
 javax.xml.bind.annotation.adapters,\
 javax.xml.bind.annotation.adapters;version="2.0",\
 javax.xml.bind.attachment,\
 javax.xml.bind.attachment;version="2.0",\
 javax.xml.bind.helpers,\
 javax.xml.bind.helpers;version="2.0",\
 javax.xml.bind.util,\
 javax.xml.bind.util;version="2.0",\
 javax.xml.crypto,\
 javax.xml.crypto;version="1.0",\
 javax.xml.crypto.dom,\
 javax.xml.crypto.dom;version="1.0",\
 javax.xml.crypto.dsig,\
 javax.xml.crypto.dsig;version="1.0",\
 javax.xml.crypto.dsig.dom,\
 javax.xml.crypto.dsig.dom;version="1.0",\
 javax.xml.crypto.dsig.keyinfo,\
 javax.xml.crypto.dsig.keyinfo;version="1.0",\
 javax.xml.crypto.dsig.spec,\
 javax.xml.crypto.dsig.spec;version="1.0",\
 javax.xml.datatype,\
 javax.xml.namespace,\
 javax.xml.parsers,\
 javax.xml.soap,\
 javax.xml.soap;version="1.3.0",\
 javax.xml.stream,\
 javax.xml.stream;version="1.0.1",\
 javax.xml.stream.events,\
 javax.xml.stream.events;version="1.0.1",\
 javax.xml.stream.util,\
 javax.xml.stream.util;version="1.0.1",\
 javax.xml.transform,\
 javax.xml.transform.dom,\
 javax.xml.transform.sax,\
 javax.xml.transform.stax,\
 javax.xml.transform.stream,\
 javax.xml.validation,\
 javax.xml.ws,\
 javax.xml.ws;version="2.1.1",\
 javax.xml.ws.handler,\
 javax.xml.ws.handler;version="2.1.1",\
 javax.xml.ws.handler.soap,\
 javax.xml.ws.handler.soap;version="2.1.1",\
 javax.xml.ws.http,\
 javax.xml.ws.http;version="2.1.1",\
 javax.xml.ws.soap,\
 javax.xml.ws.soap;version="2.1.1",\
 javax.xml.ws.spi,\
 javax.xml.ws.spi;version="2.1.1",\
 javax.xml.ws.wsaddressing;version="2.1.1",\
 javax.xml.xpath,\
 org.eclipse.virgo.kernel.equinox.extensions,\
 org.eclipse.virgo.kernel.equinox.extensions;version="3.6.1",\
 org.eclipse.virgo.kernel.equinox.extensions.hooks,\
 org.eclipse.virgo.kernel.equinox.extensions.hooks;version="3.6.1",\
 org.ietf.jgss,\
 org.omg.CORBA,\
 org.omg.CORBA_2_3,\
 org.omg.CORBA_2_3.portable,\
 org.omg.CORBA.DynAnyPackage,\
 org.omg.CORBA.ORBPackage,\
 org.omg.CORBA.portable,\
 org.omg.CORBA.TypeCodePackage,\
 org.omg.CosNaming,\
 org.omg.CosNaming.NamingContextExtPackage,\
 org.omg.CosNaming.NamingContextPackage,\
 org.omg.Dynamic,\
 org.omg.DynamicAny,\
 org.omg.DynamicAny.DynAnyFactoryPackage,\
 org.omg.DynamicAny.DynAnyPackage,\
 org.omg.IOP,\
 org.omg.IOP.CodecFactoryPackage,\
 org.omg.IOP.CodecPackage,\
 org.omg.Messaging,\
 org.omg.PortableInterceptor,\
 org.omg.PortableInterceptor.ORBInitInfoPackage,\
 org.omg.PortableServer,\
 org.omg.PortableServer.CurrentPackage,\
 org.omg.PortableServer.POAManagerPackage,\
 org.omg.PortableServer.POAPackage,\
 org.omg.PortableServer.portable,\
 org.omg.PortableServer.ServantLocatorPackage,\
 org.omg.SendingContext,\
 org.omg.stub.java.rmi,\
 org.w3c.dom,\
 org.w3c.dom.bootstrap,\
 org.w3c.dom.css,\
 org.w3c.dom.events,\
 org.w3c.dom.html,\
 org.w3c.dom.ls,\
 org.w3c.dom.ranges,\
 org.w3c.dom.stylesheets,\
 org.w3c.dom.traversal,\
 org.w3c.dom.views ,\
 org.xml.sax,\
 org.xml.sax.ext,\
 org.xml.sax.helpers,\
 sun.misc,\
 sun.reflect,\
 com.sun.tools.xjc

[MPriess]

There is already tooling out there to build OSGi integration tests for different OSGi frameworks like Felix, Equinox or Knopflerfish. I will start with a sample project which load all necessary dependencies and check how a test can be realised with pax-exam https://ops4j1.jira.com/wiki/display/PAXEXAM3/Pax+Exam

[franck102]

Yes, we use pax-exam for our testing, it is however quite a bit more involved to setup and automate... but definitely more powerful.
For antlr to load properly you will need this:

public abstract class BaseServiceTest
{
static {
System.setProperty("ANTLR_USE_DIRECT_CLASS_LOADING", "true");
}
...

[franck102]

At a higher level, I am struggling with the explorer & modeler on the OSGI front. Those two depend on zillions on non OSGI libraries, some of which would require more than simple repackaging to work nicely under OSGI (e.g. gwt) because they use unorthodox class loading patterns.

The sensible thing to do seems to be to just use the non-OSGI explorer war (my Virgo container supports that), but the explorer then doesn't share any classes with the OSGI engine (different classloaders) so it is pretty useless to me.
I will need to see if boot delegation or context class loading can help, but at this point I don't see how to resolve this.

[MPriess]

I'm satisfied if the engine is covered with a integration test in a first version and maybe the rest interface :)