Can't deploy web app as an unpacked directory inside a bundle JAR on Felix

[jmcc0nn3ll]

migrated from Bugzilla #382032
status ASSIGNED severity normal in component osgi for 7.6.x
Reported in version unspecified on platform PC
Assigned to: Jan Bartel

On 2012-06-07 13:20:10 -0400, Dan Gravell wrote:

Build Identifier:

This is Jetty 7.6.1.

I have followed the instructions (http://wiki.eclipse.org/Jetty/Feature/Jetty_OSGi) to create an OSGI bundle that, when started, both registers a Server and WebAppContext Service. Here's my code for the latter:

WebAppContext blissUiContextHandler = new WebAppContext();
Dictionary blissUiProps = new Hashtable();
blissUiProps.put(OSGiWebappConstants.SERVICE_PROP_WAR,"target/classes");
blissUiProps.put(OSGiServerConstants.MANAGED_JETTY_SERVER_NAME, serverName);
context.registerService(ContextHandler.class.getName(),blissUiContextHandler,blissUiProps);

Notice that the 'target/classes' folder is specified as the location for the WAR structure inside my bundle.

At development time, running in the Eclipse OSGi runner, this works fine. All the code for the web app goes into target/classes and it all works nicely.

At runtime, within Felix, it doesn't work. Here's what I get in way of output:

2012-06-07 18:02:32.700:WARN:oejobis.DefaultJettyAtJettyHomeHelper:No default jetty started.
2012-06-07 18:02:32.784:INFO:oejs.Server:jetty-7.6.1.v20120215
2012-06-07 18:02:32.802:INFO:oejdp.ScanningAppProvider:Deployment monitor /home/gravelld/eclipse-workspaces/bliss/com.elsten.bliss.installer/release/xyz at interval 10
2012-06-07 18:02:32.815:INFO:oejs.AbstractConnector:Started SelectChannelConnector@0.0.0.0:3220
2012-06-07 18:02:59.687:INFO:oejd.DeploymentManager:Deployable added: com.elsten.bliss.ui-1.0.0.0/target/classes
2012-06-07 18:02:59.694:INFO:oejw.WebInfConfiguration:Extract bundle://10.0:0/target/classes/ to /tmp/jetty-0.0.0.0-3220-classes--any-/webapp
2012-06-07 18:02:59.733:INFO:oejw.StandardDescriptorProcessor:NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
2012-06-07 18:02:59.745:INFO:oejsh.ContextHandler:started o.e.j.w.WebAppContext{/,file:/tmp/jetty-0.0.0.0-3220-classes--any-/webapp/},bundle://10.0:0/target/classes/

If I attempt to access the server I get a blank directory listing.

Note that /tmp/jetty-0.0.0.0-3220-classes-_-any-/webapp is empty.

I stepped through this with a debugger particularly looking at WebBundleDeployerHelper. This returns the location of the web app as bundle://10.0:0/target/classes/, whereas at runtime this is a file:// url which can be seen to exist (I don't know how the resolution of felix bundle:// URLs work although I note you have some code for this.

I inspected the bundle JAR by hand. The target/classes folder is there, fully populated, and is including in Bundle-Classpath.

Are there any workarounds? Am I initialising the WebAppContext in the wrong way?

Reproducible: Always

On 2012-06-07 21:49:49 -0400, Hugues Malphettes wrote:

Hi Dan,
It looks like the support for locating java.io.File in Felix is lagging.
You can provide your own implementation of org.eclipse.jetty.osgi.boot.utils.BundleFileLocatorHelper:

org.eclipse.jetty.osgi.boot.utils.BundleFileLocatorHelper.DEFAULT = felixAbleBundleFileLocatorHelper;

If you would consider contributing it back we would be interested.
Let us know how it goes.

On 2012-06-07 21:53:52 -0400, Hugues Malphettes wrote:

Hum, it looks like it was already solved by Caspar here: bug 372656.

On 2012-06-08 04:11:57 -0400, Dan Gravell wrote:

Thanks, I'll try that out.

On 2012-06-11 15:34:09 -0400, Dan Gravell wrote:

I took a look at this today. It took a little more work than expected to get it working, I suspect because that code was for a version of Felix before v3. Felix is ATM at 4.0.2.

I installed Casper's code (with his change in BundleLocationFilter.accept) but for me the first line in FileLocatorHelperImpl worked every time:

File retVal = DEFAULT.getBundleInstallLocation(bundle);

retVal always returned a file:// URL to the original location of the bundle (not the cache). And I assume this was still incorrect.

I disabled this line to force Casper's code to run. But the cache was never found because it relies on a bundle.location being present. It seems that since v3 of Felix the bundle data has been consolidated in "bundle.info". So I changed that to check bundle.info (I just do a readLine twice... this is probably slow / prone to error).

Put another way, http://felix.apache.org/site/apache-felix-framework-bundle-cache.html seems to be waaaaay out of date.

That wasn't all though. Initially I had the SERVICE_PROP_WAR set to a sub directory of the bundle JAR ("target/classes"). This was correctly built into the bundle JAR file but I remembered that I had this working previously with a built JAR/WAR embedded within the bundle JAR. So I changed the build to that (I'm using PDE build in Eclipse) and that worked.

Maybe you should consider this Felix versioning issue before accepting this code? Or at least adding some comments/documentation, because I don't think this will work for Felix 3+.

Here's my revised version of Casper's code. I'd welcome your comments as to improvements, because it would also be great if we could incorporate this into Jetty.

package org.eclipse.jetty.osgi.boot.utils;

import static org.osgi.framework.Constants.FRAMEWORK_STORAGE;

import java.io.File;
import java.io.FileFilter;
import java.net.URL;
import java.util.Enumeration;

import org.osgi.framework.Bundle;

public class FileLocatorHelperImpl implements BundleFileLocatorHelper {

private static final String FELIX_BUNDLE_IMP_CLASS =

"org.apache.felix.framework.BundleImpl";

@Override
public File getBundleInstallLocation(Bundle bundle) throws Exception
{
    File retVal = DEFAULT.getBundleInstallLocation(bundle);

    if(FELIX_BUNDLE_IMP_CLASS.equals(bundle.getClass().getCanonicalName())) {
        File cacheDir = new

File(bundle.getBundleContext().getProperty(FRAMEWORK_STORAGE));
retVal = findBundleCacheDir(bundle, cacheDir);
}
return retVal;
}

private File findBundleCacheDir(final Bundle bundle, File cacheDir)
{
    File[] files = cacheDir.listFiles(new BundleLocationFilter(bundle));

    if(files == null || files.length != 1) {
        return null;
    }

    File[] subdirs = files[0].listFiles(new FileFilter() {
        @Override
        public boolean accept(File file)
        {
            return file.isDirectory() &&

file.getName().startsWith("version");
}
});

    if(subdirs == null || subdirs.length != 1) {
        return null;
    }
    return new File(subdirs[0] + File.separator + "bundle.jar");
}


@Override
public File getFileInBundle(Bundle bundle, String path) throws Exception
{
    return DEFAULT.getFileInBundle(bundle, path);
}


@Override
public File[] locateJarsInsideBundle(Bundle bundle) throws Exception
{
    return DEFAULT.locateJarsInsideBundle(bundle);
}


@Override
public Enumeration<URL> findEntries(Bundle bundle, String entryPath)
{
    return DEFAULT.findEntries(bundle, entryPath);
}

}

package org.eclipse.jetty.osgi.boot.utils;

import java.io.BufferedReader;
import java.io.File;
import java.io.FileFilter;
import java.io.FileInputStream;
import java.io.IOException;
import java.io.InputStreamReader;

import org.osgi.framework.Bundle;

public class BundleLocationFilter implements FileFilter {

private static final String BUNDLE_LOCATION_FILENAME = "bundle.info";
private Bundle bundle;

BundleLocationFilter(Bundle bundle)
{
    this.bundle = bundle;
}

@Override
public boolean accept(File file)
{
    if(file.isDirectory()) {
        File originalLocation = new File(file.getAbsolutePath() +

File.separator + BUNDLE_LOCATION_FILENAME);
if(originalLocation.isFile()) {
BufferedReader reader = null;
try {
reader = new BufferedReader(new InputStreamReader(new
FileInputStream(originalLocation)));
String id = reader.readLine();
String location = reader.readLine();
if(bundle.getLocation().equals(location.trim())) {
return true;
}
} catch (IOException e) {
return false;
} finally {
if(reader != null) {
try {
reader.close();
} catch (IOException e) {}
}
}
}
}
return false;
}
}

Obvious issues:

• That code does a DEFAULT.getBundleInstallLocation(bundle) and that then gets ignored
• 2x readLines may not be optimal
• We're still reliant on an unpublished standard

On 2012-06-12 05:30:53 -0400, Dan Gravell wrote:

Additionally, I just found out that I don't even need Caspar's patch if I use an embedded JAR. So I suppose this bug is just about finding folder level resources (which are a valid WAR structure) in a bundle.

On 2012-06-12 20:40:54 -0400, Hugues Malphettes wrote:

(In reply to comment # 5)

Additionally, I just found out that I don't even need Caspar's patch if I use
an embedded JAR. So I suppose this bug is just about finding folder level
resources (which are a valid WAR structure) in a bundle.

That sounds like a good workaround.
Keep us updated.

On 2012-06-13 05:30:12 -0400, Dan Gravell wrote:

Everything appears to work fine with the embedded JAR. As far as my own application is concerned I am happy with that, although I think it's still an issue that needs to be addressed in the Jetty/Felix interaction, no?

Better still, shouldn't this be spec'd in the OSGi spec so you don't have to do the horrible URL introspection?

On 2012-06-15 03:03:33 -0400, Hugues Malphettes wrote:

(In reply to comment # 7)

Everything appears to work fine with the embedded JAR. As far as my own
application is concerned I am happy with that, although I think it's still an
issue that needs to be addressed in the Jetty/Felix interaction, no?
Yes it remains an issue.

Better still, shouldn't this be spec'd in the OSGi spec so you don't have to do
the horrible URL introspection?
Yes it is most horrible. There is no standard API to access the file system in OSGi as far as I know. Unless the OSGi spec was updated in this regard in the last year.
My naive understanding is that OSGi does not map the bundles resources to a filesystem; in order to support transparent distributed services or very small or very constrained installations.

Let us know if there is now a none proprietary API in OSGi to access the filesystem.

On 2012-06-15 05:15:05 -0400, Dan Gravell wrote:

Yeah, you're right, I wasn't thinking. By specifying a folder as the web app path I'm implying a file system. Instead, if I specify a URL to a resource with WAR structure which is in the Bundle-Classpath I guess everything just works by virtue of the OSGi classloading.

If this is true, in many ways you could see an argument for completely withdrawing support for folder locations as it's not 'OSGi' enough. However, I hope you don't because it would make development time work extremely cumbersome.

On 2012-06-15 05:56:30 -0400, Hugues Malphettes wrote:

(In reply to comment # 9)

Yeah, you're right, I wasn't thinking. By specifying a folder as the web app
path I'm implying a file system. Instead, if I specify a URL to a resource with
WAR structure which is in the Bundle-Classpath I guess everything just works by
virtue of the OSGi classloading.

If this is true, in many ways you could see an argument for completely
withdrawing support for folder locations as it's not 'OSGi' enough. However, I
hope you don't because it would make development time work extremely
cumbersome.
All my projects do want a file system returned and do many web frameworks that just won't work if the webapp is not unzipped: not anywhere near removing it.

It is here:
javax.servlet.ServletContext#getRealPath(java.lang.String)
And it won't return null in OSGi thanks to this ugly introspection.
Virgo uses a fragment specific to equinox or felix if I remember well.

On 2012-06-15 07:47:26 -0400, Dan Gravell wrote:

Cool, well thanks for your insights. I suppose, then, this remains open until Felix is supported.

On 2013-10-23 20:26:03 -0400, Jan Bartel wrote:

Hi Dan,

Sorry this issue has been open a while.

Do you know if this has actually been fixed? Our unit tests use felix 3.2.2 and 4.0.2 and it seems to work ... however, as its actually running inside eclipse I'm not confident that it would be the same as running in a standalone osgi container.

Could you possibly test this with the most recent version of jetty (something like 9.1.0.RC0) and report if this is still a problem or not?

thanks,
Jan

On 2013-10-24 03:40:11 -0400, Dan Gravell wrote:

I can try... I'm pretty sure you need to run it standalone though. If you run it in Eclipse then the Equinox URL handling for bundles will take over, which will mean it will work.

On 2014-09-19 03:14:51 -0400, Jan Bartel wrote:

Hi Dan,

Well its been a while ... where are we at with this bug? The jetty osgi code has undergone some major revisions in the interim. I've also tested jetty-2.4-SNAPSHOT with felix 4.4.1 and webapps deployed as bundles are working fine.

After all the back-and-forth on this issue, can we restate the original actual problem? Is it that you can't deploy a webapp as a packed bundle where the webapp base directory is expressed as a location relative to the root of the bundle itself?

thanks
Jan

(In reply to Dan Gravell from comment # 13)

I can try... I'm pretty sure you need to run it standalone though. If you
run it in Eclipse then the Equinox URL handling for bundles will take over,
which will mean it will work.

On 2014-11-05 14:46:56 -0500, Dan Gravell wrote:

Yes, that's right. My bundle activator was starting a Jetty instance, but the path I was specifying as OSGiWebappConstants.SERVICE_PROP_WAR, which contained a bunch of files in the standard WAR structure, was not being found.

It appeared that the OOTB behaviour just did not cater for Felix. Maybe it does now?

Personally, I continue to use the workaround detailed above.

On 2014-11-06 00:12:51 -0500, Jan Bartel wrote:

Hi Dan,

(In reply to Dan Gravell from comment # 15)

Yes, that's right. My bundle activator was starting a Jetty instance, but
the path I was specifying as OSGiWebappConstants.SERVICE_PROP_WAR, which
contained a bunch of files in the standard WAR structure, was not being
found.

Thanks for the confirmation of the situation.

It appeared that the OOTB behaviour just did not cater for Felix. Maybe it
does now?

Nope, your particular use-case (webapp as directory inside bundle) does not work.

Personally, I continue to use the workaround detailed above.

The workaround of using a webapp as a packed war embedded into the bundle I can confirm is working, so if its working for you, I suggest you keep using it for a while.

It seems to be problematic to obtain the location of the cached felix bundle, rather than the location of the bundle that was deployed.

Bundle.getLocation() on felix returns the original location of the bundle that was deployed, not the cached location. Reading your previous postings of your and Caspar's efforts to obtain the cached location, and consulting the felix source, plus observing the cache when deploying different configurations of bundles, my conclusion is that this will be difficult and messy!

For example, if I have a bundle that contains a webapp as a subdirectory (ie your case), like so:

example-bundle.jar:
META-INF/MANIFEST.MF
target/
target/classes/
target/classes/index.html
target/classes/com/
target/classes/com/acme/
target/classes/com/acme/osgi/
target/classes/com/acme/osgi/Activator.class

then felix puts it into the cache like this:
bundle38
��� bundle.info
��� version0.0
��� bundle.jar
��� bundle.jar-embedded
�   ��� target
�   ��� classes
��� revision.location

If instead I have an embedded packed war like so:

example-bundle-embedded.jar:
META-INF/MANIFEST.MF
com/
com/acme/
com/acme/osgi/
com/acme/osgi/Activator.class
embedded.war

then felix puts it into the cache like this:
bundle38
��� bundle.info
��� version0.4
��� bundle.jar
��� revision.location

I think the way forward is to use a lot of reflection to try and obtain the cached file location (rather than to hard-code an expectation of the above directory layout). However, this work will have a low priority as there is an effective workaround (using an embedded war rather than a directory).

thanks
Jan

On 2014-11-10 14:41:40 -0500, Dan Gravell wrote:

Sounds sensible.

[janbartel]

Closing this old issue, AFAIK felix works with jetty osgi. If not, please reopen and provide details.