Akka remoting does not work with Akka OSGi unless I copy its reference.conf file

[PhilAndrew]

I am trying to use Akka remoting with Akka OSGi.
Example project
https://github.com/PhilAndrew/sbt-osgi-felix-akka-blueprint-camel/tree/136f8af2a40eea8ce2c8aae066cbb08f94559c19

The example project can be used to test this problem.

This does not work, it says:

print components will not participate in quiesce operations
ERROR: Bundle default.sbt.osgi.felix.akka.blueprint.camel [21] Error starting reference:file:/D:/home/projects/sbt-osgi-
felix-akka-blueprint-camel/target/scala-2.11/classes/ (org.osgi.framework.BundleException: Activator start error in bund
le default.sbt.osgi.felix.akka.blueprint.camel [21].)
com.typesafe.config.ConfigException$Missing: No configuration setting found for key 'akka.remote.log-received-messages'
        at com.typesafe.config.impl.SimpleConfig.findKeyOrNull(SimpleConfig.java:152)
        at com.typesafe.config.impl.SimpleConfig.findOrNull(SimpleConfig.java:170)
        at com.typesafe.config.impl.SimpleConfig.findOrNull(SimpleConfig.java:176)
        at com.typesafe.config.impl.SimpleConfig.findOrNull(SimpleConfig.java:176)
        at com.typesafe.config.impl.SimpleConfig.find(SimpleConfig.java:184)
        at com.typesafe.config.impl.SimpleConfig.find(SimpleConfig.java:189)
        at com.typesafe.config.impl.SimpleConfig.getBoolean(SimpleConfig.java:214)
        at akka.remote.RemoteSettings.<init>(RemoteSettings.scala:24)
        at akka.remote.RemoteActorRefProvider.<init>(RemoteActorRefProvider.scala:112)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:422)
        at akka.actor.ReflectiveDynamicAccess$$anonfun$createInstanceFor$2.apply(DynamicAccess.scala:78)
        at scala.util.Try$.apply(Try.scala:192)
        at akka.actor.ReflectiveDynamicAccess.createInstanceFor(DynamicAccess.scala:73)
        at akka.actor.ReflectiveDynamicAccess$$anonfun$createInstanceFor$3.apply(DynamicAccess.scala:84)
        at akka.actor.ReflectiveDynamicAccess$$anonfun$createInstanceFor$3.apply(DynamicAccess.scala:84)
        at scala.util.Success.flatMap(Try.scala:231)
        at akka.actor.ReflectiveDynamicAccess.createInstanceFor(DynamicAccess.scala:84)
        at akka.actor.ActorSystemImpl.liftedTree1$1(ActorSystem.scala:626)
        at akka.actor.ActorSystemImpl.<init>(ActorSystem.scala:619)
        at akka.actor.ActorSystem$.apply(ActorSystem.scala:143)
        at akka.actor.ActorSystem$.apply(ActorSystem.scala:127)
        at akka.osgi.OsgiActorSystemFactory.createActorSystem(OsgiActorSystemFactory.scala:33)
        at akka.osgi.ActorSystemActivator.start(ActorSystemActivator.scala:42)
        at org.apache.felix.framework.util.SecureAction$Actions.run(SecureAction.java:1709)
        at java.security.AccessController.doPrivileged(Native Method)
        at org.apache.felix.framework.util.SecureAction.startActivator(SecureAction.java:688)
        at org.apache.felix.framework.Felix.activateBundle(Felix.java:2220)
        at org.apache.felix.framework.Felix.startBundle(Felix.java:2138)
        at org.apache.felix.framework.Felix.setActiveStartLevel(Felix.java:1365)
        at org.apache.felix.framework.FrameworkStartLevelImpl.run(FrameworkStartLevelImpl.java:308)
        at java.lang.Thread.run(Thread.java:745)
Terminate batch job (Y/N)?

I believe it should work, if I want it to work I have to copy the reference.conf file out of this Akka Remoting jar and put it in resources folder, that is a workaround for the problem, I should not have to do a workaround like this for Akka Remoting in OSGi to work.

[rkuhn]

Are you using the BundleDelegatingClassLoader to start the ActorSystem?

[briantopping]

Hi @PhilAndrew, apologies for the delay in getting an answer to you here. I looked through your code and couldn't find any use of BundleDelegatingClassLoader. Typesafe Config simply does a getResources on the provided (or current) ClassLoader, and resources aren't available between bundles with the standard OSGi ClassLoader implementation. That is the reason the reference.conf is not getting picked up unless you copy it. This is just the first of what will be a lot of configurations like this.

BundleDelegatingClassLoader is really a bit of a hack to get around Typesafe Config not dealing with bundle resource fencing by OSGi. It's really inefficient, but since it's only used for configuration resolution, it's not a big deal.

I'm in the process of better documenting ActorSystemActivator, so if you'd like to get involved with that in #19323, I think we could do a great job in making sure the docs answer all your questions, and in turn, those that follow you. You may have moved on from OSGi by now as well. If so, kindly close out this issue.

Thanks! Brian

[PhilAndrew]

How do I use BundleDelegatingClassLoader? I don't see any documentation on how to use it or any sample code. :(

[briantopping]

@PhilAndrew It's easier to start with OsgiActorSystemFactory to start with, then build up from there. You'll see in that process how to decompose OASF into your own code directly.

[PhilAndrew]

Thanks, I use OsgiActorSystemFactory in my code here: https://github.com/PhilAndrew/sbt-osgi-felix-akka-blueprint-camel/blob/master/src/main/scala/osgidemo/impl/AkkaBean.scala

The code I have written works, but the main problem is the reference.conf which I manually merged and created from jar files.

You can see other people have tried to merge reference.conf http://letitcrash.com/post/21025950392/howto-sbt-assembly-vs-referenceconf

As I understand, reference.conf has multiple files in multiple jars, now when I'm in Akka in OSGi and I want to run my application, it wants to find reference.conf. So my current solution is just to get all the Akka jars, extract all reference.conf files and merge each one of them manually.

I tried using BundleDelegatingClassLoader for this problem, it didn't seem to work for me, perhaps I don't understand its purpose. Is there some simple way I can change my code to run without the reference.conf file in resources directory?

[briantopping]

Did you see how BundleDelegatingClassLoader is used by OsgiActorSystemFactory? You essentially want to inline the OASF code into your own code, gaining direct access to BDCL and enhancing from there.

What you are describing is what I have also done. If you want to have configuration packaged with the OSGi bundle that it serves, there's really no other good way to handle this. Typesafe Config is immutable and the only way to set the config for an ActorSystem is during instantiation, so there's no way to change a configuration for a running AS. That may not matter in your situation, but it did in mine, so my work ended up creating a new AS for each bundle.

[PhilAndrew]

This is a bit of a unfriendly barrier to using Akka within OSGi.

[briantopping]

@PhilAndrew OSGi does not have enough traction to keep the greater team focused on it's benefits. There are no incentives for the team to learn OSGi and solve these kinds of issues because they don't see the value in OSGi for their own projects. We can't judge others on their lack of need.

I'd be happy to work with you on finding a solution to the problem, but I ran out of time to keep pushing on this myself. Would you like to contribute on a branch and work together on it?

[PhilAndrew]

Thanks for the offer of help, for me its just mostly a barrier to a destination. I just want to use Akka within OSGi in a nice way and then move on.

Then added some code as you suggested PhilAndrew/sbt-osgi-felix-akka-blueprint-camel@4920e15

Are you then suggesting I do something in OSGi to get a bundle, then get a resource from the bundle and then merge the 3 resource.conf in memory? eg val resources = bundle.getResources(name)

I see I get a org.apache.felix.framework.util.CompoundEnumeration back.

[briantopping]

Yes, it would be great if things just worked. I put a fair bit of time into getting the manifests and build cleaned up. There's only so much time in the day.

I'll check on your other questions shortly.

[PhilAndrew]

Your correct! with my last commit it no longer needs reference.conf.

Also I don't need to merge the files, I get all the bundles from the bundle context and get all the urls and return them. So refer to latest version of my github https://github.com/PhilAndrew/sbt-osgi-felix-akka-blueprint-camel

Then, really this type of code should be in Akka OSGi?

The code is just short and in the util folder. https://github.com/PhilAndrew/sbt-osgi-felix-akka-blueprint-camel/tree/master/src/main/scala/osgidemo/impl/util

[briantopping]

I haven't had a chance to look at your code, but I'd assume if you're having better luck, that's a good sign.

If you'd like me to take a look at a PR for Akka, I'm happy to review.

[PhilAndrew]

I find that changing only one method on this class BundleDelegatingClassLoader.scala, change findTransitiveBundles to the following, makes it work.

  private def findTransitiveBundles(bundle: Bundle): Set[Bundle] = {
    bundle.getBundleContext.getBundles.toList.toSet
  }

Since I don't understand what this method is trying to achieve, I'll try to work out what this method is trying to do.

I read from another ticket "findTransitiveBundles method returns a list of bundles to use when loading classes specified in the configuration" - #13977

I also worry that with bundles coming and going, shouldn't it try all bundles loaded when the resource is requested?

[PhilAndrew]

Sleuthing. I tried to track down how this method came to exist. In https://github.com/costamojan/xbean/commit/2216fe66f0dfca6bd217ed450baaa523a682b4eb there is private Map<String, Bundle> buildPackageBundleMap() { which is very similar to findTransitiveBundles. Now here https://github.com/costamojan/xbean/blob/master/xbean-bundleutils/src/main/java/org/apache/xbean/osgi/bundle/util/DelegatingBundle.java.
I believe this was introduced by this author https://github.com/jgawor.

[patriknw]

Obsolete, see #28304