New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[ZOOKEEPER-3389] Zookeeper does not export all required packages in OSGi (needed for curator) #945
base: master
Are you sure you want to change the base?
Conversation
retest this please |
Please update maven pom as well |
Hi @eolivelli , |
Let me check. @anmolnar this will be a -1 on the 3.5.5rc6 |
Is this @JiriOndrusek a "regression" ? |
Can you please try to add a fix even for the maven build ? |
@eolivelli Hi, For maven: I haven't found any occurrence of OSGi headers in main pom.xml. I can look into it later this week, is this time horizon acceptable? (I see in comment above "this will be a -1 on the 3.5.5rc6") |
Hi @JiriOndrusek , As this is not a regression, I believe rc6 will be rolled out, still short on binding +1 though. So at my current knowledge, target version for this is 3.5.6. But it needs a maven fix for sure. I'll also try to look into it, I'll let you know if I find anything. |
Hi @nkalmar , |
@JiriOndrusek I feel you can continue on this branch, we can merge all this OSGI stuff in a single commit. |
I agree with Enrico, please just continue on this branch, as maven will be used to release future 3.5.x and 3.6.x versions. |
316962a
to
cf525f6
Compare
@nkalmar @eolivelli
What do you think about it? |
There will be 2 jars upon release, zookeeper-jute and zookeeper-server. zookeeper-jute is the serialization library (just like protobuf). zookeeper-server depends on zookeeper-jute. The jute classes are generated, just like in protobuf. Unfortunately I don't know much about OSGi, so I don't know what the best solution would be here. But zookeeper-server needs zookeeper-jute to serialize the messages between nodes. |
I see you changed zookeeper-server packaging from jar to bundle. Yeah... I will check up on this sometime this week. I'm afraid this also affects other things. |
Yes changing the package name is feasible but only for 3.6+. |
cf525f6
to
60ef802
Compare
@nkalmar @eolivelli
As there is a mentioning, that changing packaging from 'jar' to 'bundle' will cause some errors elsewhere. Unfortunately I'm not aware of other way of running OSGi build, but I'll try to look into it also tomorrow. |
60ef802
to
bc59148
Compare
Maybe you can simply add raw entries to the manifest in the jar plugin configuration |
Hi @nkalmar @eolivelli, let me introduce "a better solution" for the problem with same package's names of zookeeper-jute and zookeeper-server. If I understand it correctly, maven way of solving this problem is acceptable even if it causes problems sometimes. From previous comment - "It's not ideal, and even causes problems like this one with OSGi." Maven solution is that in case that packages DOESN'T contain the same classes - it works as EXPECTED. If it contains the same classes, then classloader return class from jar based on ORDER of jars. (so even if I want clas from jute, I can get class from server) Which is typicalli zookeeper-server and then zookeeper-jute (as jute is dependency from server) I propose to use OSGi merge capability, which brings the same behavior. If someone wants class from that shared package, in case that classes are differen in both jars, this merge "feature" will behave as expected and correct class is returned. In case that there are overlapping classes, we define, which jar has advantage. (so I'd say that zookeeper-server has advantage). IMHO behavior will be a little bit better then in maven, because it will behave always in the same way. (In reality it won't robabnly happen, because the same classes in the shared packages will probably cause problems elsewhere, so everyone will try to avoid that) In other words - behavior of OSGI with merge will be the same as maven if packages contains different classes. If in the future there will be the overlapping classes, behavior in OSGi would be defined (in contrast with maven, where it would ddepend on order of jar loading - so almost "random") Possibly (I'm not sure about it in this time) I can also try to add some insurance/condition, that it will work for 3.5.x only. I mean that after upgrading to higher version of zookeeper, it will cause error during build (or at least warning), so someone will look into it and will find a comment explaining, that correct solution is to rename packages -> which is the correct solution. In case that packages won't be renamed at that time. This condition would allow us to solve problem in 3.5.x time, where packages couldn't be renamed. Just to avoid confusion, this merge capability only defines, how OSGi returns classes if the class is in package, which is shared in more jars. No actuall merging of compiled classes is not happening. |
It would be great to have some integration test which loads zookeeper client with osgi and this way we will ensure that we are delivering something that really works. We have to setup these kind of integration tests (maybe docker based) in the short term, not necessarily for thia case. Maybe you can continue with the osgi packaging stuff and I will then try to setup the integration test in a followup patch |
@eolivelli Integration test validating that OSGi is working, would be really nice. |
bc59148
to
affb736
Compare
What I meant in "private package" is that we do not expose any of this to clients. Jute is only used to serialize messages between ZooKeeper nodes, client does not use Jute for serialization. Anyway, thanks again for looking into this, the merge stuff sounds good! |
@nkalmar I've added OSGi support to zookeeper-jute. I had to change pockaging to bundle as well, but in maven environment it behaves still as jar packaging. I think that current state is, that both jars (z-server and z-jute) are also osgi bundles. Now I plan to fine tuning dependencies during manual installation. |
…SGi (needed for curator)
affb736
to
c67506e
Compare
I was able to install manually zookeeper with curator. But I had to change a little bit export-packages in zookeeper-server, which throws now warning during build about package "org.apache.zookeeper.data". Warning message looks like: Meaning of this warning is, that some classes from zookeeper (zookeeper.cli, ...) for example returns classes from package "org.apache.zookeeper.data" which is not exported from the bundle. |
@eolivelli @nkalmar
Solution:
But with the fact that in the 3.6+ version renaming is possible so we can use solution #1 from 3.6+ as it is a better solution, i would like to ask on your opinion about current version. |
We would still need to solve it for 3.5 branch. Can you write an email to the dev list @JiangJiafu with your findings? A short summary, and then you can link this PR. |
@nkalmar I've just asked for subscription to dev mailing list, then I'd be able to send an email there. |
I've sent email to zookeeper-dev mailing list: ZooKeeper and OSGi (maven) Hi, I 've created issue [1] with missing exported packages in osgi for zookeeper 3.4.10. Then I started to prepare maven OSGi packaging [2] for the higher version of ZooKeeper (in the PR for issue). I've tried to implement OSGi packaging with the low impact. So I've tried to create OSGi bundles from Zookeeper-server and from zookeeper-jute modules. But there is a problem for this solution:
-> bundles can not be deployed into osgi as two libraries are exporting the same packages Solution:
Solution #1 is a better solution, I would like to ask for your opinion about feasibility of renaming zookeeper-jute generated packages to not collide with zookeeper-server. If #1 is not acceptable, then we can go with #2. But I highly suggest to consider renaming of zookeeper-jute's packages in the nearest point in the future as possible and return to solution #1. Best regards, jiri [1] https://issues.apache.org/jira/browse/ZOOKEEPER-3389 |
@nkalmar |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
+1, tarballs unchanged, artifacts looks good.
Thanks @JiriOndrusek , I'll ping @eolivelli if he has anything to add, otherwise LGTM! I'll also check 3.5 fix... |
Excuse me for being late to the party. Let me introduce myself - I'm OSGi dinosaur and I'd be happy to help here. So
I understand that some classes of For OSGi, you'd have to do what for example Hibernate or httpclient (before silently dropping OSGi support in httpclient 5) is doing - there should be zookeeper-osgi.jar which just embeds both zookeeper and zookeeper-jute. This is best approach, because there'll be only one Maven artifact with I'll continue checking. |
Issue: https://issues.apache.org/jira/browse/ZOOKEEPER-3389
Added exported-packages which are required for Curator installation in OSGi.