New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
STORM-1202: Migrate APIs to org.apache.storm, but try to provide some form of backwards compatability #889
Conversation
Because this is a non-backwards compatible change, and the mailing lists have discussed moving the version numbers over to 1.0.0-SNAPSHOT I am happy to make that change here too, but I wanted to hold off and get feedback from others before doing so. |
I am a +1. I like this backward compatibility feature for smooth migration to package renaming. The changes look good to me. I ran some tests with topology jar compiled on 0.10.x and it works as expected. |
The test failures appear to be unrelated. |
<artifactId>asm-commons</artifactId> | ||
<version>${asmVersion}</version> | ||
</dependency> | ||
</dependencies> |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit: check indentation
Looks OK to me overall. It would be nice to have a follow-on Jira Issue or Sub-Task to remove/revert the changes we need to bootstrap the testing for this (dev-tools, etc.).
We found that with word_count, the .clj source file had been removed from the transformed jar, but we expected that this .clj would be present, but transformed. |
OK I finally tracked down some intermittent failures I was seeing when running the clojure word count example. It looks like there is an issue in clojure where how we are doing the require does not seem to be thread safe. We should be good to go now if @d2r you want to take a final look. The previous failure is not related and is one of the random storm-kafka failures. |
I just filed http://dev.clojure.org/jira/browse/CLJ-1876 for the issue I was seeing with the require. |
The one suggestion is to add to Config what the hack implementation class is (the intended value of the config) for the use case. Otherwise users might need to hunt around to find the class we provide. Once that's added, +1 |
@d2r done I updated the comment in Config. |
This is not a typical pull request, because changing the packages is a huge task, and keeping it upmerged while this is reviewed and other patches goes in is a huge task, and error prone.
Instead the actual changes to the package/namespace is done by running
./dev-tools/move_packages.sh ./
you can wipe all of the changes clean again (Including any modifications on your branch that you have not checked in) by running
./dev-tools/cleanup.sh ./
once the changes are approved/committed these two scripts should be removed.
The other code is intended to provide a single executable that can shade a user jar so instead of using
backtype.storm
andstorm.trident
it usesorg.apache.storm
andorg.apache.storm.trident
.These are off by default by can be enabled by adding
to storm.yaml. I would expect all of the code related to this to be removed when we go to the 2.0 release.
I have tested this by running a 0.10.0-SNAPSHOT storm starter topologies against a cluster running this patch (along with the move_package.sh executed).