I would appreciate the time you take to review these changes. I have tested these changes and they work on Flume fresh off of the git repo, 0.9.5-SNAPSHOT I believe. The only thing I really had to change was some byte arrays to ByteBuffers in some of the code and also the thrift generated bindings generated from cassandra.thrift from cassandra-0.8.2 with thrift-0.6.1 installed. It seems to be working just as the old plugin did on Flume-0.9.1. So far it's working great for me with the latest version of Cassandra and the latest version of Flume. It might be good to be more clear in the Docs about what version of thrift should be used in order to build. The ANT build xml points to a libthrit.jar which doesn't indicate a version of thrift, I had to download like 5 thrift versions until I found the right one, it happened to be thrift-0.2.0.jar. It might be better to include a libthrift-0.6.1.jar or something.
Thrift bindings for Thrift 0.6.0
New for thrift 0.6.0
new for thrift 0.6.0
Updated for thrift 0.6.x: changed byte's to ByteBuffers.
Updated thrift bindings for Thrift-0.6.1: Compatible with
Flume-0.9.3+, including Flume-0.9.5-SNAPSHOT.
Awesome! Thanks for tackling this. I'll review your changes as soon as I can get to them, which should be this week.
Yeah no problem. I'm new to flume/cassandra and git for that matter.. so just let me know if I didn't do something quite right.
Would you mind sharing what you needed to change in Flume and where you put the plugin jar(s) in order to get the plugin to work?
The Flume project layout and build process has changed considerably since the last time I looked at it.
Alright, so to my run configuration classpath for the master and the node in eclipse I added /flume-cassandra-plugin/build/classes (OR cassandra_plugin.jar compiled from the ant build) and jug-asl.2.0.0.jar.
Also in run configurations, an environment variable FLUME_HOME that points to flume/flume-core. And inside of the flume core directory I linked the conf directory because the flume_home var tells flume where to look for the config. You could set it to just the top flume dir too probably.
After that I just added the plugin to load in the flume-site.xml based on your instructions and then configured the node from the flume-shell within eclipse.
Also when you're building, make sure thrift-0.6.x.jar is in the build path (with jug-asl-2.0.0.jar and slf4j-api-1.5.11.jar) since the generated bindings are from thrift 0.6.1. Also since my flume-cassandra-plugin directory wasn't in flume's directory, I had to change the flume.base in the build.xml..
Finally figured it out, and it looks like the simpleCassandraSink works well. (The logsandraSink is out of date, but that's my own doing.)
Thanks for the pull request and for the tips! Very much appreciated.
Yeah no problem, anything I can do to save someone else some grief. The new docs and the download are great, thanks for doing that!