All steps to Aerospike interaction does not work properly

[alexander-myltsev]

Hi,

As Stratio Manager does not support Aerospike yet. Vagrant sandbox doesn't contain Aerospike bundles either. I had to build deep-spark from sources.

bin/stratio-deep-shell does not see anything in com.stratio.*. So, I had to manually launch Spark via bin/spark-shell --jars $(echo lib/*.jar | tr ' ' ',').

Then First Steps with Stratio Deep and Aerospike does not work as there are no MessageTestEntity and WordCount in compiled jars. And I haven't managed to implement those is plain Scala to paste it in REPL. (Something about NoSuchMethod exception and <init>).

Then I composed custom SBT project, and put all jars in lib dir to unmanaged dependencies. Now it complains:

[error] (run-main-f) java.lang.NoSuchMethodError: scala.collection.immutable.HashSet$.empty()Lscala/collection/immutable/HashSet;
java.lang.NoSuchMethodError: scala.collection.immutable.HashSet$.empty()Lscala/collection/immutable/HashSet;
        at akka.actor.ActorCell$.<init>(ActorCell.scala:336)
        at akka.actor.ActorCell$.<clinit>(ActorCell.scala)
        at akka.actor.RootActorPath.$div(ActorPath.scala:159)
        at akka.actor.LocalActorRefProvider.<init>(ActorRefProvider.scala:464)
        at akka.remote.RemoteActorRefProvider.<init>(RemoteActorRefProvider.scala:124)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  // ...

Where is the correct path in whole documentation and examples to launch deep-spark with Aerospike?

[mafernandez-stratio]

Hi Alexander,

Currently, the development of Stratio Deep is deprecated. However, Stratio Crossdata has inherited part of the features of Stratio Deep. Regarding to the use of Spark and Aerospike together, we recommend to combine:

• Stratio Crossdata
• Aerospark

I Hope it helps

[alexander-myltsev]

Thanks for fast and honest response. I really suggest you to put this information at the top of README.

Do you think both components are production ready?

I suggest combining those two components is not only about configuration files manipulation -- coding is required as well?

[mafernandez-stratio]

Hi Alexander, thanks for the suggestion.

Regarding the maturity of these two components, we have performed some operations with Aerospark, but they were functional tests so I can't tell how stable is. In the other hand, Crossdata is based on Apache Spark and the stress tests performed have been successful.

Moreover, in order to use Aerospark with Crossdata, only some SQL-like commands are required to set up the environment. Take a look at the Aerospark configuration and examples. The only one change required to use Crossdata is replace the SQLContext by the XDContext.

[alexander-myltsev]

OK, thank you. We would consider that option.