Can't serialize class org.apache.hadoop.io.DoubleWritable

[danielsan]

The branch SaveDataBackToMongoDb was created to explore and try to solve a bug when it comes to save data back to mongodb using Storm + Hadoop MongoDB Connector.

By checking the commit 5b7743a created on this branch you can see that the only change done was the removal of the '#' character which comments the line number 74.

resultRDD.saveAsNewAPIHadoopFile("file:///placeholder", outputFormatClassName, None, None, None, None, config)

The line above tries to save the RDD back to mongodb but it generates an error in the JAVA level

Can't serialize class org.apache.hadoop.io.DoubleWritable

If you follow the instruction since the Getting the code section you will be able to run the python code on Spark and see the errors by yourself.

I ran the script ./spark-ohlcbars-example.submit.sh saving the full outpu in a log file called spark-ohlcbars-example.submit.log

./spark-ohlcbars-example.submit.sh &> spark-ohlcbars-example.submit.log

You can check the full output on spark-ohlcbars-example.submit.log