Skip to content

gosubpl/akka-online

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

33 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

akka-online

Online and streaming algorithms with Akka

For good overview of streaming algos go to this debasishg's gist.

In org.apache.spark.streamdm there is code copied from Huawei Noah's Ark Lab streamDM project (the licencse is Apache 2.0 too). The code has been adapted to work with Akka streams instead of Spark streaming module by removing dependencies on the Spark stuff. Tested path is the HoeffdingTree model usage, which works best with input sourced from Arff files via SpecificationParser / ExampleParser - see see other parts of akka-online for usage examples. If you'd like to know more about HoeffdingTree on-line classifier, please read the HDT docs or go to Massive Online Analysis website that contains more information on the context.

Sample Arff files can be found in the MOA dataset repository.

For theoretical exposition to HoeffdingTree usage go to the original paper.

If you are asking yourself question why Akka, not Spark? - please read the classic bigger data, same laptop to see what we can gain going lightweight. In previous century, 4.5k records machine learning data set could be considered sizeable and hence all the growth of clustering for data processing. But with today's hardware I believe we can do better.

In the /lib directory of the project I have put suffixtree-1.0.0-SNAPSHOT.jar which is a compiled artifact of the Ukkonen's on-line Suffix Tree implementation. The licensse for this project is also Apache 2.0.

In this project I also use Guava implementation of Bloom Filter (see project dependencies) but there are other options.

About

Online and streaming algorithms with Akka

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages