Skip to content
Scalable machine learning library for Hive/Hadoop
Java Other

README.md

Hivemall: Hive scalable machine learning library

Build Status Coversity Documentation Status License

Hivemall is a scalable machine learning library that runs on Apache Hive. Hivemall is designed to be scalable to the number of training instances as well as the number of training features.

logo

Supported Algorithms

Hivemall provides machine learning functionality as well as feature engineering functions through UDFs/UDAFs/UDTFs of Hive.

Binary Classfication

  • Perceptron

  • Passive Aggressive (PA, PA1, PA2)

  • Confidence Weighted (CW)

  • Adaptive Regularization of Weight Vectors (AROW)

  • Soft Confidence Weighted (SCW1, SCW2)

  • AdaGradRDA (w/ hinge loss)

  • Factorization Machine (w/ logistic loss)

My recommendation is AROW, SCW1, AdaGradRDA, and Factorization Machine while it depends.

Multi-class Classfication

  • Perceptron

  • Passive Aggressive (PA, PA1, PA2)

  • Confidence Weighted (CW)

  • Adaptive Regularization of Weight Vectors (AROW)

  • Soft Confidence Weighted (SCW1, SCW2)

  • Random Forest Classifier

  • Gradient Tree Boosting (Experimental)

My recommendation is AROW and SCW while it depends.

Regression

My recommendation for is AROW regression, AdaDelta, and Factorization Machine while it depends.

Recommendation

k-Nearest Neighbor

  • Minhash (LSH with jaccard index)

  • b-Bit minhash

  • Brute-force search using Cosine similarity

Anomaly Detection

Natural Language Processing

  • English/Japanese Text Tokenzier

Feature engineering

System requirements

  • Hive 0.11 or later

  • Java 7 or later

    Note: It would work for Java 6 except hivemall-nlp but we recommend to use Java 7 or later.

Basic Usage

Hivemall

Find more examples on our wiki page and find a brief introduction to Hivemall in this slide.

Copyright

Copyright (C) 2015 Makoto YUI
Copyright (C) 2013-2015 National Institute of Advanced Industrial Science and Technology (AIST)  

Put the above copyrights for the services/softwares that use Hivemall.

Support

Support is through the issue list, not by a direct e-mail.

References

Please refer the following paper for research uses:

Awards

Acknowledgment

This work was supported in part by a JSPS grant-in-aid for young scientists (B) #24700111 and a JSPS grant-in-aid for scientific research (A) #24240015.

Analytics

Something went wrong with that request. Please try again.