Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP
Scalable machine learning library for Hive/Hadoop
Java Other

Merge pull request #178 from myui/feature/fix_feature_weight

Modifed extract_weight to accept a categorical feature representation
latest commit 90c70a0b06
@myui authored

README.md

Hivemall: Hive scalable machine learning library

Build Status Coversity License

Hivemall is a scalable machine learning library that runs on Apache Hive. Hivemall is designed to be scalable to the number of training instances as well as the number of training features.

logo

Supported Algorithms

Hivemall provides machine learning functionality as well as feature engineering functions through UDFs/UDAFs/UDTFs of Hive.

Classfication

  • Perceptron

  • Passive Aggressive (PA, PA1, PA2)

  • Confidence Weighted (CW)

  • Adaptive Regularization of Weight Vectors (AROW)

  • Soft Confidence Weighted (SCW1, SCW2)

  • AdaGradRDA (with hinge loss)

My recommendation is AROW, SCW1 and AdaGradRDA, while it depends.

Regression

  • Logistic Regression using Stochastic Gradient Descent

  • AdaGrad / AdaDelta (with logistic loss)

  • Passive Aggressive Regression (PA1, PA2)

  • AROW regression

My recommendation is AdaDelta and AdaGrad, while it depends.

Recommendation

  • Matrix Factorization (sgd, adagrad)

  • Minhash (LSH with jaccard index)

k-Nearest Neighbor

  • Minhash (LSH with jaccard index)

  • b-Bit minhash

  • Brute-force search using cosine similarity

Feature engineering

  • Feature hashing (MurmurHash, SHA1)

  • Feature scaling (Min-Max Normalization, Z-Score)

  • Feature instances amplifier that reduces iterations on training

  • TF-IDF vectorizer

  • Bias clause

  • Data generator for one-vs-the-rest classifiers

System requirements

  • Hive 0.11 or later

  • Hive 0.9, 0.10 [out of support]

Basic Usage

Hivemall

Find more examples on our wiki page and find a brief introduction to Hivemall in this slide.

Copyright

Copyright (C) 2015 Makoto YUI

Copyright (C) 2013-2015 National Institute of Advanced Industrial Science and Technology (AIST)

Put the above copyrights for the services/softwares that use Hivemall.

Support

Support is through the issue list, not by a direct e-mail. Put a question label to ask a question.

References

Please refer the following paper for research uses:

Awards

Acknowledgment

This work was supported in part by a JSPS grant-in-aid for young scientists (B) #24700111 and a JSPS grant-in-aid for scientific research (A) #24240015.

Analytics

Something went wrong with that request. Please try again.