myui / hivemall Archived

This is the second release candidate for Hivemall v0.4.2.

CAUTION: Please use add_bias instead of addBias for adding bias clauses from this release. If you want to use addBias, please issue the deprecated DDLs manually.

We DO NOT recommend to use this version in production yet.

Changes since v0.4.2-rc.1 are summarized as follows:

• Bug Fixes
  • Fixed a corner case bug in mf_predict [4282b6a] #310
  • Fix a bug in FMArrayModel initialization scheme [564aefe][05c8da0] #302
• Major Changes
  • Fixed feature_hashing UDF behavior for a bias feature '0' [a8fd640]
  • Cleaned up deprecated DDLs including addBias [2264ae6]
  • Changed (Fixed) the behavior of binarize_label [ab5c19e]
  • Reverted to support a function alias addBias for Treasure Data env [0e99357]
• Minor Changes
  • Updated feature OI validation scheme to use HiveUtils [612a5df]
  • Made feature UDF as GenericUDF and handle null weight [77922dc] #306 #269
  • Fixed tree_predict UDF of RandomForest to accept any Integer types [c7c5213]

This is the first release candidate for Hivemall v0.4.2. We skipped v0.4.1 since there are tremendous changes in this release.

We would like to thank all contributors of this release.

Caution: FFM and BPRMF implementations are still experimental and subject to change (so not documented). We DO NOT recommend to use this version in production yet.

Changes since v0.4.1-alpha.6 are summarized as follows:

• Major Enhancement
  • Initial support for Hivemall on Spark #274
  • Implemented Field-aware Factorization Machines #284 #286 #288 #290 #292 #293 #295 #297
  • Implemented BPR-MF (Matrix Factorization for Implicit Feedbacks) #278 [d427bd8]
    • Steffen Rendle, Christoph Freudenthaler, Zeno Gantner, Lars Schmidt-Thieme. "BPR: Bayesian Personalized Ranking from Implicit Feedback", Proc. UAI, 2009.
• Minor Enhancement
  • Added bits_collect UDAF [c78b75b][117474f]
  • Added to_bits and unbits UDF [4da12d5]
  • Added bits_or UDF [42f3db2]
    • Bitset functions usage
  • Added ndcg UDAF [4340194]
  • Added array_intersect UDF [cb7f0f7]
  • Added logloss Logarithmic Loss UDAF to evaluation functions #289
  • Added feature_hashing UDF #300
• Major Change
  • Changed the default learning rate 'eta0' of Factorization Machines [24fc383]
  • Changed the default max_init_value option value of FM/FFM [208ed62]
  • Changed the default V initialization scheme for FM/FFM classification from random to guassian [8ec5199]
• Minor Change
  • Renamed concat_array UDF to array_concat [f9561d6]
  • Fixed each_top_k to accept non-constant k for the first argument [2f18011]
  • Fixed to_order_map to support reverseOrder option [ac005fd]
  • Fixed extract_weight UDF to support FFM format [f954bad]
• Bug Fixes
  • Fixed a bug in model_id generation scheme of RandomForest #299

This is the 6th alpha release for v0.4.1. We DO NOT recommend to use this version in production yet.

Changes since v0.4.1-alpha.5 are summarized as follows:

• Minor Enhancement
  • Implemented r2 UDAF #268
• Minor Change
  • Refactored DecisionTree and RegressionTree #270
• Bug Fixes
  • Fixed a corner case bug in rf_ensemble [64f64b0]

This is the fifth alpha release for v0.4.1. We DO NOT recommend to use this version in production yet.
Note: v0.4.1-alpha.4 has been skipped.

Changes since v0.4.1-alpha.3 are summarized as follows:

• Minor Enhancement
  • Implemented binarize_label UDTF #266
• Major Change
  • Reduced memory usages of DecisionTree and RegressionTree #259 #264 #270
• Bug Fixes
  • Fixed to report progress in long running UDTFs to avoid mapreduce.task.timeout [6eacd65]

This is the third alpha release for v0.4.1. We DO NOT recommend to use this version in production yet.

Major enhancement of this release is the support for mini batch gradient descent.
Find the usage in this page.

Note that the usage/behavior of RandomForest has been changed in this release.
You can find changes in this page.

Changes since v0.4.1-alpha.2 are summarized as follows:

• Major Enhancement
  • Supported mini batch gradient descent for logistic regression #252
• Minor Enhancement
  • Supported min_samples_leaf option in RandomForest #253
  • Added inflate/deflate UDF [4f2747c]
  • Added base91/unbase91 UDFs [8c70d8d]
  • Add scritps to start/stop MIX servers #241
  • Supported classification_error for the SplitRule of RandomForest classification [19ae202]
• Major Change
  • Supported various mode types for the output prediction model of RandomForest. Due to that, the behavior of RandomForest has been changed. #249
    • Refined tree_predict UDF [57bae25]
    • Changed the default output model type [10f0ea2]
• Minor Changes
  • Fixed the default numbers of used threads for smile for non-TD env [5c8bea3]
  • Changed/enlarged buffer size for iterative training of Factorization Machine [dcf0dbc]
  • Changed formula for calculating the number of random selected features [13c3917]
• Bug Fixes
  • Fixed mf_predict logic for unkown examples [a43f793]
  • Fixed a bug that maxDepth is not set in DecisionTree [df21aba]

This is the 2nd alpha release for v0.4.1. We DO NOT recommend to use this version in production yet.

Changes since v0.4.1-alpha.1 are summarized as follows:

• Bug Fixes
  • Fix bugs in MixServer PartialResult#diffClock and add tests [0c7672d]
  • Changed the implementation of fm_predict to GenericUDAF and fixed a bug [2906b38]
  • Applied a workaround for KryoException/java.util.ConcurrentModificationException in tokenize_ja
    [06b3762]

This is a alpha release for v0.4.1. We DO NOT recommend to use this version in production yet.

For the usage of tokenize_ja, please refer this wiki page.

Changes since v0.4.0-2 are summarized as follows:

• Major Enhancement
  • Supported Japanese tokenizer UDF tokenize_ja [https://github.com//pull/227]
• Major Changes
  • Separated maven module into core and mixserv [https://github.com//pull/225]
  • Changed the default max_depth of gradient tree boosting classifier [c4ade19]
• Bug Fixes
  • Fixed fm_predict not to use Custom class for the result of terminatePartial [https://github.com//pull/230]