[HIVEMALL-120] Refactor on LDA/pLSA's mini-batch & buffered iteration logic

[takuti]

What changes were proposed in this pull request?

Refactor LDA/pLSA implementation for better mini-batch & buffered iteration logic

What type of PR is it?

Refactoring

What is the Jira issue?

https://issues.apache.org/jira/browse/HIVEMALL-120

How was this patch tested?

Unit test & manual test on EMR

How to use this feature?

Nothing has been changed

[takuti]

At first, I planned to replace String[][] miniBatch with things like FeatureValue[][] miniBatch similarly to the generic predictor implementation; that is, instead of dealing word counts as String e.g. "foo:10," I tried to represent them as FeatureValue for readability. However, since PLSAPredictionUDAF and LDAPredictionUDAF need to create OI for such word counts, Java standard string object is much more handy rather than FeatureValue objects.

Thus, document is still represented as an array (or list) of String for now.

[takuti]

@myui How do you think about adding -tol option as alias of the -eps option as I mentioned #87
?

Also, if you have further ideas which improve the topic modeling UDFs, let you share and we discuss them here!

[coveralls]

Coverage increased (+0.2%) to 40.236% when pulling bc4b9d3 on takuti:topicmodel-refactor into c06378a on apache:master.

[coveralls]

Coverage decreased (-0.06%) to 39.945% when pulling bc4b9d3 on takuti:topicmodel-refactor into c06378a on apache:master.

[myui]

@takuti Adding an alias -tol for -eps is a good idea.

[myui]

No need to replace String[][] miniBatch. It already uses FeatureVector for parsing feature vectors as thus it accept the input format of Hivemall's feature vector.

[takuti]

Adding an alias -tol for -eps is a good idea.

Sure.

it accept the input format of Hivemall's feature vector

Oh, it's not format matter by the way; String or FeatureValue in terms of type.

Current implementation repeatedly parses feature-value-formatted String at HERE for every time mini-batch is passed. If we directly pass a two-dimensional array of FeatureValue as word counts, we can avoid the unnecessary parse operations.

[myui]

ah, parsing multiple times could be avoided.

[myui]

Merged this PR. Create [HIVEMALL-120-2] for -tol alias for -eps.

[takuti]

@myui Oh, you move very fast :)

-tol alias

Sure~