How do I know the progress of Training?

[tvvignesh]

First of all, great work with this library. While it works for small data sets. I wanted to do the same for large datasets. It looks like it is stuck in trainbatch and I am not sure about the progress of the training completed so far.

How do I know how many records have been trained so far or get some feedback that its processing?

[erelsgl]

Hi Vignesh, a progress bar is currently not implemented, but it could be indeed a useful addition. *--- Erel Segal-Halevi (http://erelsgl.github.io <http://erelsgl.github.io>) אראל סגל הלוי*

…

On Mon, Aug 20, 2018 at 2:10 PM, T.V.VIGNESH ***@***.***> wrote: First of all, great work with this library. While it works for small data sets. I wanted to do the same for large datasets. It looks like it is stuck in trainbatch and I am not sure about the progress of the training completed so far. How do I know how many records have been trained so far or get some feedback that its processing? — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#60>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABZPx83h6m6UmtT1gFKRf2Pc9tdHVHovks5uSpkggaJpZM4WDzGa> .

[tvvignesh]

Hmm. I managed for now by passing debug as true to the options like so:

var TextClassifier = limdu.classifiers.multilabel.BinaryRelevance.bind(0, {
        binaryClassifierType: limdu.classifiers.SvmJs.bind(0, { C: 1.0 }),
        debug: true
});

And doing that I noticed that SvmJs gets stuck with the first input itself if the size of the input of dataset is more than 250 rows or so and does not train further (waited for around 15-20 minutes).

Hence, I had to go far WordExtractor + Winnow classifier like so:

var TextClassifier = limdu.classifiers.multilabel.BinaryRelevance.bind(0, {
        binaryClassifierType: limdu.classifiers.Winnow.bind(0, { retrain_count: 10 }),
        debug: true
});

Thanks anyways for your reply.

[erelsgl]

Interesting. So is this a bug in SvmJS? *--- Erel Segal-Halevi (http://erelsgl.github.io <http://erelsgl.github.io>) אראל סגל הלוי*

…

On Mon, Aug 20, 2018 at 5:16 PM, T.V.VIGNESH ***@***.***> wrote: Hmm. I managed for now by passing debug as true to the options like so: var TextClassifier = limdu.classifiers.multilabel.BinaryRelevance.bind(0, { binaryClassifierType: limdu.classifiers.SvmJs.bind(0, { C: 1.0 }), debug: true }); And doing that I noticed that SvmJs gets stuck with the first input itself if the size of the input of dataset is more than 250 rows or so and does not train further (waited for around 15-20 minutes). Hence, I had to go far WordExtractor + Winnow classifier like so: var TextClassifier = limdu.classifiers.multilabel.BinaryRelevance.bind(0, { binaryClassifierType: limdu.classifiers.Winnow.bind(0, { retrain_count: 10 }), debug: true }); Thanks anyways for your reply. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#60 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABZPx__B-2l3JROoAqIwb6GAEMKmLPgXks5uSsSggaJpZM4WDzGa> .

[tvvignesh]

Not sure if its a bug (since it did not throw any error or so) but there is an issue which has been raised and a pull request merged here: karpathy/svmjs#5

[tvvignesh]

@erelsgl Tried with SvmPerf and it was faster and worked great for my use case. Thanks.

Sorry for bothering you, but had few more questions to ask:

The SvmPerf application is generating a lot of model and bin files in my directory. What should I do with that? Is it possible to reuse the same without retraining?

Also, was curious how this will scale in a production environment. Any suggestions?

Btw, Great work with the library. Works like a breeze. (Needs some minor cleanups though - will see if I can help when I get the time. For instance, it was not able to find localstorage file in bayesian classifier)

[erelsgl]

Hi Vignesh, As far as I remember, the files created by SvmPerf are just input files sent to the svm_perf binary, and output files created by the svm_perf binary. So, if your training data and parameters are not changed, you can re-use them. Regarding production environment, I think the only reliable answer can be found by experimenting.. *--- Erel Segal-Halevi (http://erelsgl.github.io <http://erelsgl.github.io>) אראל סגל הלוי*

…

On Sun, Aug 26, 2018 at 10:21 PM, T.V.VIGNESH ***@***.***> wrote: @erelsgl <https://github.com/erelsgl> Tried with SvmPerf and it was faster and worked great for my use case. Thanks. Sorry for bothering you, but had few more questions to ask: The SvmPerf application is generating a lot of model and bin files in my directory. What should I do with that? Is it possible to reuse the same without retraining? Also, was curious how this will scale in a production environment. Any suggestions? Btw, Great work with the library. Works like a breeze. (Needs some minor cleanups though - will see if I can help when I get the time. For instance, it was not able to find localstorage file in bayesian classifier) — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#60 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABZPx0yH0O_MnMJPucIc9oBOgpds6_lsks5uUvUigaJpZM4WDzGa> .