-
Notifications
You must be signed in to change notification settings - Fork 128
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add classes after classifier creation #5
Comments
Hi Jake, Great question! Let me think for a second, as I haven't looked at the bayesian code for a while. Jake — On Tue, Nov 4, 2014 at 2:13 PM, JakeAustwick notifications@github.com
|
I believe all you would have to do is add a method to the Classifier struct that would look something like this (not tested):
This will add the new class and subsequent calculations will take it into account. Note this is not particularly thread-safe if you're adding new classes asynchronously because the number of classes is a parameter to the calculations. If you submit a pull request + unit test, I would be happy to merge the change. There's also some cleanup that can be done on the NewClassifier method once this method is added. Thanks, |
The above code works, I tested it on our codebase - thanks for that. Are you still open to a patch even though you state you know it wouldn't be thread safe? In our application I simply wrapped all of our interactions with the classifier with a mutex, but I'm guessing that would be undesirable to do inside the library itself? |
IMO it would be best to leave thread-safety to the user, as it adds performance overhead, etc. Just out of curiosity, what are you using it for? And how big is your data set? :) |
We're using it to identify categories for uncategorized products based on their keywords / title. As of right now, our dataset is < 10k training data items, and approx. different 250 labels. I'll look at contributing the above patch today/tomorrow along with tests, as I haven't looked at testing in Go yet. |
Cool! I'm curious about the performance on that much data, as the library does As for testing, check out this page for basic Golang testing: Thanks for your interest! Jake On Wed, Nov 5, 2014 at 1:10 PM, JakeAustwick notifications@github.com
|
That's not a terrible amount of data to load into memory in my opinion. I've loaded 2.7 million items into a struct for an in memory geocoding package and it only used about 500MB of RAM. I've also done something similar for gender detection with a much smaller list. The performance is fast in memory and unless you're storing millions of items, the memory allocation should be quite small. That said, 250 different classifications is quite a lot and I too would be curious about a benchmark in this case. |
Apologies in advance as my knowledge of Go is still somewhat limited, so this may be a naive question.
I want to expose the naive bayes classifying as a HTTP Web service, with both train and classify endpoints. I have no trouble with that, but I want the train endpoint to be able to accept new labels (labels that aren't currently in the classifier). Right now the labels are simply specified as consts and passed into the constructor. Can you think of the best way to add the ability to add labels at run-time?
The text was updated successfully, but these errors were encountered: