Application: classify core developers irc output

[karlnapf]

This should be a fun entrance task is useful for any kind of applied project with Shogun. The task is simple: Train a model that can identify core developers of our project, based on their IRC behaviour (from the logs)

Steps

• Download logs, structure them nicely
• Extract features and labels (sentences, and who wrote them)
• Pick a nice feature representation
• Optional: Implement a new feature converter for string features in Shogun
• Train a multiclass model
• Tune parameters
• Write a nice notebook

[ghost]

I'm repeatedly encountering internal server errors while opening most of the logs. Any other way I can get these logs?

[karlnapf]

We are fixing the link atm. Check back in a few days

[minxuancao]

I'm interested in this project. I'm new to open source and github. I want to confirm what 'log' means. Is it the commit history?

[karlnapf]

No it is the IRC chat logs, they are currently offline, but we might put them online again.
In order to build the system, you can use any other chat log on the web. Then, when you are done, and we have put our chat logs back online, you can just apply it to ours ....

[minxuancao]

Thanks!

[tingpan]

I am interested in this one! I will have a try!

[karlnapf]

Start with bag of words representations, see e.g. here: https://github.com/karlnapf/machine_learning_course/blob/master/classification.ipynb

[karlnapf]

Or train a NN ;)

[tboex]

Is it also possible for me to work on this one?

[karlnapf]

Everyone is free to try their ting.

[moizsajid]

Are we supposed to label the data ourselves?

[karlnapf]

Chat logs are labelled by construction

[moizsajid]

So basically our task boils down to classifying whether a given chat post came from a Shogun Core Developer or not?

[karlnapf]

Could do that. But the task is to classify who of the core devs

[moizsajid]

I have trained a Naive Bayes classifier on the 1,941 chat log files. However, I am getting a very low accuracy (less than 1%). I am searching for the possible bug in my code. https://github.com/moizsajid/shogun-core-developers

[lisitsyn]

This was supposed to be done using Shogun ;)

[moizsajid]

Found the major bug! I was not taking the log of the prior probabilities. After this update, I am getting an accuracy of 59.85%.

[moizsajid]

@lisitsyn I will also try to complete it using Shogun just to verify my results. Is there a Bag of Words representation available in Shogun?