Join GitHub today
GitHub is home to over 36 million developers working together to host and review code, manage projects, and build software together.Sign up
Shows how to write a simple data contest entry for Kaggle, using scikit-learn for machine learning algorithms http://petewarden.typepad.com/
Fetching latest commit…
Cannot retrieve the latest commit at this time.
|Type||Name||Latest commit message||Commit time|
|Failed to load latest commit information.|
MLloWorld --------- "Pete Warden" <firstname.lastname@example.org> - http://petewarden.typepad.com This is a minimal example of how to write a machine-learning algorithm to solve a prediction problem. It shows you how to create a simple prediction model from a set of training data, using real information from a contest I created on Kaggle: http://www.kaggle.com/c/PhotoQualityPrediction/ Installing scikits-learn ~~~~~~~~~~~~~~~~~~~~~~~~ Before you can run the python scripts, you'll need to install the scikits-learn machine-learning framework. You can find instructions here: http://scikit-learn.sourceforge.net/install.html Getting this code ~~~~~~~~~~~~~~~~~ To pull the latest copy of this code and enter the directory run these commands: git clone git://github.com/petewarden/MLloWorld.git cd MLloWorld/ Creating a model ~~~~~~~~~~~~~~~~ Before you can predict unknown values, you need to train up the algorithm with example data. I've packaged a set of 40,000 items as a CSV file, with each column representing an attribute of the original photo albums. You'll need to run these through the training script to build a model that can be used for prediction. Here's the command: python train.py training_data.csv storedmodel That may take ten or twenty minutes to run, but at the end you should have a file called storedmodel in the current directory. Predicting results ~~~~~~~~~~~~~~~~~~ Now that you have a model built, you can take the test set of data and predict their values: python predict.py test_data.csv storedmodel > results.csv This will also take a few minutes, but at the end you'll have a CSV file containing a list of the album ids and a prediction for each one. It's in the right format to submit to Kaggle, and if you look for the 'Full scikit-learn example' in the benchmarks at the bottom of the leaderboard, you'll see how this simple approach scored: http://www.kaggle.com/c/PhotoQualityPrediction/Leaderboard As you can see, it's not that great! If you modify the code and think you've improved its predictions, you can create a team and submit your new results to find out how well you've done. There's already stiff competition from the current teams of course! http://www.kaggle.com/c/PhotoQualityPrediction/Submissions Notes on the internal data format ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The trickiest part for me was getting the data into a format that scikit-learn's functions could understand. Because the CSV stores which words occurred for an album, the full row vector for each of them could be thousands of entries long, most of them zero. To speed up the training and save on memory, I used numpy's sparse matrix class to store the results, coo_matrix. You can see the sort of unpacking I do in the expand_to_vectors() function in mlloutils.py