Switch branches/tags
Nothing to show
Find file
Fetching contributors…
Cannot retrieve contributors at this time
45 lines (30 sloc) 2.72 KB
"Pete Warden" <> -
This is a minimal example of how to write a machine-learning algorithm to solve a prediction problem. It shows you how to create a simple prediction model from a set of training data, using real information from a contest I created on Kaggle:
Installing scikits-learn
Before you can run the python scripts, you'll need to install the scikits-learn machine-learning framework. You can find instructions here:
Getting this code
To pull the latest copy of this code and enter the directory run these commands:
git clone git://
cd MLloWorld/
Creating a model
Before you can predict unknown values, you need to train up the algorithm with example data. I've packaged a set of 40,000 items as a CSV file, with each column representing an attribute of the original photo albums. You'll need to run these through the training script to build a model that can be used for prediction. Here's the command:
python training_data.csv storedmodel
That may take ten or twenty minutes to run, but at the end you should have a file called storedmodel in the current directory.
Predicting results
Now that you have a model built, you can take the test set of data and predict their values:
python test_data.csv storedmodel > results.csv
This will also take a few minutes, but at the end you'll have a CSV file containing a list of the album ids and a prediction for each one. It's in the right format to submit to Kaggle, and if you look for the 'Full scikit-learn example' in the benchmarks at the bottom of the leaderboard, you'll see how this simple approach scored:
As you can see, it's not that great! If you modify the code and think you've improved its predictions, you can create a team and submit your new results to find out how well you've done. There's already stiff competition from the current teams of course!
Notes on the internal data format
The trickiest part for me was getting the data into a format that scikit-learn's functions could understand. Because the CSV stores which words occurred for an album, the full row vector for each of them could be thousands of entries long, most of them zero. To speed up the training and save on memory, I used numpy's sparse matrix class to store the results, coo_matrix. You can see the sort of unpacking I do in the expand_to_vectors() function in