Skip to content
Shows how to write a simple data contest entry for Kaggle, using scikit-learn for machine learning algorithms
Find file
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Failed to load latest commit information.


"Pete Warden" <> -

This is a minimal example of how to write a machine-learning algorithm to solve a prediction problem. It shows you how to create a simple prediction model from a set of training data, using real information from a contest I created on Kaggle:

Installing scikits-learn

Before you can run the python scripts, you'll need to install the scikits-learn machine-learning framework. You can find instructions here:

Getting this code

To pull the latest copy of this code and enter the directory run these commands:
git clone git://
cd MLloWorld/

Creating a model

Before you can predict unknown values, you need to train up the algorithm with example data. I've packaged a set of 40,000 items as a CSV file, with each column representing an attribute of the original photo albums. You'll need to run these through the training script to build a model that can be used for prediction. Here's the command:

python training_data.csv storedmodel

That may take ten or twenty minutes to run, but at the end you should have a file called storedmodel in the current directory.

Predicting results

Now that you have a model built, you can take the test set of data and predict their values:

python test_data.csv storedmodel > results.csv

This will also take a few minutes, but at the end you'll have a CSV file containing a list of the album ids and a prediction for each one. It's in the right format to submit to Kaggle, and if you look for the 'Full scikit-learn example' in the benchmarks at the bottom of the leaderboard, you'll see how this simple approach scored:
As you can see, it's not that great! If you modify the code and think you've improved its predictions, you can create a team and submit your new results to find out how well you've done. There's already stiff competition from the current teams of course!

Notes on the internal data format

The trickiest part for me was getting the data into a format that scikit-learn's functions could understand. Because the CSV stores which words occurred for an album, the full row vector for each of them could be thousands of entries long, most of them zero. To speed up the training and save on memory, I used numpy's sparse matrix class to store the results, coo_matrix. You can see the sort of unpacking I do in the expand_to_vectors() function in
Something went wrong with that request. Please try again.