CauseEffectPairsChallenge

Name: Diogo Moitinho de Almeida Kaggle ID: Dee5 email: diogo149@gmail.com Team: ProtoML

Software Used: arch linux (for feature creation) python 2.7.5 numpy scipy scikit-learn pandas ipython ubuntu 12.04 (for hyperparameter optimization) python 2.7.3 numpy scipy scikit-learn pandas ipython

Package Versions: -numpy 1.7.1 -scipy 0.12.0 -pandas 0.11.0 -scikit-learn 0.13.1 -ipython 0.13.2

Hardware needed: -feature creation will probably take +5GB -running on the entire dataset took several days on an 8 core machine -about 4GB of RAM per core was needed

To run with training: -open an ipython terminal -run: >>> %time %run fc_train.py

To run with testing only: -open an ipython terminal -run: >>> %time %run test_only.py

Notes: -The relevant settings can be changed in SETTINGS.py

For my 3 submissions, I use settings:

Getting leaderboard score: 0.81367
    FC_TRAIN.USE_ALL_FEAT = False
    FC_TRAIN.USE_NON_GA_FEAT = False
    FC_TRAIN.CLF = GradientBoostingRegressor(loss='huber', n_estimators=5000, random_state=1, min_samples_split=2, min_samples_leaf=1, subsample=1.0, max_features=686, alpha=0.995355212043, max_depth=10, learning_rate=np.exp(-4.09679792914))

Getting leaderboard score: 0.81279
    FC_TRAIN.USE_ALL_FEAT = True
    FC_TRAIN.USE_NON_GA_FEAT = False
    FC_TRAIN.CLF = GradientBoostingRegressor(loss='huber', n_estimators=5000, random_state=1, min_samples_split=2, min_samples_leaf=1, subsample=1.0, max_features=500, alpha=0.95, max_depth=10, learning_rate=np.exp(-3.28469694591))

Getting leaderboard score: 0.81238
    FC_TRAIN.USE_ALL_FEAT = True
    FC_TRAIN.USE_NON_GA_FEAT = False
    FC_TRAIN.CLF = GradientBoostingRegressor(loss='huber', n_estimators=5000, random_state=1, min_samples_split=2, min_samples_leaf=1, subsample=1.0, max_features=686, alpha=0.99517924408, max_depth=10, learning_rate=np.exp(-4.10031144415))