notebooks/regression_example_kaggle_allstate.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "pyLigthGBM\n",
    "=======\n",
    "\n",
    "Python wrapper for Microsoft [LightGBM](https://github.com/Microsoft/LightGBM)  \n",
    "Make sure that you have installed LightGBM [Installation-Guide](https://github.com/Microsoft/LightGBM/wiki/Installation-Guide)\n",
    "\n",
    "**GitHub      :  [https://github.com/ArdalanM/pyLightGBM](https://github.com/ArdalanM/pyLightGBM) **\n",
    "\n",
    "**Author of this notebook :** Evgeny BAZAROV <baz.evgenii@gmail.com>\n",
    "\n",
    "-------\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "%matplotlib inline\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "import os, gc\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "\n",
    "from sklearn import metrics, model_selection\n",
    "from sklearn.preprocessing import LabelEncoder\n",
    "\n",
    "from pylightgbm.models import GBMRegressor"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "DATA\n",
    "-------\n",
    "\n",
    "For this example used **data from Kaggle competition Allstate Claims Severity**  \n",
    "You can download data here : https://www.kaggle.com/c/allstate-claims-severity/data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train data shape (188318, 132)\n",
      "Test data shape (125546, 131)\n"
     ]
    }
   ],
   "source": [
    "df_train = pd.read_csv(\"data/train.csv.zip\")\n",
    "print('Train data shape', df_train.shape)\n",
    "\n",
    "df_test = pd.read_csv(\"data/test.csv.zip\")\n",
    "print('Test data shape', df_test.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Extracting `loss` from train and `id` from test"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "y = np.log(df_train['loss']+1).as_matrix().astype(np.float)\n",
    "id_test = np.array(df_test['id'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Merging train and test data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Merged data shape (313864, 132)\n"
     ]
    }
   ],
   "source": [
    "df = df_train.append(df_test, ignore_index=True)\n",
    "del df_test, df_train\n",
    "gc.collect()\n",
    "\n",
    "print('Merged data shape', df.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Droping not useful columns"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "df.drop(labels=['loss', 'id'], axis=1, inplace=True)\n",
    "feature_list = df.columns.tolist()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Transfrom categorical features `cat` from 1 to 116"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "le = LabelEncoder()\n",
    "\n",
    "for col in df.columns.tolist():\n",
    "    if 'cat' in col:\n",
    "        df[col] = le.fit_transform(df[col])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "TRAIN, VALIDATION, TEST\n",
    "-------\n",
    "Split data into train, validation (for early stopping) and test set"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "train-test split\n",
      "train-validation split\n",
      "\n",
      "Train shape (150654, 130)\n",
      "Validation shape (37664, 130)\n",
      "Test shape (125546, 130)\n"
     ]
    }
   ],
   "source": [
    "print('train-test split')\n",
    "df_train, df_test = df.iloc[:len(y)], df.iloc[len(y):]\n",
    "del df\n",
    "gc.collect()\n",
    "\n",
    "print('train-validation split\\n')\n",
    "X = df_train.as_matrix()\n",
    "X_train, X_valid, y_train, y_valid = model_selection.train_test_split(X, y, test_size=0.2, random_state=42)\n",
    "X_test = df_test.as_matrix()\n",
    "\n",
    "del df_train, df_test\n",
    "gc.collect()\n",
    "\n",
    "print('Train shape', X_train.shape)\n",
    "print('Validation shape', X_valid.shape)\n",
    "print('Test shape', X_test.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "TRAINING GBMRegressor\n",
    "-------\n",
    "List of parameters and their explanation you can find here https://github.com/Microsoft/LightGBM/wiki/Quick-Start\n",
    "\n",
    "**don't forget to change `exec_path` here**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": false,
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[LightGBM] finished load parameters\n",
      "[LightGBM] Finish loading data, use 0.711743 seconds \n",
      "[LightGBM] #data:150654 #feature:130\n",
      "\n",
      "[LightGBM] finish training init\n",
      "[LightGBM] start train\n",
      "[LightGBM] Iteration:1, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 6.961980\n",
      "[LightGBM] 0.021856 seconds elapsed, finished 1 iteration\n",
      "[LightGBM] Iteration:2, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 6.272212\n",
      "[LightGBM] 0.043370 seconds elapsed, finished 2 iteration\n",
      "[LightGBM] Iteration:3, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 5.652135\n",
      "[LightGBM] 0.065526 seconds elapsed, finished 3 iteration\n",
      "[LightGBM] Iteration:4, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 5.094611\n",
      "[LightGBM] 0.090001 seconds elapsed, finished 4 iteration\n",
      "[LightGBM] Iteration:5, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 4.593778\n",
      "[LightGBM] 0.286686 seconds elapsed, finished 5 iteration\n",
      "[LightGBM] Iteration:6, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 4.143739\n",
      "[LightGBM] 0.348910 seconds elapsed, finished 6 iteration\n",
      "[LightGBM] Iteration:7, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 3.739711\n",
      "[LightGBM] 0.489832 seconds elapsed, finished 7 iteration\n",
      "[LightGBM] Iteration:8, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 3.376981\n",
      "[LightGBM] 0.512660 seconds elapsed, finished 8 iteration\n",
      "[LightGBM] Iteration:9, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 3.051514\n",
      "[LightGBM] 0.562440 seconds elapsed, finished 9 iteration\n",
      "[LightGBM] Iteration:10, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 2.760008\n",
      "[LightGBM] 0.662212 seconds elapsed, finished 10 iteration\n",
      "[LightGBM] Iteration:11, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 2.498742\n",
      "[LightGBM] 0.680835 seconds elapsed, finished 11 iteration\n",
      "[LightGBM] Iteration:12, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 2.265063\n",
      "[LightGBM] 0.700700 seconds elapsed, finished 12 iteration\n",
      "[LightGBM] Iteration:13, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 2.056102\n",
      "[LightGBM] 0.719899 seconds elapsed, finished 13 iteration\n",
      "[LightGBM] Iteration:14, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 1.869803\n",
      "[LightGBM] 0.744201 seconds elapsed, finished 14 iteration\n",
      "[LightGBM] Iteration:15, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 1.703988\n",
      "[LightGBM] 0.768748 seconds elapsed, finished 15 iteration\n",
      "[LightGBM] Iteration:16, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 1.556266\n",
      "[LightGBM] 0.800900 seconds elapsed, finished 16 iteration\n",
      "[LightGBM] Iteration:17, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 1.425275\n",
      "[LightGBM] 0.822000 seconds elapsed, finished 17 iteration\n",
      "[LightGBM] Iteration:18, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 1.309379\n",
      "[LightGBM] 0.850259 seconds elapsed, finished 18 iteration\n",
      "[LightGBM] Iteration:19, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 1.207323\n",
      "[LightGBM] 0.880400 seconds elapsed, finished 19 iteration\n",
      "[LightGBM] Iteration:20, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 1.117490\n",
      "[LightGBM] 0.905384 seconds elapsed, finished 20 iteration\n",
      "[LightGBM] Iteration:21, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 1.038601\n",
      "[LightGBM] 0.933972 seconds elapsed, finished 21 iteration\n",
      "[LightGBM] Iteration:22, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.969808\n",
      "[LightGBM] 0.957384 seconds elapsed, finished 22 iteration\n",
      "[LightGBM] Iteration:23, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.910251\n",
      "[LightGBM] 0.981919 seconds elapsed, finished 23 iteration\n",
      "[LightGBM] Iteration:24, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.858657\n",
      "[LightGBM] 1.062642 seconds elapsed, finished 24 iteration\n",
      "[LightGBM] Iteration:25, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.814136\n",
      "[LightGBM] 1.257088 seconds elapsed, finished 25 iteration\n",
      "[LightGBM] Iteration:26, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.776074\n",
      "[LightGBM] 1.385650 seconds elapsed, finished 26 iteration\n",
      "[LightGBM] Iteration:27, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.743463\n",
      "[LightGBM] 1.412536 seconds elapsed, finished 27 iteration\n",
      "[LightGBM] Iteration:28, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.715886\n",
      "[LightGBM] 1.432164 seconds elapsed, finished 28 iteration\n",
      "[LightGBM] Iteration:29, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.692664\n",
      "[LightGBM] 1.451803 seconds elapsed, finished 29 iteration\n",
      "[LightGBM] Iteration:30, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.672695\n",
      "[LightGBM] 1.471190 seconds elapsed, finished 30 iteration\n",
      "[LightGBM] Iteration:31, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.656244\n",
      "[LightGBM] 1.495353 seconds elapsed, finished 31 iteration\n",
      "[LightGBM] Iteration:32, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.642015\n",
      "[LightGBM] 1.514429 seconds elapsed, finished 32 iteration\n",
      "[LightGBM] Iteration:33, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.629985\n",
      "[LightGBM] 1.584055 seconds elapsed, finished 33 iteration\n",
      "[LightGBM] Iteration:34, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.620077\n",
      "[LightGBM] 1.714647 seconds elapsed, finished 34 iteration\n",
      "[LightGBM] Iteration:35, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.611872\n",
      "[LightGBM] 1.747310 seconds elapsed, finished 35 iteration\n",
      "[LightGBM] Iteration:36, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.604920\n",
      "[LightGBM] 1.771540 seconds elapsed, finished 36 iteration\n",
      "[LightGBM] Iteration:37, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.599098\n",
      "[LightGBM] 1.803377 seconds elapsed, finished 37 iteration\n",
      "[LightGBM] Iteration:38, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.593893\n",
      "[LightGBM] 1.823769 seconds elapsed, finished 38 iteration\n",
      "[LightGBM] Iteration:39, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.589603\n",
      "[LightGBM] 1.853272 seconds elapsed, finished 39 iteration\n",
      "[LightGBM] Iteration:40, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.585950\n",
      "[LightGBM] 1.877874 seconds elapsed, finished 40 iteration\n",
      "[LightGBM] Iteration:41, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.582945\n",
      "[LightGBM] 2.023785 seconds elapsed, finished 41 iteration\n",
      "[LightGBM] Iteration:42, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.580280\n",
      "[LightGBM] 2.076354 seconds elapsed, finished 42 iteration\n",
      "[LightGBM] Iteration:43, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.578035\n",
      "[LightGBM] 2.098019 seconds elapsed, finished 43 iteration\n",
      "[LightGBM] Iteration:44, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.576072\n",
      "[LightGBM] 2.121360 seconds elapsed, finished 44 iteration\n",
      "[LightGBM] Iteration:45, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.574104\n",
      "[LightGBM] 2.142802 seconds elapsed, finished 45 iteration\n",
      "[LightGBM] Iteration:46, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.572589\n",
      "[LightGBM] 2.165331 seconds elapsed, finished 46 iteration\n",
      "[LightGBM] Iteration:47, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.571316\n",
      "[LightGBM] 2.190016 seconds elapsed, finished 47 iteration\n",
      "[LightGBM] Iteration:48, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.569933\n",
      "[LightGBM] 2.211532 seconds elapsed, finished 48 iteration\n",
      "[LightGBM] Iteration:49, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.568834\n",
      "[LightGBM] 2.235403 seconds elapsed, finished 49 iteration\n",
      "[LightGBM] Iteration:50, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.567688\n",
      "[LightGBM] 2.254860 seconds elapsed, finished 50 iteration\n",
      "[LightGBM] Iteration:51, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.566740\n",
      "[LightGBM] 2.285876 seconds elapsed, finished 51 iteration\n",
      "[LightGBM] Iteration:52, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.565591\n",
      "[LightGBM] 2.352138 seconds elapsed, finished 52 iteration\n",
      "[LightGBM] Iteration:53, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.564652\n",
      "[LightGBM] 2.376265 seconds elapsed, finished 53 iteration\n",
      "[LightGBM] Iteration:54, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.563783\n",
      "[LightGBM] 2.409840 seconds elapsed, finished 54 iteration\n",
      "[LightGBM] Iteration:55, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.563206\n",
      "[LightGBM] 2.523305 seconds elapsed, finished 55 iteration\n",
      "[LightGBM] Iteration:56, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.562546\n",
      "[LightGBM] 2.544789 seconds elapsed, finished 56 iteration\n",
      "[LightGBM] Iteration:57, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.561987\n",
      "[LightGBM] 2.564579 seconds elapsed, finished 57 iteration\n",
      "[LightGBM] Iteration:58, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.561262\n",
      "[LightGBM] 2.587753 seconds elapsed, finished 58 iteration\n",
      "[LightGBM] Iteration:59, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.560723\n",
      "[LightGBM] 2.638777 seconds elapsed, finished 59 iteration\n",
      "[LightGBM] Iteration:60, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.560031\n",
      "[LightGBM] 2.659677 seconds elapsed, finished 60 iteration\n",
      "[LightGBM] Iteration:61, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.559568\n",
      "[LightGBM] 2.704729 seconds elapsed, finished 61 iteration\n",
      "[LightGBM] Iteration:62, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.559140\n",
      "[LightGBM] 2.727663 seconds elapsed, finished 62 iteration\n",
      "[LightGBM] Iteration:63, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.558584\n",
      "[LightGBM] 2.749847 seconds elapsed, finished 63 iteration\n",
      "[LightGBM] Iteration:64, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.558112\n",
      "[LightGBM] 2.768331 seconds elapsed, finished 64 iteration\n",
      "[LightGBM] Iteration:65, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.557692\n",
      "[LightGBM] 2.790897 seconds elapsed, finished 65 iteration\n",
      "[LightGBM] Iteration:66, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.557303\n",
      "[LightGBM] 2.815394 seconds elapsed, finished 66 iteration\n",
      "[LightGBM] Iteration:67, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.556939\n",
      "[LightGBM] 2.892625 seconds elapsed, finished 67 iteration\n",
      "[LightGBM] Iteration:68, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.556391\n",
      "[LightGBM] 2.916651 seconds elapsed, finished 68 iteration\n",
      "[LightGBM] Iteration:69, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.556003\n",
      "[LightGBM] 2.939766 seconds elapsed, finished 69 iteration\n",
      "[LightGBM] Iteration:70, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.555597\n",
      "[LightGBM] 2.960222 seconds elapsed, finished 70 iteration\n",
      "[LightGBM] Iteration:71, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.555179\n",
      "[LightGBM] 2.982900 seconds elapsed, finished 71 iteration\n",
      "[LightGBM] Iteration:72, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.554938\n",
      "[LightGBM] 3.006531 seconds elapsed, finished 72 iteration\n",
      "[LightGBM] Iteration:73, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.554592\n",
      "[LightGBM] 3.088141 seconds elapsed, finished 73 iteration\n",
      "[LightGBM] Iteration:74, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.554282\n",
      "[LightGBM] 3.106603 seconds elapsed, finished 74 iteration\n",
      "[LightGBM] Iteration:75, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.554005\n",
      "[LightGBM] 3.127002 seconds elapsed, finished 75 iteration\n",
      "[LightGBM] Iteration:76, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.553602\n",
      "[LightGBM] 3.148345 seconds elapsed, finished 76 iteration\n",
      "[LightGBM] Iteration:77, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.553376\n",
      "[LightGBM] 3.169229 seconds elapsed, finished 77 iteration\n",
      "[LightGBM] Iteration:78, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.553013\n",
      "[LightGBM] 3.188581 seconds elapsed, finished 78 iteration\n",
      "[LightGBM] Iteration:79, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.552792\n",
      "[LightGBM] 3.208106 seconds elapsed, finished 79 iteration\n",
      "[LightGBM] Iteration:80, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.552479\n",
      "[LightGBM] 3.226874 seconds elapsed, finished 80 iteration\n",
      "[LightGBM] Iteration:81, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.552231\n",
      "[LightGBM] 3.248738 seconds elapsed, finished 81 iteration\n",
      "[LightGBM] Iteration:82, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.552009\n",
      "[LightGBM] 3.268555 seconds elapsed, finished 82 iteration\n",
      "[LightGBM] Iteration:83, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.551769\n",
      "[LightGBM] 3.288473 seconds elapsed, finished 83 iteration\n",
      "[LightGBM] Iteration:84, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.551602\n",
      "[LightGBM] 3.310157 seconds elapsed, finished 84 iteration\n",
      "[LightGBM] Iteration:85, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.551411\n",
      "[LightGBM] 3.329609 seconds elapsed, finished 85 iteration\n",
      "[LightGBM] Iteration:86, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.551085\n",
      "[LightGBM] 3.352045 seconds elapsed, finished 86 iteration\n",
      "[LightGBM] Iteration:87, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.550824\n",
      "[LightGBM] 3.372110 seconds elapsed, finished 87 iteration\n",
      "[LightGBM] Iteration:88, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.550651\n",
      "[LightGBM] 3.394026 seconds elapsed, finished 88 iteration\n",
      "[LightGBM] Iteration:89, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.550416\n",
      "[LightGBM] 3.412492 seconds elapsed, finished 89 iteration\n",
      "[LightGBM] Iteration:90, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.550145\n",
      "[LightGBM] 3.433556 seconds elapsed, finished 90 iteration\n",
      "[LightGBM] Iteration:91, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.549921\n",
      "[LightGBM] 3.454388 seconds elapsed, finished 91 iteration\n",
      "[LightGBM] Iteration:92, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.549743\n",
      "[LightGBM] 3.475955 seconds elapsed, finished 92 iteration\n",
      "[LightGBM] Iteration:93, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.549636\n",
      "[LightGBM] 3.495681 seconds elapsed, finished 93 iteration\n",
      "[LightGBM] Iteration:94, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.549469\n",
      "[LightGBM] 3.517475 seconds elapsed, finished 94 iteration\n",
      "[LightGBM] Iteration:95, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.549300\n",
      "[LightGBM] 3.532946 seconds elapsed, finished 95 iteration\n",
      "[LightGBM] Iteration:96, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.549126\n",
      "[LightGBM] 3.554787 seconds elapsed, finished 96 iteration\n",
      "[LightGBM] Iteration:97, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.548857\n",
      "[LightGBM] 3.578010 seconds elapsed, finished 97 iteration\n",
      "[LightGBM] Iteration:98, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.548668\n",
      "[LightGBM] 3.596676 seconds elapsed, finished 98 iteration\n",
      "[LightGBM] Iteration:99, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.548529\n",
      "[LightGBM] 3.617584 seconds elapsed, finished 99 iteration\n",
      "[LightGBM] Iteration:100, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.548364\n",
      "[LightGBM] 3.635129 seconds elapsed, finished 100 iteration\n",
      "[LightGBM] Iteration:101, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.548249\n",
      "[LightGBM] 3.655337 seconds elapsed, finished 101 iteration\n",
      "[LightGBM] Iteration:102, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.548083\n",
      "[LightGBM] 3.674259 seconds elapsed, finished 102 iteration\n",
      "[LightGBM] Iteration:103, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.547809\n",
      "[LightGBM] 3.697354 seconds elapsed, finished 103 iteration\n",
      "[LightGBM] Iteration:104, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.547684\n",
      "[LightGBM] 3.717541 seconds elapsed, finished 104 iteration\n",
      "[LightGBM] Iteration:105, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.547558\n",
      "[LightGBM] 3.734273 seconds elapsed, finished 105 iteration\n",
      "[LightGBM] Iteration:106, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.547449\n",
      "[LightGBM] 3.755179 seconds elapsed, finished 106 iteration\n",
      "[LightGBM] Iteration:107, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.547264\n",
      "[LightGBM] 3.773789 seconds elapsed, finished 107 iteration\n",
      "[LightGBM] Iteration:108, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.547176\n",
      "[LightGBM] 3.794435 seconds elapsed, finished 108 iteration\n",
      "[LightGBM] Iteration:109, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.546946\n",
      "[LightGBM] 3.814164 seconds elapsed, finished 109 iteration\n",
      "[LightGBM] Iteration:110, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.546777\n",
      "[LightGBM] 3.835618 seconds elapsed, finished 110 iteration\n",
      "[LightGBM] Iteration:111, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.546666\n",
      "[LightGBM] 3.857092 seconds elapsed, finished 111 iteration\n",
      "[LightGBM] Iteration:112, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.546534\n",
      "[LightGBM] 3.878972 seconds elapsed, finished 112 iteration\n",
      "[LightGBM] Iteration:113, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.546434\n",
      "[LightGBM] 3.899044 seconds elapsed, finished 113 iteration\n",
      "[LightGBM] Iteration:114, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.546254\n",
      "[LightGBM] 3.921091 seconds elapsed, finished 114 iteration\n",
      "[LightGBM] Iteration:115, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.546124\n",
      "[LightGBM] 3.938835 seconds elapsed, finished 115 iteration\n",
      "[LightGBM] Iteration:116, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.545962\n",
      "[LightGBM] 3.957124 seconds elapsed, finished 116 iteration\n",
      "[LightGBM] Iteration:117, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.545859\n",
      "[LightGBM] 3.979573 seconds elapsed, finished 117 iteration\n",
      "[LightGBM] Iteration:118, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.545758\n",
      "[LightGBM] 3.996743 seconds elapsed, finished 118 iteration\n",
      "[LightGBM] Iteration:119, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.545649\n",
      "[LightGBM] 4.016397 seconds elapsed, finished 119 iteration\n",
      "[LightGBM] Iteration:120, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.545561\n",
      "[LightGBM] 4.037353 seconds elapsed, finished 120 iteration\n",
      "[LightGBM] Iteration:121, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.545458\n",
      "[LightGBM] 4.053490 seconds elapsed, finished 121 iteration\n",
      "[LightGBM] Iteration:122, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.545343\n",
      "[LightGBM] 4.071361 seconds elapsed, finished 122 iteration\n",
      "[LightGBM] Iteration:123, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.545270\n",
      "[LightGBM] 4.090618 seconds elapsed, finished 123 iteration\n",
      "[LightGBM] Iteration:124, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.545203\n",
      "[LightGBM] 4.109759 seconds elapsed, finished 124 iteration\n",
      "[LightGBM] Iteration:125, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.545126\n",
      "[LightGBM] 4.129529 seconds elapsed, finished 125 iteration\n",
      "[LightGBM] Iteration:126, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.545031\n",
      "[LightGBM] 4.145993 seconds elapsed, finished 126 iteration\n",
      "[LightGBM] Iteration:127, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.544954\n",
      "[LightGBM] 4.162430 seconds elapsed, finished 127 iteration\n",
      "[LightGBM] Iteration:128, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.544878\n",
      "[LightGBM] 4.182834 seconds elapsed, finished 128 iteration\n",
      "[LightGBM] Iteration:129, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.544805\n",
      "[LightGBM] 4.199966 seconds elapsed, finished 129 iteration\n",
      "[LightGBM] Iteration:130, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.544653\n",
      "[LightGBM] 4.220235 seconds elapsed, finished 130 iteration\n",
      "[LightGBM] Iteration:131, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.544575\n",
      "[LightGBM] 4.240454 seconds elapsed, finished 131 iteration\n",
      "[LightGBM] Iteration:132, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.544451\n",
      "[LightGBM] 4.258494 seconds elapsed, finished 132 iteration\n",
      "[LightGBM] Iteration:133, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.544359\n",
      "[LightGBM] 4.277494 seconds elapsed, finished 133 iteration\n",
      "[LightGBM] Iteration:134, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.544301\n",
      "[LightGBM] 4.345821 seconds elapsed, finished 134 iteration\n",
      "[LightGBM] Iteration:135, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.544243\n",
      "[LightGBM] 4.361499 seconds elapsed, finished 135 iteration\n",
      "[LightGBM] Iteration:136, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.544155\n",
      "[LightGBM] 4.378303 seconds elapsed, finished 136 iteration\n",
      "[LightGBM] Iteration:137, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.544056\n",
      "[LightGBM] 4.402140 seconds elapsed, finished 137 iteration\n",
      "[LightGBM] Iteration:138, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.544012\n",
      "[LightGBM] 4.617251 seconds elapsed, finished 138 iteration\n",
      "[LightGBM] Iteration:139, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.543954\n",
      "[LightGBM] 4.810575 seconds elapsed, finished 139 iteration\n",
      "[LightGBM] Iteration:140, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.543834\n",
      "[LightGBM] 4.904122 seconds elapsed, finished 140 iteration\n",
      "[LightGBM] Iteration:141, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.543751\n",
      "[LightGBM] 4.922174 seconds elapsed, finished 141 iteration\n",
      "[LightGBM] Iteration:142, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.543666\n",
      "[LightGBM] 4.943198 seconds elapsed, finished 142 iteration\n",
      "[LightGBM] Iteration:143, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.543574\n",
      "[LightGBM] 4.962915 seconds elapsed, finished 143 iteration\n",
      "[LightGBM] Iteration:144, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.543520\n",
      "[LightGBM] 4.977955 seconds elapsed, finished 144 iteration\n",
      "[LightGBM] Iteration:145, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.543476\n",
      "[LightGBM] 4.994497 seconds elapsed, finished 145 iteration\n",
      "[LightGBM] Iteration:146, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.543396\n",
      "[LightGBM] 5.012401 seconds elapsed, finished 146 iteration\n",
      "[LightGBM] Iteration:147, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.543329\n",
      "[LightGBM] 5.032512 seconds elapsed, finished 147 iteration\n",
      "[LightGBM] Iteration:148, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.543292\n",
      "[LightGBM] 5.046175 seconds elapsed, finished 148 iteration\n",
      "[LightGBM] Iteration:149, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.543187\n",
      "[LightGBM] 5.058747 seconds elapsed, finished 149 iteration\n",
      "[LightGBM] Iteration:150, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.543112\n",
      "[LightGBM] 5.080240 seconds elapsed, finished 150 iteration\n",
      "[LightGBM] Iteration:151, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.543096\n",
      "[LightGBM] 5.121799 seconds elapsed, finished 151 iteration\n",
      "[LightGBM] Iteration:152, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.543064\n",
      "[LightGBM] 5.139937 seconds elapsed, finished 152 iteration\n",
      "[LightGBM] Iteration:153, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.543004\n",
      "[LightGBM] 5.162206 seconds elapsed, finished 153 iteration\n",
      "[LightGBM] Iteration:154, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.542979\n",
      "[LightGBM] 5.176230 seconds elapsed, finished 154 iteration\n",
      "[LightGBM] Iteration:155, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.542850\n",
      "[LightGBM] 5.197287 seconds elapsed, finished 155 iteration\n",
      "[LightGBM] Iteration:156, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.542805\n",
      "[LightGBM] 5.208938 seconds elapsed, finished 156 iteration\n",
      "[LightGBM] Iteration:157, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.542768\n",
      "[LightGBM] 5.249468 seconds elapsed, finished 157 iteration\n",
      "[LightGBM] Iteration:158, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.542690\n",
      "[LightGBM] 5.319147 seconds elapsed, finished 158 iteration\n",
      "[LightGBM] Iteration:159, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.542673\n",
      "[LightGBM] 5.451347 seconds elapsed, finished 159 iteration\n",
      "[LightGBM] Iteration:160, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.542584\n",
      "[LightGBM] 5.467711 seconds elapsed, finished 160 iteration\n",
      "[LightGBM] Iteration:161, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.542540\n",
      "[LightGBM] 5.489503 seconds elapsed, finished 161 iteration\n",
      "[LightGBM] Iteration:162, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.542535\n",
      "[LightGBM] 5.502943 seconds elapsed, finished 162 iteration\n",
      "[LightGBM] Iteration:163, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.542441\n",
      "[LightGBM] 5.521811 seconds elapsed, finished 163 iteration\n",
      "[LightGBM] Iteration:164, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.542404\n",
      "[LightGBM] 5.533414 seconds elapsed, finished 164 iteration\n",
      "[LightGBM] Iteration:165, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.542369\n",
      "[LightGBM] 5.545425 seconds elapsed, finished 165 iteration\n",
      "[LightGBM] Iteration:166, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.542297\n",
      "[LightGBM] 5.667942 seconds elapsed, finished 166 iteration\n",
      "[LightGBM] Iteration:167, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.542260\n",
      "[LightGBM] 5.680374 seconds elapsed, finished 167 iteration\n",
      "[LightGBM] Iteration:168, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.542219\n",
      "[LightGBM] 5.695811 seconds elapsed, finished 168 iteration\n",
      "[LightGBM] Iteration:169, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.542130\n",
      "[LightGBM] 5.725799 seconds elapsed, finished 169 iteration\n",
      "[LightGBM] Iteration:170, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.542091\n",
      "[LightGBM] 5.764377 seconds elapsed, finished 170 iteration\n",
      "[LightGBM] Iteration:171, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.542029\n",
      "[LightGBM] 5.780679 seconds elapsed, finished 171 iteration\n",
      "[LightGBM] Iteration:172, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.541987\n",
      "[LightGBM] 5.799397 seconds elapsed, finished 172 iteration\n",
      "[LightGBM] Iteration:173, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.541916\n",
      "[LightGBM] 5.820113 seconds elapsed, finished 173 iteration\n",
      "[LightGBM] Iteration:174, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.541894\n",
      "[LightGBM] 5.833214 seconds elapsed, finished 174 iteration\n",
      "[LightGBM] Iteration:175, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.541835\n",
      "[LightGBM] 5.854080 seconds elapsed, finished 175 iteration\n",
      "[LightGBM] Iteration:176, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.541782\n",
      "[LightGBM] 5.876464 seconds elapsed, finished 176 iteration\n",
      "[LightGBM] Iteration:177, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.541738\n",
      "[LightGBM] 5.896795 seconds elapsed, finished 177 iteration\n",
      "[LightGBM] Iteration:178, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.541699\n",
      "[LightGBM] 5.907657 seconds elapsed, finished 178 iteration\n",
      "[LightGBM] Iteration:179, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.541657\n",
      "[LightGBM] 5.920820 seconds elapsed, finished 179 iteration\n",
      "[LightGBM] Iteration:180, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.541625\n",
      "[LightGBM] 5.935172 seconds elapsed, finished 180 iteration\n",
      "[LightGBM] Iteration:181, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.541589\n",
      "[LightGBM] 5.947306 seconds elapsed, finished 181 iteration\n",
      "[LightGBM] Iteration:182, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.541582\n",
      "[LightGBM] 5.956937 seconds elapsed, finished 182 iteration\n",
      "[LightGBM] Iteration:183, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.541493\n",
      "[LightGBM] 6.078359 seconds elapsed, finished 183 iteration\n",
      "[LightGBM] Iteration:184, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.541411\n",
      "[LightGBM] 6.275286 seconds elapsed, finished 184 iteration\n",
      "[LightGBM] Iteration:185, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.541418\n",
      "[LightGBM] 6.326560 seconds elapsed, finished 185 iteration\n",
      "[LightGBM] Iteration:186, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.541389\n",
      "[LightGBM] 6.382805 seconds elapsed, finished 186 iteration\n",
      "[LightGBM] Iteration:187, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.541376\n",
      "[LightGBM] 6.460019 seconds elapsed, finished 187 iteration\n",
      "[LightGBM] Iteration:188, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.541356\n",
      "[LightGBM] 6.563450 seconds elapsed, finished 188 iteration\n",
      "[LightGBM] Iteration:189, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.541293\n",
      "[LightGBM] 6.722600 seconds elapsed, finished 189 iteration\n",
      "[LightGBM] Iteration:190, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.541248\n",
      "[LightGBM] 6.846514 seconds elapsed, finished 190 iteration\n",
      "[LightGBM] Iteration:191, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.541201\n",
      "[LightGBM] 7.002560 seconds elapsed, finished 191 iteration\n",
      "[LightGBM] Iteration:192, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.541180\n",
      "[LightGBM] 7.082383 seconds elapsed, finished 192 iteration\n",
      "[LightGBM] Iteration:193, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.541108\n",
      "[LightGBM] 7.107818 seconds elapsed, finished 193 iteration\n",
      "[LightGBM] Iteration:194, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.541018\n",
      "[LightGBM] 7.128464 seconds elapsed, finished 194 iteration\n",
      "[LightGBM] Iteration:195, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.541006\n",
      "[LightGBM] 7.140888 seconds elapsed, finished 195 iteration\n",
      "[LightGBM] Iteration:196, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.540969\n",
      "[LightGBM] 7.157751 seconds elapsed, finished 196 iteration\n",
      "[LightGBM] Iteration:197, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.540936\n",
      "[LightGBM] 7.176078 seconds elapsed, finished 197 iteration\n",
      "[LightGBM] Iteration:198, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.540919\n",
      "[LightGBM] 7.227741 seconds elapsed, finished 198 iteration\n",
      "[LightGBM] Iteration:199, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.540894\n",
      "[LightGBM] 7.238707 seconds elapsed, finished 199 iteration\n",
      "[LightGBM] Iteration:200, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.540895\n",
      "[LightGBM] 7.248360 seconds elapsed, finished 200 iteration\n",
      "[LightGBM] Iteration:201, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.540864\n",
      "[LightGBM] 7.262954 seconds elapsed, finished 201 iteration\n",
      "[LightGBM] Iteration:202, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.540821\n",
      "[LightGBM] 7.277799 seconds elapsed, finished 202 iteration\n",
      "[LightGBM] Iteration:203, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.540800\n",
      "[LightGBM] 7.291774 seconds elapsed, finished 203 iteration\n",
      "[LightGBM] Iteration:204, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.540793\n",
      "[LightGBM] 7.309328 seconds elapsed, finished 204 iteration\n",
      "[LightGBM] Iteration:205, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.540786\n",
      "[LightGBM] 7.321828 seconds elapsed, finished 205 iteration\n",
      "[LightGBM] Iteration:206, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.540768\n",
      "[LightGBM] 7.339821 seconds elapsed, finished 206 iteration\n",
      "[LightGBM] Iteration:207, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.540756\n",
      "[LightGBM] 7.351863 seconds elapsed, finished 207 iteration\n",
      "[LightGBM] Iteration:208, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.540723\n",
      "[LightGBM] 7.381111 seconds elapsed, finished 208 iteration\n",
      "[LightGBM] Iteration:209, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.540722\n",
      "[LightGBM] 7.395633 seconds elapsed, finished 209 iteration\n",
      "[LightGBM] Iteration:210, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.540707\n",
      "[LightGBM] 7.410359 seconds elapsed, finished 210 iteration\n",
      "[LightGBM] Iteration:211, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.540595\n",
      "[LightGBM] 7.430633 seconds elapsed, finished 211 iteration\n",
      "[LightGBM] Iteration:212, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.540555\n",
      "[LightGBM] 7.447134 seconds elapsed, finished 212 iteration\n",
      "[LightGBM] Iteration:213, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.540529\n",
      "[LightGBM] 7.458209 seconds elapsed, finished 213 iteration\n",
      "[LightGBM] Iteration:214, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.540529\n",
      "[LightGBM] 7.475035 seconds elapsed, finished 214 iteration\n",
      "[LightGBM] Iteration:215, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.540454\n",
      "[LightGBM] 7.588836 seconds elapsed, finished 215 iteration\n",
      "[LightGBM] Iteration:216, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.540429\n",
      "[LightGBM] 7.600363 seconds elapsed, finished 216 iteration\n",
      "[LightGBM] Iteration:217, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.540429\n",
      "[LightGBM] 7.610669 seconds elapsed, finished 217 iteration\n",
      "[LightGBM] Iteration:218, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.540436\n",
      "[LightGBM] 7.619533 seconds elapsed, finished 218 iteration\n",
      "[LightGBM] Iteration:219, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.540390\n",
      "[LightGBM] 7.638139 seconds elapsed, finished 219 iteration\n",
      "[LightGBM] Iteration:220, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.540307\n",
      "[LightGBM] 7.658040 seconds elapsed, finished 220 iteration\n",
      "[LightGBM] Iteration:221, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.540283\n",
      "[LightGBM] 7.668621 seconds elapsed, finished 221 iteration\n",
      "[LightGBM] Iteration:222, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.540243\n",
      "[LightGBM] 7.684138 seconds elapsed, finished 222 iteration\n",
      "[LightGBM] Iteration:223, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.540204\n",
      "[LightGBM] 7.703344 seconds elapsed, finished 223 iteration\n",
      "[LightGBM] Iteration:224, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.540217\n",
      "[LightGBM] 7.713761 seconds elapsed, finished 224 iteration\n",
      "[LightGBM] Iteration:225, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.540214\n",
      "[LightGBM] 7.722964 seconds elapsed, finished 225 iteration\n",
      "[LightGBM] Iteration:226, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.540208\n",
      "[LightGBM] 7.733212 seconds elapsed, finished 226 iteration\n",
      "[LightGBM] Iteration:227, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.540202\n",
      "[LightGBM] 7.747592 seconds elapsed, finished 227 iteration\n",
      "[LightGBM] Iteration:228, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.540173\n",
      "[LightGBM] 7.759592 seconds elapsed, finished 228 iteration\n",
      "[LightGBM] Iteration:229, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.540153\n",
      "[LightGBM] 7.770380 seconds elapsed, finished 229 iteration\n",
      "[LightGBM] Iteration:230, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.540151\n",
      "[LightGBM] 7.780041 seconds elapsed, finished 230 iteration\n",
      "[LightGBM] Iteration:231, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.540148\n",
      "[LightGBM] 7.971137 seconds elapsed, finished 231 iteration\n",
      "[LightGBM] Iteration:232, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.540143\n",
      "[LightGBM] 7.980352 seconds elapsed, finished 232 iteration\n",
      "[LightGBM] Iteration:233, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.540111\n",
      "[LightGBM] 7.995215 seconds elapsed, finished 233 iteration\n",
      "[LightGBM] Iteration:234, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.540102\n",
      "[LightGBM] 8.008756 seconds elapsed, finished 234 iteration\n",
      "[LightGBM] Iteration:235, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.540091\n",
      "[LightGBM] 8.027258 seconds elapsed, finished 235 iteration\n",
      "[LightGBM] Iteration:236, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.540103\n",
      "[LightGBM] 8.038257 seconds elapsed, finished 236 iteration\n",
      "[LightGBM] Iteration:237, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.540087\n",
      "[LightGBM] 8.053912 seconds elapsed, finished 237 iteration\n",
      "[LightGBM] Iteration:238, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.540066\n",
      "[LightGBM] 8.159246 seconds elapsed, finished 238 iteration\n",
      "[LightGBM] Iteration:239, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.540059\n",
      "[LightGBM] 8.172204 seconds elapsed, finished 239 iteration\n",
      "[LightGBM] Iteration:240, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.540052\n",
      "[LightGBM] 8.189843 seconds elapsed, finished 240 iteration\n",
      "[LightGBM] Iteration:241, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.539976\n",
      "[LightGBM] 8.211300 seconds elapsed, finished 241 iteration\n",
      "[LightGBM] Iteration:242, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.539981\n",
      "[LightGBM] 8.246263 seconds elapsed, finished 242 iteration\n",
      "[LightGBM] Iteration:243, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.539961\n",
      "[LightGBM] 8.278531 seconds elapsed, finished 243 iteration\n",
      "[LightGBM] Iteration:244, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.539938\n",
      "[LightGBM] 8.288509 seconds elapsed, finished 244 iteration\n",
      "[LightGBM] Iteration:245, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.539923\n",
      "[LightGBM] 8.304233 seconds elapsed, finished 245 iteration\n",
      "[LightGBM] Iteration:246, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.539923\n",
      "[LightGBM] 8.318017 seconds elapsed, finished 246 iteration\n",
      "[LightGBM] Iteration:247, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.539904\n",
      "[LightGBM] 8.361286 seconds elapsed, finished 247 iteration\n",
      "[LightGBM] Iteration:248, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.539846\n",
      "[LightGBM] 8.378991 seconds elapsed, finished 248 iteration\n",
      "[LightGBM] Iteration:249, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.539794\n",
      "[LightGBM] 8.396146 seconds elapsed, finished 249 iteration\n",
      "[LightGBM] Iteration:250, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.539777\n",
      "[LightGBM] 8.407732 seconds elapsed, finished 250 iteration\n",
      "[LightGBM] Iteration:251, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.539767\n",
      "[LightGBM] 8.426419 seconds elapsed, finished 251 iteration\n",
      "[LightGBM] Iteration:252, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.539756\n",
      "[LightGBM] 8.437063 seconds elapsed, finished 252 iteration\n",
      "[LightGBM] Iteration:253, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.539757\n",
      "[LightGBM] 8.445909 seconds elapsed, finished 253 iteration\n",
      "[LightGBM] Iteration:254, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.539750\n",
      "[LightGBM] 8.458686 seconds elapsed, finished 254 iteration\n",
      "[LightGBM] Iteration:255, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.539756\n",
      "[LightGBM] 8.469419 seconds elapsed, finished 255 iteration\n",
      "[LightGBM] Iteration:256, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.539732\n",
      "[LightGBM] 8.533498 seconds elapsed, finished 256 iteration\n",
      "[LightGBM] Iteration:257, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.539724\n",
      "[LightGBM] 8.641893 seconds elapsed, finished 257 iteration\n",
      "[LightGBM] Iteration:258, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.539726\n",
      "[LightGBM] 8.649187 seconds elapsed, finished 258 iteration\n",
      "[LightGBM] Iteration:259, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.539730\n",
      "[LightGBM] 8.656367 seconds elapsed, finished 259 iteration\n",
      "[LightGBM] Iteration:260, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.539723\n",
      "[LightGBM] 8.664829 seconds elapsed, finished 260 iteration\n",
      "[LightGBM] Iteration:261, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.539702\n",
      "[LightGBM] 8.674680 seconds elapsed, finished 261 iteration\n",
      "[LightGBM] Iteration:262, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.539689\n",
      "[LightGBM] 8.684390 seconds elapsed, finished 262 iteration\n",
      "[LightGBM] Iteration:263, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.539697\n",
      "[LightGBM] 8.692255 seconds elapsed, finished 263 iteration\n",
      "[LightGBM] Iteration:264, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.539689\n",
      "[LightGBM] 8.701459 seconds elapsed, finished 264 iteration\n",
      "[LightGBM] Iteration:265, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.539658\n",
      "[LightGBM] 8.731286 seconds elapsed, finished 265 iteration\n",
      "[LightGBM] Iteration:266, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.539643\n",
      "[LightGBM] 8.742383 seconds elapsed, finished 266 iteration\n",
      "[LightGBM] Iteration:267, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.539621\n",
      "[LightGBM] 8.752429 seconds elapsed, finished 267 iteration\n",
      "[LightGBM] Iteration:268, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.539623\n",
      "[LightGBM] 8.761838 seconds elapsed, finished 268 iteration\n",
      "[LightGBM] Iteration:269, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.539602\n",
      "[LightGBM] 8.774985 seconds elapsed, finished 269 iteration\n",
      "[LightGBM] Iteration:270, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.539592\n",
      "[LightGBM] 8.787088 seconds elapsed, finished 270 iteration\n",
      "[LightGBM] Iteration:271, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.539580\n",
      "[LightGBM] 8.799238 seconds elapsed, finished 271 iteration\n",
      "[LightGBM] Iteration:272, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.539507\n",
      "[LightGBM] 8.819215 seconds elapsed, finished 272 iteration\n",
      "[LightGBM] Iteration:273, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.539499\n",
      "[LightGBM] 8.834964 seconds elapsed, finished 273 iteration\n",
      "[LightGBM] Iteration:274, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.539488\n",
      "[LightGBM] 8.845342 seconds elapsed, finished 274 iteration\n",
      "[LightGBM] Iteration:275, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.539490\n",
      "[LightGBM] 8.853916 seconds elapsed, finished 275 iteration\n",
      "[LightGBM] Iteration:276, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.539443\n",
      "[LightGBM] 8.873427 seconds elapsed, finished 276 iteration\n",
      "[LightGBM] Iteration:277, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.539388\n",
      "[LightGBM] 8.928740 seconds elapsed, finished 277 iteration\n",
      "[LightGBM] Iteration:278, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.539376\n",
      "[LightGBM] 9.136093 seconds elapsed, finished 278 iteration\n",
      "[LightGBM] Iteration:279, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.539357\n",
      "[LightGBM] 9.276365 seconds elapsed, finished 279 iteration\n",
      "[LightGBM] Iteration:280, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.539339\n",
      "[LightGBM] 9.288837 seconds elapsed, finished 280 iteration\n",
      "[LightGBM] Iteration:281, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.539338\n",
      "[LightGBM] 9.298241 seconds elapsed, finished 281 iteration\n",
      "[LightGBM] Iteration:282, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.539316\n",
      "[LightGBM] 9.310107 seconds elapsed, finished 282 iteration\n",
      "[LightGBM] Iteration:283, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.539317\n",
      "[LightGBM] 9.325667 seconds elapsed, finished 283 iteration\n",
      "[LightGBM] Iteration:284, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.539306\n",
      "[LightGBM] 9.340356 seconds elapsed, finished 284 iteration\n",
      "[LightGBM] Iteration:285, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.539290\n",
      "[LightGBM] 9.357583 seconds elapsed, finished 285 iteration\n",
      "[LightGBM] Iteration:286, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.539280\n",
      "[LightGBM] 9.471726 seconds elapsed, finished 286 iteration\n",
      "[LightGBM] Iteration:287, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.539272\n",
      "[LightGBM] 9.480544 seconds elapsed, finished 287 iteration\n",
      "[LightGBM] Iteration:288, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.539278\n",
      "[LightGBM] 9.491781 seconds elapsed, finished 288 iteration\n",
      "[LightGBM] Iteration:289, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.539278\n",
      "[LightGBM] 9.502094 seconds elapsed, finished 289 iteration\n",
      "[LightGBM] Iteration:290, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.539278\n",
      "[LightGBM] 9.513254 seconds elapsed, finished 290 iteration\n",
      "[LightGBM] Iteration:291, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.539270\n",
      "[LightGBM] 9.524020 seconds elapsed, finished 291 iteration\n",
      "[LightGBM] Iteration:292, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.539281\n",
      "[LightGBM] 9.533233 seconds elapsed, finished 292 iteration\n",
      "[LightGBM] Iteration:293, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.539274\n",
      "[LightGBM] 9.542725 seconds elapsed, finished 293 iteration\n",
      "[LightGBM] Iteration:294, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.539255\n",
      "[LightGBM] 9.556829 seconds elapsed, finished 294 iteration\n",
      "[LightGBM] Iteration:295, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.539240\n",
      "[LightGBM] 9.570236 seconds elapsed, finished 295 iteration\n",
      "[LightGBM] Iteration:296, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.539217\n",
      "[LightGBM] 9.590220 seconds elapsed, finished 296 iteration\n",
      "[LightGBM] Iteration:297, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.539191\n",
      "[LightGBM] 9.602787 seconds elapsed, finished 297 iteration\n",
      "[LightGBM] Iteration:298, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.539143\n",
      "[LightGBM] 9.622246 seconds elapsed, finished 298 iteration\n",
      "[LightGBM] Iteration:299, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.539106\n",
      "[LightGBM] 9.639983 seconds elapsed, finished 299 iteration\n",
      "[LightGBM] Iteration:300, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.539097\n",
      "[LightGBM] 9.650265 seconds elapsed, finished 300 iteration\n",
      "[LightGBM] Iteration:301, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.539089\n",
      "[LightGBM] 9.660032 seconds elapsed, finished 301 iteration\n",
      "[LightGBM] Iteration:302, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.539087\n",
      "[LightGBM] 9.669060 seconds elapsed, finished 302 iteration\n",
      "[LightGBM] Iteration:303, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.539081\n",
      "[LightGBM] 9.684102 seconds elapsed, finished 303 iteration\n",
      "[LightGBM] Iteration:304, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.539078\n",
      "[LightGBM] 9.697104 seconds elapsed, finished 304 iteration\n",
      "[LightGBM] Iteration:305, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.539074\n",
      "[LightGBM] 9.708771 seconds elapsed, finished 305 iteration\n",
      "[LightGBM] Iteration:306, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.539074\n",
      "[LightGBM] 9.718968 seconds elapsed, finished 306 iteration\n",
      "[LightGBM] Iteration:307, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.539065\n",
      "[LightGBM] 9.728761 seconds elapsed, finished 307 iteration\n",
      "[LightGBM] Iteration:308, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.539056\n",
      "[LightGBM] 9.738583 seconds elapsed, finished 308 iteration\n",
      "[LightGBM] Iteration:309, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.539044\n",
      "[LightGBM] 9.752109 seconds elapsed, finished 309 iteration\n",
      "[LightGBM] Iteration:310, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.539044\n",
      "[LightGBM] 9.764851 seconds elapsed, finished 310 iteration\n",
      "[LightGBM] Iteration:311, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.539050\n",
      "[LightGBM] 9.775039 seconds elapsed, finished 311 iteration\n",
      "[LightGBM] Iteration:312, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.539036\n",
      "[LightGBM] 9.785636 seconds elapsed, finished 312 iteration\n",
      "[LightGBM] Iteration:313, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.539027\n",
      "[LightGBM] 9.801230 seconds elapsed, finished 313 iteration\n",
      "[LightGBM] Iteration:314, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.539020\n",
      "[LightGBM] 9.814831 seconds elapsed, finished 314 iteration\n",
      "[LightGBM] Iteration:315, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.539015\n",
      "[LightGBM] 9.825750 seconds elapsed, finished 315 iteration\n",
      "[LightGBM] Iteration:316, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.538986\n",
      "[LightGBM] 9.842238 seconds elapsed, finished 316 iteration\n",
      "[LightGBM] Iteration:317, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.538979\n",
      "[LightGBM] 9.851934 seconds elapsed, finished 317 iteration\n",
      "[LightGBM] Iteration:318, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.538976\n",
      "[LightGBM] 9.863001 seconds elapsed, finished 318 iteration\n",
      "[LightGBM] Iteration:319, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.538944\n",
      "[LightGBM] 9.876362 seconds elapsed, finished 319 iteration\n",
      "[LightGBM] Iteration:320, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.538929\n",
      "[LightGBM] 9.888547 seconds elapsed, finished 320 iteration\n",
      "[LightGBM] Iteration:321, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.538920\n",
      "[LightGBM] 9.898492 seconds elapsed, finished 321 iteration\n",
      "[LightGBM] Iteration:322, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.538893\n",
      "[LightGBM] 9.918416 seconds elapsed, finished 322 iteration\n",
      "[LightGBM] Iteration:323, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.538877\n",
      "[LightGBM] 9.930932 seconds elapsed, finished 323 iteration\n",
      "[LightGBM] Iteration:324, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.538862\n",
      "[LightGBM] 9.948159 seconds elapsed, finished 324 iteration\n",
      "[LightGBM] Iteration:325, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.538854\n",
      "[LightGBM] 9.966158 seconds elapsed, finished 325 iteration\n",
      "[LightGBM] Iteration:326, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.538825\n",
      "[LightGBM] 9.980444 seconds elapsed, finished 326 iteration\n",
      "[LightGBM] Iteration:327, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.538820\n",
      "[LightGBM] 10.028179 seconds elapsed, finished 327 iteration\n",
      "[LightGBM] Iteration:328, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.538823\n",
      "[LightGBM] 10.086198 seconds elapsed, finished 328 iteration\n",
      "[LightGBM] Iteration:329, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.538821\n",
      "[LightGBM] 10.105822 seconds elapsed, finished 329 iteration\n",
      "[LightGBM] Iteration:330, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.538812\n",
      "[LightGBM] 10.122252 seconds elapsed, finished 330 iteration\n",
      "[LightGBM] Iteration:331, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.538807\n",
      "[LightGBM] 10.134771 seconds elapsed, finished 331 iteration\n",
      "[LightGBM] Iteration:332, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.538780\n",
      "[LightGBM] 10.150330 seconds elapsed, finished 332 iteration\n",
      "[LightGBM] Iteration:333, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.538777\n",
      "[LightGBM] 10.166435 seconds elapsed, finished 333 iteration\n",
      "[LightGBM] Iteration:334, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.538769\n",
      "[LightGBM] 10.221314 seconds elapsed, finished 334 iteration\n",
      "[LightGBM] Iteration:335, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.538759\n",
      "[LightGBM] 10.357385 seconds elapsed, finished 335 iteration\n",
      "[LightGBM] Iteration:336, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.538761\n",
      "[LightGBM] 10.368089 seconds elapsed, finished 336 iteration\n",
      "[LightGBM] Iteration:337, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.538753\n",
      "[LightGBM] 10.380395 seconds elapsed, finished 337 iteration\n",
      "[LightGBM] Iteration:338, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.538728\n",
      "[LightGBM] 10.412262 seconds elapsed, finished 338 iteration\n",
      "[LightGBM] Iteration:339, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.538718\n",
      "[LightGBM] 10.421282 seconds elapsed, finished 339 iteration\n",
      "[LightGBM] Iteration:340, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.538714\n",
      "[LightGBM] 10.429935 seconds elapsed, finished 340 iteration\n",
      "[LightGBM] Iteration:341, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.538688\n",
      "[LightGBM] 10.449869 seconds elapsed, finished 341 iteration\n",
      "[LightGBM] Iteration:342, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.538682\n",
      "[LightGBM] 10.460077 seconds elapsed, finished 342 iteration\n",
      "[LightGBM] Iteration:343, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.538674\n",
      "[LightGBM] 10.471522 seconds elapsed, finished 343 iteration\n",
      "[LightGBM] Iteration:344, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.538650\n",
      "[LightGBM] 10.486395 seconds elapsed, finished 344 iteration\n",
      "[LightGBM] Iteration:345, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.538658\n",
      "[LightGBM] 10.495525 seconds elapsed, finished 345 iteration\n",
      "[LightGBM] Iteration:346, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.538650\n",
      "[LightGBM] 10.510089 seconds elapsed, finished 346 iteration\n",
      "[LightGBM] Iteration:347, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.538638\n",
      "[LightGBM] 10.520286 seconds elapsed, finished 347 iteration\n",
      "[LightGBM] Iteration:348, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.538636\n",
      "[LightGBM] 10.529352 seconds elapsed, finished 348 iteration\n",
      "[LightGBM] Iteration:349, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.538608\n",
      "[LightGBM] 10.539980 seconds elapsed, finished 349 iteration\n",
      "[LightGBM] Iteration:350, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.538596\n",
      "[LightGBM] 10.552529 seconds elapsed, finished 350 iteration\n",
      "[LightGBM] Iteration:351, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.538572\n",
      "[LightGBM] 10.563620 seconds elapsed, finished 351 iteration\n",
      "[LightGBM] Iteration:352, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.538524\n",
      "[LightGBM] 10.590800 seconds elapsed, finished 352 iteration\n",
      "[LightGBM] Iteration:353, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.538526\n",
      "[LightGBM] 10.608784 seconds elapsed, finished 353 iteration\n",
      "[LightGBM] Iteration:354, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.538491\n",
      "[LightGBM] 10.623068 seconds elapsed, finished 354 iteration\n",
      "[LightGBM] Iteration:355, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.538454\n",
      "[LightGBM] 10.644052 seconds elapsed, finished 355 iteration\n",
      "[LightGBM] Iteration:356, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.538403\n",
      "[LightGBM] 10.660688 seconds elapsed, finished 356 iteration\n",
      "[LightGBM] Iteration:357, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.538388\n",
      "[LightGBM] 10.672496 seconds elapsed, finished 357 iteration\n",
      "[LightGBM] Iteration:358, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.538382\n",
      "[LightGBM] 10.683033 seconds elapsed, finished 358 iteration\n",
      "[LightGBM] Iteration:359, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.538390\n",
      "[LightGBM] 10.699523 seconds elapsed, finished 359 iteration\n",
      "[LightGBM] Iteration:360, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.538385\n",
      "[LightGBM] 10.893703 seconds elapsed, finished 360 iteration\n",
      "[LightGBM] Iteration:361, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.538369\n",
      "[LightGBM] 10.918941 seconds elapsed, finished 361 iteration\n",
      "[LightGBM] Iteration:362, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.538347\n",
      "[LightGBM] 10.939006 seconds elapsed, finished 362 iteration\n",
      "[LightGBM] Iteration:363, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.538342\n",
      "[LightGBM] 10.959076 seconds elapsed, finished 363 iteration\n",
      "[LightGBM] Iteration:364, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.538312\n",
      "[LightGBM] 11.101232 seconds elapsed, finished 364 iteration\n",
      "[LightGBM] Iteration:365, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.538313\n",
      "[LightGBM] 11.231749 seconds elapsed, finished 365 iteration\n",
      "[LightGBM] Iteration:366, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.538273\n",
      "[LightGBM] 11.279228 seconds elapsed, finished 366 iteration\n",
      "[LightGBM] Iteration:367, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.538268\n",
      "[LightGBM] 11.288682 seconds elapsed, finished 367 iteration\n",
      "[LightGBM] Iteration:368, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.538268\n",
      "[LightGBM] 11.296513 seconds elapsed, finished 368 iteration\n",
      "[LightGBM] Iteration:369, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.538265\n",
      "[LightGBM] 11.311895 seconds elapsed, finished 369 iteration\n",
      "[LightGBM] Iteration:370, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.538256\n",
      "[LightGBM] 11.326469 seconds elapsed, finished 370 iteration\n",
      "[LightGBM] Iteration:371, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.538219\n",
      "[LightGBM] 11.348671 seconds elapsed, finished 371 iteration\n",
      "[LightGBM] Iteration:372, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.538187\n",
      "[LightGBM] 11.362759 seconds elapsed, finished 372 iteration\n",
      "[LightGBM] Iteration:373, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.538169\n",
      "[LightGBM] 11.371819 seconds elapsed, finished 373 iteration\n",
      "[LightGBM] Iteration:374, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.538172\n",
      "[LightGBM] 11.383515 seconds elapsed, finished 374 iteration\n",
      "[LightGBM] Iteration:375, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.538166\n",
      "[LightGBM] 11.391809 seconds elapsed, finished 375 iteration\n",
      "[LightGBM] Iteration:376, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.538153\n",
      "[LightGBM] 11.403380 seconds elapsed, finished 376 iteration\n",
      "[LightGBM] Iteration:377, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.538145\n",
      "[LightGBM] 11.412962 seconds elapsed, finished 377 iteration\n",
      "[LightGBM] Iteration:378, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.538129\n",
      "[LightGBM] 11.422675 seconds elapsed, finished 378 iteration\n",
      "[LightGBM] Iteration:379, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.538133\n",
      "[LightGBM] 11.432124 seconds elapsed, finished 379 iteration\n",
      "[LightGBM] Iteration:380, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.538134\n",
      "[LightGBM] 11.450892 seconds elapsed, finished 380 iteration\n",
      "[LightGBM] Iteration:381, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.538106\n",
      "[LightGBM] 11.469836 seconds elapsed, finished 381 iteration\n",
      "[LightGBM] Iteration:382, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.538100\n",
      "[LightGBM] 11.480548 seconds elapsed, finished 382 iteration\n",
      "[LightGBM] Iteration:383, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.538100\n",
      "[LightGBM] 11.493027 seconds elapsed, finished 383 iteration\n",
      "[LightGBM] Iteration:384, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.538097\n",
      "[LightGBM] 11.506745 seconds elapsed, finished 384 iteration\n",
      "[LightGBM] Iteration:385, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.538097\n",
      "[LightGBM] 11.515271 seconds elapsed, finished 385 iteration\n",
      "[LightGBM] Iteration:386, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.538085\n",
      "[LightGBM] 11.538889 seconds elapsed, finished 386 iteration\n",
      "[LightGBM] Iteration:387, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.538069\n",
      "[LightGBM] 11.559020 seconds elapsed, finished 387 iteration\n",
      "[LightGBM] Iteration:388, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.538064\n",
      "[LightGBM] 11.572432 seconds elapsed, finished 388 iteration\n",
      "[LightGBM] Iteration:389, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.538018\n",
      "[LightGBM] 11.589121 seconds elapsed, finished 389 iteration\n",
      "[LightGBM] Iteration:390, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.537980\n",
      "[LightGBM] 11.604313 seconds elapsed, finished 390 iteration\n",
      "[LightGBM] Iteration:391, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.537966\n",
      "[LightGBM] 11.620368 seconds elapsed, finished 391 iteration\n",
      "[LightGBM] Iteration:392, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.537943\n",
      "[LightGBM] 11.642072 seconds elapsed, finished 392 iteration\n",
      "[LightGBM] Iteration:393, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.537945\n",
      "[LightGBM] 11.654051 seconds elapsed, finished 393 iteration\n",
      "[LightGBM] Iteration:394, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.537924\n",
      "[LightGBM] 11.673125 seconds elapsed, finished 394 iteration\n",
      "[LightGBM] Iteration:395, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.537893\n",
      "[LightGBM] 11.686798 seconds elapsed, finished 395 iteration\n",
      "[LightGBM] Iteration:396, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.537882\n",
      "[LightGBM] 11.699036 seconds elapsed, finished 396 iteration\n",
      "[LightGBM] Iteration:397, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.537883\n",
      "[LightGBM] 11.709443 seconds elapsed, finished 397 iteration\n",
      "[LightGBM] Iteration:398, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.537877\n",
      "[LightGBM] 11.720187 seconds elapsed, finished 398 iteration\n",
      "[LightGBM] Iteration:399, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.537872\n",
      "[LightGBM] 11.729384 seconds elapsed, finished 399 iteration\n",
      "[LightGBM] Iteration:400, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.537887\n",
      "[LightGBM] 11.738616 seconds elapsed, finished 400 iteration\n",
      "[LightGBM] Iteration:401, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.537890\n",
      "[LightGBM] 11.748270 seconds elapsed, finished 401 iteration\n",
      "[LightGBM] Iteration:402, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.537897\n",
      "[LightGBM] 11.757756 seconds elapsed, finished 402 iteration\n",
      "[LightGBM] Iteration:403, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.537892\n",
      "[LightGBM] 11.768997 seconds elapsed, finished 403 iteration\n",
      "[LightGBM] Iteration:404, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.537887\n",
      "[LightGBM] 11.779787 seconds elapsed, finished 404 iteration\n",
      "[LightGBM] Iteration:405, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.537868\n",
      "[LightGBM] 11.790370 seconds elapsed, finished 405 iteration\n",
      "[LightGBM] Iteration:406, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.537854\n",
      "[LightGBM] 11.805526 seconds elapsed, finished 406 iteration\n",
      "[LightGBM] Iteration:407, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.537854\n",
      "[LightGBM] 11.822022 seconds elapsed, finished 407 iteration\n",
      "[LightGBM] Iteration:408, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.537856\n",
      "[LightGBM] 11.835247 seconds elapsed, finished 408 iteration\n",
      "[LightGBM] Iteration:409, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.537851\n",
      "[LightGBM] 11.844861 seconds elapsed, finished 409 iteration\n",
      "[LightGBM] Iteration:410, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.537852\n",
      "[LightGBM] 11.862151 seconds elapsed, finished 410 iteration\n",
      "[LightGBM] Iteration:411, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.537837\n",
      "[LightGBM] 11.894948 seconds elapsed, finished 411 iteration\n",
      "[LightGBM] Iteration:412, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.537827\n",
      "[LightGBM] 11.902790 seconds elapsed, finished 412 iteration\n",
      "[LightGBM] Iteration:413, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.537839\n",
      "[LightGBM] 11.914595 seconds elapsed, finished 413 iteration\n",
      "[LightGBM] Iteration:414, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.537830\n",
      "[LightGBM] 11.925855 seconds elapsed, finished 414 iteration\n",
      "[LightGBM] Iteration:415, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.537816\n",
      "[LightGBM] 11.936476 seconds elapsed, finished 415 iteration\n",
      "[LightGBM] Iteration:416, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.537787\n",
      "[LightGBM] 11.948352 seconds elapsed, finished 416 iteration\n",
      "[LightGBM] Iteration:417, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.537785\n",
      "[LightGBM] 11.965180 seconds elapsed, finished 417 iteration\n",
      "[LightGBM] Iteration:418, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.537778\n",
      "[LightGBM] 11.973938 seconds elapsed, finished 418 iteration\n",
      "[LightGBM] Iteration:419, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.537772\n",
      "[LightGBM] 11.986289 seconds elapsed, finished 419 iteration\n",
      "[LightGBM] Iteration:420, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.537727\n",
      "[LightGBM] 11.997849 seconds elapsed, finished 420 iteration\n",
      "[LightGBM] Iteration:421, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.537723\n",
      "[LightGBM] 12.006696 seconds elapsed, finished 421 iteration\n",
      "[LightGBM] Iteration:422, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.537721\n",
      "[LightGBM] 12.014918 seconds elapsed, finished 422 iteration\n",
      "[LightGBM] Iteration:423, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.537721\n",
      "[LightGBM] 12.024063 seconds elapsed, finished 423 iteration\n",
      "[LightGBM] Iteration:424, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.537714\n",
      "[LightGBM] 12.034107 seconds elapsed, finished 424 iteration\n",
      "[LightGBM] Iteration:425, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.537708\n",
      "[LightGBM] 12.043793 seconds elapsed, finished 425 iteration\n",
      "[LightGBM] Iteration:426, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.537701\n",
      "[LightGBM] 12.052315 seconds elapsed, finished 426 iteration\n",
      "[LightGBM] Iteration:427, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.537664\n",
      "[LightGBM] 12.070382 seconds elapsed, finished 427 iteration\n",
      "[LightGBM] Iteration:428, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.537659\n",
      "[LightGBM] 12.084006 seconds elapsed, finished 428 iteration\n",
      "[LightGBM] Iteration:429, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.537654\n",
      "[LightGBM] 12.094590 seconds elapsed, finished 429 iteration\n",
      "[LightGBM] Iteration:430, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.537649\n",
      "[LightGBM] 12.105326 seconds elapsed, finished 430 iteration\n",
      "[LightGBM] Iteration:431, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.537660\n",
      "[LightGBM] 12.113340 seconds elapsed, finished 431 iteration\n",
      "[LightGBM] Iteration:432, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.537654\n",
      "[LightGBM] 12.123459 seconds elapsed, finished 432 iteration\n",
      "[LightGBM] Iteration:433, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.537639\n",
      "[LightGBM] 12.137068 seconds elapsed, finished 433 iteration\n",
      "[LightGBM] Iteration:434, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.537631\n",
      "[LightGBM] 12.146676 seconds elapsed, finished 434 iteration\n",
      "[LightGBM] Iteration:435, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.537622\n",
      "[LightGBM] 12.156374 seconds elapsed, finished 435 iteration\n",
      "[LightGBM] Iteration:436, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.537611\n",
      "[LightGBM] 12.166170 seconds elapsed, finished 436 iteration\n",
      "[LightGBM] Iteration:437, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.537599\n",
      "[LightGBM] 12.179831 seconds elapsed, finished 437 iteration\n",
      "[LightGBM] Iteration:438, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.537589\n",
      "[LightGBM] 12.192991 seconds elapsed, finished 438 iteration\n",
      "[LightGBM] Iteration:439, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.537554\n",
      "[LightGBM] 12.211667 seconds elapsed, finished 439 iteration\n",
      "[LightGBM] Iteration:440, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.537526\n",
      "[LightGBM] 12.226523 seconds elapsed, finished 440 iteration\n",
      "[LightGBM] Iteration:441, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.537509\n",
      "[LightGBM] 12.244541 seconds elapsed, finished 441 iteration\n",
      "[LightGBM] Iteration:442, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.537492\n",
      "[LightGBM] 12.254579 seconds elapsed, finished 442 iteration\n",
      "[LightGBM] Iteration:443, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.537494\n",
      "[LightGBM] 12.264266 seconds elapsed, finished 443 iteration\n",
      "[LightGBM] Iteration:444, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.537482\n",
      "[LightGBM] 12.274440 seconds elapsed, finished 444 iteration\n",
      "[LightGBM] Iteration:445, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.537459\n",
      "[LightGBM] 12.290710 seconds elapsed, finished 445 iteration\n",
      "[LightGBM] Iteration:446, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.537446\n",
      "[LightGBM] 12.303992 seconds elapsed, finished 446 iteration\n",
      "[LightGBM] Iteration:447, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.537413\n",
      "[LightGBM] 12.318690 seconds elapsed, finished 447 iteration\n",
      "[LightGBM] Iteration:448, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.537395\n",
      "[LightGBM] 12.331088 seconds elapsed, finished 448 iteration\n",
      "[LightGBM] Iteration:449, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.537390\n",
      "[LightGBM] 12.340651 seconds elapsed, finished 449 iteration\n",
      "[LightGBM] Iteration:450, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.537384\n",
      "[LightGBM] 12.358073 seconds elapsed, finished 450 iteration\n",
      "[LightGBM] Iteration:451, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.537381\n",
      "[LightGBM] 12.368351 seconds elapsed, finished 451 iteration\n",
      "[LightGBM] Iteration:452, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.537370\n",
      "[LightGBM] 12.377560 seconds elapsed, finished 452 iteration\n",
      "[LightGBM] Iteration:453, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.537361\n",
      "[LightGBM] 12.388637 seconds elapsed, finished 453 iteration\n",
      "[LightGBM] Iteration:454, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.537358\n",
      "[LightGBM] 12.397817 seconds elapsed, finished 454 iteration\n",
      "[LightGBM] Iteration:455, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.537360\n",
      "[LightGBM] 12.407939 seconds elapsed, finished 455 iteration\n",
      "[LightGBM] Iteration:456, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.537366\n",
      "[LightGBM] 12.421532 seconds elapsed, finished 456 iteration\n",
      "[LightGBM] Iteration:457, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.537371\n",
      "[LightGBM] 12.431327 seconds elapsed, finished 457 iteration\n",
      "[LightGBM] Iteration:458, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.537368\n",
      "[LightGBM] 12.440788 seconds elapsed, finished 458 iteration\n",
      "[LightGBM] Iteration:459, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.537366\n",
      "[LightGBM] 12.453518 seconds elapsed, finished 459 iteration\n",
      "[LightGBM] Iteration:460, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.537364\n",
      "[LightGBM] 12.466761 seconds elapsed, finished 460 iteration\n",
      "[LightGBM] Iteration:461, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.537363\n",
      "[LightGBM] 12.474869 seconds elapsed, finished 461 iteration\n",
      "[LightGBM] Iteration:462, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.537351\n",
      "[LightGBM] 12.487262 seconds elapsed, finished 462 iteration\n",
      "[LightGBM] Iteration:463, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.537322\n",
      "[LightGBM] 12.500923 seconds elapsed, finished 463 iteration\n",
      "[LightGBM] Iteration:464, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.537276\n",
      "[LightGBM] 12.518967 seconds elapsed, finished 464 iteration\n",
      "[LightGBM] Iteration:465, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.537236\n",
      "[LightGBM] 12.535918 seconds elapsed, finished 465 iteration\n",
      "[LightGBM] Iteration:466, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.537224\n",
      "[LightGBM] 12.548916 seconds elapsed, finished 466 iteration\n",
      "[LightGBM] Iteration:467, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.537218\n",
      "[LightGBM] 12.561045 seconds elapsed, finished 467 iteration\n",
      "[LightGBM] Iteration:468, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.537205\n",
      "[LightGBM] 12.577229 seconds elapsed, finished 468 iteration\n",
      "[LightGBM] Iteration:469, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.537189\n",
      "[LightGBM] 12.593562 seconds elapsed, finished 469 iteration\n",
      "[LightGBM] Iteration:470, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.537193\n",
      "[LightGBM] 12.624073 seconds elapsed, finished 470 iteration\n",
      "[LightGBM] Iteration:471, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.537187\n",
      "[LightGBM] 12.657781 seconds elapsed, finished 471 iteration\n",
      "[LightGBM] Iteration:472, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.537189\n",
      "[LightGBM] 12.666865 seconds elapsed, finished 472 iteration\n",
      "[LightGBM] Iteration:473, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.537178\n",
      "[LightGBM] 12.676029 seconds elapsed, finished 473 iteration\n",
      "[LightGBM] Iteration:474, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.537189\n",
      "[LightGBM] 12.687526 seconds elapsed, finished 474 iteration\n",
      "[LightGBM] Iteration:475, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.537176\n",
      "[LightGBM] 12.705473 seconds elapsed, finished 475 iteration\n",
      "[LightGBM] Iteration:476, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.537174\n",
      "[LightGBM] 12.783040 seconds elapsed, finished 476 iteration\n",
      "[LightGBM] Iteration:477, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.537131\n",
      "[LightGBM] 12.800212 seconds elapsed, finished 477 iteration\n",
      "[LightGBM] Iteration:478, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.537097\n",
      "[LightGBM] 12.816976 seconds elapsed, finished 478 iteration\n",
      "[LightGBM] Iteration:479, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.537097\n",
      "[LightGBM] 12.830070 seconds elapsed, finished 479 iteration\n",
      "[LightGBM] Iteration:480, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.537094\n",
      "[LightGBM] 12.846662 seconds elapsed, finished 480 iteration\n",
      "[LightGBM] Iteration:481, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.537096\n",
      "[LightGBM] 12.856778 seconds elapsed, finished 481 iteration\n",
      "[LightGBM] Iteration:482, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.537092\n",
      "[LightGBM] 12.867583 seconds elapsed, finished 482 iteration\n",
      "[LightGBM] Iteration:483, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.537098\n",
      "[LightGBM] 12.892091 seconds elapsed, finished 483 iteration\n",
      "[LightGBM] Iteration:484, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.537108\n",
      "[LightGBM] 12.990557 seconds elapsed, finished 484 iteration\n",
      "[LightGBM] Iteration:485, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.537103\n",
      "[LightGBM] 13.019780 seconds elapsed, finished 485 iteration\n",
      "[LightGBM] Iteration:486, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.537085\n",
      "[LightGBM] 13.042225 seconds elapsed, finished 486 iteration\n",
      "[LightGBM] Iteration:487, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.537091\n",
      "[LightGBM] 13.054261 seconds elapsed, finished 487 iteration\n",
      "[LightGBM] Iteration:488, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.537081\n",
      "[LightGBM] 13.073595 seconds elapsed, finished 488 iteration\n",
      "[LightGBM] Iteration:489, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.537093\n",
      "[LightGBM] 13.083198 seconds elapsed, finished 489 iteration\n",
      "[LightGBM] Iteration:490, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.537088\n",
      "[LightGBM] 13.215157 seconds elapsed, finished 490 iteration\n",
      "[LightGBM] Iteration:491, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.537087\n",
      "[LightGBM] 13.224314 seconds elapsed, finished 491 iteration\n",
      "[LightGBM] Iteration:492, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.537073\n",
      "[LightGBM] 13.240534 seconds elapsed, finished 492 iteration\n",
      "[LightGBM] Iteration:493, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.537073\n",
      "[LightGBM] 13.248472 seconds elapsed, finished 493 iteration\n",
      "[LightGBM] Iteration:494, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.537066\n",
      "[LightGBM] 13.258491 seconds elapsed, finished 494 iteration\n",
      "[LightGBM] Iteration:495, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.537054\n",
      "[LightGBM] 13.269816 seconds elapsed, finished 495 iteration\n",
      "[LightGBM] Iteration:496, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.537032\n",
      "[LightGBM] 13.283366 seconds elapsed, finished 496 iteration\n",
      "[LightGBM] Iteration:497, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.537030\n",
      "[LightGBM] 13.293556 seconds elapsed, finished 497 iteration\n",
      "[LightGBM] Iteration:498, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.536992\n",
      "[LightGBM] 13.311290 seconds elapsed, finished 498 iteration\n",
      "[LightGBM] Iteration:499, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.536991\n",
      "[LightGBM] 13.326224 seconds elapsed, finished 499 iteration\n",
      "[LightGBM] Iteration:500, /tmp/tmphwxvp02b/X0_test.svm's l2 loss : 0.536986\n",
      "[LightGBM] 13.360080 seconds elapsed, finished 500 iteration\n",
      "[LightGBM] finish train\n",
      "\n",
      "[LightGBM] finished load parameters\n",
      "[LightGBM] Loaded 500 modles\n",
      "\n",
      "[LightGBM] finish predict init\n",
      "[LightGBM] start prediction for data /tmp/tmp91ip_p6w/X_to_pred.svm, and data has label\n",
      "[LightGBM] finish predict\n",
      "\n",
      "Mean Square Error: 1147.47495097\n"
     ]
    }
   ],
   "source": [
    "seed = 42\n",
    "\n",
    "gbmr = GBMRegressor(\n",
    "    exec_path='/path/to/your/LightGBM/lightgbm', # change this to your LighGBM path\n",
    "    config='',\n",
    "    application='regression',\n",
    "    num_iterations=500,\n",
    "    learning_rate=0.1,\n",
    "    num_leaves=10,\n",
    "    tree_learner='serial',\n",
    "    num_threads=4,\n",
    "    min_data_in_leaf=10,\n",
    "    metric='l2',\n",
    "    feature_fraction=1.0,\n",
    "    feature_fraction_seed=seed,\n",
    "    bagging_fraction=1.0,\n",
    "    bagging_freq=0,\n",
    "    bagging_seed=seed,\n",
    "    metric_freq=1,\n",
    "    early_stopping_round=10\n",
    ")\n",
    "\n",
    "gbmr.fit(X_train, y_train, test_data=[(X_valid, y_valid)])\n",
    "print(\"Mean Square Error:\", metrics.mean_absolute_error(y_true=(np.exp(y_valid)-1), y_pred=(np.exp(gbmr.predict(X_valid))-1)))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Best round 500\n"
     ]
    }
   ],
   "source": [
    "print('Best round', gbmr.best_round)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "FEATURE IMPORTANCE\n",
    "-------\n",
    "TOP 10"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.text.Text at 0x7f39bdc4d400>"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f39be803550>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAnoAAAGJCAYAAAAdRJ4mAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XucXXV97//Xm3CJKLeoKXKLyNEW9KSIaEmiTUyPVnoU\nD6hVQEGqtrbn4A1Qf/w4AvVwOAraIp6WtkqgRKmCUUHBQoXBFkW5BIKAIpUSRAUqAaI2CMnn/LHX\nxO04mUsye/bMmtfz8ZjH7LXWd33XZ61Zj/Dmuy47VYUkSZLaZ6t+FyBJkqTeMOhJkiS1lEFPkiSp\npQx6kiRJLWXQkyRJaimDniRJUksZ9CRJklrKoCfNMEmenOTuJId3zXtKknuSHNY178AklyZ5qPn5\ndpIPJtmpWX50kieSPNr83JXk7T2ufXGSe0dpsyzJY01Na5vfr5uAbW9I8qwt7Wcc2zs6yT9P1vZG\nMpbjLmlqMuhJM0xV/Qz4E+CsJE9tZp8BfKuqVgAkWQhcDfwz8JtVNQd4BfAE8Ntd3X29qnasqh2B\n1wIfTtK9fKIFGMtb3j/U1LVD8/uiCdj2Fr1dPsnm/Hvb9zfaJ5nF2I+7pCnGoCfNQFV1BfAl4Owk\ni+mEtD/ravIh4JNV9eGqerBZ5wdVdWpVfW0Tfd4M3AHsOzgvySHNSOBDSa5K8ltdy34rydVJ1iS5\nNcmrupb9QZLbmtG4e5O8J8n2wGXAbl0jdbuOZ7+TPCPJxUkeSPKvSY7tWvbCJF9v6rkvydlJtm6W\nXUMn7KwaHCEcbsSte9SvGVn8qyRfTrIWWJJk2yRnNqOnP2qWbzfG2u9OcnySwRo+kWRuksuSPJLk\niq7R1nlNLW9r9uW+JMd19bVtkr9s5v8gyV8k2aZZtrg55u9N8iPg08Md95GOV9ex+JMkdzZ//48P\n2Z+3Jbm96e/bSfYf7W8kafwMetLM9R5gCXAxcNxgoGsC1QJgxXg6S/JC4NnADc30c+iEhHcATwcu\nBy5NsnUTCC4FvtIsewfwqSTPbrr7BPC2ZqTwecBVVfVz4GDgh10jdT8eR31ptrkSeAbwe8A7k7ys\nabIeeBcwp9n/pTTht6oWN23+85ARwqGjXEOnDwc+WFU7ANfSCdD/CZjf/N4d+MBY9wE4rKnrN4FX\n0Tmm76dzDGfROY7dlgD7AL8PvC/J0mb+ScCLmjp+u/l8Utd6uwI7A3sBRzH8cd/k8eryX4EXNNv4\nwyQvB2gupX8AeGPzNz4E+MkY/kaSxsmgJ81QVfUwcBvwJODzXYt2ofNvw8YQleRDzcjNT5Oc2NV2\nQTNa8yhwHXBBVd3VLPtD4EtVdVVVrQfOBGYDC4GDgCdX1Yeq6omquprOCOPgfYO/AJ6bZIeqeqQZ\nLRyPE5q61iR5oJn3IuBpVXVaVa2vqn+jEyjf0ByPm6rqW9WxGvhbYPGQfjPKdocu/2JVXdf0/xjw\nNuDdzT79DPg/Xfs8FmdX1b9X1Y/oXFa/rqpWVdUv6PwNnz+k/SlVta6qvg0s69rWEcCpVfWTqvoJ\ncCrwpq711gMnV9XjTd2/ZozH6/SqWltV99K5FWD/Zv5bgA9X1U1NX99v2ryQEf5GksbPoCfNUEne\nCMwD/gn4cNeiNcAGOiMqAFTV+6pqFzphYuuutt+oqjnNqMyuwPOSnNYs2w24p6uPAn5AZxRrN2Do\nzf33NMsAXkNnNOie5vLuQePcvTOaunapqrnNvL2A3fPLh0vWAP8fMLc5Hs9O5+GTHyV5GDgNeNo4\ntzvUxn1M8nRge+DGwRrojMg9dVMrD+P+rs//Mcz0U7qmB4/3oHvoHHea36s3sQzgwap6fKRCxni8\nuuv7eVd9ewL/Oky38xjhbyRp/Ax60gyUZC7wUeCtwNuB1yVZBNBcIv0mncuEY9Zc+v0cnUuKAD+k\n8x/ubnsC9zXL9hqybK9mGVV1Y1X9NzqXJL8IfHZwM+OpaYh7ge83AXAwBO5UVYP1/jWdewz3qaqd\ngf+fkUfwfkYnuAGQ4e8X7K733+mEned21bBzVe20Bfs0ktA53oP2onPc4df/NvO6lsHol6Rh/Mer\n2710LikPN3+kv5GkcTLoSTPTx4EVVfW15n6r9wGfGLwhH3gv8EfNDflPB0iyB7D3kH42/oc9nSd4\nDwW+3cz6LPBfk7y0uS/veGAd8HU6QfJnTf9bJ1kCvBK4MMk2SY5IsmNzyXctnUuJ0BkhemqSHTdj\nn78FrG22OTvJrCTPTXJgs3wH4NGq+nk6D4386ZD1fwx0v17lFjqXl+c3D1SczAhBtBnR/DvgL7uO\n6e6D9631yP9M8qQkzwWOAf6hmX8hcFKSpyV5GvA/gQtG6Ge44z7a8RrJJ4DjkxwAkGSfJHsy+t9I\n0jgZ9KQZJsmr6dwn997BeVX1STqjaR9opq+lc3P9YuC7zWXGy+jcZ3V2V3cHNU9NPkrnfr/7aR4I\nqKo7gTfSCZUP0rkU+6rmnrzH6Yz8/QGdka6PA2+qqu81/b4JuLu5JPjHwJFNn9+lE1K+31zaG20U\n7ZczqzbQCZP7A3cDD9AJXoPh5XjgyGZf/oZfhqJBpwB/32z3tU2tfw58FbiTzj1zo3kfcBdwXbNv\nVwDPGcN6w+3XWEY3r2m2dyWde+K+2sz/X3QemllFJ7DeQOfS6/AbHv64j3a8NllvVV3cbO/Tzfqf\nB+aM4W8kaZzS+Z9MSVJbJJkHfB/YpglPkmYoR/QkqZ3Ger+cpBYz6ElSO3m5RpKXbiVJktrKET1J\nkqSW2nr0Ju2TxGFMSZI0bVTVZt13O2NH9KrKn0n8Ofnkk/tew0z78Zh7zGfCj8fcYz4TfrbEjA16\nkiRJbWfQkyRJaimDnibFkiVL+l3CjOMxn3we88nnMZ98HvPpZUa+XiVJzcT9liRJ008SyocxJEmS\n1M2gJ0mS1FIGPWkU62p9v0uQJGmzzMgXJgPs9cDl/S5B08TquQf3uwRJkjaLI3qSJEktZdCTJElq\nqSkT9JIsTrKga/olSW5M8niSw4a0vTzJmiSXbKKvjyVZ2+uaJUmSprIpE/SAJcDCrul7gKOBTw3T\n9sPAG4frJMkLgJ0BX5QnSZJmtJ4HvSRHJbklycok5yd5ZZLrmtG6K5I8Pck84O3Au5LclGRRVa2u\nqm8zTGCrqquBnw6zra2AM4ATer1fkiRJU11Pn7pNsh9wIrCgqtYk2RmoqjqoWf4W4L1VdUKSc4C1\nVfXRLdjk/wC+UFX3J9msN0hLkiS1Ra9fr7IUuKiq1gBU1cNJnpfks8AzgG2AuydiQ0meAbwOWDwR\n/UmSJE13/XiP3tnAmVX15SSLgZMnqN/nA/sAdzWjedsnubOqnjNc40fOWL7x83YL5zN70fwJKkOS\nJGnzDQwMMDAwMCF99TroXQWsSPIXVfVQkjnAjsAPm+VHd7Vd2ywbznCXYdM9v6ouA3bbuDBZu6mQ\nB7DTCcM+yyFJktRXS5YsYcmSJRunTz311M3uq6cPY1TV7cBpwDVJVgJnAqcAFye5Hniwq/mlwKGD\nD2MkOTDJvcBrgXOS3DrYMMnXgM8AS5OsTvKy4Tbfm72SJEmaHlI18/JQktrz/sv6XYamCb8CTZLU\nT0moqs16yHQqvUdPkiRJE8igJ0mS1FIGPUmSpJYy6EmSJLXUjH0YYybutzbPulrP7MzqdxmSpBnK\nhzGkHjLkSZKmK4OeJElSSxn0JEmSWsqgJ0mS1FIGPUmSpJYy6EmSJLWUQU+SJKmlDHqSJEktZdCT\nJElqKYOeJElSSxn0JEmSWsqgJ0mS1FIGPUmSpJYy6EmSJLWUQU+SJKmlDHqSJEktZdCTJsC6Wt/v\nEiRJ+jVb97uAftnrgcv7XYJaZPXcg/tdgiRJv8YRPUmSpJaatkEvybwkh3dNb5fk00lWJbktyfv7\nWZ8kSVK/TdugB+wNHNE1/QaAqpoPHAj8SZK9+lGYJEnSVNC3oJfkqCS3JFmZ5PxmhO6rSW5OcmWS\nPZp2y5KcleTaJHclOazp4nTgxUluSvJO4MfAk5PMArYHHgMe7cvOSZIkTQF9eRgjyX7AicCCqlqT\nZBfgfGBZVS1PcgxwNnBos8quVbUoyb7AJcAK4P3AcVV1SFe/bwR+BDwJeHdVPTx5eyVJkjS19GtE\nbylwUVWtAWh+LwAubJZfACzqav+Fpt0dwNzhOkxyJJ2AtyvwLOD4JM/sQe2SJEnTwlR6vUqNsOyx\nrs/ZRJtFwOeragPwYJJr6dyr92/DNX7kjOUbP2+3cD6zF80fV7GSJEm9MDAwwMDAwIT0laqR8lVv\nNJduVwALq+qhJHOAc4GLm0u3bwZeVVWvSbIMuLSqVjTrrq2qHZIcAHykql7azH8HsH9V/VGSJwPf\nAl5fVd8eZvu15/2XTcq+ambwPXqSpF5JQlVtaqBrRH0Z0auq25OcBlyT5AlgJXAscF6S44EHgWMG\nmw9dvfm9CtiQZCVwHvDXwLlJbqUz6vfJ4UKeJEnSTNGXEb1+c0RPE80RPUlSr2zJiN50fo+eJEmS\nRmDQkyRJaimDniRJUksZ9CRJklrKoCdJktRSM/ap25m43+qddbWe2ZnV7zIkSS3kU7dSnxnyJElT\nkUFPkiSppQx6kiRJLWXQkyRJaimDniRJUksZ9CRJklrKoCdJktRSBj1JkqSWMuhJkiS1lEFPkiSp\npQx6kiRJLWXQkyRJaimDniRJUksZ9CRJklrKoCdJktRSBj2pR9bV+n6XIEma4bbudwH9stcDl/e7\nBLXc6rkH97sESdIM54ieJElSSxn0JEmSWmraBr0k85Ic3jU9J8lVSdYm+Vg/a5MkSZoKpm3QA/YG\njuiaXgecBBzXn3IkSZKmlr4FvSRHJbklycok5zcjdF9NcnOSK5Ps0bRbluSsJNcmuSvJYU0XpwMv\nTnJTkndW1c+r6uvAY/3aJ0mSpKmkL0/dJtkPOBFYUFVrkuwCnA8sq6rlSY4BzgYObVbZtaoWJdkX\nuARYAbwfOK6qDunDLkiSJE15/RrRWwpcVFVrAJrfC4ALm+UXAIu62n+haXcHMHcS65QkSZq2ptJ7\n9GqEZd2XYzMRG3vkjOUbP2+3cD6zF82fiG4lSZK2yMDAAAMDAxPSV7+C3lXAiiR/UVUPJZkDfB04\nHFgOvBH4502sOxj01gI7jNJmk3Y64Y3jq1iSJGkSLFmyhCVLlmycPvXUUze7r74Evaq6PclpwDVJ\nngBWAscC5yU5HngQOGaw+dDVm9+rgA1JVgLnVdVZSe6mE/62TfJq4OVV9Z1e748kSdJUlKqRrpi2\nU5La8/7L+l2GWs6vQJMkTYQkVNVm3bo2nd+jJ0mSpBEY9CRJklrKoCdJktRSBj1JkqSWmrEPY8zE\n/dbkWlfrmZ1Z/S5DkjTN+TCGNAUZ8iRJ/WbQkyRJaimDniRJUksZ9CRJklrKoCdJktRSBj1JkqSW\nMuhJkiS1lEFPkiSppQx6kiRJLWXQkyRJaimDniRJUksZ9CRJklrKoCdJktRSBj1JkqSWMuhJkiS1\nlEFPkiSppQx6Up+sq/X9LkGS1HJb97uAftnrgcv7XYJmuNVzD+53CZKklnNET5IkqaWmfNBLsjjJ\ngq7pfZJ8LcnKJDcnObhr2dFJ7kzy3SRH9adiSZKkqWE6XLpdAvwU+EYzfRLwqar6myT7ApcBeyeZ\nA3wAOAAIcGOSL1bVI32oWZIkqe/6NqKX5KgktzQjc+cneWWS65LcmOSKJE9PMg94O/CuJDclWQT8\nCNix6WZn4L7m88uBK6rqkap6GLgCeMVk75ckSdJU0ZcRvST7AScCC6pqTZKdgaqqg5rlbwHeW1Un\nJDkHWFtVH22WrQK+keQdwPbAf2m63R24t2sz9zXzJEmSZqR+XbpdClxUVWsAqurhJM9L8lngGcA2\nwN2bWPejwCeq6i+THAQsB547GUVLkiRNJ1PpHr2zgTOr6stJFgMnb6LdIuAUgKq6LsnsJE+jM4K3\npKvdHsDVm9rYI2cs3/h5u4Xzmb1o/hYVL0mSNBEGBgYYGBiYkL5SVRPS0bg22rl0uwJYWFUPNQ9S\nXAm8tapWJjkXeGZVLU3yHmDHqjqlWfdzwCVVdX7zMMaVVbVHkl2AG+g8jLFV8/kFzf16Q7dfe95/\n2WTsqrRJvkdPkjQWSaiqbM66fRnRq6rbk5wGXJPkCWAlnVG6i5M8BFwFPLNpfmkz/xDgWOAE4Nwk\n7wY2AEc3fa5J8kE6Aa+AU4cLeZIkSTNFX0b0+s0RPU0FjuhJksZiS0b0pvwLkyVJkrR5DHqSJEkt\nZdCTJElqKYOeJElSSxn0JEmSWmrGPnU7E/dbU8u6Ws/szOp3GZKkKc6nbqVpyJAnSeo1g54kSVJL\nGfQkSZJayqAnSZLUUgY9SZKkljLoSZIktZRBT5IkqaUMepIkSS1l0JMkSWopg54kSVJLGfQkSZJa\nyqAnSZLUUgY9SZKkljLoSZIktZRBT5IkqaUMelKLrav1/S5BktRHW/e7gH7Z64HL+12C1HOr5x7c\n7xIkSX3kiJ4kSVJLGfQkSZJaasoEvSSLkyzomn5JkhuTPJ7ksCFtL0+yJsklQ+YvT/KdJKuSfCLJ\nrMmqX5IkaaqZMkEPWAIs7Jq+Bzga+NQwbT8MvHGY+cur6reqaj6wPfDWiS5SkiRpuuj5wxhJjgKO\nAzYAq4CLgJOAbYCfAEfSCWVvB55IciRwbFVd26xfQ/usqquTLB5m/le6Jr8F7DGxeyNJkjR99DTo\nJdkPOBFYUFVrkuwMVFUd1Cx/C/DeqjohyTnA2qr66ARsd2vgTcA7trQvSZKk6arXI3pLgYuqag1A\nVT2c5HlJPgs8g86o3t092O5fAdcMjgpKkiTNRP14j97ZwJlV9eXm8uvJE9l5kg8AT6uqPx6p3SNn\nLN/4ebuF85m9aP5EliFJkrRZBgYGGBgYmJC+Rg16SX4D+N/AblV1cHM5dkFVfXIM/V8FrEjyF1X1\nUJI5wI7AD5vlR3e1XdssG7aMTcz7lflJ3gr8Pp2RxBHtdMJwz3JIkiT115IlS1iyZMnG6VNPPXWz\n+xrLU7fnAf8I7NZM3wm8ayydV9XtwGnANUlWAmcCpwAXJ7keeLCr+aXAoUluSrIoyYFJ7gVeC5yT\n5NbBhkm+BnwGWJpkdZKXNYv+GpgLXNf0c9JY6pQkSWqjVP3aQ62/2iC5vqpemGRlVT2/mXdzVe0/\nKRX2QJLa8/7L+l2G1HN+BZokTX9JqKrhrm6Oaiwjej9L8lSgmo0dBDyyORuTJEnS5BnLwxjvAS4B\n9klyLfB0OpdTJUmSNIWNGPSSbAXMBhYDv0nn4YfvVtXjk1CbJEmStsCIQa+qNiT5v829ebdNUk2S\nJEmaAGN5GONM4BvAihqt8TSRpC27Io1oXa1ndmb1uwxJ0hbYkocxxhL01gJPBp4A1tG5fFtVtal3\n3k15Bj1JkjRdbEnQG/VhjKraYXM6liRJUn+N5Zsxfne4+VX1tYkvR5IkSRNlLJduL+2anA28CLix\nqkb9mrGpyku3kiRpuuj1pdtXDdnYnsBfbs7GJEmSNHnG8s0YQ/0A2HeiC5EkSdLEGss9emfTfP0Z\nnWC4P3BTL4uSJEnSlhvLV6Dd0PX5CeDCqrq2R/VIkiRpgowl6O1cVWd1z0jyzqHzJEmSNLWM5R69\no4eZ9+YJrkOSJEkTbJMjekkOB44A9k5ySdeiHYCHel2YJEmStsxIl26/DvwIeBrwka75a4FVvSxK\nkiRJW27UFya3kS9MliRJ08WWvDB51Hv0khyU5PokP03yiyTrkzy6ORuTJEnS5BnLwxgfBw4Hvgc8\nCXgr8H97WZQkSZK23Ji+GaOq7gJmVdX6qloGvKK3ZUmSJGlLjeU9ej9Psi1wc5IP03lAY3O+Ok3S\nFLCu1jM7s/pdhiRpEoz6MEaSecD9wLbAu4GdgL9qRvmmpSS15/2X9bsMqS9Wzz243yVIksZhSx7G\nGHVEr6ruSfIk4BlVdermbESSJEmTbyxP3b4KuBn4SjO9/5AXKE+IJIuTLOiafkmSG5M8nuSwIW2P\nTnJnku8mOapr/n9P8r3myeA5E12jJEnSdDKWe+1OAV4EPAxQVTcDe/egliXAwq7pe+h8/dqnuhsl\n2QX4APBC4HeAk5Ps1Cz+F+D3mnUlSZJmtLE8jPF4VT2S/Mql4TG/bbgZcTsO2EDnGzUuAk4CtgF+\nAhwJbA+8HXgiyZHAsVV1bbP+0G39PnBFVT3SLL+CzlPAn6mqW5p5m3UdW5IkqU3GEvRuS3IEMCvJ\ns4F30Pl6tFEl2Q84EVhQVWuS7AxUVR3ULH8L8N6qOiHJOcDaqvroKN3uDtzbNX1fM0+SJEldNnnp\nNskFzcd/BZ4LPAZcCDwKvGuM/S8FLqqqNQBV9TCwZ5J/TLIKOL7pW5IkSRNspBG9FyTZDXg98FLg\nI13LtgfWbeY2zwbOrKovJ1kMnDzO9e+jcz/foD2Aq4e0GfXS8iNnLN/4ebuF85m9aP44y5AkSZp4\nAwMDDAwMTEhfm3yPXpJ3AH8KPItOuNq4iM7l12eN2nnn0u0KYGFVPdQ8CXsl8NaqWpnkXOCZVbU0\nyXuAHavqlCF9LAO+VFWfa6Z3AW4ADqAzInkD8IJmtHBwnbuBA6vqJ5uoy/foacbyPXqSNL1syXv0\nNnnptqo+VlX7AudW1bO6fvYeS8hr+rgdOA24JslK4Ew6T/FenOR64MGu5pcChya5KcmiJAcmuRd4\nLXBOklubPtcAH6QT8L4JnDoY8pIc26yzO3BLkr8dz8GQJElqk1G/GaONHNHTTOaIniRNLz0Z0ZMk\nSdL0ZtCTJElqKYOeJElSSxn0JEmSWsqgJ0mS1FIz9qnbmbjfEsC6Ws/szOp3GZKkMfKpW0ljZsiT\npJnDoCdJktRSBj1JkqSWMuhJkiS1lEFPkiSppQx6kiRJLWXQkyRJaimDniRJUksZ9CRJklrKoCdJ\nktRSBj1JkqSWMuhJkiS1lEFPkiSppQx6kiRJLWXQkyRJaimDnqRJs67W97sESZpRtu53Af2y1wOX\n97sEacZZPffgfpcgSTOKI3qSJEktZdCTJElqqWkb9JLMS3J41/R/SXJDkluSXJ/kpf2sT5Ikqd+m\nbdAD9gaO6Jp+EHhlVf028Gbggn4UJUmSNFX0LeglOaoZfVuZ5PxmhO6rSW5OcmWSPZp2y5KcleTa\nJHclOazp4nTgxUluSvLOqrqlqn4MUFW3AbOTbNOn3ZMkSeq7vjx1m2Q/4ERgQVWtSbILcD6wrKqW\nJzkGOBs4tFll16palGRf4BJgBfB+4LiqOmSY/l8L3FRVj0/G/kiSJE1F/RrRWwpcVFVrAJrfC4AL\nm+UXAIu62n+haXcHMHekjpM8l85o3x9PcM2SJEnTylR6j16NsOyxrs/ZVKPmcu8K4E1V9W8jbeyR\nM5Zv/LzdwvnMXjR/bFVKkiT10MDAAAMDAxPSV6pGyle90Vy6XQEsrKqHkswBzgUubi7dvhl4VVW9\nJsky4NKqWtGsu7aqdkhyAPCRqnppM38n4BrglKr6wijbrz3vv6x3OyhpWL4wWZLGLwlVtcmBrpH0\nZUSvqm5PchpwTZIngJXAscB5SY6n8wTtMYPNh67e/F4FbEiyEjgPeAqwD/CBJCc37V5eVf/e052R\nJEmaovoyotdvjuhJ/eGIniSN35aM6E3n9+hJkiRpBAY9SZKkljLoSZIktZRBT5IkqaVm7MMYM3G/\npX5bV+uZnVn9LkOSphUfxpA0LRjyJGlyGfQkSZJayqAnSZLUUgY9SZKkljLoSZIktZRBT5IkqaUM\nepIkSS1l0JMkSWopg54kSVJLGfQkSZJayqAnSZLUUgY9SZKkljLoSZIktZRBT5IkqaUMepIkSS1l\n0JMkSWopg54kbcK6Wt/vEiRpi2zd7wL6Za8HLu93CZKmuNVzD+53CZK0RRzRkyRJaqlpG/SSzEty\neNf0EUlWJrmp+b0+yfx+1ihJktRP0zboAXsDRwxOVNWnq+r5VXUA8Cbg+1W1qm/VSZIk9Vnfgl6S\no5Lc0oy+nd+M0H01yc1JrkyyR9NuWZKzklyb5K4khzVdnA68uBnBe+eQ7g8H/mEy90eSJGmq6cvD\nGEn2A04EFlTVmiS7AOcDy6pqeZJjgLOBQ5tVdq2qRUn2BS4BVgDvB46rqkOG2cTrgeHmS5IkzRj9\nGtFbClxUVWsAmt8LgAub5RcAi7raf6Fpdwcwd6SOk7wI+FlV3T7RRUuSJE0nU+n1KjXCsse6PmeU\nft7ALwPjJj1yxvKNn7dbOJ/Zi3xuQ5Ik9d/AwAADAwMT0leqRspXvdFcul0BLKyqh5LMAc4FLm4u\n3b4ZeFVVvSbJMuDSqlrRrLu2qnZIcgDwkap6aVe/Ae4FXlxV/zbC9mvP+y/r2f5JagffoydpKkhC\nVY020DWsvozoVdXtSU4DrknyBLASOBY4L8nxwIPAMYPNh67e/F4FbEiyEjivqs4CfhdYPVLIkyRJ\nmin6MqLXb47oSRoLR/QkTQVbMqI3nd+jJ0mSpBEY9CRJklrKoCdJktRSBj1JkqSWmrEPY8zE/ZY0\nPutqPbMzq99lSJrhfBhDknrAkCdpujPoSZIktZRBT5IkqaUMepIkSS1l0JMkSWopg54kSVJLGfQk\nSZJayqAnSZLUUgY9SZKkljLoSZIktZRBT5IkqaUMepIkSS1l0JMkSWopg54kSVJLGfQkSZJayqAn\nSZLUUgY9SVJrrav1/S5B6qut+11Av+z1wOX9LkGS1GOr5x7c7xKkvnJET5IkqaUMepIkSS01ZYJe\nksVJFnRNvyTJjUkeT3LYkLZHJ7kzyXeTHNU1/5lJrmuWXZhkxl6aliRJmjJBD1gCLOyavgc4GvhU\nd6MkuwAfAF4I/A5wcpKdmsUfAj5SVc8BHgbe0uOaJUmSpqyeB70kRyW5JcnKJOcneWUz6nZjkiuS\nPD3JPODtwLuS3JRkUVWtrqpvAzWky98HrqiqR6rqYeAK4BXNsqXA55rP5wOH9nr/JEmSpqqeXtpM\nsh9wIrCgqtYk2RmoqjqoWf4W4L1VdUKSc4C1VfXRUbrdHbi3a/o+YPckTwXWVNWGZv4PgN0mcn8k\nSZKmk16wiQoyAAALvklEQVTfw7YUuKiq1gBU1cNJnpfks8AzgG2Auydwe5nAviRJkqa1fjyscDZw\nZlV9Ocli4ORxrn8fnfv5Bu0BXF1VP0myU5KtmlG9PZq2w3rkjOUbP2+3cD6zF80fZxmSJEkTb2Bg\ngIGBgQnpK1VDb4GbOM2l2xXAwqp6KMkc4ErgrVW1Msm5wDOrammS9wA7VtUpQ/pYBnypqj7XTO8C\n3AAcQOcewxuAFzSjhZ8BVlTVZ5L8NXBLVZ0zTF215/2X9Wy/JUlTgy9MVhskoao266plTx/GqKrb\ngdOAa5KsBM4ETgEuTnI98GBX80uBQwcfxkhyYJJ7gdcC5yS5telzDfBBOgHvm8CpzUMZAO8H3pPk\nTmAO8Mle7p8kSdJU1tMRvanKET1Jmhkc0VMbTNkRPUmSJPWPQU+SJKmlDHqSJEktZdCTJElqqRn7\nMMZM3G9JmmnW1XpmZ1a/y5C2iA9jSJI0DEOeZjqDniRJUksZ9CRJklrKoCdJktRSBj1JkqSWMuhJ\nkiS1lEFPkiSppQx6kiRJLWXQkyRJaimDniRJUksZ9CRJklrKoCdJktRSBj1JkqSWMuhJkiS1lEFP\nkiSppQx6kiRJLWXQkyRJM9K6Wt/vEnpu634X0C97PXB5v0uQJEl9tHruwf0uoecc0ZMkSWqpKR/0\nksxLcnjX9JwkVyVZm+Rjm1jnkiSrJq9KSZKkqWfKBz1gb+CIrul1wEnAccM1TnIo8Ogk1CVJkjSl\n9TzoJTkqyS1JViY5vxmh+2qSm5NcmWSPpt2yJGcluTbJXUkOa7o4HXhxkpuSvLOqfl5VXwceG2Zb\nTwbeDfyvXu+XJEnSVNfThzGS7AecCCyoqjVJdgHOB5ZV1fIkxwBnA4c2q+xaVYuS7AtcAqwA3g8c\nV1WHjGGTHwTOBP5jovdFkiRpuun1iN5S4KKqWgPQ/F4AXNgsvwBY1NX+C027O4C549lQkt8G9qmq\nS4A0P5IkSTNWP16vUiMs674cO96gtgB4QZLvA9sAc5NcVVVLh2v8yBnLN37ebuF8Zi+aP87NSZIk\nTbyBgQEGBgYmpK9UjZS7trDzzqXbFcDCqnooyRzgXODi5tLtm4FXVdVrkiwDLq2qFc26a6tqhyQH\nAB+pqpcO6fto4MCqOnaY7c5r+ho2vSWpPe+/bCJ3VZIkTTPT5T16SaiqzbpS2dMRvaq6PclpwDVJ\nngBWAscC5yU5HngQOGaw+dDVm9+rgA1JVgLnVdVZSe4GdgC2TfJq4OVV9Z1e7oskSdJ009MRvanK\nET1JkjQTRvSmw3v0JEmStBkMepIkSS1l0JMkSWopg54kSVJLzdiHMWbifkuSpF9aV+uZnVn9LmNU\nPowhSZI0TtMh5G0pg54kSVJLGfQkSZJayqAnSZLUUgY9TYqJ+nJmjZ3HfPJ5zCefx3zyecynF4Oe\nJoX/MEw+j/nk85hPPo/55POYTy8GPUmSpJYy6EmSJLXUjH1hcr9rkCRJGqvNfWHyjAx6kiRJM4GX\nbiVJklrKoCdJktRSrQ16SV6R5DtJ7kzyvk20+ViS7yW5Ocn+k11j24x2zJMsTvJwkpuan5P6UWeb\nJPlkkvuTrBqhjef5BBrtmHueT7wkeyS5KsltSW5N8o5NtPNcnyBjOeae6xMryXZJvplkZXPc//cm\n2o3rPN964kvtvyRbAR8Hfg/4IXB9ki9W1Xe62hwM7FNVz07yO8A5wEF9KbgFxnLMG1+rqkMmvcD2\nWgacDfz9cAs9z3tixGPe8DyfWE8A76mqm5M8BbgxyRX+m95Tox7zhuf6BKmqx5K8tKp+nmQWcG2S\nRVV17WCbzTnP2zqi9yLge1V1T1U9DvwD8OohbV5N8w91VX0T2CnJb0xuma0ylmMOsFlPDWl4VfUv\nwJoRmnieT7AxHHPwPJ9QVfXjqrq5+fxT4A5g9yHNPNcn0BiPOXiuT6iq+nnzcTs6GW3ovzXjPs/b\nGvR2B+7tmv4Bv36CDm1z3zBtNHZjOeYAC5rh5i8n2W9ySpvRPM/7w/O8R5I8E9gf+OaQRZ7rPTLC\nMQfP9QmVZKskK4EfAwNVdfuQJuM+z1t56VZT1o3AXs2w9MHAF4Dn9LkmaaJ5nvdIcwnxYuCdzSiT\nemyUY+65PsGqagPw/CQ7AlckWVxV12xJn20d0bsP2Ktreo9m3tA2e47SRmM36jGvqp8ODktX1eXA\nNknmTF6JM5Ln+STzPO+NJFvTCRwXVNUXh2niuT7BRjvmnuu9U1WPAl8GDhyyaNzneVuD3vXAf0oy\nL8m2wBuAS4a0uQQ4CiDJQcDDVXX/5JbZKqMe8+77CJK8iM4Lux+a3DJbKWz6PhnP897Y5DH3PO+Z\nc4Hbq+qsTSz3XJ94Ix5zz/WJleRpSXZqPj8JeBlw85Bm4z7PW3nptqrWJ/kfwBV0wuwnq+qOJH/S\nWVx/W1WXJfmDJHcBPwOO6WfN091Yjjnw2iR/CjwO/Afw+v5V3A5JPg0sAZ6aZDVwMrAtnuc9M9ox\nx/N8wiVZBBwJ3Nrcv1TAicA8PNd7YizHHM/1ifYM4PwkofPf0Quq6qtbml38CjRJkqSWauulW0mS\npBnPoCdJktRSBj1JkqSWMuhJkiS1lEFPkiSppQx6kiRJLWXQkzTtJLk6yQGjtHlnktld019qvlZo\nS7c9If2MY3s7Ne8qk6RxM+hJmpKal4ZuiXcB2w9OVNUrm68V2iIT1c9YJJkF7AL82WRsT1L7GPQk\nTQnN1+d9J8n5SW4F9kjysiRfT3JDks8k2X6Y9f4qybeS3Jrk5GbescBuwNVJvtrMuzvJnCSnJ/mz\nrvVPTvKe5vPxTV83D/Y1zPYG+5mX5I4ky5J8N8mnmnqvbaYP7Or/75v9+G6St3b1dUZT9y1J/rCZ\ntzjJ15J8EbgNOB3YJ8lNST6U5MlJ/qk5JrckOaTr+N2e5G+TfDvJV5Js1yzbJ8mVzX7dkGTvse6v\npGmuqvzxxx9/+v5D56uVngBe2Ew/FbgGeFIz/V7gpObz1cABzeedm99bNfOf10x/H9ilq//vA3OA\n/YGBrvm3AbvT+V7Jv2nmBbgUePEwdQ72Mw/4BbBfM/8GOl/9B3AI8Pnm88nASjpfk/ZUYDWwK3AY\n8I9Nm7nAPcBvAIuBtcBeXcdlVdf2twKe0nWMvtfV7hfAf26mPwMc0Xy+Djik+bwtMHus++uPP/5M\n759WftetpGnrnqq6vvl8ELAfcG1zGXcb4OvDrPOGJG+j893duzbrfJtOeOm+/BuAqro5ydOT7Eon\nYD1UVfcleRfwsiQ3NW2fDDwb+Jch2+vu8+6qur35fBvwT83nW+kEr0FfrKpfAD9JchXwO8CLgQub\nmh5IMgC8kE7I+1ZVrd7EMdoKOD3J7wIbgN2SzO2q59bm843AM5M8Bditqi5ptvULgCQvH+P+SprG\nDHqSppKfdX0OcEVVHbmpxkmeCRwHvKCqHk2yjM5o1WguAl5HJxh+pmt7p1fV342j3se6Pm/omt7A\nr/772v2l4mmWD9UdIH82zPJBRwJPA55fVRuS3M0v97m7nvVd84e733Fz9lfSNOM9epKmku5Ach2w\nKMk+AEm2T/LsIe13BH4KrE3yG8DBXcsebZYP57PAG4DX0Al9AP8I/FGSJzfb2y3J08dR70henWTb\nJE+lc2n2euCfgdcn2arZzkuAbw2z7lpgh67pnYAHmpD3Un515PDX6qmqnwL3Jnl1s1/bJnkSm7e/\nkqYZR/QkTSUbR76q6t+TvBm4sHmooICTgO8NtquqVUluBu4A7uVXLzv+HfCVJPdV1e8N6fv2JDsA\nP6iq+5t5Vyb5LeAbzQO/a4E3Ag9uqsYRPg+1Chigc0/dn1fVj4HPJzkIuIXOCN8JzSXcfX9lY1UP\nNQ94rAIuBz4EfCnJLXTuC7xjDDUcBfxNkj+ncx/f68axv5KmsVSN9G+TJGlLNE+zrq2qj/a7Fkkz\nj5duJUmSWsoRPUmSpJZyRE+SJKmlDHqSJEktZdCTJElqKYOeJElSSxn0JEmSWsqgJ0mS1FL/Dxus\nH4id4tYhAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f39bdc736d8>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "feature_dict = dict(zip(range(len(feature_list)), feature_list))\n",
    "df_fi = pd.DataFrame(gbmr.feature_importance(), columns=['feature', 'importance'])\n",
    "df_fi = df_fi.replace({\"feature\": feature_dict})\n",
    "\n",
    "del feature_dict, feature_list\n",
    "\n",
    "top = 10\n",
    "\n",
    "plt.figure()\n",
    "df_fi.head(top).plot(kind='barh',\n",
    "                     x='feature',\n",
    "                     y='importance',\n",
    "                     sort_columns=False,\n",
    "                     legend=False,\n",
    "                     figsize=(10, 6),\n",
    "                     facecolor='#1DE9B6',\n",
    "                     edgecolor='white')\n",
    "\n",
    "plt.title('XGBoost Feature Importance')\n",
    "plt.xlabel('relative importance')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "SUBMISSION\n",
    "-------\n",
    "Predicting Test set"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[LightGBM] finished load parameters\n",
      "[LightGBM] Loaded 500 modles\n",
      "\n",
      "[LightGBM] finish predict init\n",
      "[LightGBM] start prediction for data /tmp/tmp35nwyz05/X_to_pred.svm, and data has label\n",
      "[LightGBM] finish predict\n",
      "\n"
     ]
    }
   ],
   "source": [
    "y_test_preds = gbmr.predict(X_test)\n",
    "y_test_preds=(np.exp(y_test_preds)-1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Make submission file"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "df_submission = pd.read_csv('data/sample_submission.csv.zip')\n",
    "df_submission['loss'] = y_test_preds"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Save submission file"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "df_submission.to_csv('submission.csv',index=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### This submission file scored 1138.06444 on LB"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.4.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}