make FTRL more c-style and faster

[stegben]

It's about 10% ~ 30% faster when interaction=False.

You may use the following script to profile the performance. But before compiling, add # cython: linetrace=True at the header of ftrl.pyx.

import cProfile

import numpy as np
np.random.seed(1234)
import scipy.sparse as sps

from kaggler.online_model import FTRL


DATA_NUM = 5e7


class customCSR(object):
    def __init__(self, csr_matrix):
        self.data = []
        self.shape = csr_matrix.shape
        for row in range(self.shape[0]):
            self.data.append(csr_matrix[row])
    def __getitem__(self, idx):
        return self.data[idx]


def main():
    print('create y...')
    y = np.random.randint(0, 1, DATA_NUM)
    print('create x...')
    row = np.random.randint(0, 100000, DATA_NUM)
    col = np.random.randint(0, 10, DATA_NUM)
    data = np.ones(DATA_NUM)
    x = sps.csr_matrix((data, (row, col)), dtype=np.int8)
    x = customCSR(x)
    
    print('train...')
    profiler = cProfile.Profile(subcalls=True, builtins=True, timeunit=0.001,)
    clf = FTRL(interaction=False)
    profiler.enable()
    clf.fit(x, y)
    profiler.disable()
    profiler.print_stats()
    print(clf.predict(x))


if __name__ == '__main__':
    main()

[stegben]

And as you may already know, the main overhead is scipy sparse matrix, not what I fixed here. And I would like to hear about your thoughts of it.

[stegben]

Ah, sorry, I didn't follow some PEP8 and docstring format. Plz do not merge for a while.

[stegben]

done

[stegben]

Just curious, are you participating in Outbrain competition?

[jeongyoonlee]

@stegben Thanks again! I will complete the code review later because it's holiday here. I'm not participating in the Outbrain competition. I'm considering to start the annual Santa competition instead. :)

Also, you may find other modules such as preprocessing and data_io useful other than online_model. Please take a look.

[stegben]

I'll take a look at those modules, thank you and Merry Christmas!

[stegben]

Wait, plz don't merge it now, I'm working on #21 which I think should consider first. If that is OK and merged, I'll rebase this branch based on that.

[stegben]

#22