CUDA matrix factorization

[benfred]

Add CUDA kernels and code to train the implicit ALS model on the GPU.

[maciejkula]

Let me just say, this is pretty great! I've been meaning to get into CUDA programming for a while, and now I have a good example to motivate me.

[benfred]

Thanks! It was fun thing to learn.

I actually initially experimented with BPR for this (using sgd only - no item biases etc), and came up with something like

__global__ void bpr_update_kernel(int samples, int * userids, int * likedids, int * dislikedids,
                                  int item_count, int user_count, int factors,
                                  float * X, float * Y, float * item_bias,
                                  float learning_rate, float reg) {
    extern __shared__ float shared_memory[];
    float * temp = &shared_memory[0];

    for (int i = blockIdx.x; i < samples; i += gridDim.x) {
        float * user = &X[userids[i] * factors + threadIdx.x],
              * liked = &Y[likedids[i] * factors + threadIdx.x],
              * disliked = &Y[dislikedids[i] * factors + threadIdx.x];
 
        float user_val = *user, liked_val = *liked, disliked_val = * disliked;

        temp[threadIdx.x] = liked_val - disliked_val;
        float z = 1.0 / (1.0 + exp(dot(user, temp)));

        // update the factors
        *user     += learning_rate * ( z * (liked_val - disliked_val) - reg * user_val);
        *liked    += learning_rate * ( z * user_val - reg * liked_val);
        *disliked += learning_rate * (-z * user_val - reg * disliked_val);
    }
}

But since BPR can already be done efficiently on the GPU with your spotlight package (and realistically I would need to switch from sgd to adam or something for optimizing this) - I didn't bother finishing =)