Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Function to make top-k recommendations to all users in a fast way for ALS model. #179

Merged
merged 18 commits into from
Jun 20, 2019

Conversation

Tych0n
Copy link
Contributor

@Tych0n Tych0n commented Jan 14, 2019

Function to make top-k recommendations to all users in a fast way for ALS model. Main feature is custom C++ argsort for numpy 2d ndarray that runs in parallel.

For users_items == ~11kk x 130k it requires ~45 minutes to evaluate on 80-CPU server and is 10-20 times faster than looping model.recommend. I think it closes #29 and #72 . May be it can be improved by ideas from #126.

Please feel free to criticise. I can move it anywhere in code to be more consistent from your point of view with the library structure.

	modified:   implicit/evaluation.cpp
	modified:   implicit/evaluation.pyx
	new file:   implicit/topnc.cpp
	new file:   implicit/topnc.h
	modified:   setup.py

 Function to make top-k recommendations to all users in a fast way. Doesn't have "exclude already liked items" functionality yet.
BuildTools and others added 5 commits March 29, 2019 11:57
Filter already liked items from recommendations before selecting top@k.
Added docstring and comments
@ita9naiwa
Copy link
Collaborator

I'll check the code soon...

@ita9naiwa
Copy link
Collaborator

ita9naiwa commented Jun 10, 2019

I briefly checked the code(I didn't understand the detail of cython code) but

(It's totally my opinion)

(1) This function is very helpful for both actual recommendation and evaluation.

I think it should be a method of MatrixFactorizationBase because BPR model also generates exactly the same way as ALS does.

class MatrixFactorizationBase(RecommenderBase):

@ita9naiwa
Copy link
Collaborator

Oh, I've just seen that the owner wants it to be in evaluation.py at #72
hmm...

@Tych0n
Copy link
Contributor Author

Tych0n commented Jun 11, 2019

We don't have evaluation.py yet, but we have evaluation.pyx instead)

@ita9naiwa
Copy link
Collaborator

That's not the point btw. Anyway, could you please consider moving that function into the method of MatrixFactorizationBase?

@ita9naiwa
Copy link
Collaborator

ita9naiwa commented Jun 14, 2019

one last thing...
its great if you add some unittests for this.

1. Recommend will filter items like default method.
2. C++ types strike back. Fix in batch calculation.
3. Moved unit test to proper place.
@ita9naiwa
Copy link
Collaborator

Thanks!!

@ita9naiwa ita9naiwa merged commit 805a9b3 into benfred:master Jun 20, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[question] Getting recommendations for 14M+ users quickly
2 participants