New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reading: Field-aware Factorization Machine #11

Closed
atkm opened this Issue Oct 9, 2018 · 1 comment

Comments

1 participant

@atkm atkm created this issue from a note in Models (To Do) Oct 9, 2018

@atkm

This comment has been minimized.

Copy link
Owner

atkm commented Oct 10, 2018

This paper by Rendle introduces Factorization Machines.
Summary:

  • FMs model all interactions between features. The order of interactions can be large.
  • FMs is efficient for high-dimensional sparse data, which SVM is bad at.

How Factorization Machines Work (Amazon) (link).

Field-aware Factorization Machine:

  • In order to model interactions better, instead of learning one latent vector of length N for a feature, learn multiple latent vectors of lengths N1, ... , Nk (sum(Ni) = N), where k is the number of features. Nj determines how the feature interacts with feature j.

    Model coefficients of interaction terms as a matrix, which can be approximated by a product of two vectors, which we refer to as latent vectors. In FFM, think of this matrix as a sum of block matrices, where each block corresponds to an interaction of two features (each of which has multiple columns, whose number equals the number of categories in it). FFM solves for latent vectors for each block matrix.

  • https://arxiv.org/abs/1701.04099

Summary of https://www.csie.ntu.edu.tw/~cjlin/papers/ffm.pdf :

  • FFM is a generalization of PITF (pairwise interaction tensor factorization), which is a variant of FM.
    Introduced in "Ensemble of collaborative filtering and feature engineered model for click through rate prediction".
  • Their FFM trains faster than other models. Their implementation of FM trains faster than LIBFM (author Rendle).
  • The best FFM model scores 0.06 points better than the best logistic model, and less than 0.02 points better than the best FM model.
  • Solutions on github: guestwalk.

Summary of Ensemble of collaborative filtering and feature engineered model for click through rate prediction:

Packages:

  • getty/tffm
  • aksnzhy/xlearn

Robust Factorization Machine https://dl.acm.org/citation.cfm?id=3186148

(how?)FMs generalize Logistic Regression, SVM and Matrix Factorization.

@atkm atkm closed this Oct 10, 2018

@atkm atkm moved this from To Do to In Progress in Models Oct 11, 2018

@atkm atkm moved this from In Progress to Done in Models Oct 11, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment