Algorithm wish list

[perimosocordiae]

Methods that we haven't implemented yet, but would like to. In no particular order:

• Probablistic Global Distance Metric Learning
  • Probabilistic extension of MMC (pMMC?)
• Discriminative Component Analysis (DCA)
  • reference implementation
• Localized Distance Metric Learning (LDM)
  • reference implementation
• DistBoost and KernelBoost
  • reference implementation
• Active Distance Metric Learning
  • reference implementation
• Large Scale Metric Learning from Equivalence Constraints (KISSME)
  • website (includes reference implementation)
  • python implementation (unverified)
• Logistic Discriminant Metric Learning (LDML)
  • reference implementation
• Metric Learning for Kernel Regression (MLKR)
  • reference implementation
  • added in PR Added MLKR algorithm #28
• Mahalanobis Metric for Clustering (MMC)
  • reference implementation
  • added in PR Implementation of MMC #61

[bhargavvader]

Hello, any tips or further documentation on how to go about doing this? And which one of these would be highest priority?

[perimosocordiae]

I don't personally have a priority, but if you want to rank them you could look at their citation counts on Google Scholar. That should give you an idea of what's more popular in the literature. I would start with one that seems easy to implement, though.

Each method should have its own Python file in the metric_learn directory, which defines a class that inherits from BaseMetricLearner. This subclass should implement a .fit() method, as well as one of the .metric() or .transformer() methods. As a really simple example, you can start with the Covariance class.

[perimosocordiae]

Once you have something that works, add a class to test/metric_learn_test.py which exercises the new algorithm. You should also verify that your implementation matches the reference code, if possible.

[bhargavvader]

Thanks for all the details! The paper by Xing, et al is I think the most popular (and one of the first in Metric Learning) algorithm/paper among the list, I'll give it a shot.

[devashishd12]

Hey @perimosocordiae can I take up the MLKR implementation? I was thinking of using the transformer() method to use MLKR for dimensionality reduction and hence return X_new as the output. Also, I was thinking of having the A matrix stored as an attribute after the fit() operation thus enabling the user to access it independently also if needed. Does this seem fine to you? I'll take the output dimension as a parameter for __init__() to achieve the extra optimisation as done in the matlab code.

[perimosocordiae]

@dsquareindia Sure, MKLR is a good choice. The transformer() method should return the matrix A (as defined the paper), and the transform() method will take care of the dimension reduction without needing to be overridden. In this way, the base class metric() method will also work as intended (corresponding to M in the paper).

[aschuman]

KISSME metric learning would be a nice addition (no optimization).
You can find a reference Matlab implementation and the paper here: https://lrs.icg.tugraz.at/research/kissme/
There is also a python implementation (which I have not verified) here: https://github.com/Cysu/dgd_person_reid/blob/master/eval/metric_learning.py

[perimosocordiae]

@aschuman thanks, I've added it to the list.

[anirudt]

LDML

[RishabGoel]

hi @perimosocordiae
I would like to take up Probabilistic Global Distance Metric Learning.
thanks :)

[perimosocordiae]

@anirudt Thanks, added.

@RishabGoel Go for it! I look forward to your pull request.

[bhargavvader]

Hello, just wanted to know where the short form PGDM came up - I can't seem to find it in the paper linked (which is the paper just called "Distance metric learning, with application to clustering with side-information"), by Xing et al.

[RishabGoel]

@bhargavvader Probably the wrong link. The metric is a non probabilistic one in the link. But find the PRobabilistic one in the following link:
http://www.cs.cmu.edu/~liuy/frame_survey_v2.pdf
I am bit busy right now so I am not able to do much, but will submit a PR soon.
Hope this helps!
Regards

[perimosocordiae]

@bhargavvader, @RishabGoel: yep, that's correct. I pulled the citation from the survey paper without looking too closely at it. I'll fix the link now.

[Callidior]

The method by Xing et al. (2003) is also sometimes referred to as "MMC", e.g., in the LMNN paper by Weinberger et al. I guess, "MM" stands for "Mahalanobis Metric Learning" and "C" for "Clustering".

[souravsingh]

@perimosocordiae I would like to take up DistBoost and KernelBoost

[perimosocordiae]

@souravsingh Great! Feel free to open a PR whenever you'd like to get feedback. I just noticed the paper links for those methods are broken, so if you find working links to the papers let me know.

[souravsingh]

@perimosocordiae I think these are the links to the papers-

DistBoost- http://www.cs.huji.ac.il/~daphna/papers/distboost-icml.pdf
KernelBoost- http://leibniz.cs.huji.ac.il/tr/919.pdf

I couldn't find the reference implementations accompanying these algorithms. There is an implementation for DistBoost here- http://www.scharp.org/thertz/code.html

[perimosocordiae]

@souravsingh thanks, I've updated the list with your links.

[bellet]

Another suggestion: the triplet-based method proposed in the paper below (heavily cited, very useful for information retrieval applications):
https://www.cs.cornell.edu/people/tj/publications/schultz_joachims_03a.pdf

Given that we currently do not have any triplet-based weakly supervised approach I think we could make this a priority. A GD or SGD implementation should not be very difficult

[hansen7]

How about add the extension to the existed LSML algorithm, in the original paper and code, there is the extension of the vanilla algorithm called spLSML, which seems to be is just to add a L1 norm of the learned Mahalanobis matrix in the loss function, thus with a novel gradient update method, it should be quick to implement.