Gromov-Wasserstein distance #23

ncourty · 2017-08-28T12:44:58Z

Hi everyone,

This is a new implementation of the Gromov-Wasserstein distance, mostly programmed by Erwan Vautier and myself. In the next commit, I will add a new example on how to compute barycenters and also tests for this new functionality.

rflamary · 2017-08-28T12:48:31Z

README.md

@@ -16,7 +16,7 @@ It provides the following solvers:
 * Conditional gradient [6] and Generalized conditional gradient for regularized OT [7].
 * Joint OT matrix and mapping estimation [8].
 * Wasserstein Discriminant Analysis [11] (requires autograd + pymanopt).
-
+* Gromov-Wasserstein distances [12]


and barycenters

agramfort · 2017-08-29T16:42:28Z

README.md

@@ -182,3 +182,5 @@ You can also post bug reports and feature requests in Github issues. Make sure t
 [10] Chizat, L., Peyré, G., Schmitzer, B., & Vialard, F. X. (2016). [Scaling algorithms for unbalanced transport problems](https://arxiv.org/pdf/1607.05816.pdf). arXiv preprint arXiv:1607.05816.

 [11] Flamary, R., Cuturi, M., Courty, N., & Rakotomamonjy, A. (2016). [Wasserstein Discriminant Analysis](https://arxiv.org/pdf/1608.08063.pdf). arXiv preprint arXiv:1608.08063.
+
+[12] Peyré, Gabriel, Marco Cuturi, and Justin Solomon, [Gromov-Wasserstein averaging of kernel and distance matrices](http://proceedings.mlr.press/v48/peyre16.html)  International Conference on Machine Learning (ICML). 2016.


Gabriel Peyré to be consistent

agramfort · 2017-08-29T16:42:38Z

examples/plot_gromov.py

+"""
+====================
+Gromov-Wasserstein example
+====================


not enough ===

agramfort · 2017-08-29T16:42:54Z

examples/plot_gromov.py

+import numpy as np
+
+import ot
+import matplotlib.pylab as pl


import pl before ot

agramfort · 2017-08-29T16:43:13Z

examples/plot_gromov.py

+
+"""
+Sample two Gaussian distributions (2D and 3D)
+====================


not enough ===

it won't render well in sphinx

agramfort · 2017-08-29T16:43:33Z

examples/plot_gromov.py

+For demonstration purpose, we sample two Gaussian distributions in 2- and 3-dimensional spaces.
+"""
+
+n = 30  # nb samples


n -> n_samples

you won't need to write # nb samples :)

agramfort · 2017-08-29T16:44:16Z

ot/gromov.py

+    Returns the value of L(a,b)=(1/2)*|a-b|^2
+    """
+
+    return (1 / 2) * (a - b)**2


1 / 2 will be 0 python 2

agramfort · 2017-08-29T16:44:48Z

ot/gromov.py

+        return b
+
+    tens = -np.dot(h1(C1), T).dot(h2(C2).T)
+    tens = tens - tens.min()


tens -= tens.min()

agramfort · 2017-08-29T16:45:23Z

ot/gromov.py

+
+    Parameters
+    ----------
+    C1 : np.ndarray(ns,ns)


C1 : ndarray, shape (ns, ns)

is the standard of numpydoc

agramfort · 2017-08-29T16:46:34Z

ot/gromov.py

+    cpt = 0
+    err = 1
+
+    while (err > stopThr and cpt < numItermax):


avoid while loops. Use for with break. It's much safer to avoid infinite loops

you can use for else syntax to capture the absence of a break

Ok for this one I will keep the consistency with the rest of the optimization method (especially those in Bregman module)

…test function

agramfort · 2017-08-31T15:07:19Z

examples/plot_gromov_barycenter.py

+"""
+
+
+def smacof_mds(C, dim, maxIter=3000, eps=1e-9):


maxIter -> max_iter

agramfort · 2017-08-31T15:07:57Z

examples/plot_gromov_barycenter.py

+
+    Parameters
+    ----------
+    C : np.ndarray(ns,ns)


C : ndarray, shape (ns , ns)

agramfort · 2017-08-31T15:08:05Z

examples/plot_gromov_barycenter.py

+    ----------
+    C : np.ndarray(ns,ns)
+        dissimilarity matrix
+    dim : Integer


Integer -> int

agramfort · 2017-08-31T15:08:29Z

examples/plot_gromov_barycenter.py

+        dissimilarity matrix
+    dim : Integer
+          dimension of the targeted space
+    maxIter : Maximum number of iterations of the SMACOF algorithm for a single run


max_iter : int Maximum number of iterations of the SMACOF algorithm for a single run

agramfort · 2017-08-31T15:12:29Z

examples/plot_gromov_barycenter.py

+Ct01 = [0 for i in range(2)]
+for i in range(2):
+    Ct01[i] = ot.gromov.gromov_barycenters(N, [Cs[0], Cs[1]], [
+                                           ps[0], ps[1]], p, lambdast[i], 'square_loss', 5e-4, numItermax=100, stopThr=1e-3)


numItermax -> max_iter?

agramfort · 2017-08-31T15:16:59Z

examples/plot_gromov_barycenter.py

+triangle = spi.imread('../data/triangle.png').astype(np.float64) / 256
+fleche = spi.imread('../data/coeur.png').astype(np.float64) / 256
+
+shapes = [carre, rond, triangle, fleche]


I guess you meant : square, circle, triangle and arrow :)

agramfort · 2017-08-31T15:18:58Z

@ncourty please go over the full diff about docstrings and naming. If you're ok with me bugging you :) I'll do one more pass when you did it.

agramfort · 2017-09-01T12:54:34Z

ot/gromov.py

+                        'It.', 'Err') + '\n' + '-' * 19)
+                print('{:5d}|{:8e}|'.format(cpt, err))
+
+        cpt = cpt + 1


agramfort · 2017-09-01T14:04:13Z

examples/plot_gromov_barycenter.py

+square = spi.imread('../data/carre.png').astype(np.float64) / 256
+circle = spi.imread('../data/rond.png').astype(np.float64) / 256
+triangle = spi.imread('../data/triangle.png').astype(np.float64) / 256
+arrow = spi.imread('../data/coeur.png').astype(np.float64) / 256


can you rename maybe the png files? also I see arrow = coeur. Is this a bug?

agramfort · 2017-09-01T14:04:59Z

test/test_gromov.py

+
+    xs = ot.datasets.get_2D_samples_gauss(n_samples, mu_s, cov_s)
+
+    xt = xs[::-1]


I would have written:

xt = xs[::-1].copy()

and removed the array below

agramfort · 2017-09-01T14:06:40Z

examples/plot_gromov_barycenter.py

+    npos : ndarray, shape (R, dim)
+           Embedded coordinates of the interpolated point cloud (defined with one isometry)
+
+


remove unnecessary empty lines here and one before Returns

agramfort · 2017-09-01T14:06:55Z

examples/plot_gromov.py

+"""
+Sample two Gaussian distributions (2D and 3D)
+=============================================
+The Gromov-Wasserstein distance allows to compute distances with samples that do not belong to the same metric space.


line too long

agramfort · 2017-09-01T14:07:33Z

ot/gromov.py

+    tens : ndarray, shape (ns, nt)
+           \mathcal{L}(C1,C2) \otimes T tensor-matrix multiplication result
+
+


remove empty lines

agramfort · 2017-09-12T20:33:07Z

examples/plot_gromov_barycenter.py

+=====================================
+Gromov-Wasserstein Barycenter example
+=====================================
+This example is designed to show how to use the Gromov-Wassertsein distance


Wassertsein -> Wasserstein

agramfort · 2017-09-12T20:34:10Z

examples/plot_gromov_barycenter.py

+
+def smacof_mds(C, dim, max_iter=3000, eps=1e-9):
+    """
+    Returns an interpolated point cloud following the dissimilarity matrix C using SMACOF


line too long.

see pep257 https://www.python.org/dev/peps/pep-0257/

especially for multiline docstings

agramfort · 2017-09-12T20:34:45Z

examples/plot_gromov_barycenter.py

+           Embedded coordinates of the interpolated point cloud (defined with one isometry)
+    """
+
+    rng = np.random.RandomState(seed=3)


you should expose the random_state and use check_random_state like sklearn does.

wait it's an example maybe it's not necessary...

agramfort · 2017-09-12T20:36:21Z

ot/gromov.py

+    ----------
+    p  : ndarray, shape (N,)
+         weights in the targeted barycenter
+    lambdas : list of the S spaces' weights


agramfort · 2017-09-12T20:37:14Z

ot/gromov.py

+         sample weights in the S spaces
+    p  : ndarray, shape(N,)
+         weights in the targeted barycenter
+    lambdas : list of the S spaces' weights


agramfort · 2017-09-12T20:37:49Z

ot/gromov.py

+    lambdas = np.asarray(lambdas, dtype=np.float64)
+
+    # Initialization of C : random SPD matrix
+    xalea = np.random.randn(N, 2)


expose random_state to make results deterministic if one wants

ncourty · 2017-09-12T23:04:42Z

Thanks for the careful reading @agramfort . And congrats for your NIPS paper :) See you in LA ?

rflamary · 2017-09-13T06:47:46Z

Hello @ncourty ,

I think we should merge shortly since it has converged, could you please update from master ?

agramfort · 2017-09-13T07:42:05Z

@ncourty thx :) yes see you in LA !

rflamary

This looks good to me, you have taken into account all comments and the contribution is very nice for the toolbox.

I think we can merge.

Gromov-Wasserstein distance

7ab9037

rflamary reviewed Aug 28, 2017

View reviewed changes

gromov:flake8 and other

0a68bf4

agramfort reviewed Aug 29, 2017

View reviewed changes

examples/plot_gromov.py Outdated

"""

====================

Gromov-Wasserstein example

====================

Copy link

Collaborator

agramfort Aug 29, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not enough ===

agramfort reviewed Aug 29, 2017

View reviewed changes

examples/plot_gromov.py Outdated

import numpy as np

import ot

import matplotlib.pylab as pl

Copy link

Collaborator

agramfort Aug 29, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

import pl before ot

agramfort reviewed Aug 29, 2017

View reviewed changes

ot/gromov.py Outdated

Returns the value of L(a,b)=(1/2)*|a-b|^2

"""

return (1 / 2) * (a - b)**2

Copy link

Collaborator

agramfort Aug 29, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

1 / 2 will be 0 python 2

agramfort reviewed Aug 29, 2017

View reviewed changes

ot/gromov.py Outdated

return b

tens = -np.dot(h1(C1), T).dot(h2(C2).T)

tens = tens - tens.min()

Copy link

Collaborator

agramfort Aug 29, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

tens -= tens.min()

agramfort reviewed Aug 29, 2017

View reviewed changes

ot/gromov.py Outdated

Parameters

----------

C1 : np.ndarray(ns,ns)

Copy link

Collaborator

agramfort Aug 29, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

C1 : ndarray, shape (ns, ns)

is the standard of numpydoc

agramfort reviewed Aug 29, 2017

View reviewed changes

rflamary added enhancement new feature and removed enhancement labels Aug 30, 2017

Minor corrections suggested by @agramfort + new barycenter example + …

3007f1d

…test function

agramfort reviewed Aug 31, 2017

View reviewed changes

examples/plot_gromov_barycenter.py Outdated

"""

def smacof_mds(C, dim, maxIter=3000, eps=1e-9):

Copy link

Collaborator

agramfort Aug 31, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maxIter -> max_iter

agramfort reviewed Aug 31, 2017

View reviewed changes

examples/plot_gromov_barycenter.py Outdated

Parameters

----------

C : np.ndarray(ns,ns)

Copy link

Collaborator

agramfort Aug 31, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

C : ndarray, shape (ns , ns)

agramfort reviewed Aug 31, 2017

View reviewed changes

minor corrections

bc68cc3

ncourty and others added 6 commits September 1, 2017 01:25

Merge branch 'master' into gromov

986f46d

remove linewidth error message

e89f09d

first proposal for OT wrappers

f469205

small modifs according to NG proposals

fa36e77

integrate AG comments

aa19b6a

own BaseEstimator class written + rflamary comments addressed

5ab5035

agramfort reviewed Sep 1, 2017

View reviewed changes

ot/gromov.py Outdated

'It.', 'Err') + '\n' + '-' * 19)

print('{:5d}|{:8e}|'.format(cpt, err))

cpt = cpt + 1

Copy link

Collaborator

agramfort Sep 1, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cpt += 1

Nicolas Courty added 2 commits September 1, 2017 15:37

docstrings + naming

53e1115

docstrings + naming

8ea74ad

agramfort reviewed Sep 1, 2017

View reviewed changes

Corrections on Gromov

36bf599

ncourty force-pushed the gromov branch from 503aa11 to 36bf599 Compare September 12, 2017 20:05

Corrections on Gromov

24784ed

agramfort reviewed Sep 12, 2017

View reviewed changes

agramfort approved these changes Sep 12, 2017

View reviewed changes

Corrections on Gromov

84c2723

Corrections on Gromov

55db350

ncourty added 4 commits September 13, 2017 10:07

Corrections on Gromov

5a2ebfa

Merge branch 'gromov' of https://github.com/rflamary/POT into gromov

7e5df4c

Merge branch 'master' into gromov

c86cc4f

Merge branch 'master' into gromov

c7eef9d

rflamary approved these changes Sep 13, 2017

View reviewed changes

rflamary merged commit e70d542 into master Sep 14, 2017

rflamary deleted the gromov branch June 19, 2018 11:06


		xs = ot.datasets.get_2D_samples_gauss(n_samples, mu_s, cov_s)

		xt = xs[::-1]

		npos : ndarray, shape (R, dim)
		Embedded coordinates of the interpolated point cloud (defined with one isometry)

		tens : ndarray, shape (ns, nt)
		\mathcal{L}(C1,C2) \otimes T tensor-matrix multiplication result

Gromov-Wasserstein distance #23

Gromov-Wasserstein distance #23

Uh oh!

Conversation

ncourty commented Aug 28, 2017

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ncourty Sep 1, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

agramfort commented Aug 31, 2017

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ncourty commented Sep 12, 2017

Uh oh!

rflamary commented Sep 13, 2017

Uh oh!

agramfort commented Sep 13, 2017

Uh oh!

rflamary left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

ncourty Sep 1, 2017 •

edited

Loading