Support MixUpSoftLabelDataset. #497

fukatani · 2017-12-17T05:53:31Z

Between class learning is simple and effective technique, I think it is valuable to merge to chainercv.
https://arxiv.org/abs/1711.10284

yuyu2172 · 2017-12-21T07:15:47Z

Hi thanks for a timely PR. I am interested in supporting this feature.
I checked BC paper, and I found that it is slightly different from a more popular MixUp paper https://arxiv.org/abs/1710.09412.

MixUp takes positive pairs, but BC only takes negative pairs.
BC proposes two formulas for mixing two samples. The first one is the same as MixUp. The second one is different.

Since MixUp is simpler, it is easier to include in a library.
I would suggest to only add MixUp, and not BC to ChainerCV.
However, if there is a strong advantage for BC (note: I would like to know this!), it can be integrated into ChainerCV together with MixUp.

Having said that, I have several design comments.

Can you change the name to MixUpSoftLabelDataset? We call a dataset *LabelDataset when it returns an image and an integer. Since, MixUp returns an array of size (n_class,) as label, it needs to be differentiated. Thus, I would name it *SoftLabelDataset. Also, MixUp is easier to recognize, and I prefer the name better.
I found that it is common in CV to fetch two samples from two datasets. Thus, I made a dataset class
called SiameseDataset that supports this functionality in a different PR Add SiameseDataset #505. Could you to wait until that one is merged to ChainerCV, then change your code to use SiameseDataset? Sorry for inconvenience with this point.

yuyu2172 · 2017-12-21T07:17:12Z

chainercv/datasets/between_class_dataset.py

+
+    """
+
+    def __init__(self, dataset, dtype=numpy.float32, label_dtype=numpy.float32,


Could you remove dtype and label_dtype from arguments?
By default, images and soft-labels are dtype=np.float32.
If different dtypes are needed, transforms can be used.

fukatani · 2017-12-22T15:17:16Z

Thank you for your quick and attentive response.

OK, I will change BetweenClassLabeledDataset to MixUpSoftLabelDataset and implement it as MixUp paper.

After merging this PR and if I can confirm advantage fo BC, I may suggest BC-like mixing as option.

Sorry for inconvenience with this point.

Don't mind. Actually, I will take time to restart this PR.

yuyu2172 · 2018-03-21T02:09:21Z

chainercv/datasets/mixup_dataset.py

@@ -0,0 +1,60 @@
+import numpy


We use np in ChainerCV. Could you change it?

yuyu2172 · 2018-03-21T02:12:04Z

chainercv/datasets/__init__.py

@@ -15,6 +15,7 @@
 from chainercv.datasets.cub.cub_utils import cub_label_names  # NOQA
 from chainercv.datasets.directory_parsing_label_dataset import directory_parsing_label_names  # NOQA
 from chainercv.datasets.directory_parsing_label_dataset import DirectoryParsingLabelDataset  # NOQA
+from chainercv.datasets.mixup_dataset import MixUpSoftLabelDataset  # NOQA


Can you change the name of the file to mixup_soft_label_dataset?
Also, could you change the name of the test file as well?

yuyu2172 · 2018-03-21T02:16:07Z

chainercv/datasets/mixup_dataset.py

+
+    """
+
+    def __init__(self, dataset, max_label):


How about using n_class instead of max_label?
n_class is widely used in other parts of ChainerCV.

In that case the doc would be

n_class (int): The number of classes in the base dataset.

You can remove `+1 in line 57 if you do this.

yuyu2172 · 2018-03-21T02:20:47Z

chainercv/datasets/mixup_dataset.py

+            >>> mixed_image, mixed_label = dataset[0]
+
+    Args:
+        dataset: The underlying dataset. dataset should returns two image


daset should return ... can be written more explicitly.

The dataset returns :obj:`img_0, label_0, img_1, label_1`, which is a tuple containing two pairs of an image and a label.

yuyu2172 · 2018-03-21T02:22:23Z

chainercv/datasets/mixup_dataset.py

+    Args:
+        dataset: The underlying dataset. dataset should returns two image
+            and their label. Typically, dataset is `SiameseDataset`.
+            More over, each element of each dataset should have same shape.


Moreover, ... can be rewritten as...

The shapes of images and labels should be constant.

yuyu2172 · 2018-03-21T02:24:13Z

chainercv/datasets/mixup_dataset.py

+            >>> mnist, _ = get_mnist()
+            >>> base_dataset = SiameseDataset(mnist, mnist)
+            >>> dataset = MixUpSoftLabelDataset(base_dataset)
+            >>> mixed_image, mixed_label = dataset[0]


How about printing the shape? This is an important details.

>>> print(mixed_label.shape, mixed_label.dtype) # ((10,), numpy.float32)

yuyu2172 · 2018-03-21T02:25:34Z

tests/datasets_tests/test_mixup_dataset.py

+        self.assertEqual(example[1].dtype, np.float32)
+        self.assertEqual(example[1].ndim, 1)
+        self.assertEqual(example[1].shape[0], self.n_class + 1)
+        self.assertAlmostEqual(example[1].sum(), 1.0)


Can you check that the label is nonnegative?

yuyu2172 · 2018-03-21T02:26:25Z

chainercv/datasets/mixup_dataset.py

+    dataset respectively by weighted average.
+
+    Unlike `LabeledImageDatasets`, label is a one-dimensional float array with
+    at most two nonzero weights (i.e. soft label). The summed weights is one.


The sum of the two weights is one

yuyu2172 · 2018-03-21T02:26:40Z

chainercv/datasets/mixup_dataset.py

+    Unlike `LabeledImageDatasets`, label is a one-dimensional float array with
+    at most two nonzero weights (i.e. soft label). The summed weights is one.
+
+    The base dataset `__getitem__` should return image and label. Please see


This is incorrect. We can remove this.

yuyu2172 · 2018-03-21T02:28:25Z

chainercv/datasets/mixup_dataset.py

+
+    """Dataset which returns mixed images and labels for mixup learning[1].
+
+    `MixUpSoftLabelDataset` mix two images and labels which is chosen by base


:class:`MixUpSoftLabelDataset` mixes two pairs of labeled images fetched from the base dataset.

yuyu2172 · 2018-03-21T02:31:10Z

Thank you for a nice PR!
I am so sorry for the late response.

yuyu2172 · 2018-03-21T02:32:52Z

chainercv/datasets/mixup_dataset.py

+            >>> from chainercv.datasets import MixUpSoftLabelDataset
+            >>> mnist, _ = get_mnist()
+            >>> base_dataset = SiameseDataset(mnist, mnist)
+            >>> dataset = MixUpSoftLabelDataset(base_dataset)


You need to supply the number of class to MixupSoftLabelDataset

yuyu2172 · 2018-03-21T02:34:19Z

The link to the paper is not compiled properly.

fukatani · 2018-03-25T04:58:03Z

Thank you for your attentive review.
(And sorry for my poor doc)

I addressed your comments.

fukatani · 2018-03-25T04:59:26Z

I confirmed document was compiled successfully. (And changed reference style as other arxiv paper in chainercv)

yuyu2172 · 2018-03-26T00:31:58Z

docs/source/reference/datasets.rst

@@ -18,6 +18,10 @@ SiameseDataset
 ~~~~~~~~~~~~~~
 .. autoclass:: SiameseDataset

+MixUpSoftLabelDataset


Can you move this before SiameseDataset?
We are ordering docs in alphabetical order.

yuyu2172 · 2018-03-26T00:33:21Z

chainercv/datasets/mixup_soft_label_dataset.py

+        return len(self._dataset)
+
+    def get_example(self, i):
+        image1, label1, image2, label2 = self._dataset[i]


How about changing the names of the variables to img_0, label_0, img_1, label_1 so that they are consistent with the documentation?

fukatani · 2018-03-26T14:15:06Z

Thanks! I addressed your comments.

yuyu2172 · 2018-04-04T06:16:23Z

LGTM. Thank you for your contribution!

yuyu2172 reviewed Dec 21, 2017

View reviewed changes

fukatani force-pushed the between-class branch from c9c99ae to 66f981f Compare March 3, 2018 06:16

fukatani changed the title ~~[WIP] Add BetweenClassLabeledImageDatasets.~~ Support MixUpSoftLabelDataset. Mar 3, 2018

yuyu2172 reviewed Mar 21, 2018

View reviewed changes

yuyu2172 self-assigned this Mar 21, 2018

yuyu2172 reviewed Mar 21, 2018

View reviewed changes

fukatani added 11 commits March 25, 2018 13:49

WIP add BetweenClassLabeledImageDatasets.

e16f061

Use siamese dataset and rename to mixup from between class

bdc7a4c

rename

4ecf721

flake8

438df0e

fix doc

4098854

fix doc

619f4dd

addressed review comments

59dece5

Addressed review comments

66b3776

addressed review comments

0a039a7

fix doc

913a24d

fix doc

7d9fed5

fukatani force-pushed the between-class branch from 409209e to 7d9fed5 Compare March 25, 2018 04:51

fix doc

5ef37fd

yuyu2172 reviewed Mar 26, 2018

View reviewed changes

fukatani added 2 commits March 26, 2018 23:12

change argument name as doc

2ac2b38

doc keep alphabetical order

b4415df

yuyu2172 approved these changes Apr 4, 2018

View reviewed changes

yuyu2172 merged commit 74de140 into chainer:master Apr 4, 2018

yuyu2172 added this to the v0.9 milestone Apr 17, 2018

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Support MixUpSoftLabelDataset. #497

Support MixUpSoftLabelDataset. #497

fukatani commented Dec 17, 2017

yuyu2172 commented Dec 21, 2017 •

edited

yuyu2172 Dec 21, 2017

fukatani commented Dec 22, 2017

yuyu2172 Mar 21, 2018

yuyu2172 Mar 21, 2018

yuyu2172 Mar 21, 2018

yuyu2172 Mar 21, 2018

yuyu2172 Mar 21, 2018

yuyu2172 Mar 21, 2018

yuyu2172 Mar 21, 2018

yuyu2172 Mar 21, 2018

yuyu2172 Mar 21, 2018

yuyu2172 Mar 21, 2018

yuyu2172 Mar 21, 2018

yuyu2172 commented Mar 21, 2018

yuyu2172 Mar 21, 2018

yuyu2172 commented Mar 21, 2018

fukatani commented Mar 25, 2018

fukatani commented Mar 25, 2018 •

edited

yuyu2172 Mar 26, 2018

yuyu2172 Mar 26, 2018

fukatani commented Mar 26, 2018

yuyu2172 commented Apr 4, 2018


		"""

		def __init__(self, dataset, dtype=numpy.float32, label_dtype=numpy.float32,


		"""Dataset which returns mixed images and labels for mixup learning[1].

		`MixUpSoftLabelDataset` mix two images and labels which is chosen by base

Support MixUpSoftLabelDataset. #497

Support MixUpSoftLabelDataset. #497

Conversation

fukatani commented Dec 17, 2017

yuyu2172 commented Dec 21, 2017 • edited

Choose a reason for hiding this comment

fukatani commented Dec 22, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

yuyu2172 commented Mar 21, 2018

Choose a reason for hiding this comment

yuyu2172 commented Mar 21, 2018

fukatani commented Mar 25, 2018

fukatani commented Mar 25, 2018 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

fukatani commented Mar 26, 2018

yuyu2172 commented Apr 4, 2018

yuyu2172 commented Dec 21, 2017 •

edited

fukatani commented Mar 25, 2018 •

edited