Add ConvNet tutorials #2337

mitmul · 2017-02-27T06:59:53Z

Convolutional Network for Visual Recognition Tasks
- How to write a convolutional network that has a large number of components (ResNet)

bkvogel

Thanks for the tutorial! I think the content looks good, so I have only suggested some grammatical and wording changes.

I also wonder if it could be useful for one of the examples to use the parameter shape placeholder feature so that users can know about it, but I don't think it is required. You can decide.

bkvogel · 2017-04-20T03:15:58Z

docs/source/tutorial/convnet.rst

+
+After reading this section, you will be able to:
+
+* Write your original convolutional network in Chainer


Consider rewording:

Write your own original convolutional network in Chainer

bkvogel · 2017-04-20T03:46:01Z

docs/source/tutorial/convnet.rst

+* Write your original convolutional network in Chainer
+
+A convolutional network (ConvNet) is mainly comprised of convolutional layers.
+This type of network is oftenly used for various visual recognition tasks,


This type of network is commonly used for

bkvogel · 2017-04-20T23:29:05Z

docs/source/tutorial/convnet.rst

+- :math:`H` and :math:`W` denote the height and width of those images,
+
+respectively. Then, it typically outputs a fixed-sized vector as membership
+probability over the target object classes. It also can output a set of feature


Then, it typically outputs a fixed-sized vector of membership probabilities over the target object classes.

bkvogel · 2017-04-20T23:34:58Z

docs/source/tutorial/convnet.rst

+LeNet5
+''''''
+
+Here, let's start from defining LeNet5 [LeCun98]_ in Chainer.


Here, let's start by defining LeNet5 [LeCun98]_ in Chainer.

bkvogel · 2017-04-20T23:35:46Z

docs/source/tutorial/convnet.rst

+
+Here, let's start from defining LeNet5 [LeCun98]_ in Chainer.
+This is a ConvNet model that has 5 layers comprised of 3 convolutional layers
+and 2 fully-connected layers. This has been proposed to classify hand-written


This was proposed

bkvogel · 2017-04-21T01:39:47Z

docs/source/tutorial/convnet.rst

+In :class:`BottleNeck` class, it switches whther ``conv1x1r`` to fit the
+channel dimension of the input ``x`` and the output ``h`` and ``bn_r`` for it
+should be added or not with respect to :attr:`~BottleNeck.proj` attribute.
+Writing the building block in this way gains the re-usability of a class a lot.


In the :class:BottleNeck class, depending on the value of the proj argument supplied to the initializer, it will conditionally also compute the batch normalization of a conv1x1r layer before the final ReLU layer. Writing the building block in this way improves the re-usability of a class.

@bkvogel Thank you for great re-wording! In this sentence, I'd like to say the computation of conv1x1r itself is also conditional. So can I say as the below? :

it will conditionally compute a convolutional layer conv1x1r and the following batch normalization layer before the final ReLU layer

I'd appreciate it if you could check this part again :)

Sorry, additionally, I'd like to say the conv1x1r is used to make the number of channels of the input x same as the number of channels of the output of conv1x1c. This part "conv1x1r to fit the channel dimension of the input x and the output h" intended to say that. Could you advise me how to add this information to this paragraph? How about this?:

it will conditionally compute a convolutional layer conv1x1r which is to make the number of channels of the input x same as the number of channels of the output of conv1x1c, and the following batch normalization layer before the final ReLU layer.

How about:

it will conditionally compute a convolutional layer conv1x1r which will extend the number of channels of the input x to be equal to the number of channels of the output of conv1x1c, and followed by a batch normalization layer before the final ReLU layer.

bkvogel · 2017-04-21T01:40:28Z

docs/source/tutorial/convnet.rst

+usage would be efficient compared to the case when it registers both anyway and
+just ignore them if :attr:`proj` is ``False``.
+
+Using nesting :class:`~chainer.Chain` s and :class:`~chainer.ChainList` for


Using nested

bkvogel · 2017-04-21T01:44:04Z

docs/source/tutorial/convnet.rst

+very useful as general feature extractor for many different tasks from image
+classification. So Chainer provides you the pre-trained models of VGG16 and
+ResNet-50/101/152 models with a simple API. You can use those models in this
+way:


Various ways to write your models were described above. It turns out that VGG16 and ResNet are very useful as general feature extractors for many different image classification tasks. So, Chainer provides you with the pre-trained VGG16 and ResNet-50/101/152 models with a simple API. You can use these models as follows:

Sorry, I intended to say "those models are useful not only for classification tasks but also for many different tasks other than classification tasks". So can I say this as follows?:

It turns out that VGG16 and ResNet are very useful as general feature extractors noy only for image classification tasks but also for many different tasks other than classification.

I'd appreciate it if you could check this part again :)

How about:

Various ways to write your models were described above. It turns out that VGG16 and ResNet are very useful as general feature extractors for many kinds of tasks, including but not limited to image classification. So, Chainer provides you with the pre-trained VGG16 and ResNet-50/101/152 models with a simple API. You can use these models as follows:

bkvogel · 2017-04-21T01:45:35Z

docs/source/tutorial/convnet.rst

+    model = VGG16Layers()
+
+When :class:`~chainer.links.VGG16Layers` is instantiated, the pre-trained
+parameters is automatically downloaded from the author's server. So you can


the pre-trained parameters are automatically downloaded

bkvogel · 2017-04-21T01:46:46Z

docs/source/tutorial/convnet.rst

+:class:`chainer.links.VGG16Layers`.
+
+In the case of ResNet models, there are 3 variation in terms of the number of
+layers. We have :class:`chainer.links.ResNet50`,


In the case of ResNet models, there are three variations differing in the number of layers.

mitmul · 2017-04-21T03:01:02Z

@bkvogel Thank you for reviewing! I'd like to confirm two points replied in the above comments. I'd appreciate it if you could check them again :)

bkvogel · 2017-04-21T05:02:48Z

docs/source/tutorial/convnet.rst

+        def __call__(self, x, train):
+            h = F.relu(self.bn_a(self.conv1x1a(x), test=not train))
+            h = F.relu(self.bn_b(self.conv3x3b(x), test=not train))
+            h = self.bn_c(self.conv1x1c(x), test=not train)


Is this code correct? It looks like you keep overwriting h because the value is not used in the following lines (it keeps reading from x only).

Sorry, this code is wrong. I'll fix it. Thank you for pointing out it!

mitmul · 2017-04-21T07:03:27Z

Thank you for replying my comments. I fixed all points by following your suggestions :)

bkvogel · 2017-04-21T10:08:10Z

LGTM!

mitmul added 2 commits February 27, 2017 15:58

Create files

7cff00b

Add LeNet5

373f3a1

mitmul added cat:document Documentation such as function documentations, comments and tutorials. reviewer-team labels Mar 8, 2017

mitmul and others added 3 commits March 13, 2017 10:04

Merge branch 'master' into add-conv-tutorial

a1715c0

Merge remote-tracking branch 'origin/master' into add-conv-tutorial

f1e357e

Update conv tutorial

e88fee0

This was referenced Apr 17, 2017

Reorgnize tutorial & add new contents #2576

Closed

Improve All Docs toward v2 #1867

Closed

mitmul added 4 commits April 18, 2017 13:14

Finished LeNet5 part

ec120e9

Reorganize tutorial part

239cd1d

Show second depth contents for better index

9455495

Finished writing

591849a

mitmul changed the title ~~[WIP] Add ConvNet tutorials~~ Add ConvNet tutorials Apr 18, 2017

mitmul added 2 commits April 18, 2017 16:01

Fix a link

312a7a2

Fix a link again

180e27c

bkvogel self-assigned this Apr 18, 2017

bkvogel requested changes Apr 21, 2017

View reviewed changes

bkvogel reviewed Apr 21, 2017

View reviewed changes

Revised to follow reviews

84d3e3d

bkvogel approved these changes Apr 21, 2017

View reviewed changes

bkvogel merged commit 71fc7b8 into master Apr 21, 2017

bkvogel deleted the add-conv-tutorial branch April 21, 2017 10:09

beam2d added this to the v1.24.0 milestone May 11, 2017

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add ConvNet tutorials #2337

Add ConvNet tutorials #2337

mitmul commented Feb 27, 2017 •

edited

bkvogel left a comment

bkvogel Apr 20, 2017

bkvogel Apr 20, 2017

bkvogel Apr 20, 2017

bkvogel Apr 20, 2017

bkvogel Apr 20, 2017

bkvogel Apr 21, 2017

mitmul Apr 21, 2017 •

edited

mitmul Apr 21, 2017 •

edited

bkvogel Apr 21, 2017

bkvogel Apr 21, 2017

bkvogel Apr 21, 2017

mitmul Apr 21, 2017

bkvogel Apr 21, 2017

bkvogel Apr 21, 2017

bkvogel Apr 21, 2017

mitmul commented Apr 21, 2017

bkvogel Apr 21, 2017

mitmul Apr 21, 2017

mitmul commented Apr 21, 2017

bkvogel commented Apr 21, 2017


		After reading this section, you will be able to:

		* Write your original convolutional network in Chainer

Add ConvNet tutorials #2337

Add ConvNet tutorials #2337

Conversation

mitmul commented Feb 27, 2017 • edited

bkvogel left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mitmul Apr 21, 2017 • edited

Choose a reason for hiding this comment

mitmul Apr 21, 2017 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mitmul commented Apr 21, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mitmul commented Apr 21, 2017

bkvogel commented Apr 21, 2017

mitmul commented Feb 27, 2017 •

edited

mitmul Apr 21, 2017 •

edited

mitmul Apr 21, 2017 •

edited