Added first version of episode 4 #58
Conversation
| > {: .solution} | ||
| {: .challenge} | ||
|
|
||
|
|
There was a problem hiding this comment.
I would cut the lesson here. We should try to focus on the exercises at this point and perhaps "train" an entire network with convolutions for the fun of it. I also end my review here.
There was a problem hiding this comment.
I kinda agree that training a NN with convolutions here would be nice.
Introduce a concept and use it, then move on to the next concept.
I do feel that the drop out is very useful to have in this lesson as it helps a lot against overfitting.
There was a problem hiding this comment.
But would you suggest then to not introduce the pooling layers? Or only introduce them after we first train a CNN without pooling layers?
I think CNNs almost always include some sort of pooling and they are easier to understand than convolutional layers, so I prefer to keep them in the same episode.
But I will try to add an exercise in between, before we introduce the pooling layer.
|
@dafnevk this is quite some material. I stopped my review after the convolution part. Reading through the remaining material with MaxPooling and Dropout, cutting the lesson after the convolutions appears quite natural to me. A subsequent lesson could then add more types of layers for our onion and motivate them. I suggest to work towards merging this in for now and then start a separate PR on a split. |
Co-authored-by: Peter Steinbach <p.steinbach@hzdr.de>
There was a problem hiding this comment.
I think this is very nice episode, but I have added a few small comments.
I see you're using the active voice a lot, which on the one had is good, but sometimes the sentence does not flow very well with it. For instance: 'We have a look at the CIFAR10 dataset', and 'We take a small sample of the data...'.
| plt.title(class_names[train_labels[i,0]]) | ||
| plt.show() | ||
| ~~~ | ||
| {: .language-python} |
There was a problem hiding this comment.
I think this code is not correct, from the 2nd row onwards the labels do not match the pictures
There was a problem hiding this comment.
Hm that is strange! This requires some further investigations.
For me, the output pictures match the labels, but depending on the matplotlib version (I think, could also be a different package that causes this) the images are shown upside down.
| > > We have 100 matrices with 3 * 3 * 3 = 27 values each so that gives 27 * 100 = 2700 weights. This is a magnitude of 100 less than the fully connected layer with 100 units! | ||
| > {: .solution} | ||
| {: .challenge} | ||
|
|
There was a problem hiding this comment.
I would mention/reitterate something about the performance or expressiveness of convolutional layers or allude to it for later. Because now I had the though: "Ok it's a lot smaller, but that doesn't mean much"
| > {: .solution} | ||
| {: .challenge} | ||
|
|
||
|
|
There was a problem hiding this comment.
I kinda agree that training a NN with convolutions here would be nice.
Introduce a concept and use it, then move on to the next concept.
I do feel that the drop out is very useful to have in this lesson as it helps a lot against overfitting.
First version of episode 4, using the CIFAR 10 dataset6. See #45 .