Better question and answer for deep learning FAQ

[BoltzmannBrain]

No description provided.

[rasbt]

Hi there. Thanks for the pull request. I agree that the way this was written is maybe not ideal -- the origin was (I believe), an email response to someone who asked me this question, which is the reason for this "brevity" -- it's not meant to be a comprehensive article (like a blog article I would write); but I quite literally copy & pasted my answer here in hope that it could be hopeful to others.

Deep learning refers to neural network structures with many layers and parameters that learn features in data.

I agree with this, but I think it is also a bit too general. I'd say that deep learning is not just a multi-layer perceptron with many layers, for example. Deep learning is more about algorithms to process data using multiple processing layers to learn a representation of the data as input for e.g., a MLP.

Yeah, we could think of it as a fancy, new term for neural networks, or a "rebranding." However, I think the key difference between the multi-layer neural networks e.g., 20-50 years ago and now is the "learning the feature representation part" (vs. handcrafted features)

[BoltzmannBrain]

Okay thanks for the explanation. Would you like me to edit my PR then, or are you going to handle the FAQ section on your own with these "email responses"? And in general it is unclear whether or not you want others to contribute to this book...

[rasbt]

And in general it is unclear whether or not you want others to contribute to this book...

I definitely appreciate any sort of feedback and suggestions, and pull requests for fixes! :). I should maybe say that I created this repository to "just" share the code examples. However, I got a couple of requests and questions, which is why I added this FAQ section as some sort of bonus material. I.e., these are the more general questions I was asked, and I thought it maybe be useful to other to share them (also it's easier for me to provide a link to an answer I have already given if someone asks a similar question). Here, improvements are always welcome :).

Regarding the other contents that are more specific to the book, for example, the code notebooks ... Here, the contents shouldn't be changed except for additional notes or bugfixes since it would become inconsistent with the book's content otherwise (and I really have no control about it anymore since it is in the publisher's hand ... unless there'll be a 2nd edition, of course).

Would you like me to edit my PR then, or are you going to handle the FAQ section on your own with these "email responses"?

I noticed that there was a merge conflict, which is probably due to a typo fix #12 that happened shortly before your edit. If you have some time to edit your pull request and your are okay with my suggestions below, I'd appreciate it and be happy to merge it.

What's an intro to deep learning and how does it relate to usual machine learning?

I'd suggest "What is the relationship between deep learning and regular machine learning on a conceptual level?" as a title here.

The tl;dr version of this is: Deep learning refers to neural network structures with many layers and parameters that learn features in data. Deep learning methods can be applied to many machine learning problems.

Deep learning refers to neural network structures with many layers and parameters that learn features in data. Deep learning methods can be applied to many machine learning problems.

Before giving an intro to deep learning, I'll answer the second part of the question. Deep learning encompasses a set of machine learning algorithms, specifically that employ artificial neural network architectures with many hidden layers. Deep learning is yet another tool in the machine learning practitioner's toolbox to solve problems like classification and prediction.

And I'd replace the previous three paragraphs by this one:

In essence, deep learning offers a set of techniques and algorithms that help us to parameterize deep neural network structures -- artificial neural networks with many hidden layers and parameters.
One of the key ideas behind deep learning is to extract high level features from the given dataset. Thereby, deep learning aims to overcome the challenge of the often tedious feature engineering task and to overcome challenges in parameterizing traditional neural networks with many layers.

Now to introduce deep learning I'll go through a concrete example:

And change the sentence above to

Now, to introduce deep learning, let us take a look at a more concrete example involving multi-layer perceptrons (MLPs).

[rasbt]

If you don't mind, I just fixed another typo and applied these changes alongside. Thanks again for taking the time for providing feedback, it's honestly appreciated!

[BoltzmannBrain]

Sorry for the delayed response. Thank you for the detailed answers, and your changes look good 👍