Skip to content
Deep Learning Resources and Tutorials using Keras and Lasagne
Branch: master
Clone or download
Latest commit d85a40b Jun 1, 2018
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
keras Merge pull request #20 from mehrdadscomputer/master Jan 2, 2018
lasagne fiw link recurrent.md + code lasagne recurrent + update readmes Jan 27, 2016
yelp_data format Jun 1, 2018
.gitignore black May 30, 2018
License LICENSE -> License Jan 12, 2016
README.md
aws_gpu.md
recurrent.md fiw link recurrent.md + code lasagne recurrent + update readmes Jan 27, 2016
setup.md +moving Dec 13, 2016
theano.md +moving Dec 13, 2016
useful_resources.md

README.md

MOVING

I'm moving the tutorial to vict0rsch.github.io to make it nicer for you guys. This is an ongoing process and your feedbacks are welcome! Basically it will be betetr organized and there will be comments possible. This repo will stay active for the code, issues, pull requests and collaborations.

Deep-Learning

Table Of Contents

Presentation

How to Learn from this Tutorial

The Toothbrush Technique

Disclaimer

Presentation

This repository contains Deep Learning implementations tutorials. For more general knowledge regarding Machine/Deep Learning, have a look at useful_resources.md.

Lasagne and Keras are Theano-based so I recommend you get familiar with Theano before starting these ones.

However Keras is way closer to usual Python than Lasagne so it requires a weaker understanding of Theano. The main thing to understand to get started with Keras is Theano's graph structure.

We concentrate, in theano.md, on a few features of Theano that will be needed in Lasagne mostly and just a little in Keras. You will not learn Theano there but get a glimpse at how it works and how it can be used in a Deep Learning context. Theano is about much more than this, especially regarding GPU calculation and Automatic Differentiation. If you use TensorFlow (as a backend for Keras for instance) you can still go through the Theano introduction as Tensorflow's phylosophy on using graphs is very similar : you declare "sessions" instead of "compiling" the graph but the underlying process is conceptually the same.

See the official Theano tutorial here.

I have not worked a lot with Convolutional Networks so I won't mention them here, for now.

Set up Theano Keras Lasagne Recurrent Resources AWS + GPU Lose Time
i1 i2 i3 i4 i6 i5 i7 i6

Amazon Instances

You will find that Neural Network computations are very expensive and slow on CPUs. This is why all(?) such frameworks are GPU-accelerated. What if you don't have access to a GPU? You can still use Amazon's computers for ~1€/hour. See my attempt to a tutorial here.

How to Learn from this Tutorial

Machine learning is a vast area. Time and concentration are the two things you need the most to get into it. Don't jump to the next step if you're not sure you're clear with the present one's outcomes.

  1. Learn about Machine Learning -> Resources -> Starting with Machine Learning

    • Requirements: None. Except basic knowledge in maths
    • Outcomes: Understand what ((un)supervised)learning and training mean, what are some of the most famous techniques and the importance of data (feature selection/extraction, overfitting).
  2. Learn about Deep Learning Theory and feedforward networks (your best bet may very well be M. Nielsen's blog) -> Starting with Deep Learning

    • Requirements: Python, very basic linear algebra and analysis (vector products and differenciation basically) + outcome (1)
    • Outcomes: Understand how neural networks are built, trained, improved. Both on the theory and the implementation side. You'll also understand how networks are coded to get a sense of how frameworks work.
  3. Get familiar with Theano -> Theano

    • Requirements: Python
    • Outcomes: Be able to understand Theano code and write functions relying on (shared) variables.
  4. Get into some code

    a - Start easy with Keras and feedforward networks -> Keras

    * **Requirements**: Python + outcomes (1) and (2)
    * **Outcomes**: Understand how the Keras framework can be used and therefore implement any dense feedforward network you like.
    

    b - Go into the details with Lasagne (still with feedforward networks) -> Lasagne

    * **Requirements**: Python + outcomes (1), (2) and (3)
    * **Outcomes**: Understand how the Lasagne framework can be used and therefore implement any dense feedforward network you like. Understand the differences with Keras.
    
  5. Dig into Recurrent Networks -> Resources

    • Requirements: outcomes (1) and (2) (strong)
    • Outcomes: Understand the core concepts and usage of recurrent nets. Get the variety of structures.
  6. Spend some time understanding the handling of dimensions in recurrent nets -> Recurrent

    • Requirements: outcomes (1), (2), one of (4), (5)
    • Outcomes: Be able to create the appropriate dataset and format your data according to the task you seek.
  7. Get back to code

    • Requirements:
    • Outcomes:

The Toothbrush Technique

The Toothbrush technique is used to debug code. The concept is easy: pick up your toothbrush, a pen or a spoon and walk it through your code as if they understood it. Better yet use a friend or coworker: you won't need their brain, rather their ears.

The thing is that debugging can be hard and the error might very well be silly. However looking as someone else's code is often hard and/or laborious, so asking a friend/coworker to debug it is hardly possible.
On the other hand, explaining it to your toothbrush makes you rethink the whole coding process you went through and hopefully find that (silly?) mistake or incoherence.

Contact Clément to learn more or see the Feynman technique, the Nobel prize's famous technique to understand and remember things.

Disclaimer

I am currently an MSc. student so I don't have much time to improve regularly this tutorial and solve issues but I'll do my best. If you have some experience and would like to contribute get in touch and anyway feel free to send pull requests. See more at vict0rsch.github.io


Icons made by Freepik and Gregor Cresnar from Flaticon licensed by CC BY 3.0 and Favicon basic license

You can’t perform that action at this time.