Deep Learning

Welcome to the repository for my Deep Learning course at Sharif University, taught by Dr. Fatemizadeh.

About the Course

• Course Name: Deep Learning
• Instructor: Dr. Fatemizadeh
• Semester: 2023

This course is designed to provide a comprehensive understanding of deep learning and its applications. Throughout the semester, students will work on various assignments and projects that cover a wide range of topics within the field of deep learning.

Syllabus

This course covers a wide range of topics in deep learning, providing students with both theoretical understanding and practical implementation skills. Here are the key subjects covered in the course:

Introduction/Overview

• Importance and applications of machine learning
• Basics of machine learning concepts
• Rapid review of key mathematical foundations

Neural Networks

• Single layer and multilayer perceptrons
• Error backpropagation and key theorems
• Regularization, normalization, and optimization techniques (e.g., stochastic gradient descent)

Convolutional Neural Networks (CNNs)

• History, architecture, and learning of CNNs
• Application of CNNs in computer vision
• Key CNN models (e.g., AlexNet, ResNet)

Recurrent Neural Networks (RNNs)

• RNNs, LSTM, GRUs, and Transformers
• Applications in natural language processing (NLP) and signal/image processing

Unsupervised Learning

• Autoencoders
• Variational autoencoders
• Conditional variational autoencoders

Generative Models

• Generative adversarial networks (GANs)
• Conditional GANs
• Important architectures and applications
• Diffusion models

The syllabus aims to provide students with a comprehensive understanding of major deep learning architectures, concepts, and applications. It includes a balance of theory and practical implementation to ensure a well-rounded learning experience.

Repository Structure

• Assignment 1: Review of Essential ML Concepts
• Assignment 2: Neural Networks, Optimization, Regression
• Assignment 3: Convolutional Neural Networks
• Assignment 4: RNNs, LSTM, VAEs, AEs, GPT2
• Project: GAN-BERT
• Slides
• Exams

Assignment 1: Review of Essential ML Concepts

The first assignment focuses on reviewing essential machine learning concepts, serving as a foundational exercise to ensure a strong understanding of core principles and techniques used in machine learning. It includes theoretical questions and practical coding tasks covering decision trees, PCA, and classification problems using datasets like MNIST and the Heart Disease Dataset.

Assignment 2: Neural Networks, Optimization, Regression

This assignment covers building neural networks, optimization, and regression. It includes theoretical questions on neural network convergence and coding tasks such as implementing fully connected neural networks and a novel "forward-forward propagation" algorithm, tested on the MNIST dataset.

Assignment 3: Convolutional Neural Networks

This assignment focuses on Convolutional Neural Networks (CNNs) for computer vision tasks. It covers theory and coding sections, including topics like CNN architectures (e.g., ResNet), advanced techniques such as deformable convolutions, and change detection between images. Theoretical questions explore concepts like Densely Connected CNNs and U-Net, while coding tasks involve implementing and evaluating models on datasets like CIFAR-10.

Assignment 4: RNNs, LSTM, VAEs, AEs, GPT2

This assignment covers a diverse range of deep learning models, including RNNs, LSTM, autoencoders, and GPT2. Tasks include applying LSTM and BiLSTM to molecular strings, implementing Variational Autoencoders on MNIST, fine-tuning GPT2 on poetry generation, and testing Vector-Quantized Variational Autoencoders on MNIST and its color version.

Project: GAN-BERT

The GAN-BERT project focuses on text classification using a GAN-BERT model on the SemEval-2024 dataset. It involves training a BERT classifier on labeled data and then integrating a GAN architecture for text classification. The GAN-BERT model consists of a Discriminator and Generator, trained adversarially alongside the BERT model.

Contact

If you have any questions or want to connect regarding the assignments or deep learning, feel free to reach out:

Mohammad Javad Amin

• M.j.amin200@gamil.com

I hope you find the assignments and code in this repository insightful and informative. Happy learning!