COL 870: Special Topics in Machine Learning - Deep Learning
(Semester II, 2020-21)



General Information

Instructor: Parag Singla (email: parags AT cse.iitd.ac.in)

Class Timings/Venue:
  • Slot C. Tuesday, Wednesday, Friday: 8 am - 9 am.
  • Venue: Online (link to be provided)

Teaching Assistants:
To Be Decided

Semester

Semester II, 2020-21

Pre-requisites

A foundational course in AI or ML.

Course Objective and Content

Obective: This course is meant to be the first graduate level course in deep learning. Deep Learning is an emerging area of Machine Learning which has revolutionized the progress in the field during last few years with applications found in NLP, Vision and Speech to name a few domains. This course is intended to give a basic overview of the mathematical foundations of the field, and present the standard techniques/arhitectures which become basis for more advanced ones. About a 3rd of the course will focus on latest research topics in the area. Without an implementation, no deep learning class can be complete. Students will get to implement some of the architectures on a GPU to test on large datasets.

Content: Basics: Introduction. Multi-layered Perceptrons. Backpropagation. Regularization: L1-L2 Norms. Dropouts. Optimization: Challenges. Stochastic Gradient Descent. Advanced Optimization Algorithms. Convolutional Networks (CNNs). Recurrent Architectures. Dropout, Batch Normalization. Generative Architectures. Advanced Architectures for Vision. Advanced Architectures for NLP. More Recent Advances in the field.

Week-Wise Schedule

NOTE: The exact list of topics below is tentative (until we are past that week). We will update it as we go through the lectures in each week. So, stay tuned!

WeekTopic Book ChaptersSupplementary Material
1 Introduction, Motivation
2 Mulit-layered Perceptrons, Backpropagation Goodfellow et al. Chapter 6
3 Regularization, Optimization Goodfellow et al. Chapter 7, Chapter 8
4 Convolutional Networks (CNNs) Goodfellow et al. Chapter 9
5 Recurrent Architectures Goodfellow et al. Chapter 10
6 Generative Architectures
7 Dropout, Batch Normalization
8 Deep Learning for Vision
9 Deep Learning for NLP
10 Advanced Topics
11 Advanced Topics
12 Advanced Topics
13 Advanced Topics
14 Advanced Topics


Additional Reading

Review Material

Chapters 1 - 5, Goodfellow et al.

References

  1. Deep Learning. Ian Goodfellow, Yoshua Bengio and Aaron Courville. 2016.

Assignment Submission Instructions

  1. You are free to discuss the problems with other students in the class. But the final solution/code that you produce should come through your individual efforts.
  2. Required code should be submitted using Moodle Page.
  3. Honor Code: Any cases of copying will be awarded a zero on the assignment. Additional penalities will be imposed based on the severity of copying. Any copying cases run the chances of being escalated to the Department/DISCO.
  4. Late policy: You are allowed a total of 5 buffer days for the three programming assignments. There is no penalty if your submission stays withing the limit of the 5 buffer days (total). For each additional day beyond the allowed 5 buffer days, you will lose 10% of the score for every late day in submission.

Assignments

Grading Secheme (Tentative)
Assignment 1 10%
Assignment 2 15%
Assignment 3 15%
Minor 25%
Major 35%