
COL 775: Deep Learning (Semester II, 202223)
Semester: Sem II, 202223.
Instructor: Parag Singla (email: parags AT cse.iitd.ac.in)
Class Timings/Venue:
 Slot C. Tuesday, Wednesday, Friday: 8 am  9 am. [Any makeup classes will happen on Saturdays 8 am  9 am].
 Venue: LHC 623
Teaching Assistants:
Anshul Mittal (Email: anz198717 AT sit), Daman Arora (Email: cs5180404 AT cse)
Registration Class List
Fill the following form if you are interested in registering for the course (and not yet registered)  only for CSE/ScAI/SIT students at this point.
Registration Form
Class List and TA Assignment: Coming soon!
Sign up for Piazza
Code: same as the course number (in small letters, with no spaces).
Prerequisites
A foundational course in AI or ML.
Announcements
[Jan 7, 2023]: Makeup class organized.
Obective:
This course is meant to be the first graduate level course in deep learning. Deep Learning is
an emerging area of Machine Learning which has revolutionized the progress in the field during
last few years with applications found in NLP, Vision and Speech to name a few domains. This
course is intended to give a basic overview of the mathematical foundations of the field, and
present a number of techniques/arhitectures which have become defacto standard for a large number
of applications in vision, NLP and other areas. We will also touch upon some of the recent advances
in the field. become basis for more advanced ones. Without an implementation, no
deep learning class can be complete. Students will get to implement some of the architectures on
a GPU to test on large datasets.
Content: Basics: Introduction. Multilayered Perceptrons. Backpropagation. Regularization:
L1L2 Norms. Dropouts. Optimization: Challenges. Stochastic Gradient Descent. Advanced Optimization
Algorithms. Batch and Layer Normalization. Convolutional Networks (CNNs). Pretrained networks.
Recurrent Architectures. Transformers. Generative Architectures. Graph Neural Networks. Deep Reinforcement
Learning. Advanced Topics.
WeekWise Schedule (to be refined as we progress in the semester)
NOTE: The exact list of topics below is tentative (until we are past that week).
We will update it as we go through the lectures in each week. So, stay tuned!
Week  Topic  Book Chapters  Supplementary Material 
1  Introduction, Motivation  

2  Mulitlayered Perceptrons, Backpropagation  Goodfellow et al. Chapter 6 

3  Regularization  L1/L2, Other Techniques  Goodfellow et al. Chapter 7 

4  Optimization Techniques. Normalization  Goodfellow et al. Chapter 8 

5  Deep Learning for Vision  Basic Models (CNNs)  Goodfellow et al. Chapter 9 

6  Deep Learning for Vision (Architectures), PreTrained Models  
See list of papers 
7  Deep Learning for NLP  Basic Models (RNNs,LSTM)  Goodfellow et al. Chapter 10 

8  Deep Learning for NLP (Attention), Transformers  
See list of papers 
9  Transformers  continued  
See list of papers 
10  Generative Models  
See class notes, list of papers 
11  Generative Models  continued  
See class notes, list of papers 
11  Graph Neural Networks  

12  Deep Reinforcement Learning  

14  Advanced Topics  

Class Notes:
Jan 4, Jan 6, Jan 7,
Jan 10, Jan 11,Jan 13,
Jan 17, Jan 20,Jan 21,
Jan 24, Jan 25,Jan 28,
Jan 31,Feb 1,Feb 3,
Feb 10
Link to list papers covered (primarily in second half)
Videos:
Jan 7th Class, Apr 1st Class.
Chapters 1  5, Goodfellow et al. 
Books/References
 Deep Learning. Ian Goodfellow, Yoshua Bengio and Aaron Courville. 2016.
Assignment Submission Instructions
 You are free to discuss the problems with other students in the class. But the final solution/code that you
produce should come through your individual efforts.
 Required code should be submitted using Moodle Page.
 Honor Code: Any cases of copying will be awarded a zero on the assignment. Additional
penalities will be imposed based on the severity of copying. Any copying cases run the chances of being
escalated to the Department/DISCO.
 Late policy: You are allowed a total of 3 buffer days for the first programming assignment.
There is no penalty if your submission stays within the limit of the 3 buffer days
(total). For each additional day beyond the allowed 3 buffer days, you will lose 10% of the score for every
late day in submission. Since second assignment will likely have a competitive part (to be communicated later), buffer days may not be
applicable. We will update with more details as get closer to assignments.
Practice Questions
Assignments
 Assignment 1
Due Date: Wednesday March 15, 2022, 11:50 pm.
Grading Secheme (Tentative)
Assignment 0 (tentative)  0% 
Assignment 1 (Part A, Part B)  15% 
Assignment 2 (Project)  20% 
Minor  25% 
Major  40% 
