|
COL 865: Special Topics in Computer Applications - Deep Learning
General Information
Instructor: Parag Singla (email: parags AT cse.iitd.ac.in)
Class Timings/Venue:
Slot H. Time (Modified): Monday - 2:00 pm - 3:20 pm. Thursday - 12:00 noon - 1:20 pm.
- Slot H. Time: Monday, Wednesday - 11:00 am - 11:50 pm. Thursday - 12:00 noon - 12:50 pm.
- Venue: Bharti 305.
Office Hours:
Teaching Assistants:
Ankit Anand (csz138105 AT cse), Akshay Gupta (cs5130275 AT cse)
Announcements
- [October 2, 2017] Check out the updated grading policy!
- [September 22, 2017] We are planning to use Microsoft Azure Credits for part of the assignments in the course. Thanks to Microsoft for the credits!
- [September 22, 2017] Course Webpage is finally up! Yay!!
Obective:
This course is meant to be the first graduate level course in deep learning. Deep Learning is
an emerging area of Machine Learning which has revolutionized the progress in the field during
last few years with applications found in NLP, Vision and Speech to name a few domains. This
course is intended to give a basic overview of the mathematical foundations of the field, and
present the standard techniques/arhitectures which become basis for more advanced ones. About a
3rd of the course will focus on latest research in the area through research paper dicussions.
Without an implementation, no deep learning class can be complete. Students will get to implement
some of the architectures (CNNs/LSTMs etc.) on a GPU to test on large datasets. Students will also
likely get some experience with cloud computing facilities such as Microsoft Azure and/or other
HPC systems.
For 2017-18 Sem I offering: We plan to use Data Science Virtual Machine (DSVM) service provided
by Microsoft Azure so that students can collaborate with each other for building deep learning
models on large amount of data.
Content: Basics: Introduction. Multi-layered Perceptrons. Backpropagation. Regularization:
L1-L2 Norms. Dropouts. Optimization: Challenges. Stochastic Gradient Descent. Advanced Optimization
Algorithms. Convolutional Neural Networks (CNNs). Advanced Architectures for Vision. Recurrent Neural
Networks. Long Short Term Memory (LSTMs). Gated Recurrent Units (GRUs). Attention. Word Vectors.
Generative Adversarial Networks (GANs). Deep Re-inforcement Learning. More Recent Advances in
the field.
Week-Wise Schedule
Week | Topic | Book Chapters | Class Notes/ Supplementary Material
| 1 | Introduction, Motivation | |
|
2 | Mulit-layered Perceptrons, Backpropagation | Goodfellow et al. Chapter 6 |
|
3 | Regularization Techniques | Goodfellow et al. Chapter 7 |
|
4,4.5 | Optimization | Goodfellow et al. Chapter 8 |
|
5 | Convolutional Neural Networks (CNNs) | Goodfellow et al. Chapter 9 |
|
6 | CNNs - Advanced Architectures |
Slides |
|
7 | Recurrent Neural Networks (RNNs) | Goodfellow et al. Chapter 10 |
|
8 | LSTMs, GRUs | Goodfellow et al. Chapter 10 |
|
8.5 | Word-2-Vec |
Slides |
9 | Generative Adversarial Learning (GANs) | |
|
10 | Research Paper Discussions | |
|
11 | Deep Reinforcement Learning | |
|
12 - 14 | Research Paper Discussions | |
|
Guidelines & Detailed Schedule
Chapters 1 - 5, Goodfellow et al. |
References
- Deep Learning. Ian Goodfellow, Yoshua Bengio and Aaron Courville. 2016.
Assignment Submission Instructions
- You are free to discuss the problems with other students in the class. But the final solution/code that you
produce should come through your individual efforts.
- Required code should be submitted using Moodle Page.
- Honor Code: Any cases of copying will be awarded a zero on the assignment. Additional
penalities will be imposed based on the severity of copying. Any copying cases run the chances of being
escalated to the Department/DISCO.
- Late policy:
You will lose 20% of the score for every late day in submission. Maximum of two late
days are allowed for any given assignment. You are allowed a total of 2 buffer days for the two
programming assignments. There is no penalty if your submission stays withing the limit of the 2 buffer days
(total). Any delay beyond 2 days will result in a zero on your submission(s).
Assignments
-
Assignment
1: Implement AlexNet on a Subset of classes in the Imagenet Dataset. Implement an LSTM Cell (tentative) [Weight : 10%]. Due: Sunday October 15, 11:50 pm.
-
Assignment 2. Open ended implementation of a deep learning system. Details to be decided.
We are planning to use Microsoft Azure Credits for part of the above assignments. Thanks to Microsoft for the credits!
Grading Secheme (Tentative)
Assignment 1 | 10% |
Assignment 2 | 15% |
Class Presentation + Reviewing | 15% (Presentation:6% + Reviewing:9%) |
Minor 1/Assignment 0 | 6% |
Minor 2 | 20% |
Major | 34% |
|