COL774: Machine Learning



General Information

Semester: Sem II, 2018-19.

Instructor: Parag Singla (email: parags AT cse.iitd.ac.in)

Class Timings (Slot F):
  • Tue, 11:00 am - 11:50am
  • Thu, 11:00 am - 11:50am
  • Fri, 11:00 am - 11:50am
Venue: LHC 527.

Teaching Assistants (TAs)

TA Assignment

Sign up for Piazza
Code: As sent over email.

Announcements
  • [Apr 18, 2019]: Notes for the last class are also up. Check out the last two files in the notes section!
  • [Apr 26, 2019]: Class notes (until Apr 25) and last two years' major question papers are up now!
  • [Mar 28, 2019]: Updated Assignment 3 is now uploaded!
  • [Mar 15, 2019]: Assignment 3 is out!
  • [Mar 3, 2019]: Assignment 2 (second question) is out!
  • [Feb 15, 2019]: Assignment 2 (first question) is out!
  • [Feb 1, 2019]: Last year's Minor 1 exam is up for practice.
  • [Feb 1, 2019]: Class notes up to Feb 1 are up.
  • [Jan 22, 2019]: Assignment 1 is out!
  • [Jan 22, 2019]: Class notes up to Jan 18 are up.
  • [Jan 14, 2019] Sign-up for the course page on Piazza (code as sent over email).
  • [Jan 14, 2019]: Course website is finally up!

    Course Content

    NOTE: The exact list of topics below is still tentative (until we are past that week). We will update it as we go through the lectures in each week. So, stay tuned!
    Week Topic Supplementary Notes Book Chapters
    1 Introduction
    2,3 Linear Regression (and Its Variants) lin-log-reg.pdf Bishop, Chapter 3.1
    4 Logistic Regression, Generalized Linear Models lin-log-reg.pdf Bishop, Chapter 3.1
    5 Gaussian Discriminant Analysis (GDA), Naive Bayes gda_nb.pdf Bishop, Chapter 4
    6,7 Support Vector Machines svm.pdf Bishop, Chapter 7.1
    8 Decision Trees, Random Forests dtrees.pdf Mitchell, Chapter 3. Online Resource: Random Forests
    9 Neural Networks nnets.pdf nnets-hw.pdf Mitchell, Chapter 4
    10 Deep Learning deep_learning_slides.pdf
    cnn.pdf
    Online Resource:
    Convolutional Neural Networks
    11 K-Means, Gaussian Mixture Models kmeans.pdf gmm.pdf em.pdf pca.pdf
    12 Expectation Maximiation (EM), Principal Component Analysis (PCA) kmeans.pdf gmm.pdf em.pdf pca.pdf
    13 Learning Theory, Model Selection theory.pdf model.pdf Mitchell, Chapter 7
    14 Advanced Topics

    Class Notes (Date-Wise)

    jan1.pdf, jan8.pdf, jan9.pdf, jan10.pdf, jan11.pdf, jan15.pdf, jan17.pdf, jan18.pdf jan22.pdf jan24_25.pdf jan29.pdf jan31.pdf feb1.pdf feb8.pdf feb12.pdf feb14.pdf feb19.pdf feb21.pdf feb22.pdf feb26.pdf feb28.pdf mar1.pdf mar12.pdf mar14.pdf mar15.pdf mar16.pdf mar19.pdf mar29.pdf apr2.pdf apr5.pdf apr6.pdf apr9.pdf apr11.pdf apr12.pdf apr16.pdf apr18.pdf apr23.pdf apr25_26.pdf apr26.pdf

    Additional Resources

    Review Material

    Topic Notes
    Probability prob.pdf
    Linear Algebra linalg.pdf
    Gaussian Distribution gaussians.pdf
    Convex Optimization (1) convex-1.pdf

    References

    1. Machine Learning: A Probabilistic Perspective. Kevin Murphy. MIT Press, 2012.
    2. Pattern Recognition and Machine Learning. Christopher Bishop. First Edition, Springer, 2006.
    3. Pattern Classification. Richard Duda, Peter Hart and David Stock. Second Edition, Wiley-Interscience, 2000.
    4. Machine Learning. Tom Mitchell. First Edition, McGraw-Hill, 1997.

    Assignment Submission Instructions
    1. You are free to discuss the assignment problems with other students in the class. But all your code should be produced independently without looking at/referring to anyone else's code.
    2. Python (preferred) and MATLAB are the default programming languages for the course. You should use one of these two languages for programming your assignments unless otherwise explicitly allowed.
    3. Code should be submitted using Moodle Page. Make sure to include commenrs for readability.
    4. Create a separate directory for each of the questions named by the question number. For instance, for question 1, all your submissions files should be put in the directory named Q1 (and so on for other questions). Put all the Question sub-directories in a single top level directory. This directory should be named as "yourentrynumber_firstname_lastname". For example, if your entry number is "2016anz7535" and your name is "Nitika Rao", your submission directory should be named as "2016anz7535_nitika_rao". You should zip your directory and name the resulting file as "yourentrynumber_firstname_lastname.zip" e.g. in the above example it will be "2016anz7535_nitika_rao.zip". This single zip file should be submitted online.
    5. Honor Code: Any cases of copying will be awarded a zero on the assignment and a penalty of -10. More severe penalties may follow.
    6. Late Policy: You are allowed a total of 5 late days acorss all the assignments. You are free to decide how you would like to use them. You will get a zero on an assignment once you exceed the (total) allowed exemption of 5 days.

    Practice Questions

    Assignments
    1. Assignment 3
      Due Date: Friday April 5, 2019. 11:50 pm.
    2. Assignment 2 (Part A)
      Dataset (Naive Bayes): ass2_data.zip
      Assignment 2 (Part B)
      Due Date (for both parts): March 12 March 13, 2019. 11:50 pm.
    3. Assignment 1
      Datasets: ass1_data.zip
      Due Date: Tuesday February 12, 2019. 11:50 pm.

    Grading Policy (Tentative)

    Assignments (1-4) 30-35% (total)
    Minors (1,2) 30-35% (total)
    Major 30-35%