S.No.

Topics

Lectures

Instructor

References

0

Introduction to Machine Learning

0101

SDR



Flavours of Machine Learning: Unsupervised, Supervised,
Reinforcement, Hybrid models. Decision Boundaries: crisp, and
noncrisp, optimisation problems. Examples of unsupervised learning.

01 Jan (Tue) {lecture#01}

SDR


1

Unsupervised Learning:
KMeans, Gaussian Mixture Models, EM

0206

SDR

[Bishop Chap.9],
[Do: Gaussians],
[Do: More on Gaussians],
[Ng: KMeans],
[Ng: GMM],
[Ng: EM],
[Smyth:
EM]


The KMeans algorithm: two flavours. Algorithms: history,
flavours. A mathematical formulation of the KMeans algorithm.
The Objective function to minimise. The basic KMeans algorithm,
computation complexity issues: each step, overall.
Limitations of KMeans.

02 Jan (Wed) {lecture#02}

SDR



KMeans: Alternate formulation with a distance threshold.
Gaussian Mixture Models. The Bayes rule, and Responsibilities.
Maximum Likelihood Estimation. Parameter estimation for a mixture
of Gaussians, starting with a simple 1D single Gaussian case.

04 Jan (Fri) {lecture#03}

SDR

{Early start: 7:30am9:00am class}


 No class 

08 Jan (Tue) {lecture#xx}





 No class 

09 Jan (Wed) {lecture#xx}





 No class 

11 Jan (Fri) {lecture#xx}





MLEstimation: the simple case of one 1D Gaussian,
to the general case of K Ddimensional Gaussians.

15 Jan (Tue) {lecture#04}

SDR

{Early start: 7:30am9:00am class}


The general case of K Ddimensional Gaussians.
Getting stuck, using Lagrange Multipliers.
The EM Algorithm for Gaussian Mixtures.
Application: Assignment 1:
The
Stauffer and Grimson Adaptive Background Subtraction
Algorithm.
An introduction to the basic set of interesting heuristics!

16 Jan (Wed) {lecture#05}

SDR

{Early start: 7:30am9:00am class}
[CVPR'99], [PAMI'00]


The Stauffer and Grimson algorithm (contd)

18 Jan (Fri) {lecture#06}

SDR

{Early start: 7:30am9:00am class}

2

Unsupervised Learning:
EigenAnalysis:
PCA, LDA and Subspaces

0709

SDR

[Ng: PCA],
[Ng: ICA],
[Burges: Dimension Reduction],
[Bishop Chap.12]


Introduction to Eigenvalues and Eigenvectors.
Properties of Eigenvalues and Eigenvectors

22 Jan (Tue) {lecture#07}

SDR

{Early start: 7:30am9:00am class}


GramSchmidt Orthogonalisation, other properties.
The KL Transform

23 Jan (Wed) {lecture#08}

SDR

{Early start: 7:30am9:00am class}


The SVD and its properties

25 Jan (Fri) {lecture#09}

SDR

{Early start: 7:30am9:00am class}

3

Linear Models for Regression, Classification

1014

SDR

[Bishop Chap.3],
[Bishop Chap.4],
[Ng: Supervised, Discriminant Analysis],
[Ng: Generative],


General introduction to Regression and Classification. Linearity
and restricted nonlinearity. Maximum Likelihood.

29 Jan (Tue) {lecture#10}

SDR

{Early start: 7:30am9:00am class}


Maximum Likelihood and Least Squares.
The MoorePenrose Pseudoinverse.

30 Jan (Wed) {lecture#11}

SDR



 No class 

01 Feb (Fri) {lecture#xx}

SDR




Minor I

05 Feb (Tue), LH310, 11am12pm






Regularised Least Squares.
Classification (an introduction).

08 Feb (Fri) {lecture#12}

SDR



OpenCV Introduction

08 Feb (Fri) {lecture#xx}

SC

(EE Committee Room, 03:00pm04:00pm)


Linear Discriminant Functions: 2 classes, and K classes.
Fisher's Linear Discriminant.

12 Feb (Tue) {lecture#13}

SDR



Fisher's Linear Discriminant (contd.)

13 Feb (Wed) {lecture#14}

SDR


4

SVMs

1528

SDR

[Bishop Chap.7],
[Alex: SVMs],
[Ng: SVMs],
[Burges: SVMs],
[Bishop Chap.6],
[Khardon: Kernels]


Introduction to SVMs

19 Feb (Tue) {lecture#15}

SDR



The basic optimisation problem: margin maximisation

20 Feb (Wed) {lecture#16}

SDR



Optimisation criteria.
Getting the physical significance of the y = +1 and y = 1 lines.
The two `golden' regions for the 2class pefectly separable case.
The generalised canonical representation in terms of one inequation.

22 Feb (Fri) {lecture#17}

SDR



The basic SVM optimisation

26 Feb (Tue) {lecture#18}

SDR



The basic SVM optimisation: the primal and the dual problems.
An illustration of the kernel trick

27 Feb (Wed) {lecture#19}

SDR



Lagrange Multipliers and the KKT Conditions

01 Mar (Fri) {lecture#20}

SDR



Lagrange Multipliers and the KKT Conditions (contd).
The SoftMargin SVM.

12 Mar (Tue) {lecture#21}

SDR



The SoftMargin SVM (contd).

13 Mar (Wed) {lecture#22}

SDR



The SoftMargin SVM (contd).

15 Mar (Fri) {lecture#23}

SDR



The SoftMargin SVM (contd).:
A common inequation, the primal problem, getting the KKT
conditions for the softmargin problem.

19 Mar (Tue) {lecture#24}

SDR



Recap: Lagrange Multipliers and the KKT Conditions.
The softmargin SVM (contd).

20 Mar (Wed) {lecture#25}

SDR

{Early start: 7:30am9:00am class}


Introduction to Kernels.

22 Mar (Fri) {lecture#26}

SDR




Minor II

26 Mar (Tue), LH310, 11am12pm






Kernels in Regression

29 Mar (Fri) {lecture#27}

SDR

{Early start: 7:30am9:00am class}


 No class 

30 Mar (Sat) {lecture#xx}





Kernel Functions: properties, construction

02 Apr (Tue) {lecture#28}

SDR


5

Neural Networks

3340

SDR

[Bishop Chap.5],
[Alex: NNRep],
[Alex: NNLearn]


Introduction to Neural Networks: the MultiLayer Perceptron:
Conventions, restricted nonlinearity

03 Apr (Wed) {lecture#29}

SDR



Basic Perceptron, Optimisation

05 Apr (Fri) {lecture#30}

SDR



Optimisation: Estimating network parameters for a regression problem

09 Apr (Tue) {lecture#31}

SDR



Optimisation: Estimating network parameters on a classification
problem.
The Binomial distribution, the Bernoulli distribution
Physical Significance of the NN solution visavis the linear
method done before

10 Apr (Wed) {lecture#32}

SDR



Basic Optimisation basics: local quadratic approximation,
geometric interpretation, computing the gradient

12 Apr (Fri) {lecture#33}

SDR



The Backpropagation Algorithm

16 Apr (Tue) {lecture#34}

SDR



Backpropagation vs numerical evaluation of the gradient
Invariance issues: linear transformation of the input and outputs
4 Basic Invariancehandling techniques

19 Apr (Fri) {lecture#35}

SDR



Some details on Tangent Propagation, CNNs

23 Apr (Tue) {lecture#36}

SDR




24 Apr (Wed) {lecture#37}

SDR




26 Apr (Fri) {lecture#38}

SDR




Major

28 Apr (Sun),
LH310,
10:30am12:30pm





5

Feature Selection

xxxx

SDR


6

Logistic Regression

2829

SDR

[Alex: LogReg]

xx

Mathematical Basics for Machine Learning

xxxx

xx

[Burges: Math for ML],
[Burges: Math Slides],
[Do,
Kolter: Linear Algebra Notes],
