ECE1510/CSC2535: Advanced Inference Algorithms/Advanced Machine Learning, University of Toronto

University of Toronto
Department of Electrical and Computer Engineering

ECE1510/CSC2535: Advanced Inference Algorithms/Advanced Machine Learning

Instructor: Brendan J. Frey
Email: frey psi toronto edu
Office: 4136, Bahen Centre, 40 St. George St.

Lecture time:Wednesday, 1:10pm-3:00pm
Location: Bahen 1220
Office hours: Wednesday, Feb 29, 3.15pm-5.00pm
Course information

FINAL EXAM: Wednesday, April 18, 1.10pm-3.00pm
Click here for the final exam from 2004

Due dates (email pdf file by midnight on date indicated)
Project proposal (2 pages, 10%): March 4
Extended proposal, including initial results (8 pages, 10%): March 25
-- 150 word abstract (20%), problem description (20%), state of the art (20%), proposed methods (20%), fallback plans (eg, simpler methods) (20%), initial evaluation required (20%)
Draft of project report (8 pages, 15%): April 25
-- 150 word abstract (20%), problem description (10%), state of the art (10%), proposed methods (20%), evaluation (20%), discussion and conclusions (20%)
Final project report (8 pages, 25%): May 4
-- Polished paper and grammar: 150 word abstract (20%), problem description and state of the art (20%), proposed methods (20%), evaluation (20%), discussion and conclusions (20%). 2% will be deducted for each grammatical or spelling error

Textbook:
C.M. Bishop. Pattern Recognition and Machine Learning, Springer, 2006.
Readings:
Review paper: A comparison of algorithms for inference and learning

Topics

Graphical models
Bayesian Networks, Markov blankets, conditional independence
Markov Random Fields, HC theorem
Factor graphs, BNs and MRFs as factor graphs

Exact inference in graphical models
Variable elimination
Junction tree algorithm
Belief propagation/message passing
Sum-product and max-product algorithms

Approximate inference
ICM or gradient descent
Free energy and variational methods
Sampling methods, Gibbs sampling and more advanced methods
Loopy belief propagation and how to construct good graphs
Expectation propagation

Bayesian methods
Nonparametric methods
Hyperparameters
Sparse priors
Gaussian processes and kernel methods

Extras
Outlier detection
Sparse methods and sparse models
Optimization, stochastic and conjugate gradient descent
Energy-based models, RBMs, DBNs, deep autoencoders
Bias-variance decomposition
Dimensionality Reduction and manifold Learning
Semi-supervised learning