FALL 2002 Class Information:
Lectures: Tuesdays and Thursdays 1:00 - 2:30, E15-054
Textbook: Pattern Classification by Duda, Hart, and Stork, with
other readings
Recitations: Fridays, 1:00 - 2:00, E15-054
Staff | Announcements | Assignments| Syllabus | Policies
Instructor:
Prof. Rosalind W. Picard
Office: E15-020g, Office hours: Tuesdays, 2:30-4:00 or by appointment
Phone: 253-0611
picard@media.mit.edu
Teaching Assistants:
Mr. Ashish Kapoor
E15-120d, Office hours: Mondays, 5:00-6:00 or by appointment
253-5437
ash@media.mit.edu
Mr. Yuan Qi
E15-120d, Office hours: Thursday, 11:00-12:00 or by appointment
253-5437
yuanqi@media.mit.edu
Support Staff:
Ms. Vickey McKiernen
E15-020a
253-0369
vm@media.mit.edu
Staff | Announcements | Assignments| Syllabus | Policies
9/5/02 First day of class. (Visitors welcome.)
9/19/02 Warning: if you have the first printing of the 2nd Edition of DHS, then please discard section 2.11 entirely and replace it with today's handout. If you have a later printing, you'll notice that today's handout (from fourth printing) is probably very close, but I've hand corrected some parts. -- Roz
10/3/02 Correction to DHS p. 127: when no data is missing, x_4=[2 4]', then theta=[5/4 2 11/16 2]', not what is given in the book. However, when x_4=[1 4] then theta=[1 2 0.5 2]'.
10/8/02 Next week's schedule: Next week is "Media Lab Sponsor Week" -- an all-consuming time for most of us who work in the ML. Mon Oct 14 and Tues Oct 15 are MIT Holidays. Ashish will still hold office hours Monday. Tuesday there are no classes, and Roz will be in sponsor meetings during what would ordinarily be her office hours, so there will not be Tuesday office hours. (Note from Roz: I will be around all next week, just in and out of constant sponsor meetings, but if you see me around at breaks/etc, feel free to ask me things -- I am happy to help and your questions will be a pleasurable break from the sponsor meetings. Vickey can help you identify when you are most likely to find me; I expect to have limited time for email so asking her first is the best way to reach me next week.). Yuan will have his usual office hours on Thursday but class is cancelled on Thursday since about half the class and the professor are involved in sponsor meetings. Friday's recitation will happen, but in a new location TBA.
10/11/02 The programming problem (problem 2) needs to take care of some
implementation issues for HMMs. These are very well discussed in section 6.12 of the Rabiner and Juang Handout (Theory and Implementation of
HMMs). I will go through it today in the recitation. In case you cannot make the
recitation you are welcome to come to the office hours next week or schedule an appointment.
10/20/02 PS4 deadline is now extended to being due at the start of class on 10/24. The quiz will happen as scheduled on 10/29, covering PS1-4 and related materials presented in class. For the quiz you will be allowed to bring an 8.5" x 11" sheet of paper, two-sided, with notes, and a calculator.
11/04/04 Problem two requires you to compute the evidence curve for up to
20 degrees of the polynomial. If you use the formula given in the book to compute the
psuedoinverse of Y (Y_dagger=inv(Y_t*Y)*Y_t, you will run into numerical problems.
Instead you can use pinv function of MATLAB to compute Y_dagger = pinv(Y), to avoid the numerical problems.
Use: help pinv , for more details
Staff | Announcements | Assignments| Syllabus | Policies
9-05 Lecture1 Reading: DHS Chap 1, A.1-A.2
9-10 Lecture2 Reading: DHS Chap A.2-A.4
9-12 Lecture3 Reading: DHS Chap A.5, 2.1-2.4 (can skip 2.3.1, 2.3.2)
(due 09/17) Problem Set 1, Solutions, Data set, MATLAB Tutorial
9-17 Lecture4 Reading: DHS Chap 2.5-2.6
9-19 Lecture 5 Reading: DHS Chap 2.8.3, 2.11, Breese & Ball Handout (to illustrate an application), Independence Diagram handout (pls. download and at least read this lightly now; we will probably revisit it later in the course as well), Cowell article (This goes into more on Bayes Nets than we will cover, but is a good introduction that goes beyond DHS. Pls read pp. 9-18 and give at least a quick glance at the rest, so you'll know what other topics it covers for possible future reference.)
9-24 Lecture 6 Reading: DHS Chap 2.9, 3.1-3.2
(due 9-26) Problem Set 2, Solutions (Hardcopy available)
9-26 Lecture 7 Reading: DHS Chap 3.3-3.4
10-1 Lecture 8 Reading: DHS Chap 3.5, 3.7-3.8, Belhumeur et al paper
10-3 Lecture 9 Reading: DHS Chap 3.8-3.9
(due 10-8) Problem Set 3, Solutions (Hardcopy available) , Data set, Matlab Code Problem 1
10-8,10 Lectures 10&11 Reading: Rabiner & Juang 6.1-6.5 and 6.12, optional: DHS Chap 3.10
(due 10-24 at start of class) Problem Set 4, Solutions, Data set, hmm_demo.m, learn_hmm_param.m, log_prob_obs_hmm.m, viterbi.m
(No Lecture 12 due to Media Lab Sponsor Week)
10-22 Lecture 13 Reading: DHS Chap 4.1-4.4, 4.5 pp 177-178 and 4.5.4, 4.6.1
10-24, 10-31 Lectures 14+15 Reading: DHS Chap 5.1-5.5.1, 5.8-5.8.3, 5.11, 6.1
11-5,7 Lectures 16+17 Reading: DHS Chap 6.2-6.3, 8.1-8.2, 10.1-10.4.2
(due 11-7) Problem Set 5, Solutions, Data set, PS5.m, pazen_gaussian.m, linear_discriminant.m, knn.m
11-12,14 Lectures 18+19 Reading: DHS Chap 8.3-8.4, Chap 10.4.3, 10.6-10.10
(due 11-19) Problem Set 6, Solutions, Data set Problem 1, ocr_train, ocr_test
11-19 Lecture 20 Reading: DHS Chap 9, and "Election Selection: Are we using the worst voting procedure?" Science News, Nov 2 2002.
11-21 Lecture 21: Guest lecture by Yuan Qi. Reading: Chapter 14 of Jordan & Bishop's book on Kalman Filtering and Tom Minka's short paper relating this to HMM's.
11-26 Lecture 22: Guest lecture by Ashish Kapoor. Reading: "An Introduction to Kernel Based Learning Algorithms" - Muller et al. in IEEE Trans on Neural Networks.
12-3 Lecture 23: Combined "final" lecture: Yuan Qi introduces Bayes Point Machines and Junction Trees (for more information see Chapter 16 of Jordan & Bishop's book) and also the Cowell article from 9/19 (above). Finally, Roz wraps up with a brief course overview.
12-5 Project Presentations: Face and Music/Artist data sets.
Note: all presentations are due online by 11:00 a.m. today. For ML students, please put them into
the directory at \\www\courses\2002fall\mas622j\proj\students\$Student-NAME. For non-ML students, please mail
your files to yuanqi and ash, who will put them online.
12-10 Project Presentations: PAF and special topics.
Staff | Announcements | Assignments| Syllabus | Policies
Fall 2002 Syllabus: (not necessarily in this order)
Intro to pattern recognition, feature detection, classification
Review of probability theory, conditional probability and Bayes rule
Random vectors, expectation, correlation, covariance
Review of linear algebra, linear transformations
Decision theory, ROC curves, Likelihood ratio test
Linear and quadratic discriminants, Fisher discriminant
Sufficient statistics, coping with missing or noisy features
Template-based recognition, eigenvector analysis, feature extraction
Training methods, Maximum likelihood and Bayesian parameter estimation
Linear discriminant/Perceptron learning, optimization by gradient descent, SVM
k-nearest-neighbor classification
Non-parametric classification, density estimation, Parzen estimation
Unsupervised learning, clustering, vector quantization, K-means
Mixture modeling, optimization by Expectation-Maximization
Hidden Markov models, Viterbi algorithm, Baum-Welch algorithm
Linear dynamical systems, Kalman filtering and smoothing
Bayesian networks, independence diagrams
Decision trees, Multi-layer Perceptrons
Combination of multiple classifiers "Committee Machines"
Staff | Announcements | Assignments | Syllabus | Policies
30% Project with approximate due dates:
10% Your presence and interaction in lectures (especially the last two days), in recitation, and with the staff outside the classroom.
The midterm will be closed-book, but we will allow a cheat sheet.