MAS 622J/1.126J: PATTERN RECOGNITION AND ANALYSIS


FALL 2002 Class Information:

Lectures: Tuesdays and Thursdays 1:00 - 2:30, E15-054
Textbook: Pattern Classification by Duda, Hart, and Stork, with other readings

Recitations: Fridays, 1:00 - 2:00, E15-054 


Staff | Announcements | Assignments| Syllabus | Policies

Instructor:

Prof. Rosalind W. Picard
Office: E15-020g, Office hours: Tuesdays, 2:30-4:00 or by appointment
Phone: 253-0611
picard@media.mit.edu
 

Teaching Assistants:

Mr. Ashish Kapoor
E15-120d, Office hours: Mondays, 5:00-6:00 or by appointment
253-5437
ash@media.mit.edu

Mr. Yuan Qi
E15-120d, Office hours: Thursday, 11:00-12:00 or by appointment
253-5437
yuanqi@media.mit.edu
 

Support Staff:

Ms. Vickey McKiernen
E15-020a
253-0369
vm@media.mit.edu
 
 



 
 

Staff | Announcements | Assignments| Syllabus | Policies





9/5/02 First day of class. (Visitors welcome.)

9/19/02 Warning: if you have the first printing of the 2nd Edition of DHS, then please discard section 2.11 entirely and replace it with today's handout. If you have a later printing, you'll notice that today's handout (from fourth printing) is probably very close, but I've hand corrected some parts. -- Roz

10/3/02 Correction to DHS p. 127: when no data is missing, x_4=[2 4]', then theta=[5/4 2 11/16 2]', not what is given in the book. However, when x_4=[1 4] then theta=[1 2 0.5 2]'.

10/8/02 Next week's schedule: Next week is "Media Lab Sponsor Week" -- an all-consuming time for most of us who work in the ML. Mon Oct 14 and Tues Oct 15 are MIT Holidays. Ashish will still hold office hours Monday. Tuesday there are no classes, and Roz will be in sponsor meetings during what would ordinarily be her office hours, so there will not be Tuesday office hours. (Note from Roz: I will be around all next week, just in and out of constant sponsor meetings, but if you see me around at breaks/etc, feel free to ask me things -- I am happy to help and your questions will be a pleasurable break from the sponsor meetings. Vickey can help you identify when you are most likely to find me; I expect to have limited time for email so asking her first is the best way to reach me next week.). Yuan will have his usual office hours on Thursday but class is cancelled on Thursday since about half the class and the professor are involved in sponsor meetings. Friday's recitation will happen, but in a new location TBA.

10/11/02 The programming problem (problem 2) needs to take care of some implementation issues for HMMs. These are very well discussed in section 6.12 of the Rabiner and Juang Handout (Theory and Implementation of HMMs). I will go through it today in the recitation. In case you cannot make the recitation you are welcome to come to the office hours next week or schedule an appointment.

10/20/02 PS4 deadline is now extended to being due at the start of class on 10/24. The quiz will happen as scheduled on 10/29, covering PS1-4 and related materials presented in class. For the quiz you will be allowed to bring an 8.5" x 11" sheet of paper, two-sided, with notes, and a calculator.

11/04/04 Problem two requires you to compute the evidence curve for up to 20 degrees of the polynomial. If you use the formula given in the book to compute the psuedoinverse of Y (Y_dagger=inv(Y_t*Y)*Y_t, you will run into numerical problems. Instead you can use pinv function of MATLAB to compute Y_dagger = pinv(Y), to avoid the numerical problems. Use: help pinv , for more details



 
 

Staff | Announcements | Assignments| Syllabus | Policies





9-05 Lecture1 Reading: DHS Chap 1, A.1-A.2

9-10 Lecture2 Reading: DHS Chap A.2-A.4

9-12 Lecture3 Reading: DHS Chap A.5, 2.1-2.4 (can skip 2.3.1, 2.3.2)

(due 09/17) Problem Set 1Solutions,  Data setMATLAB Tutorial 

9-17 Lecture4 Reading: DHS Chap 2.5-2.6

9-19 Lecture 5 Reading: DHS Chap 2.8.3, 2.11, Breese & Ball Handout (to illustrate an application), Independence Diagram handout (pls. download and at least read this lightly now; we will probably revisit it later in the course as well), Cowell article (This goes into more on Bayes Nets than we will cover, but is a good introduction that goes beyond DHS. Pls read pp. 9-18 and give at least a quick glance at the rest, so you'll know what other topics it covers for possible future reference.)

9-24 Lecture 6 Reading: DHS Chap 2.9, 3.1-3.2

(due 9-26) Problem Set 2, Solutions (Hardcopy available)

9-26 Lecture 7 Reading: DHS Chap 3.3-3.4

10-1 Lecture 8 Reading: DHS Chap 3.5, 3.7-3.8, Belhumeur et al paper

10-3 Lecture 9 Reading: DHS Chap 3.8-3.9

(due 10-8) Problem Set 3, Solutions (Hardcopy available) , Data set, Matlab Code Problem 1

10-8,10 Lectures 10&11 Reading: Rabiner & Juang 6.1-6.5 and 6.12, optional: DHS Chap 3.10

(due 10-24 at start of class) Problem Set 4, Solutions, Data set, hmm_demo.m, learn_hmm_param.m, log_prob_obs_hmm.m, viterbi.m

(No Lecture 12 due to Media Lab Sponsor Week)

10-22 Lecture 13 Reading: DHS Chap 4.1-4.4, 4.5 pp 177-178 and 4.5.4, 4.6.1

10-24, 10-31 Lectures 14+15 Reading: DHS Chap 5.1-5.5.1, 5.8-5.8.3, 5.11, 6.1

11-5,7 Lectures 16+17 Reading: DHS Chap 6.2-6.3, 8.1-8.2, 10.1-10.4.2

(due 11-7) Problem Set 5, Solutions, Data set, PS5.m, pazen_gaussian.m, linear_discriminant.m, knn.m

11-12,14 Lectures 18+19 Reading: DHS Chap 8.3-8.4, Chap 10.4.3, 10.6-10.10

(due 11-19) Problem Set 6, Solutions, Data set Problem 1, ocr_train, ocr_test

11-19 Lecture 20 Reading: DHS Chap 9, and "Election Selection: Are we using the worst voting procedure?" Science News, Nov 2 2002.

11-21 Lecture 21: Guest lecture by Yuan Qi. Reading: Chapter 14 of Jordan & Bishop's book on Kalman Filtering and Tom Minka's short paper relating this to HMM's.

11-26 Lecture 22: Guest lecture by Ashish Kapoor. Reading: "An Introduction to Kernel Based Learning Algorithms" - Muller et al. in IEEE Trans on Neural Networks.

12-3 Lecture 23: Combined "final" lecture: Yuan Qi introduces Bayes Point Machines and Junction Trees (for more information see Chapter 16 of Jordan & Bishop's book) and also the Cowell article from 9/19 (above). Finally, Roz wraps up with a brief course overview.

12-5 Project Presentations: Face and Music/Artist data sets.
Note: all presentations are due online by 11:00 a.m. today. For ML students, please put them into the directory at \\www\courses\2002fall\mas622j\proj\students\$Student-NAME. For non-ML students, please mail your files to yuanqi and ash, who will put them online.

12-10 Project Presentations: PAF and special topics.

2002 Class Projects Page



 
 

Staff | Announcements | Assignments| Syllabus | Policies



Fall 2002 Syllabus: (not necessarily in this order)

Intro to pattern recognition, feature detection, classification

Review of probability theory, conditional probability and Bayes rule

Random vectors, expectation, correlation, covariance

Review of linear algebra, linear transformations

Decision theory, ROC curves, Likelihood ratio test

Linear and quadratic discriminants, Fisher discriminant

Sufficient statistics, coping with missing or noisy features

Template-based recognition, eigenvector analysis, feature extraction

Training methods, Maximum likelihood and Bayesian parameter estimation

Linear discriminant/Perceptron learning, optimization by gradient descent, SVM

k-nearest-neighbor classification

Non-parametric classification, density estimation, Parzen estimation

Unsupervised learning, clustering, vector quantization, K-means

Mixture modeling, optimization by Expectation-Maximization

Hidden Markov models, Viterbi algorithm, Baum-Welch algorithm

Linear dynamical systems, Kalman filtering and smoothing

Bayesian networks, independence diagrams

Decision trees, Multi-layer Perceptrons

Combination of multiple classifiers "Committee Machines"



 
 

Staff | Announcements | Assignments | Syllabus | Policies

Grading:

35% Homework/Mini-projects, due every 1-2 weeks up until 3 weeks before the end of the term. These will involve some programming (Matlab or equiv.) assignments.

30% Project with approximate due dates:

25% Midterm: 10/29

10% Your presence and interaction in lectures (especially the last two days), in recitation, and with the staff outside the classroom.

Late Policy:

Assignments are due by 5:00 p.m. on the due date, in the TA's office. You are also free to bring them to class on the due date. If you are late, you will get a zero on the assignment. However, the lowest assignment grade will be dropped in computing the final grade.
 

Collaboration/Academic Honesty:

The goal of the assignments is to help you learn, not to see how many points you can get. Grades in graduate school do not matter very much: what you learn really does matter. Thus, if you stumble across old course material with similar-looking problems, please try not to look at their solutions, but rather work the problem(s) yourself. Please feel free to come to the staff for help, and also to collaborate on the problems and projects with each other. Collaboration should be at the "whiteboard" level: discuss ideas, techniques, even details - but write your answers independently. This includes writing Matlab code independently, and not copying code or solutions from each other or from similar problems from previous years. If you are caught violating this policy it will result in an automatic F for the assignment AND may result in an F for your grade for the class. If you team up on the final project, then you may submit one report which includes a jointly written and signed statement of who did what.

The midterm will be closed-book, but we will allow a cheat sheet.

Course feedback:

The staff welcomes your comments on the course at any time. There is an anonymous service for providing feedback . Please feel free to send us comments -- in the past, we have obtained helpful remarks that allow us to make improvements mid-course. We want to maximize the value of this course for everyone and welcome your input, positive or negative.

Attendance:

All students are expected to attend all project presentations the last two days of class; these tend to be very educational experiences, and thus attendance these last two days will contribute to your final grade.