MAS 622J/1.126J: PATTERN RECOGNITION AND ANALYSIS


FALL 2004 Class Information:

Lectures: Tuesdays and Thursdays 1:00 - 2:30, E51-063 (at Sloan)
Recitations: Fridays, 1:00 - 2:00, E51-063
Textbook: Pattern Classification by Duda, Hart, and Stork, with other readings

Staff | Announcements | Assignments| Syllabus | Policies

Instructor:

Prof. Rosalind W. Picard
Office: E15-020g, Office hours: Tuesdays, 2:30-4:00 or by appointment
Phone: 253-0611
picard ( @ media dot mit dot edu)
 

Teaching Assistants:

Josh Lifton
E15-352, Office hours: TBA
452-5627 (office), 452-5615 (lab)
lifton ( @ media dot mit dot edu)

Larissa Welti-Santos
E15-320P, Office hours: Fridays, 11:00-12:00 or by appointment
324-1679
lwelti ( @ media dot mit dot edu)
 

Support Staff:

Ms. Lynne Lenker
E15-020a
253-0369
llenker ( @ media dot mit dot edu)
(works part-time; usually in on Tues and Thurs)


 

Staff | Announcements | Assignments| Syllabus | Policies


09SEP2004 First day of class. (Visitors welcome.) The first recitation will be held Friday 9/10/04 in the same room as class and will cover the software tools used in this class.

09SEP2004 This class has traditionally made heavy use of MATLAB. Here is a MATLAB Tutorial you may find helpful. This semester, in addition to MATLAB, we will be using a suite of numerical and visualization tools based on Python. Here is a Python Quick Start Guide to installing all the necessary software and collecting basic documentation. You are free to use either MATLAB or Python, or both.

10SEP2004 Here are the MATLAB notes of the first recitation.

14SEP2004 Tom Minka (my former student) has written a short insightful piece on nuances in the use of probability.

14SEP2004 Here is a list of Useful Python Links. The list will be added to as new links are found. Please let us know of any you find useful.

24SEP2004 Here are the main points covered during recitation 3.

24SEP2004 Problem Set 2 had some typos --an email was sent explaining what was incorrect. When defining the domain of x in terms of chi, priors were missing! Epsilon_1 should be integrated over chi_2 and epsilon_2 should be integrated over chi_1. The file has been corrected.

11OCT2004 Reminder: Prof Picard will not be available for office hours tomorrow because of the need for her to guest lecture another MIT class 2:30-4:30. Josh Lifton has kindly agreed to be available in his office during this time.

12OCT2004 Correction to DHS p. 127: when no data is missing, x_4=[2 4]', then theta=[5/4 2 11/16 2]', not what is given in the book. However, when x_4=[1 4] then theta=[1 2 0.5 2]'.

14OCT2004 Here is the 2002 Class Projects Page.

22OCT2004 The HMM handout does not show you the formula to compute P(O | lamda) when using the backward algorithm. You need to consider the probability of starting at each node and observe O_1 having obtained beta_1. That is:P(O | lamda)= sum over all the j's of (Pi_j* b_j(O_1)*beta_1(j) .

3NOV2004 Here is the 2004 Class Projects Page. You will find links to "standard" datasets on this page.

23NOV2004 Prof. Picard will be in Asia next week with limited Email access (and no office hours). We will have two guest lectures who are former TA's of the class, covering very useful and important topics: Ashish Kapoor (a PhD student in Picard's group) talking about clustering and how it can be used for new semi-supervised learning techniques; and Alan Qi (recently graduated PhD from Picard's group) talking about feature selection, classic methods and new Bayesian methods. Hope you enjoy these special lectures, covering both classic problems and state-of-the art research approaches to solving them.


 
 

Staff | Announcements | Assignments| Syllabus | Policies


**Note: Future assignments, beyond what has been stated in class, are predictions; thus, they are likely to be subject to revision.** "If you must predict, predict often." -- Prof. Paul Samuelson, 1970 Nobel Prize in Economics


9-09 Lecture 1 Reading: DHS Chap 1, A.1-A.2

(due 09-21) Problem Set 1 Data set Solutions

9-14 Lecture 2 Reading: DHS Chap A.2-A.4

9-16 Lecture 3 Reading: DHS Chap A.5, 2.1-2.4 (can skip 2.3.1, 2.3.2)

9-21 Lecture 4 Reading: DHS Chap 2.5-2.6

(due 09-30) Problem Set 2 Solutions

9-23 Lecture 5 Reading: DHS Chap 2.8.3, 2.11, Breese & Ball Handout (to illustrate an application), Independence Diagram handout (optional but good to give at least a skim), Introduction to inference for bayesian networks by Cowell.

9-28 Lecture 6 Reading: DHS 2.9.1, 3.1-3.2

9-30 Lecture 7 Reading: DHS 3.3-3.4

(due 10-12) Problem Set 3 Data Sets Solutions

10-5 Lecture 8 Reading: DHS 3.5, 3.7, 3.8, Eigenfaces vs. Fisherfaces paper (Guest lecture: Larissa Welti)

10-7 Lecture 9 Reading: DHS 3.8-3.9

10-12 Lecture 10 Rabiner & Juang handout 6.1-6.5 and 6.12, optional: DHS Chap 3.10

(due 10-26) Problem Set 4 Data Sets Solutions Demo Learning LogProbObs Viterbi

10-14 Lecture 11 Rabiner & Juang handout 6.1-6.5 and 6.12, optional: DHS Chap 3.10

10-19 NO CLASS: Media Lab Sponsor Events (MAS students should attend the sponsor events, where they can watch for examples of pattern recognition and learn a lot about putting this material to practice.)

10-21 Lecture 12 Reading: DHS Chap 4.1-4.4, 4.5 pp 177-178 and 4.5.4, 4.6.1 Special Guest Lecture by Prof. Jake Aggarwal, recent recipient of K.S.Fu Prize in Pattern Recognition.

10-26 Lecture 13 Reading: DHS Chap 5.1-5.5.1, 5.8-5.8.3

(due 11-09) Problem Set 5 Data Set

10-28 Lecture 14 Reading: DHS (from above)

11-2 MIDTERM QUIZ Optional Reading (not covered on quiz, but a short and very interesting article given today is Election Day) "Election Selection: Are we using the worst voting procedure?" Science News, Nov 2 2002.

11-4 Lecture 15 DHS Chap 5.11, 6.1-6.3

11-9 Lecture 16 DHS Chap 6.3, 10.1 - 10.4.2

(due 11-18) Problem Set 6 Data Set Solutions

11-11 NO CLASS: Veteran's Day Holiday

11-16 Lecture 17 Guest lecture: Josh Lifton, Kalman Filtering

11-18 Lecture 18 DHS 8.1-8.4

11-23 Lecture 19 DHS 9.1-9.3, 9.5-9.7; Kapoor, R. W. Picard and Y. Ivanov (2004), "Probabilistic Combination of Multiple Modalities to Detect Interest," International Conference on Pattern Recognition, Cambridge, U.K. August 2004. PDF

11-25 NO CLASS: Thanksgiving Holiday

11-30 Lecture 20 Guest lecture: Ashish Kapoor, Clustering(Unsupervised) and Introduction to Semi-supervised Learning DHS 10.4.3, 10.6-10.7, 10.9, 10.11

12-2 Lecture 21 Guest lecture: Alan Qi, Feature Selection

12-7 Project Presentations: Everyone required to attend class today

12-9 Last Day of Class/Project Presentations: Everyone required to attend class today


 
 

Staff | Announcements | Assignments| Syllabus | Policies



Fall 2004 Syllabus: (not necessarily in this order; also, one of these topics is likely to be dropped given the semester is 2 days shorter than usual.)

Intro to pattern recognition, feature detection, classification

Review of probability theory, conditional probability and Bayes rule

Random vectors, expectation, correlation, covariance

Review of linear algebra, linear transformations

Decision theory, ROC curves, Likelihood ratio test

Linear and quadratic discriminants, Fisher discriminant

Sufficient statistics, coping with missing or noisy features

Template-based recognition, eigenvector analysis, feature extraction

Training methods, Maximum likelihood and Bayesian parameter estimation

Linear discriminant/Perceptron learning, optimization by gradient descent, SVM

k-nearest-neighbor classification

Non-parametric classification, density estimation, Parzen estimation

Unsupervised learning, clustering, vector quantization, K-means

Mixture modeling, optimization by Expectation-Maximization

Hidden Markov models, Viterbi algorithm, Baum-Welch algorithm

Linear dynamical systems, Kalman filtering and smoothing

Bayesian networks, independence diagrams

Decision trees, Multi-layer Perceptrons

Combination of multiple classifiers "Committee Machines"


 
 

Staff | Announcements | Assignments | Syllabus | Policies

Grading:

35% Homework/Mini-projects, due every 1-2 weeks up until 3 weeks before the end of the term. These will involve some programming (Matlab or equiv.) assignments.

30% Project with approximate due dates:

25% Midterm Quiz: 11/2 (Drop Date is 11/20)

10% Your presence and interaction in lectures (especially the last two days), in recitation, and with the staff outside the classroom.

Late Policy:

Assignments are due by 5:00 p.m. on the due date, in Josh Lifton's office, E15-352. You are also free to bring them to class on the due date. If you are late, you will get a zero on the assignment. However, the lowest assignment grade will be dropped in computing the final grade.
 

Collaboration/Academic Honesty:

The goal of the assignments is to help you learn, not to see how many points you can get. Grades in graduate school do not matter very much: what you learn is what matters. Thus, if you stumble across old course material with similar-looking problems, please try not to look at their solutions, but rather work the problem(s) yourself. Please feel free to come to the staff for help, and also to collaborate on the problems and projects with each other. Collaboration should be at the "whiteboard" level: discuss ideas, techniques, even details - but write your answers independently. This includes writing Matlab code independently, and not copying code or solutions from each other or from similar problems from previous years. If you are caught violating this policy it will result in an automatic F for the assignment AND may result in an F for your grade for the class. If you team up on the final project, then you may submit one report which includes a jointly written and signed statement of who did what.

The midterm will be closed-book, but we will allow a cheat sheet.

Course feedback:

The staff welcomes your comments on the course at any time. There is an anonymous service for providing feedback . Please feel free to send us comments -- in the past, we have obtained helpful remarks that allow us to make improvements mid-course. We want to maximize the value of this course for everyone and welcome your input, positive or negative.

Attendance:

All students are expected to attend all project presentations the last two days of class; these tend to be very educational experiences, and thus attendance these last two days will contribute to your final grade.