FALL 2010 Class Information:
Lectures: Tuesdays and Thursdays 9:30-11:00, 8-205
Recitations: Fridays 10-11, 8-205
Textbook: Pattern Classification by Duda, Hart, and Stork, with other readings
Prof. Rosalind W. Picard
Office hours: I'm available Tuesdays 11-12 pm, weekly, and for additional hours during project planning season: Nov 5, Nov 9 and Nov 16. Outside of posted hours you can meet with me by scheduling an appointment with Alex at email below. I'm also usually available right after class walking back to the lab & always happy to try to answer your questions in person (I usually don't have enough hours to answer by email).
picard ( @ media dot mit dot edu)
Office hours: Monday 5-6pm
djmcduff ( @ media dot mit dot edu)
Javier Hernandez Rivera
Office hours: Wednesday 10-11am
javierhr ( @ mit dot edu)
lillian ( @ media dot mit dot edu)
R 9-9 First day of class. (Visitors welcome, although class
size is limited to 20.)
The first recitation will be held Friday Sep 10, providing an
overview for those who would like help with MATLAB, probability, or
the first problem set (PS). This class has traditionally made heavy use of
MATLAB. Here is
Tutorial you may find helpful. You might also enjoy this short
insightful (optional) piece
nuances in the use of probability by Media Lab graduate Tom Minka,
There is a useful website for the text, which includes not only ppt slides for the chapters, but also errata for each edition of the text. Please correct the errors in your text now to save you time later when you are reading and trying to understand.
The IAPR Pattern Recognition Education Resources web site
was initiated by the Internation Association for
Pattern Recognition (http://www.iapr.org/).
The goal was a web site that can support students, researchers and staff.
Of course, advances in pattern recognition and its subfields means that
developing the site will be a never-ending process.
What resources does the IAPR Education web site have?
The most important resources are for students, researchers and educators.
These include lists with URLs to:
- Tutorials and surveys
- Explanatory text
- Online demos
- Book lists
- Free code
- Course notes
- Lecture slides
- Course reading lists
- A list of course web pages at many universities
"If you must predict, predict often." -- Prof. Paul Samuelson, 1970 Nobel Prize in Economics
R 9-9 Lecture 1 Introduction, Reading: DHS Chap 1, A.1-A.2
T 9-14 Lecture 2 Reading: DHS Chap A.2-A.5.
Recommended readings on probability:
- Alvin W. Drake. Fundamentals of applied probability theory.
- William Feller. An introduction to probability theory and its applications.
- Richard W. Hamming. The art of probability for scientis and engineers.
- Andrew's Moore's tutorials: [ probabilistic analytics, probability densities, and gaussians ]
R 9-16 Lecture 3 Reading: DHS Chap 2.1-2.4 (can skip 2.3.1, 2.3.2) Notes (courtesy by Rob Speer)
F 9-17 Recitation 2
M 9-20 First submission PS 1
T 9-21 Lecture 4 Reading: DHS Chap 2.5-2.7 Notes
R 9-23 Lecture 5 Reading: DHS Chap 2.8.3, 2.9, 3.1-3.2
Final submission PS 1
F 9-24 Recitation 3
T 9-28 Lecture 6 DHS 3.1-3.5.1 Slides
R 9-30 Lecture 7 Guest lecture from Javier Hernandez Rivera: Dimensionality Reduction DHS 3.7.1-.3, 3.8 Slides
F 10-1 Recitation 4
M 10-4 Final submission PS 2
T 10-5 Lecture 8 HMMs, reading: A Tutorial On Hidden Markov Models and Selected Applications in Speech Recognition, L.R. Rabiner, Proceedings of the IEEE, Vol 77 No 2, Feb. 1989.; optional extra reading: DHS 3.10 (beware, as DHS uses non-standard notation in this section).
R 10-7 Lecture 9 HMMs, same reading. Notes
F 10-8 Recitation 5
T 10-12 Lecture 10 DHS 2.10 and 3.9, Missing Data and Expectation Maximization, brief intro to Bayes Nets, 2.11 Notes
First submission PS 3 (prob. 1, 2 and 3)
R 10-14 Lecture 11 (Picard away co-directing TTT meeting) Inference on Bayes Nets and Dynamic Bayes Nets, guest lecture from Dr. Rana el Kaliouby, Reading: Introduction to Graphical Models and Bayesian Networks, Kevin Murphy, 1998. (DBN Lecture PDF slides)
First submission PS 3 (prob. 4 and 5) + solution
F 10-15 Recitation 6 (during 25th anniversary sponsor/alumni event)
M 10-18 Final submission PS 3
T 10-19 Lecture 12 DHS 10.2-10.4.3 Mixture densities, K-means clustering, Quiz review
R 10-21 Lecture 13 DHS 10.4.3, 10.4.4, 10.6-10.10 Clustering
F 10-22 Recitation 7 (Midterm 2008)
T 10-26 Lecture 14 DHS 4.5.1, 4.5.4, K-nn classifier, DHS 5.1-5.3, 5.8.1 Linear Discriminants. Slides
W 10-27 Final submission PS 4
R 10-28 MIDTERM QUIZ, Covers Lectures 1-13 (Nov 17, 2010 = DROP DATE)
F 10-29 Recitation 8
T 11-2 Lecture 15 (Picard in UK), Provided Data for Projects Ready
Presentations on project data available, Optional Readings (short and
very interesting articles to discuss with your friends, given Election
"Election Selection: Are we using the worst voting procedure?" Science News, Nov 2 2002.
Range voting: Best way to select a leader?
Slides on Class Project
R 11-4 Lecture 16 Guest lecture on regression by Sophia Yuditskaya Slides
Project Plan Due if you're using your own data
F 11-5 Recitation 9
M 11-8 First submission PS 5 + solution
T 11-9 Lecture 17 DHS 6.1-6.6, 6.8 Multilayer Neural Nets
Project Plan Due if you're using class-provided data
R 11-11 Veteran's Day Holiday - No Class
F 11-12 Recitation 10
Final submission PS 5
T 11-16 Lecture 18 Feature Selection, webpage shown in class: http://ro.utia.cz/fs/fs_guideline.html DHS reading: Entropy/Mutual information A.7, Decision Trees, 8.1-8.4
R 11-18 Lecture 19 Project Progress Presentations/ Critique Day/ Attendance counts toward grade today
F 11-19 Recitation 11
T 11-23 Lecture 20 Project Progress Presentations/ Critique Day/ Attendance counts toward grade today
W 11-24 Final submission PS 6
R 11-25 Thanksgiving Holiday - No Class
F 11-26 No Recitation, Thanksgiving Vacation
T 11-30 Lecture 21 5.11 SVM, 9.1-9.2.1 No free lunch, 9.2.3-9.2.5 MDL, Occam; 9.3 Bias and Variance, 9.5 Bagging, Boosting, Active Learning, 9.6 Estimating and Comparing Classifiers, 9.7 Classifier Combination
R 12-2 Lecture 22 Guest lecture from Dan McDuff: Gaussian Processes. Slides
F 12-3 Recitation 12 Project help session - your staff has lots of experience - get our input and help
T 12-7 Final Project Presentations: All students required to attend: Attendance counts significantly for grade today.
R 12-9 Final Project Presentations, Last Day of Class: All students required to attend; attendance counts significantly for grade today.
Fall 2010 Syllabus: (subject to adjustment)
Intro to pattern recognition, feature detection, classification
Review of probability theory, conditional probability and Bayes rule
Random vectors, expectation, correlation, covariance
Review of linear algebra, linear transformations
Decision theory, ROC curves, Likelihood ratio test
Linear and quadratic discriminants
Sufficient statistics, coping with missing or noisy features
Template-based recognition, feature extraction
Eigenvector and Fisher linear discriminant analysis
Independent component analysis
Training methods, Maximum likelihood and Bayesian parameter estimation
Linear discriminant/Perceptron learning, optimization by gradient descent
Support Vector Machines
Non-parametric classification, density estimation, Parzen estimation
Unsupervised learning, clustering, vector quantization, K-means
Mixture modeling, Expectation-Maximization
Hidden Markov models, Viterbi algorithm, Baum-Welch algorithm
Decision trees, Multi-layer Perceptrons
Optional other topics toward end of term
30% Homework/Mini-projects, due every 1-2 weeks up until 3 weeks before the end of the term. These will involve both programming (Matlab) and non-programming assignments.
New homework submission and grading policy:
It is NOT ALLOWED to look at old homeworks before handing in your first pass at your homework. If we catch you doing this you will get a zero on the homework. This year we want to help you get as far as possible on your own or working with the other students collaboratively, and then after you hand in that work, we will hand out the solutions and let you fix your homework and improve your grade. This policy has multiple goals: (1) it forces you to learn by figuring things out, which develops your abilities better, (2) you are allowed to collaborate, which also facilitates discussion-based learning and meeting other people in the class (although you should not copy anyone's answers - and don't let them copy yours - write your own please, (3) it levels the playing field and is fair to those who don't have access to old homeworks, and (4) if many of you don't do well, it shows us we need to teach better, giving us important feedback so we can make the course better. Finally, because looking at solutions can raise your grade and be educational, we will hand out the solutions in a fair way to everyone after you hand your problem set in, and if you fix your answers by the grading deadline, we will regrade your homework and give you the average of the two grades, before and after the solutions.
Here is how it works: Homework will be due by 5pm on the due date. On the day when the homework is due, submit the best efforts you have made and photocopy or otherwise make a copy for yourself of what you handed in. We will hand out the homework solution and you will have a chance to refine your work given the solution. Submit your revised homework by the date and time specified (please do not be late - it is not fair to our graders and TAs). Do not copy your revised answers directly from the solution sheet: directly copying answers from the solution sheet will not be counted. Our goal is to help you learn and understand the ideas in the course materials. Optimizing your grades in grad school is not as important as getting depth of knowledge and developing your learning skills. Feel free to add comments to your solution to make the graders aware of specific things you have learnt from the 'second pass' - this will improve your chance of achieving the maximum bonus.
30% Project with approximate due dates:
25% Midterm Quiz: R 10-28 (Drop Date is W 11-17)
15% Your presence and interaction in lectures (especially your
presence during the two days of project critiques and two days of
final project presentations, which is 10%), in recitation, and with
the staff outside the classroom.
The midterm will be closed-book, but we will allow a cheat sheet.