As I was doing these tests, I began to notice that there were a few stray points for each training class when they were projected on the the Fisher basis:
Yellow: Smiles -- Green:NonSmiles
Red:Smiles Test Set -- Blue:NonSmiles Test Set
Since I had classified the data by hand, I went to take a look that the images that these points represent to make see if there was some obvious difference. There was; the two stray green points were the only two NonSmile images for that subject in the training set, and the stray yellow point was an image that was ambiguous even to me.
I decided to go back and re-classify the images, but this time into three groups, which I believed would lead to a more accurate classifier over all. For one thing, it would be easier for me to determine whether each picture was a smile with mouth open, smile with mouth closed, or non smile than it had been to divide all the data into only two groups; also this way I could include some of the ambiguous data I had to reject before.
I used a similar technique of computing PCA basis and coefficients, as well as a Fisher bases, and the data separated fairly well:
Yellow: SmilesOpen -- Green:SmilesClosed
Blue:NonSmiles
The smiles with mouth closed group is the least well grouped, which makes sense because it has far fewer samples than the other two groups.
I used the same technique as before and drew a random sampling of my training set to use to test with, and I removed that set before calculating anything. I then created 3 linear boundaries using support vector machines, and classified the training data via a vote of the SVM classifiers. It looked like this:
YellowDots: SmilesOpen -- GreenDots:SmilesClosed -- BlueDots:NonSmiles
RedDots: SmilesOpen -- DarkBlueDots:SmilesClosed -- PurpleDots:NonSmiles
GreenLine:NonSmile/SmilesClosed -- RedLine:SmilesOpen/SmilesClosed -- BlueLine:NonSmiles/SmilesOpen
The classification rate overall was 88%, less than I'd hoped. However the rate of detecting just a smile with mouth open (That is the probability that point p is classified as SmileOpen if it is a SmileOpen and classified as anything else if it is not a SmileOpen) correctly was 94%, as was the rate of detecting a smile with mouth closed. The rate of detecting non-smiles was 88%.
Future Work: I wanted to take advantage of the time based nature of this data; that is, I wanted to utilize the fact that I know these frames come one after another and are fairly close in time help detect smiles (perhaps the difference from one frame to the next will help with the classification).