The purpose of this project was to endow Roco with a smile detector. Roco is a robot currently under construction which will (with any luck) resemble an iMac as seen on TV; that is, it has a flat screen attached by a moveable arm to a fixed base and it has the ability to move its screen around on its own, using 5 degrees of freedom. In order to be interactive, Roco perceives the world through a number of channels, including a camera mounted on its screen.

The camera is running a system designed by Ashish that can , among other things, detect a user's eye positions. This data is used to draw a box around the user's mouth, resulting in an image like this:

The image is then further clipped, resulting in an image that is designed to contain only the user's mouth and the area immediately surrounding:

One very nice property of this method of getting the image of the user's mouth is that the box's top edge is parallel to the line connecting the user's eyes, so all the images of the mouths will always be in the same orientation even though the user may be tilting their head.

 

NEXT