MAS 630: AFFECTIVE COMPUTING

SPRING 2005 Class Information:

Fridays 1:00-3:00
Room E15-335
Textbook: Affective Computing by Rosalind Picard (MIT Press), and other readings

Instructor:

Prof. Rosalind W. Picard
Office: E15-020g
Office hours: Fridays 11-12 through 4/15 or by appointment. After 4/15, Office hours will be on: Tues 4/19 and Tues 4/26 from 1:30-2:30, and Fri 5/6 from 11-12, or by appointment.
Office hours: extra before project proposals due: Thursday March 3 10:00-12:00 and Thursday March 17 10:00 - 12:00
Phone: 253-0611
picard (at) media (dot) mit (dot) edu
 

Support Staff:

Ms. Lynne Lenker
E15-020a
253-0369
llenker (at) media (dot) mit (dot) edu
 
 

Announcements:

2/4/05 First day of class. (Visitors welcome.) Overview by Prof. Picard.

2/4/05 Projects: I will say much more about these in the coming classes. Feel free to browse projects from the last three years: 2001 Projects , 2002 Projects , and 2004 Projects.

2/11/05 Upcoming Seminar of Interest: Larry Barsalou, from Emory University in Atlanta, speaking in Bartos Theatre (lower level of Media Lab) at 2:00 p.m. Thursday February 17. (2/14/05: NOTE TIME CHANGE, now this is at 2 PM).

2/18/05 I'm now putting AV/ppt materials from class on the password part of the web site -- e.g., Week 1 talk slides.

2/21/05 The Anonymous feedback web site allows you to anonymously email me how long you spent on readings/hwk this week (and any other comments you might have on the course. The guidelines for a 9 hour class are 2 hours in class plus 7 out of class each week. I appreciate your feedback, anonymous or not, and also always welcome you to come by office hours or make an appointment to meet.

2/24/05 PPT slides related to the last two weeks are online at Weeks 2 & 3 slides.

2/25/05 I'm pleased to announce Hugo Liu will join me in leading the second half of class next week. Also today's talk slides are online.

3/15/05 I'm pleased to announce Dr. Tim Bickmore will join me in leading our discussion in class this week. Tim developed the main ideas behind relational agents and has built the most successful one to date that I know of.

3/18/05 There is no class next Friday -- have a great Spring Break! Note that the next time class meets, April 1, will be in the Bartos Theatre (lower level of the Media Lab) for Prof. Nalini Ambady's presentation at 1pm. We will join Pentland and Donath's class after the lecture for a discussion.

4/14/05 Next week I will be away at a board meeting but Prof. Cynthia Breazeal, expert in human-robot interaction, will be leading our class. I've asked her to present her work on emotion and robots. Pentland's class has asked to join in and we agreed that was ok, so we will need to meet in a larger room, and will meet in 384 from 1-3 instead of in 335.

 

Final 2005 Class Projects

Due in class Fri Apr 22:

READ:
Robot Emotions: A Functional Perspective by Breazeal and Brooks, in Who Needs Emotions? Ed by Fellous and Arbib
Robot Science Meets Social Science: An Embodied Computational Model of Social Referencing by Andrea Lockerd Thomaz, Matt Berlin, and Cynthia Breazeal.
Mirror neurons and the simulation theory of mind-reading by Vittorio Gallese and Alvin Goldman.
What imitation tells us about social cognition: rapprochement between developmental psychology and cognitive neuroscience by Andrew Meltzoff and Jean Decety.

WRITE:
Please hand Cynthia at the start of class a single page with your name on it and with a short description of your project progress to date (1 paragraph will do). This will serve both to give me an update of where you're at and also to help Cynthia know who from my class is participating. (Since our class will be merged with Pentland's, it will be hard for her to get everybody's name; also, class participation counts even in my absence, so this will help with records).

Due in class Fri Apr 15:

READ:
R. McCraty et al., The Effects of Emotions on Short-Term Power Spectrum Analysis of HRV (Handout)
R. B. Williams, Psychological factors in coronary artery disease: epidemiologic evidence, Circulation, 1987. (ok to skim the detail about studies of type A behavior) (Handout)
R. M. Frankel, Emotion and the Physician-Patient Relationship, (Handout)
D. Goleman, Emotional Intelligence, Chapter 11 (Handout)
Selected news articles:
Hostility, Anger, and Heart Disease 2004.
Heartfelt Fear Science News, February 12, 2005.

WRITE (OPTIONAL MAKE-UP HOMEWORK, THIS WRITING PART IS NOT REQUIRED):
Suppose that you are a type who is subject to anxiety, depression, or chronic hostility -- the "big three" affective states that interact with health. Pick one of these and describe a system design for helping yourself reduce the likelihood of getting sick. It doesn't have to be doable with today's technology, and you do not need to build it this week, or provide precise technical details, but I would like you to describe some ways it would work. For example, put yourself in the shoes of someone who needs to reduce their risk of heart disease by better managing/reducing hostile tendencies. Try to design an affective computing system that would help you in this case. What would you like your interactions with it to be like? What would you want it to measure? Give a couple scenarios of its use, being clear how you would like it to respond to your needs.

EMAIL:
Please email your written response 24 hours before class (inline/no attachments please) with SUBJECT: mas630-homework TO: picard@media.mit.edu

Due 1 p.m. Thurs Apr 7:

READ:
Picard, AC Chapter 4: Potential Concerns
"Computers that recognise and respond to user emotion: theoretical and practical implications," R. W. Picard and J. Klein, Interacting with Computers 14, (2002) 141-169.
The Naked Face
Lie Detection - Recovery of the Periorbital Signal through Tandem Tracking and Noise Suppression in Thermal Facial Video P. Tsiamyrtzis , J. Dowdall , D. Shastri , I. Pavlidis, M.G. Frank , P. Eckman, SPIE Conference (2005).
And this short news article:
Desktop computers to counsel users to make better decisions

WRITE:
1) Suppose you are called upon to help design the affect-related part of a lie detection system for use in airports. When you check in as a passenger, it is already routine at the check-in counter to ask each passenger questions such as "Have your bags been out of your immediate control since you packed them? Has any stranger given you a package or present to carry?" It is also routine to walk through a metal detector, and the metal-detection systems in Boston also gather video of you as you walk through. Describe how you might augment one of these interactions (or some other part of the experience, if you prefer) to help catch people who might have malicious life-threatening intent. What additional questions would you ask if your system involves questioning? What sensors would you use? What behavior is your system aimed at detecting? Be very specific, and give a couple examples. What do you think would be the strengths of your system? What do you expect would be its weaknesses? What concerns does your system raise?
2) Please give me a short paragraph describing what parts of your project have been completed or are significantly underway.

EMAIL:
Please email your response (inline/no attachments please) with SUBJECT: mas630-homework TO: picard@media.mit.edu

Due 1 p.m. Thurs Mar 31:

READ:
Ambady, N., LaPlante, D., Nguyen. T., Rosenthal, R., & Levinson, W. (2002). Surgeon’s tone of voice: A clue to malpractice history.Surgery, 132, 5-9.
Ambady, N., & Rosenthal, R. (1997). Judging Social Behavior Using Thin Slices. Chance (American Statistical Association), 10, 12-18. (Handout)
(optional:) Feel free to browse related publications at Ambady's publications page

WRITE:
1) Jot two questions you would like to ask Ambady based on the readings.

EMAIL:
Please email your response (inline/no attachments please) with SUBJECT: mas630-homework TO: picard@media.mit.edu

Due 1 p.m. Thurs March 17:

READ:
"This Computer Responds to User Frustration: Theory, Design, and Results," J. Klein, Y. Moon, R. W. Picard, Interacting with Computers 14 (2002) 119-140;
"Towards Caring Machines," T. Bickmore and R. W. Picard, Proceedings of the Conference on Computer-Human Interaction, Vienna, 2004.
"Establishing and Maintaining Long-term Human-Computer Relationships," T. Bickmore and R. W. Picard, to appear in Transactions on Computer-Human Interaction
"Embedded Empathy in Continuous, Interactive Health Assessment," K. Liu and R. W. Picard, CHI Workshop on HCI Challenges in Health Assessment
Computers that Care: investigating the effects of orientation of emotion exhibited by an embodied computer agent, S. Brave, C. Nass and K. Hutchinson, IJHCS 623 (2005).

WRITE:
1) Please hand in a ~1-2 page proposal for your project What are you thinking of building/testing/investigating? What resources would you need to do this? What do you expect to learn from it? If you are partnering on this, how do you plan to divide up what has to be done?
2) What are the key elements of a successful empathetic exchange?
3) Write a short dialogue illustrating good/effective use of empathy. Write a second short dialogue illustrating a well-intentioned response, but one that falls short of being empathetic.
4) Listen for a situation this week where you hear somebody say something about their feelings, and try to be empathetic in response. Do you think you came across as empathetic as you intended? What happened?

EMAIL:
Please email your response (inline/no attachments please) with SUBJECT: mas630-homework TO: picard@media.mit.edu

Due 1 p.m. Thurs March 10:

READ:
Forgas and Moylan 87 on Going to the Movies (Handout)
Clore 92 on Feelings and Judgment (Handout)
Isen et al 87 on Affect and Creativity (Handout)
Isen et al 91 on Affect and Clinical Problem Solving (Handout)

WRITE:
1) Please hand in a ~1-2 page "draft" description of your project idea(s) or ideas you are considering. What are you thinking of building/testing/investigating? What resources would you need to do this? What do you expect to learn from it? If you haven't narrowed it down, that's ok -- feel free to mention more than one project possibility. After I see these on Thursday Mar 10 I can give you feedback as well as make some possible matchmaking suggestions in class on Fri March 11.
2) Jack, the please-his-boss pollster, has been given ten questions with which he must canvas people's opinions. The questions relate to the overall satisfaction that people perceive with his party's politicians and their impact both locally and nationwide. He is not allowed to modify the questions, but he is willing to modify how the poll is conducted in subtle ways to make his party's political candidates look as good as possible. He plans to poll 1000 people nationally by phone and 1000 locally, in person, by some "random" process. Describe three ways Jack can try to bias the opinions he collects by manipulating affect-related factors. State how you think these manipulations would affect their opinions.
3) You've read about "feelings of knowing," "feelings of ease," "feelings of uncertainty," "feelings of familiarity" and other internal signals that are perceived as feelings, but seem to have primarily cognitive roles. Pick one of these internal signals and argue why it might be important to implement in a machine. Give an example of its possible improvement over existing machine capability. Pick another one and argue against its implementation, providing an example why it might be undesirable.

EMAIL:
Please email your response (inline/no attachments please) with SUBJECT: mas630-homework TO: picard@media.mit.edu

Due 1 p.m. Thurs March 3:

READ:
Picard, AC Chapters 5&6
A Model of Textual Affect Sensing using Real-World Knowledge by Hugo Liu, Henry Lieberman and Ted Selker
"Toward Machine Emotional Intelligence: Analysis of Affective Physiological State" by Picard, Vyzas, Healey (OK to just skim pattern recognition details in sections 3 and 4)
Toward Computers that Recognize and Respond to Emotion by R. W. Picard

WRITE:
1) Pick an example of text, about half a page, from either an expressive mail (please check for permission with sender before sharing) or from another expressive source. Label the text (by hand) according to the categories in Liu's paper.
2) What was most difficult about this task?
3) How appropriate were the six "basic" categories? If they weren't a perfect fit, how would you modify them to make them fit better?
4) Suppose you are asked to design an emotion-sensing system to explore people's emotional responses to eating food. Describe some things you might measure and the sensors you might use. Feel free to be imaginative.

EMAIL:
Please email your response (inline/no attachments please) with SUBJECT: mas630-homework TO: picard@media.mit.edu
 

Due 1 p.m. Thurs Feb 24:

READ:
Schlosberg54
Dawson90
Galvactivator FAQ
Picard & Scheirer

WRITE:
1) Briefly describe an experience where the galvactivator brightness changed significantly while you or somebody you know was wearing it and had a clear change in emotional state.
2) Briefly describe an experience where the galvactivator brightness changed significantly while you or somebody you know was wearing it and did not have any obvious change in emotional state.
3) Schlossberg's "Attention-Rejection" (A-R) axis can be seen as a third axis for the arousal/activity - valence space. Another commonly used third axis is the "dominance" axis: a raging forest fire dominates you, whereas an ant is dominated by you. Construct a scenario involving an affective computer where the A-R axis might be useful. Construct an affective computing scenario where the "dominance" axis might be useful.

EMAIL:
Please email your response (inline/no attachments please) with SUBJECT: mas630-homework TO: picard@media.mit.edu
 

Due 1 p.m. Thurs Feb 17:

READ: 
Barsalou et al.
Hatfield et al. (Handout)
Nass and Brave (Handout)
Riskind (Handout)
Strack et al. (Handout)

WRITE:
(1) For decades, cognitive scientists, computer scientists, and engineers have been interested in designing computer systems to minimize "cognitive load" (usually load is increased by steadily increasing the perceptual or other cognitive demands of a task --e.g, by increasing the number of airplanes an air traffic controller has to track). What do this week's readings suggest about what one might call "emotional load?" Comment briefly on how emotional and cognitive load might interact. (Note that the relationship between load and performance is not usually linear for all values of load.)
(2) Summarize the affective states mentioned in these papers where a "matching" affective response was found to be beneficial in some way. Name an affective state where you think this approach would backfire and say why you think this would be the case. What response would you recommend a computer provide if it detected the state you named?
(3) Pick at least one of the physical manipulations in the readings and try it on yourself, on a task of your choice. For example, try browsing something neutral with a pen in your mouth in the two different positions, or try criss-crossing physical affect manipulations with a task having agreeing or opposing affective demands. (You can pick something not in the readings as long as it's in the spirit of them.) Of course, you know what the physical manipulations aim to do and you are a biased participant, so your results cannot be expected to be the same as those of a naive subject, or can they? Play around. (It's ok to just have fun with this item). Jot a little something about what you tried and what you learned. If you have time, try one of these manipulations on somebody who doesn't know what is supposed to happen.

EMAIL:
Please email your response (inline/no attachments please) with SUBJECT: mas630-homework TO: picard@media.mit.edu
 

Due 1 p.m. Thurs Feb 10:

READ:
Affective Computing: Introduction, Chapter 1
Bechara et al.
Lerner et al.

WRITE:
1. Construct a specific example of a human-human interaction that clearly involves affect, and how it might have an "equivalent" human-computer interaction with an affective computer; Describe the interaction and the affective information in both the human-human and human-computer cases.
2. Construct another example interaction involving affect, but this time try to think of a human-human interaction that may not carry over to the human-computer "equivalent". In what way does "the media equation" hold (or not) for your example?
3. Argue for or against this statement: "Emotions are just special kinds of thoughts."
4. If time is short, what one question/issue would you most like to discuss based on these readings?

EMAIL:
A typical response is from 1-3 pages. Please send your response TO: picard@media.mit.edu, SUBJECT: mas630-homework by the deadline above (24 hours before class, so I have time to read through your responses and prepare for class). Please place your response inline in the email (no attachments, please; ascii or html email is fine) so it is more efficient for me to go through all the homeworks carefully.
 
 

Grading:

20% Classroom participation
40% Nine assignments (reading/response)
40% Project and presentation (proposal draft due March 10, final proposal due by March 17, presentations due April 29 and May 6)
 

Policy: All students are expected to attend all classes and all project presentations. Please contact Prof. Picard in advance if you will have to miss class. Unexcused absence will affect your grade. The final project presentations are especially important for everyone to attend; please do not plan to leave for summer until after the last day of class May 6.
 
 

SPRING 2005 Syllabus:
 

Neuroscience findings related to emotion

Emotion and cognition: perception, decision-making, and creativity

Emotion and learning

Physiology of emotion

Affect recognition by machines (incl. wearable systems)

Measuring frustration/stress for usability feedback

Responding to user emotion to reduce user frustration

Inducing emotion

Robots/agents that "have" emotion

Emotion and behavior

Expression of emotion by machines/agents/synthetic characters

Philosophical, social, ethical implications of affective computing