MAS 963: Curious Machines

Instructor: Bruce Blumberg, Cynthia Breazeal, and Deb Roy

Wed, 2-4pm, E15-054

It is ironic that we are witness today to an abundance of machines that learn, yet we would not consider any of them to be curious. Curiosity is a trait of those natural learning systems, such as people and animals, that learn what they ought to learn when they ought to learn it. It implies a pro-active system that is motivated to learn. It also implies a reflective aspect to the learning process: when to learn, what to learn, from whom, how, and why? How can we build machines that are as curious learners as natural systems? How can we build systems that have a deeper understanding of the learning process beyond turning the statistical crank of a learning algorithm? How can synthetic systems interact with and leverage rich environments that include other agents?

This course examines the issues, principles, and challenges toward building curious machines through lecture, lively discussion, critique of course readings, and student projects. Both natural and synthetic systems are explored to investigate how to build machines that are:

Curious, self-motivated and pro-active learners.

Reflect upon their learning process in order to initiate learning what they ought to learn when they ought to learn it.

Have learning behaviors that are transparent and understandable to a human instructor. They must be easy to train and teach based on their observable behavior.

Grading:

Summaries / critiques 25%

Class participation / presentations 25%

Project/paper 50%

Weekly readings:

Two components: summary and critique
One page per reading maximum
Due at noon, Monday before each class

Summaries should be emailed to curious-fac@media.mit.edu

Feb 12 (summaries / critiques due Monday at noon):

Reading 1 J. Call & M. Tomasello "The effect of humans on the cognitive development of
apes". In Reaching into Thought (eds. A.E. Russou, K. A. Bard and S. T.
Parker). Cambridge University Press (1996). pp 371--403.

Reading 2. A. Meltzoff and A. Gopnik "The role of imitation in understanding persons
and developing a theory of mind". In Understanding other minds: perspectives
from autism (eds. S. Baron-cohen, H. Tager-Flusberg, D. Cohen) Oxford
University Press (1993). Chapter 16 pp 335--366.

Feb 19 (summaries / critiques due Tues at noon):

Reading 1: Pongracz, Miklosi, A., Kubinyi, E., Gurobi, K., Topal, J., and Csanyi, V. , Social learning in dogs: the effect of a human demonstrator on the performance of dogs in a detour task. ANIMAL BEHAVIOUR, 2001, 62, 1109–1117.

Reading 2: Slabbert, J.M. and Rasa A. E., Observational learning of an acquired maternal behavior pattern by working dog pups: an alternative training method? Applied Animal Behavior Science 53 (1997) 309-316

Reading 3: Hare, B., Brown, M., Williamson, C., Tomasello, M. The Domestication of Social Cognition in Dogs, Science Vol 298, pg 1634-1636

Feb 26 (summaries / critiques due Tues at noon):

Reading 1: Dennett, D. "True Believers: The Intentional Strategy and Why It Works." in Mind Design II, J. Haugeland (ed.), MIT Press, 1997. (Originally 1981).
Reading 2: Bloom, P., (2002). Mindreading, communication, and the learning of the names for things. Mind and Language, 17, 37-54.

March 5

Reading 1: Zukow-Goldring, P., M.A. Arbib and E. Oztop. Language and the Mirror System: A Perception/Action Based Approach to Communicative Development

Reading 2: Gallese, V. and A. Goldman. (1998). Mirror neurons and the simulation theory of mind-reading. Trends in Cognitive Sciences, 2(12).

Reading 3: Williams, J., A. Whiten, T. Suddendorf, and D.I. Perrett. (2001). Imitation, mirror neurons and autism. Neuroscience and Biobehavioural Review 25:287-295.

March 12

Reading 1: Arkin, R., Fujita, M., Takagi, T., and Hasegawa, R. (2003). An Ethological and Emotional Basis for Human-Robot Interaction, Robotics and Autonomous Systems , 42 (3-4), March 2003.

Reading 2: Hall, W.G., and C.L. Williams. (1983). Suckling isn't feeding, or is it? A search for developmental continuities. Advances in the Study of Behaviour, 13: 219-254.

March 19: No class (TTT)

March 26: No class (Spring break)

April 2

Reading 1: Minksy, M. (In preparation). The emotion machine, Chapter 2: Attachments and Goals.

Reading 2: Sloman, A. (2001). Varieties of affect and the CogAff architecture schema. Proceedings of the AISB'01 Symposium on Emotion, Cognition, and Affective Computing.

Reading 3: Norman, D. A., Ortony, A., & Russell, D. M. (2003). Affect and machine design: Lessons for the development of autonomous machines. IBM Systems Journal, 42 (1), 38-44.

April 9: No class

April 16

Reading 1: Killeen, P.R. (2001). Modeling games from the 20th century. Behaviroural Processes 54: 33-52.

Reading 2: Blumberg, B., M. Downie, Y. Ivanov, M. Berlin, M. Johnson, and B. Tomlinson.

April 23

Reading 1: Schaal S, A. Ijspeert, and A. Billard. (2003). Computational approaches to motor learning by imitation. Philosophical Transaction of the Royal Society of London: Series B, Biological Sciences 358: 537-547.

Reading 2: Schaal, S (1999). Is imitation learning the route to humanoid robots? Trends in Cognitive Sciences 3:233-242.

April 30

Reading 1: Goldstone, R. L., & Rogosky, B. J. (2002). Using relations within conceptual systems to translate across conceptual systems, Cognition, 84, 295-320.

Reading 2: Fass, D. and Feldman, J. (2002) Categorization under complexity: a unified MDL account of human learning of regular and irregular categories. Advances in Neural Information Processing Systems.

Reading 3: Feldman, J. (2000) Minimization of Boolean complexity in human concept learning. Nature, 407, 630-633.

May 7

Reading 1: C. R. Gallistel, C.R. (In press). The Principle of Adaptive Specialization as It Applies to Learning and Memory. To appear in R. H. Kluwe, G. Lüer & F. Rösler (Eds.) Principles of Human Learning and Memory. Basel: Birkenaeuser

Presentation of readings:

Presentation assignments made by Friday of each week
Presentation should include summary of work, critique, and connections to relevant other literature

Projects:

Analytical paper with original perspectives on literature, OR build and evaluate computational model, and connect to literature
Proposals due: Wed, April 9
Presentations: Wed, May 14
Final conference style paper (8 page max) due: Wed, May 14

Reading List (Tentative and Partial)

Interaction and Learning in Natural Systems

Catchpole, C.K. & Slater, P.J.B. (1995) Bird Song: Biological themes and variations. Cambridge: Cambridge University Press (selected chapter)

Clark and Schaefer. (1989). Contributing to Discourse. Cognitive Science 12: 259-294.

Dautenhahn, K. and C. Nehaniv (eds). (2002) . Imitation in Animals and Artifacts. MIT Press.

Diamond, A. (1990). Developmental time course in human infants and infant monkeys, and the neural bases of inhibitory control in reaching, The development and neural bases of higher cognitive functions (ed. Adele Diamond). Annals of the New York Academy of Sciences vol 608.

Gould, J. and C. Gould (1999). The Animal Mind. New York, N.Y., W.H. Freeman. (chapters 3 & 4)

Halliday, M.A.K. (1975). Learning to Mean: Explorations in the Development of Language, Elsevier.

Kreithen, M. (1983). Orientation Strategies in birds: A tribute to WT Keeton, Behavioral Energetics: the cost of survival in vertebrates, Ohio State University press, pp 3—28.

Lorenz, K. and P. Leyhausen (1973). Motivation of Human and Animal Behavior: An Ethological View. New York, NY., Van Nostrand Reinhold Co (Chapter 73)

Putnam, H. (1975). The meaning of 'meaning.' In H. Putnam, Mind, language, and reality: Philosophical papers, vol. 2. Cambridge: Cambridge University Press.

Ramirez, K., Ed. (1999). Animal Training: Successful Animal Management Through Positive Reinforcement. Chicago, Il., Shedd Aquarium. (selected chapters)

Von Hofsten, C. (1991). Structuring of early reaching movements: A longitudinal study. Journal of Motor Behavior, vol. 23, no 4 pp 280—292.

Whiten, A. (forthcoming). The dissection of socially mediated learning.

Architectures of Mind

Isla, D., R. Burke, et al. (2001). A Layered Brain Architecture for Synthetic Creatures. Proceedings of The International Joint Conference on Artificial Intelligence. Seattle, WA.

McCarthy, J., M. Minsky, A. Sloman, L. Gong, T. Lau, L. Morgenstern, E. Mueller, D. Riecken, M. Singh, and P. Singh. (2002). An architecture of diversity for commonsense reasoning”. IBM Systems Journal, 41(3), pp. 530-539.
Norman, D. A., Ortony, A., & Russell, D. M. (forthcoming, 2003). Affect and machine design: Lessons for the development of autonomous machines. In press, IBM Systems Journal.

Sloman, A. and B. Logan. (2002). Evolvable architectures for human-like minds. In Affective Minds, Ed. Giyoo Hatano, Elsevier.

Interaction and Learning in Synthetic Systems

Agre, P. and D. Chapman. (1987). Pengi: An Implementation of a Theory of Activity, AAAI National Conference.

Blumberg, B. (2002). D-Learning: What learning in dogs tells us about building characters that learn what they ought to learn. In: Exploring Artificial Intelligence in the New Millenium. G. Lakemeyer and B. Nebel. San Francisco, Morgan Kaufmann Publishers.

Blumberg, B., M. Downie, et al. (2002). "Integrated Learning for Interactive Synthetic Characters." Transactions on Graphics 21, 3(Proceedings of ACM SIGGRAPH 2002).

Breazeal, C. (forthcoming, 2002). Emotion and sociable humanoid robots". In International Journal of Human Computer Interaction.

Brooks, R. A., "Intelligence Without Representation", Artificial Intelligence Journal (47), 1991, pp. 139–159.

Humphrys, Mark (1997), Action Selection methods using Reinforcement Learning, Unpublished PhD Thesis, University of Cambridge.

Mataric, M. (1992). Integration of Representation Into Goal-Driven Behavior-Based Robots, IEEE Transactions on Robotics and Automation, 8(3).

Roy, D. and A. Pentland. (2002). Learning Words from Sights and Sounds: A Computational Model. Cognitive Science, 26(1), 113-146.

Todd, P.M., and Miller, G.F. (1991). Exploring adaptive agency II: Simulating the evolution of associative learning. In J.-A. Meyer and S.W. Wilson (Eds.), From animals to animats: Proceedings of the First International Conference on Simulation of Adaptive Behavior (pp. 306-315). Cambridge, MA: MIT Press/Bradford Books.