MAS 963: Curious Machines 

Instructor: Bruce Blumberg, Cynthia Breazeal, and Deb Roy 

Wed, 2-4pm, E15-054 

It is ironic that we are witness today to an abundance of machines that learn, yet we would not consider any of them to be curious. Curiosity is a trait of those natural learning systems, such as people and animals, that learn what they ought to learn when they ought to learn it. It implies a pro-active system that is motivated to learn. It also implies a reflective aspect to the learning process: when to learn, what to learn, from whom, how, and why? How can we build machines that are as curious learners as natural systems? How can we build systems that have a deeper understanding of the learning process beyond turning the statistical crank of a learning algorithm? How can synthetic systems interact with and leverage rich environments that include other agents?

This course examines the issues, principles, and challenges toward building curious machines through lecture, lively discussion, critique of course readings, and student projects. Both natural and synthetic systems are explored to investigate how to build machines that are:


Weekly readings:

Two components: summary and critique
One page per reading maximum
Due at noon, Monday before each class

Summaries should be emailed to

Feb 12 (summaries / critiques due Monday at noon):

Feb 19 (summaries / critiques due Tues at noon):

Feb 26 (summaries / critiques due Tues at noon):

March 5

March 12

March 19: No class (TTT)

March 26: No class (Spring break)

April 2

April 9: No class

April 16

  • Reading 1: Killeen, P.R. (2001). Modeling games from the 20th century. Behaviroural Processes 54: 33-52.
  • Reading 2: Blumberg, B., M. Downie, Y. Ivanov, M. Berlin, M. Johnson, and B. Tomlinson.

April 23

  • Reading 1: Schaal S, A. Ijspeert, and A. Billard. (2003). Computational approaches to motor learning by imitation. Philosophical Transaction of the Royal Society of London: Series B, Biological Sciences 358: 537-547.
  • Reading 2: Schaal, S (1999). Is imitation learning the route to humanoid robots? Trends in Cognitive Sciences 3:233-242.

April 30

  • Reading 1: Goldstone, R. L., & Rogosky, B. J. (2002). Using relations within conceptual systems to translate across conceptual systems, Cognition, 84, 295-320.
  • Reading 2: Fass, D. and Feldman, J. (2002) Categorization under complexity: a unified MDL account of human learning of regular and irregular categories. Advances in Neural Information Processing Systems.
  • Reading 3: Feldman, J. (2000) Minimization of Boolean complexity in human concept learning. Nature, 407, 630-633.

May 7

Presentation of readings:

Presentation assignments made by Friday of each week
Presentation should include summary of work, critique, and connections to relevant other literature


Analytical paper with original perspectives on literature, OR build and evaluate computational model, and connect to literature
Proposals due: Wed, April 9
Presentations: Wed, May 14
Final conference style paper (8 page max) due: Wed, May 14

Reading List (Tentative and Partial)

Interaction and Learning in Natural Systems

Catchpole, C.K. & Slater, P.J.B. (1995) Bird Song: Biological themes and variations. Cambridge: Cambridge University Press (selected chapter)

Clark and Schaefer. (1989). Contributing to Discourse. Cognitive Science 12: 259-294.

Dautenhahn, K. and C. Nehaniv (eds). (2002) . Imitation in Animals and Artifacts. MIT Press.

Diamond, A. (1990). Developmental time course in human infants and infant monkeys, and the neural bases of inhibitory control in reaching, The development and neural bases of higher cognitive functions (ed. Adele Diamond). Annals of the New York Academy of Sciences vol 608.

Gould, J. and C. Gould (1999). The Animal Mind. New York, N.Y., W.H. Freeman. (chapters 3 & 4)

Halliday, M.A.K. (1975). Learning to Mean: Explorations in the Development of Language, Elsevier.

Kreithen, M. (1983). Orientation Strategies in birds: A tribute to WT Keeton, Behavioral Energetics: the cost of survival in vertebrates, Ohio State University press, pp 3—28.

Lorenz, K. and P. Leyhausen (1973). Motivation of Human and Animal Behavior: An Ethological View. New York, NY., Van Nostrand Reinhold Co (Chapter 73)

Putnam, H. (1975). The meaning of 'meaning.' In H. Putnam, Mind, language, and reality: Philosophical papers, vol. 2. Cambridge: Cambridge University Press.

Ramirez, K., Ed. (1999). Animal Training: Successful Animal Management Through Positive Reinforcement. Chicago, Il., Shedd Aquarium. (selected chapters)

Von Hofsten, C. (1991). Structuring of early reaching movements: A longitudinal study. Journal of Motor Behavior, vol. 23, no 4 pp 280—292.

Whiten, A. (forthcoming). The dissection of socially mediated learning.

Architectures of Mind

Isla, D., R. Burke, et al. (2001). A Layered Brain Architecture for Synthetic Creatures. Proceedings of The International Joint Conference on Artificial Intelligence. Seattle, WA.

McCarthy, J., M. Minsky, A. Sloman, L. Gong, T. Lau, L. Morgenstern, E. Mueller, D. Riecken, M. Singh, and P. Singh. (2002). An architecture of diversity for commonsense reasoning”. IBM Systems Journal, 41(3), pp. 530-539.
Norman, D. A., Ortony, A., & Russell, D. M. (forthcoming, 2003). Affect and machine design: Lessons for the development of autonomous machines. In press, IBM Systems Journal.

Sloman, A. and B. Logan. (2002). Evolvable architectures for human-like minds. In Affective Minds, Ed. Giyoo Hatano, Elsevier.

Interaction and Learning in Synthetic Systems

Agre, P. and D. Chapman. (1987). Pengi: An Implementation of a Theory of Activity, AAAI National Conference.

Blumberg, B. (2002). D-Learning: What learning in dogs tells us about building characters that learn what they ought to learn. In: Exploring Artificial Intelligence in the New Millenium. G. Lakemeyer and B. Nebel. San Francisco, Morgan Kaufmann Publishers.

Blumberg, B., M. Downie, et al. (2002). "Integrated Learning for Interactive Synthetic Characters." Transactions on Graphics 21, 3(Proceedings of ACM SIGGRAPH 2002).

Breazeal, C. (forthcoming, 2002). Emotion and sociable humanoid robots". In International Journal of Human Computer Interaction.

Brooks, R. A., "Intelligence Without Representation", Artificial Intelligence Journal (47), 1991, pp. 139–159.

Humphrys, Mark (1997), Action Selection methods using Reinforcement Learning, Unpublished PhD Thesis, University of Cambridge.

Mataric, M. (1992). Integration of Representation Into Goal-Driven Behavior-Based Robots, IEEE Transactions on Robotics and Automation, 8(3).

Roy, D. and A. Pentland. (2002). Learning Words from Sights and Sounds: A Computational Model. Cognitive Science, 26(1), 113-146.

Todd, P.M., and Miller, G.F. (1991). Exploring adaptive agency II: Simulating the evolution of associative learning. In J.-A. Meyer and S.W. Wilson (Eds.), From animals to animats: Proceedings of the First International Conference on Simulation of Adaptive Behavior (pp. 306-315). Cambridge, MA: MIT Press/Bradford Books.