MAS 964: Meaning Machines

Prof. Deb Roy

Tues and Thurs, 2-4pm
3-0-9 (H), E15-468

description : format : grading : reading list

 
Description
We use words to do things, ranging from expressing ideas and feelings to making requests and demands. Words have meaning because they are about things in the world. By learning the conventions by which words map to experiences, we are able to use them to achieve our goals. In contrast, current natural language processing systems represent words as ungrounded symbols that have meaning solely due to their relations to other symbols. As a result, words processed by computers are not about anything, as far as computers are concerned. Words are merely shapeless colorless symbols that are manipulated for the human user who reads actual meaning into them.

The goal of this class is to develop new computational methods that move us towards machines that use language with meaning. We will study theories of language and meaning from psychology and philosophy of language/mind with an emphasis on the role of goal driven behavior, physical embodiment, and social interaction. We will develop a new synthesis of these theories, drawing from methods of artificial intelligence to make our ideas computationally precise. Through programming assignments and a final project, students will put these ideas to use by implementing human-machine communication systems.
Format
Classes will alternate between student presentations of papers (Tuesdays) and lectures by the instructor (Thursdays). For weekly readings, students will be required to write one-page summaries / critiques.
Grading
15% Weekly paper critiques.
15% Paper presentations and class participation.
30% Bi-weekly assignments.
40% Term project / paper.
Reading List
Meaning Machines: Setting the Stage
  1. Grice, H. P., (1975), Logic and Conversation, in P. Cole and J. Morgan, eds., Syntax and Semantics , vol. 3, Academic Press, pp. 41-58.
  2. Searle, J. (1969). Speech Acts, An Essay in the Philosophy of Language,
    Cambridge: Cambridge University Press. (excerpts)
  3. Jackendoff, R. (2002). Foundations of Language. Oxford University Press. Chapter 9 & 10.
  4. Barwise, J. and Perry, J. Situations and Attitudes. MIT-Bradford, 1983. (excerpts)
   Looking In
 
  1. Minsky, M. (1986). Society of Mind. Simon and Schuster.
  2. Tinbergen, N. (1950). The Hierarchical Organization of Nervous Mechanisms Underlying Instinctive Behaviour. Symposium for the Society for Experimental Biology 4: 305 - 12.
  3. Rosenblueth, A., Wiener, N. and Bigelow, J. (1943). Behavior, purpose
    and teleology. Philosophy of Science, 10, pp. 18-24.
   Looking Between
 
  1. Brooks, R. A. (1991). Intelligence Without Representation, Artificial Intelligence Journal 47: 139–159.
  2. Smith, B.C.. (1996). On the origin of objects. MIT Press. (excerpts)
  3. Grush, Rick (2001). Self, World and Space: On the Meaning and Mechanisms of Egocentric and Allocentric Spatial Representation. Brain and Mind 1(1):59-92.
  4. Kosslyn, S. (1994). Image and Brain, MIT Press.
  Looking Out from the Inside
 
  1. Dennett, D. True Believers: The Intentional Strategy and Why It Works. in Mind Design II, J. Haugeland (ed.), MIT Press, 1997. (Originally 1981).
  2. Gopnik, A. (1996). The Scientist as Child. Philosophy of Science, 63(4), 485-514.
  Coordination, Cooperation, Communication
 
  1. Goldstone, R. L., & Rogosky, B. J. (2002). Using Relations within Conceptual Systems to Translate Across Conceptual Systems, Cognition, 84, 295-320.
  2. Gallese, V. and A. Goldman. (1998). Mirror Neurons and the Simulation Theory of Mind-reading. Trends in Cognitive Sciences, 2(12).
  3. Clark, H. (1996). Using Language. Cambridge University Press. (excerpts)
  Grounding Language
 
  1. Harnad, S. (1990) The Symbol Grounding Problem. Physica D 42: 335-346.
  2. Hofstadter, D. (1995). A Review of Mental Leaps: Analogy in Creative Thought. AI Magazine, Fall 1995, 75-80 .
  3. Gardenfors, P. (1997). Symbolic, Conceptual and Subconceptual Representations, pp. 255-270 in Human and Machine Perception: Information Fusion, ed. by V. Cantoni, V. di Gesù, A. Setti and D. Tegolo, Plenum Press, New York.
  4. Regier, T. and Carlson, L. (2001). Grounding Spatial Language in Perception: An Empirical and Computational Investigation. Journal of Experimental Psychology: General, vol. 130(2), 273-298.
  5. Siskind, J. Grounding the Lexical Semantics of Verbs in Visual Perception Using Force Dynamics and Event Logic, Technical Report 2000-105, NEC Research Institute, Inc., July 2000.
  6. Narayanan, S. (1999). Reasoning About Actions in Narrative Understanding. Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI '99), pp. 350-358.
   Language and Meaning Acquisition
 
  1. Bloom, P., (2002). Mindreading, Communication, and the Learning of the Names for Things. Mind and Language, 17, 37-54.
  2. Tomasello. M. and W. Merriman. (1994). Beyond Names for Things: Young Children’s Acquisition of Verbs. Lawrence Erlbaum. (excerpts)
  3. Roy, D. (2003). Grounded Spoken Language Acquisition: Experiments in Word Learning. IEEE Transactions on Multimedia.
  4. Yu, C. and D. Ballard. (2003). Exploring the Role of Attention in Modeling Embodied Language Acquisition. Fifth International Conference on Cognitive Modeling.
  Situated Language Machines
 
  1. Roy, D., K. Hsiao, N. Mavridis, and P. Gorniak. (2003). Ripley, Hand Me the Cup! (Sensorimotor Representations for Grounding Word Meaning). IEEE Automatic Speech Recognition and Understanding Workshop
  2. Dourish, P. (2001). Seeking a Foundation for Context-Aware Computing. J. Human-Computer Interaction, Volume 16, no. 2-4.