Readings

3

Neuronal Object Representations: Spatial Aspect of the Code

1) How sparse versus distributed are the neural codes for objects?

2) How physically/spatially localized are cortical object representations?

3) Are the answers to (1) and (2) different for different categories of objects?

Required Readings

Barlow, H. "The neuron doctrine in perception." In The Cognitive Neurosciences. Edited by M. Gazzaniga. Cambridge, MA: MIT Press, 1995, pp. 415-435. ISBN: 0262071576.

Tanaka, K. "Columns for complex visual object features in the inferotemporal cortex: clustering of cells with similar but slightly different stimulus selectivities." Cereb Cortex 13 (2003): 90-99.

Tsao, D. Y., W. A. Freiwald, T. A. Knutsen, J. B. Mandeville, and R. B. Tootell. "Faces and objects in macaque cerebral cortex." Nature Neuroscience 6, no. 9 (2003): 989-995.

Supplemental Readings (Required to read at least two 'to be presented' papers, indicated by *)

Sparse vs. Distributed Representations:

*Olshausen, B. A., and D. J. Field. "Sparse coding of sensory inputs." Curr Opin Neurobiol 14 (2004): 481-487.

Simoncelli, E. P. "Vision and the statistics of the visual environment." Curr Opin Neurobiol 13 (2003): 144-149.

Olshausen, B. A., and D. J. Field. "Emergence of simple-cell receptive field properties by learning a sparse code for natural images." Nature 381 (1996): 607-609. (See comments.)

Peters, R. J., F. Gabbiani, and C. Koch. "Human visual object categorization can be described by models with low memory capacity." Vision Research 43 (2003): 2265-2280.

Okada, M. "Notions of Associative Memory and Sparse Coding." Neural Netw 9 (1996): 1429-1458.

Physiology:

*Rolls, E. T, and M. J. Tovee. "Sparseness of the neuronal representation of stimuli in the primate temporal visual cortex." Journal of Neurophysiology 73 (1995): 713-726.

*Tsunoda, K., Y. Yamane, M. Nishizaki, and M. Tanifuji. "Complex objects are represented in macaque inferotemporal cortex by the combination of feature columns." Nature Neuroscience 4 (2001): 832-838.

Baddeley, R., L. F. Abbott, M. C. Booth, F. Sengpiel, T. Freeman, E. A. Wakeman, and E. T. Rolls. "Responses of neurons in primary and inferior temporal visual cortices to natural scenes." Proc R Soc Lond B Biol Sci 264 (1997): 1775-1783.

Young, M. P., and S. Yamane. "Sparse population coding of faces in the inferotemporal cortex." Science 256 (1992): 1327-1331.

Vinje, W. E., and J. L. Gallant. "Sparse coding and decorrelation in primary visual cortex during natural vision." Science 287 (2000): 1273-1276.

Weliky, M., J. Fiser, R. H. Hunt, and D. N. Wagner. "Coding of natural scenes in primary visual cortex." Neuron 37 (2003): 703-718.

fMRI:

Haxby, J. V., M. I. Gobbini, M. L. Furey, A. Ishai, J. L. Schouten, and P. Pietrini. "Distributed and overlapping representations of faces and objects in ventral temporal cortex." Science 293 (2001): 2425.

*Carlson, T. A., P. Schrater, and S. He. "Patterns of activity in the categorical representations of objects." Journal of Cognitive Neuroscience 15, no. 5 (2003): 704-717.

Spiridon, M., and N. Kanwisher. "How distributed is visual category information in human occipito-temporal cortex? An fMRI study." Neuron 35, no. 6 (2002): 1157-1165.

Cox, D. D., and R. L. Savoy. "Functional magnetic resonance imaging (fMRI) "brain reading": detecting and classifying distributed patterns of fMRI activity in human visual cortex." Neuroimage 19 (2003): 261-270.

Levy, I., U. Hasson, and R. Malach. "One picture is worth at least a million neurons." Curr Biol 14, no. 11 (2004): 996-1001.

Spatially Localized Representations and Maps:

  Kanwisher, N. Chalupa, and Werner. The Visual Neurosciences. Cambridge, MA: MIT Press, November 2003, chapter 79. ISBN: 0262033089.

Kohonen, T., and R. Hari. "Where the abstract feature maps of the brain might come from." Trends in Neurosciences 22 (1999): 135-139.

Kohonen, T. "Cortical maps." Nature 346, no. 24 (1990).

Gawne, T. J., T. W. Kjaer, J. A. Hertz, and B. J. Richmond. "Adjacent visual cortical complex cells share about 20% of their stimulus-related information." Cereb Cortex 6 (1996): 482-489.

5

Adaptation Effects: fMRI and Neurophysiology

1) Under what conditions and time scales do neural responses to visuallypresented objects adapt?

2) What are the mechanisms underlying such adaptation?

3) What do the answers to (1) and (2) say about the validity of fMRI adaptation as a tool for studying object representations?

Required

Kourtzi, Z., and K. Grill-Spector. "fMR-adaptation: a tool for studying visual representations in the primate brain." In Fitting the Mind to the World: Aftereffects in High-Level Vision. Edited by Clifford, and Rhodes. New York, NY: Oxford University Press. (In press.)

Ringo, J. L. "Stimulus specific adaptation in inferior temporal and medial temporal cortex of the monkey." Behavioural Brain Research 76 (1996): 191-197.

Supplemental Readings (Required to read at least two 'to be presented' papers, indicated by *)

Behavior:

*Leopold, D. A., A. J. O'Toole, T. Vetter, and V. Blanz. "Prototype-referenced shape encoding revealed by high-level aftereffects." Nature Neuroscience 4, no. 1 (2001): 89-94.

Moradi, F., C. Koch, and S. Shimojo. "Face adaptation depends on seeing the face." Neuron 45, no. 1 (2005): 169-175.

Rhodes, G., L. Jeffery, T. L. Watson, E. Jaquet, C. Winkler, and C. W. Clifford. "Orientation-contingent face aftereffects and implications for face-coding mechanisms." Curr Biol 14, no. 23 (2004): 2119-2123.

Watson, T. L., and C. W. Clifford. "Pulling faces: an investigation of the face-distortion aftereffect." Perception 32, no. 9 (2003): 1109-1116.

Physiology:

*Lueschow, A., E. K. Miller, and R. Desimone. "Inferior temporal mechanisms for invariant object recognition." Cereb Cortex 4, no. 5 (1994): 523-531.

*Li, L., E. K. Miller, and R. Desimone. "The representation of stimulus familiarity in anterior inferior temporal cortex." Journal of Neurophysiology 69, no. 6 (1993): 1918-1929.

Kohn, A., and J. A. Movshon. "Adaptation changes the direction tuning of macaque MT neurons." Nature Neuroscience 7, no. 7 (2004): 764-772.

Suzuki, W. "The long and the short of it: Memory signals in the medial temporal lobe." Neuron 24 (1999): 295-298.

Muller, J. R., A. B. Metha, J. Krauskopf, and P. Lennie. "Rapid adaptation in visual cortex to the structure of images." Science 285, no. 5432 (1999): 1405-1408.

fMRI:

*Rotshtein, P., R. N. Henson, A. Treves, J. Driver, and R. J. Dolan. "Morphing Marilyn into Maggie dissociates physical and identity face representations in the brain." Nature Neuroscience 8, no. 1 (2005): 107-113.

Winston, J. S., R. N. Henson, M. R. Fine-Goulden, and R. J. Dolan. "fMRI-adaptation reveals dissociable neural representations of identity and expression in face perception." Journal of Neurophysiology 92, no. 3 (2004): 1830-1839.

Henson, R. N., A. Rylands, E. Ross, P. Vuilleumeir, and M. D. Rugg. "The effect of repetition lag on electrophysiological and haemodynamic correlates of visual object priming." Neuroimage 21, no. 4 (2004): 1674-1689.

Boynton, G. M., and E. M. Finney. "Orientation-specific adaptation in human visual cortex." Journal of Neuroscience 23, no. 25 (2003): 8781-8787.

Henson, R., T. Shallice, and R. Dolan. "Neuroimaging evidence for dissociable forms of repetition priming." Science 287, no. 5456 (2000): 1269-1272.

Kourtzi, Z., A. S. Tolias, C. F. Altmann, M. Augath, and N. K. Logothetis. "Integration of local features into global shapes: monkey and human FMRI studies." Neuron 37, no. 2 (2003): 333-346.

7

The Tolerances (invariances) of Neuronal Representations

Part 1: Affine Transformations: Position and Scale

1) What aspects of the retinal image are neurons/voxels able to ignore or otherwise discount or re-code?

2) Specifically, what is the tolerance to changes in object position and size.? Is position and size information discarded or somehow re-coded? Binding problem?

Required Readings

Ashbridge, E., and D. I. Perrett. "Generalizing across object orientation and size." In Perceptual Constancy. Edited by V. Walsh, and J. Kulikowski. Cambridge, UK: Cambridge University Press, 1998, pp. 192-209. ISBN: 0521460611.

Rolls, E. T. "Functions of the primate temporal lobe cortical visual areas in invariant visual object and face recognition." Neuron 27 (2000): 205-218.

Supplemental Readings (Required to read at least two 'to be presented' papers, indicated by *)

Behavior:

*Nazir, T. A., and J. K. O'Regan. "Some results on translation invariance in the human visual system." Spat Vis 5 (1990): 81-100.

Biederman, I., and E. E. Cooper. "Evidence for complete translational and reflectional invariance in visual object priming." Perception 20 (1991): 585-593.

———. "Size invariance in visual object priming." J Exp Psychol [Hum Percept and Perform] 18 (1992): 121-133.

Dill, M., and M. Fahle. "Limited translation invariance of human pattern recognition." Perception & Psychophysics 60 (1998): 65-81.

Some Theoretical Approaches:

*Riesenhuber, M., and T. Poggio. "Hierarchical models of object recognition in cortex." Nature Neuroscience 2 (1999): 1019-1025.

Olshausen, B. A., C. H. Anderson, and D. C. Van Essen. "A neurobiological model of visual attention and invariant pattern recognition based on dynamic routing of information." Journal of Neuroscience 13 (1993): 4700-4719.

Salinas, E., and L. F. Abbott. "Invariant visual responses from attentional gain fields." Journal of Neurophysiology 77 (1997): 3267-3272.

Physiology:

Logothetis, N. K., J. Pauls, and T. Poggio. "Shape representation in the inferior temporal cortex of monkeys." Curr Biol 5 (1995): 552-563.

Tovée, M. J., E. T. Rolls, and P. Azzopardi. "Translation invariance in the responses to faces of single neurons in the temporal visual cortical areas of the alert monkey." Journal of Neurophysiology 72 (1994): 1049-1060.

*Ito, M., H. Tamura, I. Fujita, and K. Tanaka. "Size and position invariance of neuronal responses in monkey inferotemporal cortex." Journal of Neurophysiology 73 (1995): 218-226.

DiCarlo, J. J., and J. H. R. Maunsell. "Anterior Inferotemporal Neurons of Monkeys Engaged in Object Recognition Can be Highly Sensitive to Object Retinal Position." Journal of Neurophysiology 89 (2003): 3264-3278.

Op de Beeck, H., and R. Vogels. "Spatial sensitivity of macaque inferior temporal neurons." J Comp Neurol 426 (2000): 505-518.

Gross, C. G., and M. Mishkin. "The neural basis of stimulus equivalence across retinal translation." In Lateralization in the Nervous System. Edited by S. Harnad. New York, NY: Academic Press, 1977, pp. 109-122. ISBN: 0123257506.

Desimone, R., T. D. Albright, C. G. Gross, and C. Bruce. "Stimulus-selective properties of inferior temporal neurons in the macaque." Journal of Neuroscience 4 (1984): 2051-2062.

fMRI:

*Grill-Spector, K., T. Kushnir, S. Edelman, G. Avidan, Y. Itzchak, and R. Malach. "Differential processing of objects under various viewing conditions in the human lateral occipital complex." Neuron 24, no. 1 (1999): 187-203.

Andrews, T. J., and M. P. Ewbank. "Distinct representations for facial identity and changeable aspects of faces in the human temporal lobe." Neuroimage 23, no. 3 (2004): 905-913.

9

Effects of Training/Experience on Object Representations

1) Should we view neuronal representations in IT/LOC as largely fixed or plastic?

2) What are the mechanisms that allow us to improve our object discrimination abilities?

3) Under what conditions and time scales do changes in behavior and neuronal representations take place?

4) Are the invariances of object representations learned or automatic?

Required Readings

Miyashita, Y. "Inferior temporal cortex: where visual perception meets memory." Ann Rev Neurosci 16 (1993): 245-263.

Supplemental Readings (Required to read at least two 'to be presented' papers, indicated by *)

Computation:

*Wallis, G., and H. Bulthoff. "Learning to recognize objects." Trends in Cognitive Sciences 3 (1999): 22-31.

Foldiak, P. "Learning invariance from transformation sequences." Neural Comp 3 (1991): 194-200.

Karklin, Y., and M. S. Lewicki. "Learning higher-order structures in natural images." Network 14 (2003): 483-499.

Behavior:

*Furmanski, C. S., and S. A. Engel. "Perceptual learning in object recognition: object specificity and size invariance." Vision Research 40, no. 5 (2000): 473-484.

Gold, J., P. J. Bennett, and A. B. Sekuler. "Signal but not noise changes with perceptual learning." Nature 1402, no. 6758 (1999): 176-178.

Ahissar, M., and S. Hochstein. "The reverse hierarchy theory of visual perceptual learning." Trends in Cognitive Sciences 8, no. 10 (2004): 457-464.

Fabre-Thorpe, M., A. Delorme, C. Marlot, and S. Thorpe. "A limit to the speed of processing in ultra-rapid visual categorization of novel natural scenes." Journal of Cognitive Neuroscience 13, no. 2 (2001): 171-180.

Physiology:

Miyashita, Y. "Neuronal correlate of visual associative long-term memory in the primate visual cortex." Nature 335 (1988): 817-820.

Kobatake, E., G. Wang, and K. Tanaka. "Effects of shape-discrimination training on the selectivity of inferotemporal cells in adult monkeys." Journal of Neurophysiology 80 (1998): 324-330.

*Baker, C. I., M. Behrmann, and C. R. Olson. "Impact of learning on representation of parts and wholes in monkey inferotemporal cortex." Nature Neuroscience 5 (2002): 1210-1216.

Erickson, C. A., B. Jagadeesh, and R. Desimone. "Clustering of perirhinal neurons with similar properties following visual experience in adult monkeys." Nature Neuroscience 3 (2000): 1143-1148.

fMRI Papers:

*Op de Beeck, H., C. I. Baker, J. J. DiCarlo, and N. K. Kanwisher. (Submitted)

Janzen, G., and M. van Turennout. "Selective neural representation of objects relevant for navigation." Nature Neuroscience 7 (2004): 673-677.

Furmanski, C. S., D. Schluppeck, and S. A. Engel. "Learning strengthens the response of primary visual cortex to simple patterns." Current Biology 14 (2004): 573-578.

Gauthier, I., P. Skudlarski, J. C. Gore, and A. W. Anderson. "Expertise for cars and birds recruits brain areas involved in face recognition." Nature Neuroscience 3, no. 2 (2000): 191-197.

George, et. al. "Contrast polarity and face recognition in the human fusiform gyrus." Nature Neuroscience 2, no. 6 (1999): 574-804.

11

Perceptual Awareness

1) How do neural representations of objects differ when we are aware of them and when we are not?

2) What are the neural necessary conditions for perceptual awareness? For example, what if anything is the role of top-down feedback in perceptual awareness?

3) What are the processing consequences of perceptual awareness?

Required Redings

Crick, F., and C. Koch. "A framework for consciousness." Nature Neuroscience 6 (2003): 119-126.

Dehaene, S., C. Sergent, and J. P. Changeux. "A neuronal network model linking subjective reports and objective physiological data during conscious perception." Proc Natl Acad Sci U S A 100, no. 14 (2003): 8520-8525.

Optional Readings

Hochstein, S., and M. Ahissar. "View from the top: hierarchies and reverse hierarchies in the visual system." Neuron 36 (2002): 791-804.

Kanwisher, N. "Neural Events and Perceptual Awareness." Cognition 79 (2001): 89-113.

Sewards, T. V., and M. A. Sewards. "On the neural correlates of object recognition awareness: relationship to computational activities and activities mediating perceptual awareness." Conscious Cogn 11 (2002): 51-77.

Supplemental Readings (Required to read at least two 'to be presented' papers, indicated by *)

Physiology:

*Logothetis, N. K., D. A. Leopold, D. L. Sheinberg. "What is rivalling during binocular rivalry?" Nature 380, no. 6575 (1996): 621-624.

*Kreiman, G., I. Fried, and C. Koch. "Single-neuron correlates of subjective vision in the human medial temporal lobe." Proc Natl Acad Sci U S A 99, no. 12 (2002): 8378-8383.

Sheinberg, D. L., N. K. Logothetis. "Noticing familiar objects in real world scenes: the role of temporal cortical neurons in natural vision." Journal of Neuroscience 21, no. 4 (2001): 1340-1350.

fMRI:

*Pasley, B. N., L. C. Mayes, and R. T. Schultz. "Subcortical discrimination of unperceived objects during binocular rivalry." Neuron 42, no. 1 (2004): 163-172.

*Marois, R., D. J. Yi, and M. M. Chun. "The neural fate of consciously perceived and missed events in the attentional blink." Neuron 41 (2004): 465-72.

Dehaene, S., L. Naccache, L. Cohen, D. L. Bihan, J. F. Mangin, J. B. Poline, and D. Riviere. "Cerebral mechanisms of word masking and unconscious repetition priming." Nature Neuroscience 4, no. 7 (2001): 752-758.

Silvanto, J., A. Cowey, N. Lavie, and V. Walsh. "Striate cortex (V1) activity gates awareness of motion." Nature Neuroscience, no. 2 (2005): 143-144.

Pegna, A. J., A. Khateb, F. Lazeyras, and M. L. Seghier. "Discriminating emotional faces without primary visual cortices involves the right amygdala." Nature Neuroscience 8, no. 1 (2005): 24-25.