How does the brain bind perception into one picture?
Colour, shape and motion are processed in different parts of the brain. Why do we see one red ball rolling?
The question
A red ball rolls across a green lawn. You see it at once, whole: the red belongs to the ball, the green to the grass, and the movement to the ball, not the lawn. It seems too obvious to need explaining.
But the brain doesn't see this way. Visual information is split up almost immediately: colour is processed largely in one area, motion in another, shape in others still, and sound, touch and the meaning of what we see somewhere else again. There is no place in the brain where all these pieces meet on one screen. So how does the brain know which colour goes with which shape and which movement, and how do separate signals become a single experience? This is the binding problem.
What we know
- Specialisation is real. Damage to one area can leave a person unable to see colour while seeing shapes perfectly; damage to another can leave them unable to perceive motion, so that pouring tea looks like a series of frozen images.
- Binding can fail. In 1980 Anne Treisman proposed that attention is needed to combine features. When people see coloured letters for a moment while their attention is busy elsewhere, they sometimes report "illusory conjunctions": a red X when the X was blue and something else was red. A patient with damage to both parietal lobes made such errors all the time, even when looking for as long as he wanted.
- The different features of the same object aren't even perceived at exactly the same moment. Experiments suggest that colour is registered a few tens of milliseconds earlier than motion, although we never notice the gap.
- In 1989 Wolf Singer's team found that neurons responding to the same object can fire in synchrony, in rhythms of about 40 times a second, while neurons responding to different objects don't. This seemed for a while to be the answer.
Main hypotheses
Binding by synchrony
Neurons that represent features of the same object fire at the same moments, and this shared timing tells the rest of the brain that the features belong together. The idea is elegant, but critics point out that synchrony could be a consequence of perception rather than its cause, and that it's not clear how other neurons would read such a timing code.
Binding by attention
Attention links features at one location: whatever is at the place being attended to is bound into one object. This explains illusory conjunctions well, but we perceive whole scenes, not one object at a time.
Convergence and population codes
Perhaps there is no separate binding step. Neurons higher up in the brain respond to combinations of features, and distributed patterns across many neurons represent whole objects. Critics reply that there are far too many possible combinations to have a dedicated neuron for each.
No single problem
Some researchers think "binding" covers several different problems: linking features in vision, linking the senses, linking words into sentences, and the unity of experience as a whole. Each may have its own solution.
Where understanding ends
We know where the brain processes different features and roughly how attention helps combine them. We don't know what physical event the binding is: what in the activity of neurons makes "red" and "round" belong to the same thing.
The question splits into two layers. The first is computational: how the brain keeps track of what goes with what, a problem that even artificial neural networks struggle with, sometimes mixing up which colour belongs to which object. The second is the unity of experience itself: why we have one picture rather than separate streams of colour and shape. Here the binding problem merges into the problem of consciousness, and here understanding stops.
Anaesthesia offers a clue: many theories of how it switches off the mind say it breaks communication between brain regions, exactly what binding would need. But a clue isn't an explanation.
What would settle it
- Experiments that manipulate a specific candidate mechanism, such as synchrony, and show that binding fails while the separate features are still perceived.
- A detailed model that explains the timing and errors of human binding, including illusory conjunctions, and predicts new ones.
- A theory that connects how features are combined with why the result is experienced as one scene.
Sources
- Treisman, A. M., & Gelade, G. (1980). A feature-integration theory of attention. Cognitive Psychology, 12(1), 97–136.
- Treisman, A., & Schmidt, H. (1982). Illusory conjunctions in the perception of objects. Cognitive Psychology, 14(1), 107–141.
- Gray, C. M., König, P., Engel, A. K., & Singer, W. (1989). Oscillatory responses in cat visual cortex exhibit inter-columnar synchronization which reflects global stimulus properties. Nature, 338(6213), 334–337.
- Friedman-Hill, S. R., Robertson, L. C., & Treisman, A. (1995). Parietal contributions to visual feature binding: evidence from a patient with bilateral lesions. Science, 269(5225), 853–855.
- Moutoussis, K., & Zeki, S. (1997). A direct demonstration of perceptual asynchrony in vision. Proceedings of the Royal Society of London. Series B: Biological Sciences, 264(1380), 393–399.
- Roskies, A. L. (1999). The binding problem. Neuron, 24(1), 7–9.
- Shadlen, M. N., & Movshon, J. A. (1999). Synchrony unbound: a critical evaluation of the temporal binding hypothesis. Neuron, 24(1), 67–77.
- Feldman, J. (2013). The neural binding problem(s). Cognitive Neurodynamics, 7(1), 1–11.