Binding problem


The binding problem is a term used at the interface between neuroscience, cognitive science and philosophy of mind that has multiple meanings.
Firstly, there is the segregation problem: a practical computational problem of how brains segregate elements in complex patterns of sensory input so that they are allocated to discrete "objects". In other words, when looking at a blue square and a yellow circle, what neural mechanisms ensure that the square is perceived as blue and the circle as yellow, and not vice versa? The segregation problem is sometimes called BP1.
Secondly, there is the combination problem: the problem of how objects, background and abstract or emotional features are combined into a single experience. The combination problem is sometimes called BP2.
However, the difference between these two problems is not always clear. Moreover, the historical literature is often ambiguous as to whether it is addressing the segregation or the combination problem.

The segregation problem

Definition

The segregation problem, also known as binding problem 1, is the problem of how brains segregate elements in complex patterns of sensory input so that they are allocated to discrete "objects".
John Raymond Smythies defined BP1 in these terms: "How is the representation of information built up in the neural networks that there is one single object 'out there' and not a mere collection of separate shapes, colours and movements?" Revonsuo refers to this as the problem of "stimulus-related binding" – of sorting stimuli. Although usually referred to as a problem of binding, the computational problem is arguably one of discrimination. Thus, in the words of Canales et al.: "to bind together all the features of one object and segregate them from features of other objects and the background". Bartels and Zeki describe it as "determining that it is the same stimulus which is activating different cells in a given visual area or in different visual areas".

Experimental work

Most experimental work is on vision, where it is known that humans and other mammals process different aspects of perception by separating information about those aspects and processing them in distinct regions of the brain. For example, Bartels and Zeki have shown that different areas in the visual cortex specialize in processing the different aspects of colour, motion, and shape. This type of modular coding has been claimed to yield a potential for ambiguity. When humans view a scene containing a blue square and a yellow circle, some neurons signal in response to blue, others signal in response to yellow, still others to a square shape or a circle shape. Here, the binding problem is the issue of how the brain correctly pairs colour and shape, i.e. indicates that blue goes with square, rather than yellow.

Synchronization theory

A popular hypothesis perhaps first suggested by Milner has been that features of individual objects are bound/segregated via synchronisation of the activity of different neurons in the cortex. The theory, called binding-by-synchrony, is that when two feature-neurons fire synchronously they are bound, while when they fire out of synchrony they are unbound. Empirical testing of the idea was given impetus when von der Malsburg proposed that feature binding posed a special problem that could not be covered simply by cellular firing rates. A number of studies suggested that there is indeed a relationship between rhythmic synchronous firing and feature binding. This rhythmic firing appears to be linked to intrinsic oscillations in neuronal somatic potentials, typically in the gamma range close to 40 Hz. The positive arguments for a role for rhythmic synchrony in resolving the segregational object-feature binding problem have been summarized by Singer. There is certainly extensive evidence for synchronization of neural firing as part of responses to visual stimuli.
However, there is inconsistency between findings from different laboratories. Moreover, a number of recent reviewers, including Shadlen and Movshon and Merker have raised concerns. Thiele and Stoner found that perceptual binding of two moving patterns had no effect on synchronisation of the neurons responding to the two patterns. In the primary visual cortex, Dong et al. found that whether two neurons were responding to contours of the same shape or different shapes had no effect on neural synchrony. Revonsuo reports similar negative findings.
Shadlen and Movshon, raise a series of doubts about both the theoretical and the empirical basis for the idea of segregational binding by temporal synchrony. Firstly, it is not clear that binding does pose a special computational problem of the sort proposed by von der Malsburg. Secondly, it is unclear how synchrony would come to play a distinct role in terms of local computational logic. Thirdly, it is difficult to envisage a situation in which pre-synaptic firing rate and synchrony could be usefully interpreted independently by a post-synaptic cell, since the two are interdependent over plausible time scales.
Another point that has been raised is that within standard time frames for neuronal firing very few distinct phases of synchrony would be distinguishable even under optimal conditions. However, this would only be significant if the same pathways are potentially fed spike trains in multiple phases. In contrast, Seth describes an artificial brain-based robot that demonstrates multiple, separate, widely distributed neural circuits, firing at different phases, suggesting that synchrony may assist the establishment of discrete object-related re-entrant circuits in a system exposed to randomly timed stimuli.
Goldfarb and Treisman point out that a logical problem appears to arise for binding solely via synchrony if there are several objects that share some of their features and not others. When viewing a display of variously coloured letters, internal representation of a red X, a green O, a red O and a green X cannot be accounted for purely by synchrony of signals for red and X shape, for instance. At best synchrony can facilitate segregation supported by other means.
A number of neuropsychological studies suggest that the association of colour, shape and movement as "features of an object" is not simply a matter of linking or "binding". Purves and Lotto give extensive evidence for top-down feedback signals that ensure that sensory data are handled as features of postulated objects early in processing. In many illusions data appear as if pre-consciously adjusted in accordance with object expectations. Pylyshyn has also emphasized the way the brain seems to pre-conceive objects to which features are to be allocated and which are attributed continuing existence even if features like color change.

Feature integration theory

In her feature integration theory, Treisman suggested that binding between features is mediated by the features' links to a common location. Psychophysical demonstrations of binding failures under conditions of full attention provide support for the idea that binding is accomplished through common location tags.
An implication of these approaches is that sensory data such as colour or motion may not normally exist in "unallocated" form. For Merker: "The 'red' of a red ball does not float disembodied in an abstract color space in V4." If colour information allocated to a point in the visual field is converted directly, via the instantiation of some form of propositional logic into colour information allocated to an "object identity" postulated by a top-down signal as suggested by Purves and Lotto no special computational task of "binding together" by means such as synchrony may exist.
How signals in the brain come to have propositional content, or meaning, is a much larger issue. However, both Marr and Barlow suggested, on the basis of what was known about neural connectivity in the 1970s that the final integration of features into a percept would be expected to resemble the way words operate in sentences.
The role of synchrony in segregational binding remains controversial. Merker has recently suggested that synchrony may be a feature of areas of activation in the brain that relates to an "infrastructural" feature of the computational system analogous to increased oxygen demand indicated via MRI. Apparent specific correlations with segregational tasks may be explainable on the basis of interconnectivity of the areas involved. As a possible manifestation of a need to balance excitation and inhibition over time it might be expected to be associated with reciprocal re-entrant circuits as in the model of Seth et al.
If it transpires that synchronized activity plays at most an infrastructural role in segregative computational "binding", the question arises as to whether we need another explanation. The implication both of Shadlen and Movshon's and of Merker's analyses seems to be that there may be no special binding problem in this sense. The problem may be merely an integral part of the more general problem of the computational logic used by neurons, or what is often referred to as the "neural code". In particular it may be inappropriate to analyse binding in perception without taking into account the way features are bound in memory, as addressed by Zimmer and colleagues, and how that informs the way the brain pre-conceives objects.

The combination problem

Definition

defines the combination problem, also known as binding problem 2, as "How do the brain mechanisms actually construct the phenomenal object?". Revonsuo equates this to "consciousness-related binding", emphasizing the entailment of a phenomenal aspect. As Revonsuo explores in 2006, there are nuances of difference beyond the basic BP1:BP2 division. Smythies speaks of constructing a phenomenal object but philosophers such as Descartes, Leibniz, Kant and James have typically been concerned with the broader unity of a phenomenal experience – which, as Bayne illustrates may involve features as diverse as seeing a book, hearing a tune and feeling an emotion. Further discussion will focus on this more general problem of how sensory data that may have been segregated into, for instance, "blue square" and "yellow circle" are to be re-combined into a single phenomenal experience of a blue square next to a yellow circle, plus all other features of their context. There are a wide range of views on just how real this "unity" is, but the existence of medical conditions in which it appears to be subjectively impaired, or at least restricted, suggests that it is not entirely illusory.

History

Early philosophers such as Descartes and Leibniz noted that the apparent unity of our experience is an all-or-none qualitative characteristic that does not appear to have an equivalent in the known quantitative features, like proximity or cohesion, of composite matter. William James, in the nineteenth century, considered the ways the unity of consciousness might be explained by known physics and found no satisfactory answer. He coined the term "combination problem", in the specific context of a "mind-dust theory" in which it is proposed that a full human conscious experience is built up from proto- or micro-experiences in the way that matter is built up from atoms. James claimed that such a theory was incoherent, since no causal physical account could be given of how distributed proto-experiences would "combine". He favoured instead a concept of "co-consciousness" in which there is one "experience of A, B and C" rather than combined experiences. A detailed discussion of subsequent philosophical positions is given by Brook and Raymont. However, these do not generally include physical interpretations. James remained concerned about the absence of a "single physical thing", other than an atom, that could be co-conscious, echoing Leibniz.
Whitehead proposed a fundamental ontological basis for a relation consistent with James's idea of co-consciousness, in which many causal elements are co-available or "compresent" in a single event or "occasion" that constitutes a unified experience. Whitehead did not give physical specifics but the idea of compresence is framed in terms of causal convergence in a local interaction consistent with physics. Where Whitehead goes beyond anything formally recognized in physics is in the "chunking" of causal relations into complex but discrete "occasions". Even if such occasions can be defined, Whitehead's approach still leaves James's difficulty with finding a site, or sites, of causal convergence that would make neurobiological sense for "co-consciousness". Sites of signal convergence do clearly exist throughout the brain but there is a concern to avoid re-inventing what Dennett calls a Cartesian Theater or single central site of convergence of the form that Descartes proposed.
Descartes's central "soul" is now rejected because neural activity closely correlated with conscious perception is widely distributed throughout the cortex. The remaining choices appear to be either separate involvement of multiple distributed causally convergent events or a model that does not tie a phenomenal experience to any specific local physical event but rather to some overall "functional" capacity. Whichever interpretation is taken, as Revonsuo indicates, there is no consensus on what structural level we are dealing with – whether the cellular level, that of cellular groups as "nodes", "complexes" or "assemblies" or that of widely distributed networks. There is probably only general agreement that it is not the level of the whole brain, since there is evidence that signals in certain primary sensory areas, such as the V1 region of the visual cortex, do not contribute directly to phenomenal experience.

Modern theories

Dennett has proposed that our sense that our experiences are single events is illusory and that, instead, at any one time there are "multiple drafts" of sensory patterns at multiple sites. Each would only cover a fragment of what we think we experience. Arguably, Dennett is claiming that consciousness is not unified and there is no phenomenal binding problem. Most philosophers have difficulty with this position. Dennett's view might be in keeping with evidence from recall experiments and change blindness purporting to show that our experiences are much less rich than we sense them to be – what has been called the Grand Illusion. However, few, if any, other authors suggest the existence of multiple partial "drafts". Moreover, also on the basis of recall experiments, Lamme has challenged the idea that richness is illusory, emphasizing that phenomenal content cannot be equated with content to which there is cognitive access.
Dennett does not tie drafts to biophysical events. Multiple sites of causal convergence are invoked in specific biophysical terms by Edwards and Sevush. In this view the sensory signals to be combined in phenomenal experience are available, in full, at each of multiple sites. To avoid non-causal combination each site/event is placed within an individual neuronal dendritic tree. The advantage is that "compresence" is invoked just where convergence occurs neuro-anatomically. The disadvantage, as for Dennett, is the counter-intuitive concept of multiple "copies" of experience. The precise nature of an experiential event or "occasion", even if local, also remains uncertain.
The majority of theoretical frameworks for the unified richness of phenomenal experience adhere to the intuitive idea that experience exists as a single copy, and draw on "functional" descriptions of distributed networks of cells. Baars has suggested that certain signals, encoding what we experience, enter a "Global Workspace" within which they are "broadcast" to many sites in the cortex for parallel processing. Dehaene, Changeux and colleagues have developed a detailed neuro-anatomical version of such a workspace. Tononi and colleagues have suggested that the level of richness of an experience is determined by the narrowest information interface "bottleneck" in the largest sub-network or "complex" that acts as an integrated functional unit. Lamme has suggested that networks supporting reciprocal signaling rather than those merely involved in feed-forward signaling support experience. Edelman and colleagues have also emphasized the importance of re-entrant signaling. Cleeremans emphasizes meta-representation as the functional signature of signals contributing to consciousness.
In general, such network-based theories are not explicitly theories of how consciousness is unified, or "bound" but rather theories of functional domains within which signals contribute to unified conscious experience. A concern about functional domains is what Rosenberg has called the boundary problem; it is hard to find a unique account of what is to be included and what excluded. Nevertheless, this is, if anything is, the consensus approach.
Within the network context, a role for synchrony has been invoked as a solution to the phenomenal binding problem as well as the computational one. In his book, The Astonishing Hypothesis, Crick appears to be offering a solution to BP2 as much as BP1. Even von der Malsburg, introduces detailed computational arguments about object feature binding with remarks about a "psychological moment". The Singer group also appear to be interested as much in the role of synchrony in phenomenal awareness as in computational segregation.
The apparent incompatibility of using synchrony to both segregate and unify might be explained by sequential roles. However, Merker points out what appears to be a contradiction in attempts to solve the phenomenal unification problem in terms of a functional rather than a local biophysical, domain, in the context of synchrony.
Functional arguments for a role for synchrony are in fact underpinned by analysis of local biophysical events. However, Merker points out that the explanatory work is done by the downstream integration of synchronized signals in post-synaptic neurons: "It is, however, by no means clear what is to be understood by 'binding by synchrony' other than the threshold advantage conferred by synchrony at, and only at, sites of axonal convergence onto single dendritic trees..." In other words, although synchrony is proposed as a way of explaining binding on a distributed, rather than a convergent, basis the justification rests on what happens at convergence. Signals for two features are proposed as bound by synchrony because synchrony effects downstream convergent interaction. Any theory of phenomenal binding based on this sort of computational function would seem to follow the same principle. The phenomenality would entail convergence, if the computational function does.
Although BP1 and BP2 are different, this need not invalidate the assumption, implicit in many of the quoted models, that computational and phenomenal events, at least at some point in the sequence of events, parallel each other in some way. The difficulty remains in identifying what that way might be. Merker's analysis suggests that either both computational and phenomenal aspects of binding are determined by convergence of signals on neuronal dendritic trees, or that our intuitive ideas about the need for "binding" in a "holding together" sense in both computational and phenomenal contexts are misconceived. We may be looking for something extra that is not needed. Merker, for instance, argues that the homotopic connectivity of sensory pathways does the necessary work.
The nature of, and solution to, BP2 remains a matter of controversy.

Recent developments

A new study published in Psychological Review and Interface Focus 2018 the Royal Society's cross-disciplinary journal at the interface between the physical and life sciences, sheds new light on how the visual system may represent which features are bound together as part of the same object.
The research team, led by Dr Simon Stringer from University of Oxford, performed bio-inspired Spiking Neural Network simulations of the primate ventral visual system to address this enduring question. By training the biophysical model on a set of visual stimuli, has been observed the emergence of a subpopulation of neurons, called polychronous neuronal groups, that exhibits regularly repeating spatio-temporal patterns of spikes. The underlying phenomenon of such spatio-temporal patterns responses is known as polychronization. The main point of this proposed novel approach is that within these PNGs exist neurons, called binding neurons, that learn to represent the hierarchical binding relationships between lower and higher level visual features at every spatial scale and across the entire visual field. These binding neurons have been firstly formulated by Christoph von der Malsburg, however, it has not previously been shown how these neurons may develop naturally through a biologically plausible process of visually guided learning and self-organization of polychronous neuronal groups. This latest research describe that such binding neurons emerge automatically within the PNGs during visually training when key properties of the visual cortex are incorporated into the model. This finding is consistent with the hierarchical nature of primate vision depicted by John Duncan & Glyn W. Humphreys:
"A fully hierarchical representation is created by repeating segmentation at different levels of scale. Each structural unit, contained by its own boundary, is further subdivided into parts by the major boundaries within it. Thus, a human body may be subdivided into head, torso, and limbs, and a hand into palm and fingers. Such subdivision serves two purposes. The description of a structural unit at one level of scale must depend heavily on the relations between the parts defined within it. Then, at the next level down, each part becomes a new structural unit to be further described with its own properties, defined among other things by the relations between its own subparts. At the top of the hierarchy may be a structural unit corresponding to the whole input scene, described with a rough set of properties."

Moreover, this hierarchy feature binding theory proposes that information about visual features at every spatial scale, including the binding relations between these features, would be projected upwards to the higher layers of the network, where spatial information would be available for readout by later brain systems to guide behavior. This mechanism has been called the holographic principle. Lastly, by representing the hierarchical binding relationships between visual features at every spatial scale across a visual scene, these kinds of binding neurons could underpin visual consciousness itself, the capacity of the visual brain to perceive and make sense of its visuospatial world. Therefore, this work may represent a significant advancement towards the future development of Artificial General Intelligence and Machine Consciousness, opening up new perspectives on building machines endowed with human-level intelligence.