The evolution of complex sensory systems in mammals

Determining how sensory-perceptual systems evolved is a difficult process.

Since little can be determined from the fossil record other than brain sizes and the proportions of major parts, most of the changes that have taken place in the brains of mammals over the last 200 million years must be inferred from the results of detailed and comprehensive studies of the organizations of the brains of extant mammals (Northcutt, 1984). Of course, what is postulated must also be consistent with the fossil record. We know that early mammals had small brains with little neocortex, and that the sizes of brains and the amounts of neocortex increased independently in many lines of descent (e.g. Jenson, 1973). In other lines of descent, however, the sizes of brains and neocortex did not increase markedly. Thus, extant mammals vary in how much the brain has expanded relative to body size from very little to many-fold. In addition, brain structures vary in histological or architectonic differentiation (Kaas, 1987a) from being rather homogeneous internally and quite similar in a number of species to being highly laminated and uniquely organized in others. Such species differences in brain structure and size suggest that many primitive features of brain organization have been preserved and retained to varying extents in living species. This is fortunate, since the evolution of complex from simple systems can be more easily understood in the context of intermediate stages.

Early investigators were limited to noting differences in brain size and histological structure. While many distinctions were obvious, and they led to important and lasting conclusions, large regions of the brain, especially much of the neocortex, were so similar in architectonic appearance that there was little agreement over how they were organized (Lashley & Clark, 1946). However, we now have technical capabilities for reliably determining how brains are organized into parts, how the parts are interconnected, and how connections relate to the response properties of neurons. Microelectrode recording and mapping methods, powerful and reliable methods for determining neural connections, and a range of histological and histochemical procedures have markedly changed the situation (Kaas, 1987a), so that it is possible to postulate in some detail how brain systems are organized in a particular species, rapidly evaluate postulates with a variety of methods, and come to major agreements across groups of investigators. Unfortunately, the brains of only a few species have been studied extensively, and many uncertainties remain. Nevertheless, enough is known to support firmly a number of conclusions.

Traditional architectonic studies demonstrated that the neocortex can be divided into large subdivisions called areas, and that the dorsal thalamus can be divided into equivalent units called nuclei. These subdivisions have been characterized as the ‘organs of the brain’ (Brodmann, 1909) and, as parts of sensory-perceptual systems, areas and nuclei generally ‘represent’, in some systematic way, the receptors or a functional subset of receptors from a peripheral sensory sheet (the retina, skin or cochlear hair cells). Areas and nuclei were further subdivided into layers in these early studies. However, it was not obvious from the early architectonic studies that nuclei and areas are often subdivided into other, small structural units composed of clusters of neurons with common physiological properties (Sur et al. 1981; DeYoe & Van Essen, 1988). In the cortex these subdivisions have commonly been called columns, although they might form bands, blobs or other shapes. Since these subdivisions are thought to consist of arrays of separated groups of cells, with each mediating some sort of calculation, they have been characterized as processing modules (Szentagothai, 1975; Mount-castle, 1978). Such modules appear to exist not only within the cortex but within nuclei as well.

Comparative studies indicate that brains differ in the lamination and the modular differentiation of areas and nuclei, and in the numbers of areas and nuclei. How did these differences come about in evolution?

In mammals such as hedgehogs and opossums, with relatively little forebrain and neocortex, different thalamic nuclei and cortical areas are very similar in structural appearance (Ebner, 1969). In most areas of cortex, cell layers do not differ much in the densities, the sizes and shapes, or the staining characteristics of neurons. Thus, there is not much morphological evidence of cell or laminar specialization. In the thalamus, cell distributions across and within nuclei are relatively similar as well. As a result, it can be difficult to distinguish nuclei, areas and layers in the brains of these mammals. In many mammals with expanded forebrains, however, cortical layers are more distinct, with clear differences in cell densities, cell sizes and staining characteristics, and some cortical areas and thalamic nuclei, mainly those most directly related to sensory inputs, are more differentiated from the surrounding cortex or thalamus. In addition, at least one sensory nucleus, the dorsal lateral geniculate nucleus, commonly demonstrates morphologically distinct layers. The types of differentiation that are found can vary among mammals, and similar types of differentiations can occur independently in different mammals. For example, the segregation of groups of cells activated by one eye or the other in ocular dominance bands in layer IV of the primary visual cortex is found in some but not other primates (Florence et al. 1986) in a pattern that indicates that such segregations have evolved independently in different lines. There are many other examples of such parcellations of groups of cells by physiological response properties (and therefore connections) into columns, bands, layers or modules within nuclei and areas (Kaas, 1982, 1987a). These observations imply that, in early mammals, different cortical areas and nuclei were populated throughout with neurons of rather similar functions, and that nuclei and areas did not differ much in basic functions. From these beginnings, nuclei and areas commonly became more distinct, acquired more cellular differences, and parcellated into groups of cells with different morphological and physiological characteristics. The fact that areas and nuclei can differ greatly in appearance has complicated the recognition of homologous structures across species, and early investigators, relying only on appearance and relative position, made many mistakes (Kaas, 1987b).

Another problem is that changes in the brain, as for other structures, have occurred in a mosaic pattern, so that in the same mammal some areas and nuclei have changed greatly, and others have not. In tarsiers, for example, the primary visual cortex is perhaps more conspicuously laminated than in any other mammal, yet these primates have only moderately expanded forebrains, and other areas of cortex are not notably specialized. Thus, the brains of such mammals as hedgehogs may be primitive with regard to the expansion and general differentiation of the forebrain, and yet have a number of specializations. Distinguishing generalized from specialized brain features can be difficult.

It is not yet certain how the differences within areas and nuclei evolved, but it seems likely that they can be explained largely within the framework of current understandings of how brains develop, and how the course of development can be altered. Many of the cellular changes that have occurred in areas and nuclei over the course of evolution could be the result of alterations in sequences of migration and differentiation of neurons. Thus, more classes of neurons could differentiate, and changes in migration could more tightly confine neuron types to layers. In addition, even small changes in the ways that developing neural systems are exposed to sensory stimuli could alter the functional parcellations of nuclei and areas (Kaas, 1988).

A model for the formation of ocular dominance columns in the visual cortex has been evaluated in experiments where two eyes rather than one are induced to innervate one optic tectum in frogs (Constantine-Paton, 1982). Under these conditions, inputs from the two eyes segregate from each other into a sequence of alternating bands of neurons crossing the tectum much like ocular dominance bands in the cortex. The bands are thought to result from the interaction of the tendency of retinal projections from each of the two eyes to terminate in the same particular locations in the tectum and the tendency of tectal neurons to select inputs from axons with temporally correlated activity patterns, with axons from adjoining ganglion cells in the retina of each eye having the highest correlations. According to this model, differences in the relative strengths of these two factors would produce ocular bands of varying widths and numbers, or even super-imposed inputs from the two eyes and no bands. The same type of model can account for many of the observed variations in internal organizations of nuclei and areas. For example, from the same skin location, slowly adapting (SA) receptor afferents would be more correlated with each other than with rapidly adapting (RA) afferents, and vice versa. Thus, selection for correlations based on receptor transduction properties could result in the parcellation of the ventroposterior nucleus (Kaas & Pons, 1988) and somatosensory cortex (Sur et al. 1981) of monkeys into cell clusters related to SA inputs and those related to RA inputs. As another example, neurons in the lateral geniculate nucleus typically have either ‘on center’ or ‘off center’ receptive fields (they respond to light or dark in the receptive field center). The on-and off-cells are mixed together in the same layers in cats, but separated into different layers in other carnivores such as ferrets (Stryker & Zahs, 1983). This difference in the organization of the lateral geniculate nucleus could relate to something as simple as the developmental stage at which ferrets and cats are born. Ferrets are born earlier in development than cats, and thus the photoreceptors are subject to on-and off-(light and dark) influences at an earlier age. In cats, a segregation at the neuronal level could occur if the early stages of synapse selection were based on correlated activity stemming from retinal location rather than visual stimuli, so that the initially large axon arbors would be reduced to relate effectively to no more than a few adjacent cells. Later, when correlations based on the development of receptor transduction properties becomes a factor, further synapse selection could produce local segregations so that neighboring cells become either on-or off-types. In ferrets, however, where the dorsal geniculate layers are subdivided into layers of on-cells and layers of off-cells, correlations based on receptor transduction properties may be important earlier in development when axon arbors are larger. The process of arbor restriction could initially involve arbor retraction of inputs in the on-class from zones slightly favored by the off-class and vice versa. Thus, separate zones or layers of on-or off geniculate cells could develop. The point here is not to account for the lamination pattern of the lateral geniculate nucleus in carnivores, but to suggest, in a general way, that slight changes in developmental sequences and the interplay of selections for correlated activity and for terminations in specific locations can result in many different possible cell classes in terms of response properties, and in many different types of groupings of such cells. Basic to this idea is the assumption that the central neurons are not intrinsically on-or off-cells (or SA or RA, or right-eye or left-eye cells), but happen to become so. Of course, this does not mean that neurons have the potential to acquire any or all characteristics. For example, the X and Y classes of geniculate cells in cats are morphologically distinct, and the morphological features of Y geniculate cells develop even when they have been experimentally induced to accept inputs from ganglion cells of the X class (Garraghty & Sur, 1988).

Brodmann (1909) and others at that time argued, from studies of cortical architecture, that the number of areas in the neocortex varies across mammalian species from as few as 10-15 fields in more generalized mammals with relatively little neocortex to over 50 areas in humans. Such conclusions have been debated until rather recently because many of the structural features that distinguish subdivisions of the neocortex are unimpressive in standard Nissl-and myelin-stained preparations, and the significance of observed changes was usually uncertain. As a result of these uncertainties, various investigators disagreed on the number of functionally distinct areas in the human brain (from 10 or so to over 100) and on how or whether species differed in numbers of areas. Fortunately, current methods allow subdivisions to be determined with great assurance and accuracy. Microelectrode mapping methods are especially useful for sensory Representations, but considerable progress has also resulted from studies of patterns of connections, using the many new tracing procedures, as well as from the application of a range of histochemical reactions and tissue stains. Of course, subdivisions of the brain are defined with greatest assurance when results from a number of procedures reinforce the same conclusion.

Applications of current methods have confirmed the identity of a few previously obvious architectonic subdivisions of the cortex, defined other areas with great certainty, and provided evidence for the probable or possible existence of a number of other areas in the brains of the few select species of mammals that have been investigated. Although much more research is needed, several major conclusions are already possible.

Perhaps the most important conclusion is that Brodmann (1909) was correct when he surmised that mammals vary in numbers of cortical areas from a few to many. However, Brodmann (1909) was largely wrong in how he subdivided brains and homologized areas across species (Kaas, 1987a,b). A second conclusion is that, whether few or many, most of the subdivisions of the neocortex are sensory-perceptual in nature. This may not be too surprising when one considers that brains, while having many regulatory functions, contain the computational machinery for guiding behavior with reliable inferences about the nature of the outside world. Bigger and more differentiated brains have more machinery for making inferences, and are thus better able to deal with the complexities of the external world. Bigger brains have larger sensory areas, but they also have more sensory areas (Kaas, 1989). In addition, a few of the sensory areas are found in most or all mammals. Again this is not surprising. The first mammals evidently had several sensory and sensorimotor areas of neocortex, and these basic areas have been retained in all or nearly all lines of descent. These areas apparently include the first and second visual areas, V-I and V-II, and the first and second somatosensory areas, S-I and S-II, since these subdivisions have been identified in a wide range of mammals (Kaas, 1980,1983; Krubitzer et al. 1986). Less is known about the auditory cortex, but a primary area, A-I, has been described in a number of eutherian mammals and in metatherian opossums (see Luethke et al. 1988, for a review), suggesting that this field is basic to all mammals. Quite commonly, one or more additional auditory fields have been described, so that it is possible and even likely that most mammals have two or more auditory fields in common. A separate motor or motor-sensory area, M-I, has been defined in many but not all mammalian species (Lende, 1963; Saraiva & Magalhaes-Castro, 1975; Royce et al. 1975; Bohringer & Rowe, 1977; Haight & Neylon, 1979; Ulinski, 1984). At present, it is not clear if M-I evolved early in mammals or premammals and has been retained (but not always discovered) in most extant mammals, has been lost in several lines of descent, or has evolved independently in a number of branches of mammalian phylogeny. Thus, a few basic sensory areas, those with the most direct relays of sensory information from the receptors, are present in all or nearly all mammals. In addition, other cortical regions basic to mammals include the entorhinal cortex, relaying to the hippocampus and amygdala, and the limbic cortex of the medial wall of the cerebral hemisphere. These fields relate to motivation, attention and learning (Mishkin, 1982; Mesulam, 1981) and, together with motor fields and subcortical stations, they allow sensory information to be used effectively in behavior. Thus, these limbic fields can be considered ‘end-stations’ of cortical sensory processing sequences (Mishkin, 1982; Wall, 1988; Kaas, 1989).

Two further conclusions follow from the statements above. Complex systems involving large numbers of sensory areas have evolved independently at least several times. For example, the complex cortical visual systems of cats with at least 12 visual areas (Rosenquist, 1985) and of New World monkeys with 15 or more visual areas (Kaas, 1989) evolved independently from common ancestors with fewer visual areas. Finally, new sensory areas have been added to the middle levels of cortical processing hierarchies. Thus, additional areas have been acquired between the primary and secondary fields, with the most direct inputs from sensory receptors, and the motor and limbic fields that are critical in molding behavior.

The above discussion has concentrated on conclusions about the evolution of species differences in the organization of cortex, but similar conclusions are likely to be valid for the thalamus. For example, the number of visual nuclei or separate representations of the retina in the dorsal thalamus has probably increased independently in the lines leading to present-day cats and New World monkeys (Kaas & Huerta, 1988). However, we currently have a much better understanding of cortical than of thalamic organization in these mammals.

The most common proposal for how the number of cortical areas increases is that a single existing area gradually differentiates into two or more areas (Brodmann, 1909; von Economo, 1929; Lende, 1969; Diamond & Hall, 1969; Kaas, 1982; Ebbesson, 1984). More specifically, overlapping inputs and outputs in a field gradually segregate to produce two or more fields of differing inputs and outputs. We have raised questions about this possibility (Kaas, 1984). Simply put, it would seem difficult for a given orderly sensory representation to differentiate gradually into two adjoining representations, since adjoining representations typically have matched or congruent borders and often form mirror images of each other in topographic order. How could such a gradual change take place without disrupting existing topographic order and function? Thus, we proposed and supported an alternative that, owing to a mutation, development is altered so that an existing area is duplicated (Allman & Kaas, 1971; Kaas, 1982). The duplication releases one or, in part, both areas from the constraints of subserving existing functions, and one or both areas can gradually be modified to subserve new functions. However, the gradual differentiation of one field into two remains a likely alternative, and the two alternatives are not mutually exclusive. Thus, it seems reasonable to consider in more detail how one area could gradually become wo or more.

One possibility is that instead of one area gradually dividing in half (or into some other proportions) to become two, inputs gradually segregate into separate, repeating, but widely distributed clusters, clumps or columns of cells. Examples of such parcellation within areas include the blob and nonblob regions in V-I and the bandlike subdivisions of V-II of monkeys (for reviews see DeYoe & Van Essen, 1988; Livingstone & Hubel, 1988) and the less regular bandlike shapes of SA and RA neuronal groups in S-I (area 3b) of monkeys (Sur et al. 1981). The functionally distinct sets of modules, once formed, by the process of segregation of inputs (and outputs), could gradually coalesce to form adjacent parallel representations. For example, alternating bands of functionally distinct neurons in an area, such as those in V-II of primates, could gradually thicken on the inner border while the other set of alternating bands could thicken on the outer border. As a further step in the process, one set of bands would fuse on the outer border, the other set would merge on the inner border, and a gradual withdrawal of the inserting ends of the bands would ultimately result in two functionally distinct, complete and orderly parallel representations of the visual field. Thus, a sequence of gradual changes could produce two separate maps of visual space while maintaining local order and function at each step (Fig. 1).

Such a process may have occurred in the evolution of separate nuclei in the somatosensory thalamus. In most investigated mammals, the ventroposterior nucleus (VP), which contains an orderly representation of cutaneous receptors, projects to both the first and the second somatosensory areas of the cortex, S-I and S-II (see Krubitzer & Kaas, 1987, for a review). However, in monkeys (Friedman & Murray, 1986; Krubitzer & Kaas, 1986) and at least some carnivores (Herron, 1983), VP projects to S-I and a separate nucleus of smaller neurons, the ventroposterior inferior nucleus (VPI), projects to S-II. Furthermore, in some primates, such as marmosets (Krubitzer & Kaas, 1986), it is clear that narrow fingers of small cells extend from VPI up into VP, and that these small cells also project to S-II. It seems possible that, early in mammalian evolution, VP contained cells projecting to both S-I and S-II, a mixture of cells projecting to one target or the other, or a combination of these types of relay cells. In some lines of evolution, however, a gradual segregation of neuron types may have occurred so that VP contained a mixture of S-I-projecting cells and S-II-projecting cells, each with different inputs from the brainstem and spinal cord. A further stage may have resulted in the formation of small groups of cells of one or the other type, so that two interdigitated maps existed in VP. Ultimately, the groups of neurons projecting to S-II may have coalesced along the ventral margin of VP to form VPI. If so, this apparently happened independently in primate and carnivore fines.

If new sensory areas and nuclei are added to basic systems in evolution, it follows that connections between areas and nuclei change. That is, if new structures are added to the middle levels of processing hierarchies, then connection patterns need to be modified to incorporate the new structures. For example, more direct pathways to motor structures may be replaced by less direct pathways to take advantage of the further processing. However, in complex systems, new connections that bypass steps in a processing sequence may also evolve to speed up processing.

Species differences in the connections of sensory structures have not been extensively studied, largely because only a few structures have been identified as homologous across a wide range of species with any degree of certainty. Furthermore, caution is needed, since apparent species differences may only reflect technical complications and the sensitivities of various anatomical procedures. Nevertheless, there is some evidence for modified connections. For example, the generalized ventroposterior nucleus (VP) projects to both S-I and S-II, but in monkeys a separate nucleus, the ventroposterior inferior nucleus, provides the thalamic input to S-II. Interestingly, the projections from VPI to S-II are no longer capable of activating neurons in S-II, since S-II is deactivated after removing S-I (Pons et al. 1987; Garraghty et al. 1988). Thus, S-II appears to become less influenced by early thalamic stages of processing and more dependent on cortical stages of processing in higher primates.

The VP connections appear to have changed in another way in the evolution of higher primates. In addition to S-I (area 3b), monkeys have representations of cutaneous receptors in area 1 and in area 2 (see Kaas 1983; Kaas & Pons, 1988, for reviews). A dual projection pattern evolved in primates so that VP projects in a systematic pattern to the area 3b representation and in a parallel manner to the area 1 representation in monkeys. In addition, VP projects to at least the representation of the hand in area 2 of Old World macaque monkeys (Pons & Kaas, 1986). Since area 3b provides feedforward (activating) projections to area 1, and area 1 feeds forward to area 2, VP provides parallel inputs to three successive levels of a cortical processing hierarchy in monkeys.

As another example of phylogenetic changes in connections, some rodents such as rats (Miller & Vogt, 1984) have direct projections from the primary visual cortex, V-I, to motor or premotor fields in the frontal lobe and limbic cortex of the medial wall. In contrast, V-I projects only to nearby visual areas in primates. This difference supports our view that mammals with few cortical processing stations (Kaas et al. 1989) have more direct pathways to motor, memory and motivational centers.

The mechanisms for changing connection patterns in the brain are only partly understood. Research on development has led to an emphasis on what has been called regressive events (Cowan et al. 1984), the elimination of connections through neuron death and the retraction or loss of neuron branches. Certainly these factors are susceptible to manipulation in evolution to create new patterns of connections (Ebbesson, 1984). However, it is also important to consider the evidence, some mentioned above, that, in the course of evolution, axons became altered in growth so that they come to innervate or ‘invade’ new targets (Fritzsch, 1984). Observations that visual projections can be experimentally induced to form in the auditory thalamus (Sur et al. 1989) provide further support for the view that, over the course of evolution, development can be altered in ways that produce new connections.

I thank P. Garraghty, S. Florence, L. Krubitzer and A. Morel for comments on the manuscript.