Binocular Spatial Localization in the Praying Mantis

Many insects possess binocular vision, that is to say the fields of view of the two compound eyes overlap by a substantial amount (e.g. Burkhardt, Darnhofer-Demar & Fischer, 1973 ; Horridge, 1978). This raises the question of the extent to which the two eyes interact to coordinate monocular information and to provide a means for depth discrimination based on disparity. In particular the latter aspect of binocular vision has been a topic of considerable interest, yet it has remained controversial in insect physiology. It has often been supposed that binocular vision is important for distance estimation, because one-eyed raptors do not strike their prey (e.g. Baldus, 1926; Friederichs, 1931; Maldonado & Barrós-Pita, 1970; Cloarec, 1978). But doubts have been raised because connections between the two eyes were assumed to be insufficient for mediating extensive binocular interactions (e.g. Horridge, 1977, 1978; Wehner, 1981). The main obstacle to understanding the role of binocularity in insects is that only a few possess appropriate behavioural responses and are amenable to the experimental techniques for critical tests. One of these insects is the praying mantis.

Mantids are ambush predators hanging on vegetation in wait for prey. When a target appears in the field of view, the animal rapidly turns its head for a closer binocular inspection with its forward looking foveas (Mittelstaedt, 1957; Levin & Maldonado, 1970; Lea & Mueller, 1977; Rossel, 1979, 1980). Prey which moves within the appropriate distance is captured by a rapid promotion of the forelegs, known as the strike (Roeder, 1960). Thus visual prey localization according to direction and distance is expressed by two distinct behavioural responses, the fixational head movements and the strike. Both responses are performed by a stationary animal, which makes them easily accessible to observations and experiments.

This paper describes how in mantids the two eyes collaborate during the localization of prey. It is shown that the visual system coordinates monocular directional information, such that the direction a mantis fixates is intermediate between the image directions in the two eyes. This suggests that mantids possess some sort of binocular single vision - at least with respect to the direction of objects in space. Also it is shown that mantids strike short when viewing prey through base-out prisms, indicating that depth is perceived by means of binocular disparity.

Some of the results reported here have already been published in a preliminary form (Rossel, 1983a,b).

Observations and experiments were performed on the Australian mantid, Tenodera australasiae, and the European species, Mantis religiosa. The animals were kept individually in Plexiglas cages until use. They were fed on flies and cricket nymphs every second day, but were starved for at least 3 days prior to an experiment. Only mantids which then responded vigorously to prey were tested.

The prothorax of a mantis selected for an experiment was firmly attached to a holder, such that the animal could hang upside down from a movable cardboard disc (Mittelstaedt, 1957 ; Rossel, 1983a). The head and the legs were thus free to move. The insect was then centred in a circular arena which consisted of a white cardboard cylinder (800mm in diameter and 400mm high). The arena was located close to the large laboratory window. Daylight usually provided sufficient illumination, but on very overcast days light from incandescent lamps was also used. Records of the mantid’s behaviour, and the position of prey, were obtained using a National video camera and a National video recorder adapted for single frame analysis.

Visual fixation was analysed using the fruitfly, Drosophila, as a target. The fly was glued to the tip of a glass pipette which could be moved within the mantid’s horizontal field of view by means of a micromanipulator. A shutter consisting of a piece of white cardboard was interposed between the mantid’s head and the fly, in order to control when and where the target would be seen. Target distance varied from between 12 and 14mm, so that the target subtended a visual angle of about 10° on the mantis retina. This image size is optimal for eliciting saccadic head movements (Maldonado & Rodriguez, 1972; Rossel, 1980). Targets were presented no further than 30° to the body axis in order to avoid the fixation errors that occur with more peripheral targets (see Rossel, 1980). Successive target presentations were arranged, such that in each set of experiments equal numbers of fixational head movements were directed towards and away from the body axis.

Records were taken from below with the camera looking through a hole in the floor of the test arena. The small, sharp pseudopupils in the dorsal eye region of the mantid were then clearly visible on the television monitor and were used to determine the mantid’s direction of gaze. Mantids were temporarily deprived of binocular vision by painting the left or the right eye black, leaving only the dorsal eye region (pseudopupil) open. The paint could easily be removed from the eye surface, so that individuals could be tested monocularly and binocularly.

In some experiments the target was periodically obscured by the cardboard shutter which was interposed between the mantis head and the fly, so that the latter was visible for 1 s and disappeared for another. The mantids then often turned their head when the target had disappeared, so that the open-loop characteristics of the saccadic head movements could be examined.

The blowfly, Sarcophaga, was used to release strikes. The flies, glued vertically to a metal pin, were moved towards the mantis by means of an X/Y recorder (Rossel, 1983a). Jerky sideways movements (maximum amplitude: 3mm) were superimposed on the forward motion in order to attract the mantid’s attention. The speed of the target was varied with its distance to keep retinal image velocity, as produced by the forward motion, approximately constant (∼ 0·2° s^-1). After a strike had been released the target was returned to its starting point, approximately 200 mm from the mantis head.

The prisms used in these experiments were fixed on a separate holder and placed exactly 2 mm in front of the mantid’s eyes. Video recordings of both the mantis strike and the target were taken from above. A detailed figure of the experimental set-up is given in Rossel (1983a).

Observations of the pseudopupils show that the eyes of Tenodera and Mantis are very similar. In both species, the large compound eyes are almost completely panoramic. The binocular field has a maximum overlap of about 70° in the frontal parts of the eyes. There is a distinct fovea in the forward looking eye region with interommatidial angles as small as 0·7°, compared with more than 2° in the peripheral eye (Fig. 1A). The horizontal separation between the two foveas (interocular distance) varies from one individual to another and values ranging between 3·6 mm and 5·3 mm have been measured under the microscope (for a full account of the optical performance of the eyes of Tenodera seeRossel, 1979).

Fig. 1B presents a map of lines of sight in the binocular field of view and defines the parameters which are of potential importance for visual spatial localization. Each line of sight is separated by 10° from its neighbour, and has been calculated from the interommatidial angles and the facet diameters given in Fig. 1A. Notice that when the lines of sight are extrapolated back into the eyes, they do not meet at a single point. This is because the interommatidial angles and the facet diameters vary across the eye. For each eye the direction of an object in space is defined in terms of the angle between the line of sight of the forward looking fovea and that through the centre of the object. These angles are denoted by α_L and α_Rfor the left and the right eye, respectively. In terms of binocular interactions, two quantitative measures are of interest: the ‘binocular eccentricity’ of the target, defined as the mean between α_R and[(α_L + α_R)/2], and the binocular disparity, given by the difference between at and α_R (α_L – α_R).

In practice α_L and α_R were determined in two different ways, depending on the distance of the target from the mantis head. For relatively large distances (the catching distance and beyond) the varying origin of at and OR within the eyes can be neglected and the two reginal position angles can be estimated rather precisely by applying simple trigonometry to the triangle given by the interocular distance (b), the target distance (d), and the angular deviation of the target from the midline of the head (β) (see Fig. 1B). At very close range, however, the point of origin of α_L and α_R within the eyes is critical. In this case, therefore, the triangle given by b, d and β was transposed to a map with a plot of lines of sight, each separated by 1° from its neighbour, and α_L and α_R were determined graphically from the map.

Binocular disparity was manipulated by base-out prisms placed in front of the eyes (Fig. 2). The prisms deviate the line of sight in such a way that disparity is increased. Thus for an insect which uses binocular disparity for distance estimation the target appears closer than it really is. The deviation of the line of sight on the prism faces is given by the formula :

The way in which visual directions are discriminated by the praying mantis can be deduced from its fixation behaviour. As in other animals, saccadic fixational movements in mantids are pre-programmed events, specified exclusively by the direction of the target before the turn (Mittelstaedt, 1957; Lea & Mueller, 1977; Rossel, 1980). This implies that perceived visual directions are expressed by saccade amplitudes. Thus to investigate whether and how the two eyes collaborate in determining visual directions, saccades were analysed under monocular and binocular conditions.

To study monocularly driven saccades one eye was painted over and a fly was shown in different horizontal directions relative to the forward looking fovea of the seeing eye (see Materials and Methods). The amplitudes of such saccades imply that they are made to place the target on the fovea, i.e. the saccade amplitudes are closely correlated with the angle made by the line of sight of the fovea with the line of sight through the target (Fig. 3). This holds true whether or not the target is visible during the execution of the saccade, which confirms the open-loop characteristics of this behavioural response. Note, however, that foveal fixation is not precise. In fact, the saccades tend to undershoot the target, particularly when the latter projects onto the median region of the eye. This effect does not depend on the direction of the saccade (towards or away from the midline of the body). One possible reason for the fixation errors is the inhomogeneous sampling of the target in the subfoveal region of the eye. Here there is a considerable variation of the interommatidial angles, resulting in a progressive increase of the sampling density towards the fovea (see Fig. 1). This effect is most distinct in the median region of the eye where the largest fixation errors occur. One might suppose, therefore, that the perceived visual direction of the target is shifted towards the region of increased sampling density, causing the visual system to programme a smaller saccade amplitude than predicted.

Basically the same results as in Fig. 3 were obtained when the target was shown in the vertical rather than in the horizontal field of view (Rossel, 1980, unpublished data). Thus it can be concluded that monocular mantids perceive the direction of objects in space relative to a single region of fixation which is associated with the forward looking fovea.

Based on this finding a critical test can be made regarding the binocular coordination of directional information; by presenting targets at very close range, so that they are seen from markedly different directions by the left and the right eye, and then testing what visual direction is expressed by the saccade amplitude. The open-loop saccadic head movements place the target neither on the left nor on the right fovea but on some point in between (Fig. 4). The scatter of the data is small, compared with the large disparities involved, and does not exceed that of monocular mantids.

While these findings demonstrate that binocular interactions occur, a closer inspection of the data reveals that the visual system does not simply calculate the mean between the two monocular position angles of the target images. It seems as though the smaller position angle (in these experiments α_R) is weighted more heavily than the larger one, because the saccades fall consistently short of the mean. This discrepancy can partly be explained by the fact that, even in the monocular case, the saccades are slightly smaller than predicted (Fig. 3). A better prediction of binocular fixation is thus achieved when α_L– α_R are expressed by the saccade amplitudes, as performed by monocular mantids. The remaining discrepancy should probably be attributed to some dominance of that eye which views the target closer to the fovea.

If mantids were to make use of binocular disparity for distance estimation, then they should strike at the apparent binocular image when they view prey through base-out prisms (Fig. 2). This is indeed the case as documented for one individual in Fig. 5 and for four others in Table 1. In quantitative terms the effect is correlated with the prism strength, such that strikes are released when the disparity of the retinal images of the approaching prey reaches some fixed value. None of the monocular depth cues potentially available to the praying mantis could account for this observation. In fact, image size as well as motion parallax are defined by the real prey and have not been altered by the prisms. Thus it can be concluded that mantids use binocular disparity as a source of distance information.

Image size is important in these experiments, as being one parameter determining the attractiveness of prey (Rilling, Mittelstaedt & Roeder, 1959). In the control situation in Fig. 5A, i.e. when no prisms were used, the smallest disparity to elicit a strike was 9·8° when the target subtended a visual angle of about 18° and was highly attractive. However, with the prisms used in Fig. 5C, the prey at that disparity would be about 3°. Prey of such small size are readily detected and fixated by adult mantids, but strikes rarely occur. This implies that the smallest disparities in the sample of Fig. 5A do not show up in the one of Fig. 5C. As a consequence, the mean disparity to elicit a strike slightly increases with increasing prism strength, as can be seen from the data presented.

Mantids which are free to move their heads usually fixate the prey before catching it. In the experiments described above fixation could be performed with precision because the prey slowly approached the mantid keeping close to the animal’s longitudinal axis. In this case, therefore, binocular estimation of prey distance was accomplished with the target restricted to a small region of the eye close to the fovea. The next experiments, therefore, were designed to test whether disparity measurements can also be made when the target is viewed by the extra foveal region of the eyes. For this purpose the heads of a number of mantids were firmly glued to their thoraxes, thus preventing the fixation of prey deliberately presented in eccentric positions. The frequency of striking then decreased with increasing eccentricity of the prey and no strikes were released when the eccentricity exceeded an angle of 20°. However, within this range of eccentricity, strikes occurred at the same binocular disparity. Tests with base-out prisms confirmed this finding (Table 2). The mantid’s strike undershot the prey but the disparity remained constant over the whole range of eccentricities tested. Thus it can be concluded that binocular distance estimation is not restricted to the fovea, but operates over much of the binocular field of view.

The final point of interest concerns the range of distances (or disparities) over which stereopsis operates. The experiments so far demonstrate the use of binocular disparity only over the narrow range corresponding to the catching distance.

However, there is a sequence of actions related to prey capture and each component is associated with a particular range of distances. Prey at about 100 mm and beyond usually evokes no more than fixational head movements. When the prey distance is less than 100 mm, the mantis brings its forelegs into the typical striking posture. When the prey is almost within the reach of the strike, the forelegs often twitch a little, indicating somehow the tension which precedes the prey capture. Strikes occur at prey lying within a range of about 30 mm to 15 mm. At closer distances the mantid no longer attacks. Instead it displays a series of typical defensive strikes with the intention of pushing the object away. All these behavioural responses are also performed at the appropriate disparities when prey are viewed through prisms. It seems, therefore, that disparity provides distance information over a wider range than is expressed by the strike behaviour.

The data in this study demonstrate that visual spatial localization relies on inputs from both eyes. On the basis of this evidence several points may now be made regarding the characteristics of binocular vision in mantids.

Binocular vision provides the praying mantis with a powerful means of distance estimation. This is presumably the only function of binocularity in this insect. In fact important visual tasks such as the detection, fixation and identification of biologically relevant objects are readily performed by monocular mantids, and it is only the estimation of range which requires both eyes (Maldonado & Barrós-Pita, 1970; Maldonado, Benko & Isem, 1970; Maldonado & Rodriguez, 1972; Rossel, 1980).

The gain of binocular vision as provided by stereopsis is accompanied by the need to coordinate information between the two eyes, so that a unified control of visual behaviour can be achieved. This could be done either by combining the two monocular inputs or by inhibiting either of them. In the case of visual fixation there is evidence demonstrating that directional information from the two eyes is combined such that the direction the mantid fixates is intermediate between the image directions in the two eyes. Obviously, an inhibition mechanism of binocular coordination could not account for these results. According to such a mechanism only one image direction would be effective at a time, and, as a consequence, would be associated either with the left or the right fovea as shown in monocular fixation.

Apart from the fact that the two eyes collaborate with each other, it is most remarkable that they do this over most of the binocular field of view. Obviously, this is important for the praying mantis because its major visual task is the localization of highly mobile prey against a structured background. As shown previously (Rossel, 1980), mantids then track saccadically, where the saccade amplitudes usually exceed an angle of 10°. This means that after a saccade the target will soon be lost to the fovea until the next saccade occurs. Thus the limitation of binocular interactions to a narrow foveal region of the eyes would be a severe disadvantage, because then the location of mobile prey could not be continuously monitored by the insect.

More difficult to assess is the range of distance over which disparity provides a depth cue for the praying mantis, because the strike, as the most informative indicator, only relates to the catching distance. However, as mentioned in the Results, mantids display additional behavioural responses which suggest that stereopsis is not restricted to the catching distance. In this context one series of experiments is of special interest (Maldonado et al. 1970). When confronted with a potential predator such as a bird, mantids often display a typical defensive posture in order to avoid attack. Maldonado exploited this behavioural response to study the role of binocular vision in estimating the range of far distant objects. The birds used in these experiments differed in size and were presented at distances to a maximum of 750 mm. It then turned out that the frequency with which the defensive posture was released gradually decreased with increasing distance in both monocular and binocular mantids. However, while monocular mantids confused image size and distance, the binocular mantids responded almost identically to different-sized birds over the whole range of target distances tested. Although the experiments provide no direct evidence that retinal disparity was involved in assessing the distance of the birds, it is difficult to think of an alternative binocular depth cue which could explain the performance of the binocular mantids. Thus with the evidence at hand, it seems possible that retinal disparity provides distance information well beyond the range which is usually important for prey capture.

Nevertheless, there are a number of limitations which must be taken into account. Geometry tells us that the sensitivity of retinal disparity as a depth cue will decline with increasing object distance (e.g. Burkhardt et al. 1973). For instance, with an interocular separation of 4 mm the distance interval corresponding to a given disparity interval will grow by a factor of 20 when the object distance is increased from 25 mm to 750 mm. Such geometrical constraints are inescapable, even though the range and the accuracy of binocular depth vision will ultimately depend on the precision with which disparity can be measured. The interommatidial angle is an important factor in this respect (e.g. Burkhardt et al. 1973), but this parameter is not necessarily a direct index of the insect’s acuity. The point is that objects of interest such as prey and enemy are considerably larger than the interommatidial angles, so that many visual units are potentially involved in the sampling of positional information. This holds true even for far distant objects with a small angular subtense, because this is the very range where the sampling density of the forward looking foveas is most effective (Rossel, 1979). It seems possible, therefore, that the precision of positional judgements is limited by the capacity of the visual system to integrate local position information from extensive retinal images, thus providing the insect with some form of hyperacuity, as is realized in higher vertebrates (e.g. Westheimer, 1979). At present, there is no evidence in support of this possibility. The present data suggest only that disparity measurements are performed with the same precision over most of the binocular field of view, in spite of the different interommatidial angles involved in the acuity task (compare scatter of data in Table 2 with variation of interommatidal angles in Fig. 1A). However, what is needed is an estimate of the absolute acuity with which distance can be measured by the insect. Possibly the appropriate information could be obtained by analysing the characteristics of the strike when prey is presented at slightly different distances. In fact, Corrette (1980) has already shown that the temporal and spatial characteristics of the strike do vary with the target distance, but further work is needed to quantify this.

Binocular spatial localizations as performed by the praying mantis requires neuronal interactions between the two eyes. By analogy with binocular neuronal integration in higher vertebrates (e.g. Bishop, 1973), this could be achieved within a binocular retinotopic map, such that adjacent points in physical space are in some way represented by groups of neighbouring cells within the map. Given the large range of eccentricities and disparities over which binocular interactions occur in the praying mantis, this method of representing binocular visual space would certainly demand an enormous amount of neuronal wiring (e.g. Wehner, 1981; Collett & Harkness, 1982). At present, with our limited knowledge of the anatomy and physiology of the mantis brain, we do not know whether such complex patterns of retinal projections exist. Doubts may arise from anatomical findings in other insects, mostly flies, where it has been shown that only a relatively small number of neurones connect the two optic lobes (e.g. Pichka, 1976; Strausfeld & Nässel, 1981). Thus it seems worthwhile speculating whether there may be simpler ways of combining information from the two retinal images.

Our binocular machinery enables us to obtain a detailed three-dimensional ‘picture’ of the outside world. In contrast, the mantid seems mainly interested in the recognition and localization of single objects such as prey, predator and mate, where the recognition of the latter is supported by chemical sex attractants (Robinson & Robinson, 1979). This is hardly surprising since the sexual activity of mantids usually occurs in dim light between dusk and dawn. Among the visual cues involved in the recognition of prey (and predator), movement and size are most decisive. Shape is also important, but only in combination with movement (e.g. moving legs and wings increase the attractivity of prey) (Rilling et al. 1959). Based on this evidence it has been suggested that the mantid’s visual system is sensitive to moving objects as a whole and does not bother with the spatial details of the image (Horridge & Duelli, 1979). If true, this would have severe repercussions for the ‘correspondence’ - and ‘matching’ - problem of binocular vision (e.g. Poggio & Poggio, 1984). One of the major problems in mammalian binocular vision is concerned with how the spatial details of the images of the two eyes are matched with each other. This problem is circumvented in mantids if there is no point-by-point matching of the two retinal images of a single object in space, and a simple hypothesis can be offered to account for binocular spatial localization.

The first step is to suppose that from the detailed retinal image each optic lobe selects the few key features which characterize the potential object of interest. High impulse rate in corresponding feature detectors then means that the trigger feature is present in the outside world (e.g. an object the typical size of prey which performs jerky movements). Up to this level monocular data processing is expected to occur in parallel pathways which preserve the spatial organization of the retina. The next step is to suppose that the mean position of active feature detectors is coded in the form of x/y coordinates whose origin is at the fovea^*. To accomplish this x and y positiondetecting neurones are needed whose impulse rate varies in proportion to the angular

Notice that once the position coding system has intervened there is no further need to preserve the spatial organization of the retina. Thus each class of feature detectors from both optic lobes could be fed onto a single collecting neurone which carries the information to the brain.

One of the characteristics of the model is that it allows only one object to be perceived at a time. From a biological point of view, this is hardly a disadvantage, because visually controlled behaviour such as prey capture and predator avoidance is usually directed towards a single object. However, the question is whether each eye can safely assume that it is dealing with the same object should there be several available within the field of view. In this case the model assumes that each optic lobe selects the most attractive object (i.e. the one which produces the highest impulse rate in appropriate feature detectors) and suppresses the visual processing of all others. Since the most attractive object is selected on the basis of the same ‘attention’ values in either eye, no confusion should arise in this respect.

In summary, the aim of the model has been to explain the phenomena of binocular vision encountered in the praying mantis with the fewest possible neuronal connections. Simplicity, of course, is at the expense of information processing capacity. Thus, as it stands, the model cannot deal with binocular pattern matching nor can it explain the simultaneous recognition and localization of several objects. These limitations, however, may be justified, because the visual system of the praying mantis is not necessarily trying to achieve a complete representation of the outside world, but rather to respond to a few features of objects which are biologically relevant. Thus if it is important to limit the number of neuronal connections between the optic lobes, without sacrificing the range and the precision of binocular spatial localization, this is the very place where economy measures can best be applied. Nevertheless, at present the model is nothing more than speculation, and it is readily admitted that it may over-simplify the issue. However, its main purpose is to show that the simple visual recognition tasks performed by the praying mantis may allow equally simple mechanisms of binocular interactions. This is worthwhile because previous thinking on the issue was heavily influenced by the complex neuronal principles realized in higher vertebrates, and, as a consequence, it has repeatedly been concluded that binocular spatial vision might be too complex a visual task to be performed by the tiny brain of an insect. The present study, however, has shown that this is not the case.

I would like to thank Drs T. S. Collett and M. V. Srinivasan for their critical comments on the manuscript. The work was supported by Swiss NSF grant No. 3.073 · 0.81 to Professor R. Wehner.