Template-matching describes visual pattern-recognition tasks in the weakly electric fish Gnathonemus petersii

It is difficult to identify a visual object or an environmental landmark from any possible viewpoint since its retinal appearance may vary drastically with the chosen viewpoint. Primates and advanced technical systems therefore process the retinal image to extract invariants that remain unchanged with changes in viewpoint (e.g. Ullman, 1996). Several insects, however, can identify landmarks or objects using simple template-matching systems that are not viewpoint-invariant (Wehner, 1972; Cartwright and Collett, 1983; Gould, 1985; Dill et al., 1993; Zeil, 1993; Dill and Heisenberg, 1995; Collett, 1995; Ronacher and Duft, 1996). For instance, in learning the visual surroundings of a nest or a feeding place, bees and wasps appear to store views taken at a limited number of viewpoints on a defined path. In their later returns to the nest or feeding place, they follow the same path and are thus able to match, at the corresponding viewpoints, the actual and the stored retinal appearance of each landmark. This does not allow the insects to recognize the landmarks from arbitrary positions, but works well as long as the set path is kept to and stored templates are retrieved from the same points. Template-matching has been viewed as one of the most basic and probably oldest forms of pattern vision (e.g. Dill and Heisenberg, 1995; Heisenberg, 1995), and we wondered whether it would also be employed, under specific conditions, by vertebrates.

In this study, we report evidence that a vertebrate, the weakly electric fish Gnathonemus petersii, uses template-matching to recognize visual patterns that it had previously viewed from a fixed vantage point. This fish is nocturnal, lives in small streams in Western Africa (Niger to Zaire basins) (Boulenger, 1909) and uses active electrolocation to find its way about in the dark. By discharging its electric organ, it sends probing currents through its skin and measures, using electroreceptors, the object-induced distortions in both the spatial distribution and the time course of its self-generated probing current to derive an image of its environment (see Heiligenberg, 1977; Moller, 1995; von der Emde, 1999). Its vision is, however, likely to contribute to its foraging success (von der Emde and Bleckmann, 1998) and is important during social interactions (Moller et al., 1982). In particular, the distinct vertical double-banding of Gnathonemus petersii is likely to play a role as a visual signal in social communication.

Gnathonemus petersii possesses a fovealess bundle-type retina (McEwan, 1938) in which the elongated rods and cones are grouped in bundles ensheathed in pigment cells (McEwan, 1938; Ciali et al., 1997). No melanin appears to migrate between the photoreceptors even in fish that have been kept under bright illumination, and the large quantity of guanine in the pigment cells reflects unabsorbed photons back to the photoreceptor bundle (McEwan, 1938). The fish appears to be well adapted to vision in dim light, and we expected a low visual acuity. Important visual brain areas, such as the visual area of the diencephalon, an important processing and relay station, are greatly reduced in Gnathonemus petersii compared with non-electrosensory teleosts (Lazar et al., 1984; Northcutt and Wullimann, 1988; Wullimann and Northcutt, 1990). With this in mind, we hypothesized that Gnathonemus petersii might employ simple mechanisms of visual pattern recognition.

Three juvenile Gnathonemus petersii (Gthr.) of standard length 7–8 cm were kept individually in tanks (60 cm×30 cm×30 cm) in which all training and testing was performed. The fish were fed on red bloodworms. Water conductivity was 300 μS cm^–1, pH was 6–7 and temperature was 25°C. Experiments were performed in a lightproof room with artificial light only. A 12 h:12 h L:D cycle was maintained, and training and testing were performed during the light phase, at least 2 h after light onset. Daytime light intensity was, unless stated otherwise, 9.6 lx at the water surface (measured above the front end of the shelter, see below and Fig. 1).

A partition (grey polyvinylchloride, PVC) divided each tank into a front and back section (Fig. 1). A rectangular opening (4.5 cm×4.5 cm) in the centre of the partition extended into the back section with a shelter (PVC tube, cross-section 4.5 cm×4.5 cm, length 15 cm) in which the fish spent most of the day. The front of the shelter could be opened and closed by sliding a transparent Plexiglas screen through which the fish could view the patterns that were fixed in a pattern holder on the tank’s front screen outside the water (distance from the Plexiglas screen 30 cm in two tanks and 20 cm in a third tank). The visual patterns were printed black-on-white paper cards, and the cards were laminated. The centres of the patterns were both at equal height with the shelter’s centre and displaced by 6.5 cm to the left and right of it. A PVC partition, fixed inside the tank at the front window in the midline of the patterns, forced the fish to move towards only one of the two patterns and prevented it from viewing both patterns from much closer viewpoints. Feeders were placed on the left and right sides of the front window, 10 cm above the floor.

Patterns were presented and feeders placed in the tank when the fish was in its shelter, behind the Plexiglas screen, and faced the front window. The screen was then lifted, and the fish swam towards one side. Touching the feeder on this side was scored as a decision for the pattern presented on that side. During training, the feeder on the side with the rewarded pattern always contained a red bloodworm; the feeder on the side with the unrewarded pattern remained empty. During testing, both feeders were empty. During both training and testing, the pattern holder was quickly removed after a decision had been scored.

The side on which the rewarded pattern was shown varied randomly. The level of correct discrimination of the training patterns in unrewarded presentations was independent of whether the rewarded pattern was shown on the right or left side of the fish. Thus, the fish did not appear to learn the discrimination using predominantly one eye. For instance, in the tests in Fig. 6 (top row ‘Testing’), fish B (fish C) chose the disk in 40 out of 50 (36 out of 48) presentations in which it was on the fish’s right side and in 42 out of 50 (38 out of 52) presentations in which it appeared on the left side of the fish.

Unrewarded tests could not be conducted in series, one after the other, but had to be interspersed with a large number of rewarded presentations of the training patterns so that no two tests immediately followed each other. Generally, in a series of 10 presentations, only three or four were unrewarded tests, and the others were rewarded presentations of the training patterns. This necessity for continuous rewarded training considerably slowed down progress during the actual testing phase of the present study. It was, however, generally maintained since it was necessary to prevent the fish from losing interest in the visual patterns and to maintain a high performance level. Transfer tests, in which the patterns shown differed from the training patterns, were only started after a level of at least 70 % correct choices had been reached. In the third year of the present study, fish A and C died of an unknown disease: fish A died during the training stage of the experiments shown in Fig. 6 (but after finishing the experiments in Fig. 8); fish C died 1 month after completion of the experiments shown in Fig. 6.

Throughout this paper, statements of significance levels (e.g. P<0.05) without further details refer to the result of χ²-tests to decide whether or not the fish’s response was random.

To establish whether Gnathonemus petersii can be trained to use visual cues, all three fish were first trained to discriminate a black disk (rewarded; diameter 11.4° for fish A and B and 17.1° for fish C) from a white card (unrewarded). Initially, this training was attempted at a daytime light level of 990 lx at which the fish had previously been kept and at which fish B had been successfully trained in an electrosensory task (discrimination of a plastic from a metal rod). After 2 months, all fish had learned to leave their shelter when the screen was lifted, but fish A and B still randomly approached one of the patterns. Only fish C, trained to the larger disk, learned the task, but it reached an apparently stable level of only 59 % correct choices (400 tests, P<0.001).

After this failure, the daytime light intensity was lowered to only 43 lx. After 1 month of training, unrewarded tests were interspersed with the normal training sessions (see Materials and methods) to assess the learning success. Again, only fish C mastered the task (77 % correct choices in 490 tests; P≪0.001) while the other two fish had still not learned it (fish A, 52 % correct choices in 300 tests; fish B, 50 % correct choices in 170 tests). Therefore, the daytime light level was lowered further to only 9.6 lx and the training continued at that level. After 2 months of training at 9.6 lx, fish A and B had reached stable levels of 71.6 % (950 tests; fish A) and 69.7 % (770 tests; fish B) correct choices, both significantly (P≪0.001) different from random choice.

After the pretraining phase had shown that the fish could be trained to use visual cues and had indicated that working in dim light conditions might be crucial, the actual training phase on visual pattern discrimination could be started at 9.6 lx. For all three fish, the black disk was kept as the rewarded pattern but the unrewarded white card was exchanged for another pattern that differed for each fish. These patterns, a large open disk, a small filled square and a large triangle, are illustrated in Fig. 3 (top row). After approximately 2 months, all three fish were successfully trained to their respective discrimination tasks. In the unrewarded tests, interspersed among rewarded presentations (see Materials and methods), fish reached levels of 86.6 % (780 tests, fish A), 75.9 % (660 tests, fish B) and 85 % (100 tests, fish C) correct choices (P≪0.001 for each fish).

The difficulties in the initial pretraining at high light intensities suggested that the fish might be able to discriminate visual patterns only under dim light conditions. The experiments of Fig. 2A were made to test this hypothesis. After the three fish had successfully performed in their respective discrimination tasks at the training light intensity (9.6 lx; marked with a grey bar), they were tested at other light levels. Each new daytime light level was kept for several days until testing had been finished. Daytime light intensities were tested in the order 9.6 lx, 43 lx, 990 lx, 26 lx and 1 lx. No tests were performed on the first day after a change to a new daytime intensity; thereafter, testing was performed every day for approximately 1 week. Our results indicate that the fish were able to discriminate their training patterns only at low ambient light intensities (Fig. 2A). At light intensities higher than 9.6 lx, their performance deteriorated significantly (e.g. different performance at 9.6 lx and at 43 lx, P<0.01 in fish B, P<0.05 in fish C; χ²-tests), whereas it was unaffected at still lower intensity down to 1 lx. At 990 lx, the choices of all three fish did not differ significantly from random choice. All three fish chose regularly even on the first day of testing after a change to a higher daytime light intensity, so the deterioration in performance at higher light intensity was not due to increased shyness at the higher light intensities. Also note that the unrewarded tests were always interspersed with rewarded presentations of the training patterns (see Materials and methods) to maintain the continuous interest of the fish.

To analyze the effects of pattern size, experiments were performed with fish B and C at the standard daytime light intensity 9.6 lx (Fig. 2B). After each fish had mastered the discrimination of two patterns, its performance was tested when these patterns were shown at reduced sizes. Again, these tests were interspersed with rewarded presentations of the training patterns at their original sizes. Performance deteriorated strongly when the patterns were shown at reduced size. For the smallest sizes tested, all choice levels no longer differed from random choice, indicating that pattern discrimination requires patterns subtending a visual angle larger than approximately 3°.

All three fish could successfully be trained to discriminate the visual patterns shown in Fig. 3 (top row). To determine the basis on which the fish had performed this discrimination, a series of transfer tests was performed. In these, the fish were tested with patterns other than those to which they had been trained. The patterns used are shown in Fig. 3 below the respective training patterns. Interestingly, the preferences observed in these tests can be explained in a simple way by a template-matching mechanism. In such a mechanism, the fish would have stored a snapshot of the training pattern(s) and compared it with the actual image of the patterns that it viewed from its shelter. Fig. 4 illustrates the mechanism. We assume that the fish is able to align the stored image (a disk) with the actual image (an upright triangle) so that their centres of mass coincide. The fish could then quantify the degree of the matching between the actual and the stored image by taking into account the matching area L and/or the non-matching areas T, M, where L is the amount of overlap between template and actual image, M is the total area of the actual image that remains unmatched and T is the total area of the template that remains unmatched

Table 1 presents a detailed comparison of the findings of Fig. 3 with predictions of various template-matching mechanisms. The mechanisms that were considered differ in whether a template of the rewarded (‘Template +’) or of the unrewarded (‘Template –’) figure is stored and, for each of these two possibilities, the way in which the quality of matching is assessed. Three possibilities were considered for the latter: (i) the actual overlap L; (ii) the total non-overlapping area T+M; and (iii) the relative overlap L/(L+T+M) (see Fig. 4 for a graphical illustration of the quantities L, T and M). The pattern pairs shown in the transfer tests of Fig. 3 are illustrated in the first column. In each pair, the preferred pattern is depicted at the top. The lines separate the tests made with different fish (top, fish A; bottom, fish C; cf. Fig. 3). For each of the patterns, the three quantities (i–iii) are given as appropriate for a template of the pattern that was rewarded or unrewarded in training (see Fig. 3 for the training figures). The quantities L, T and M were measured simply by printing the relevant patterns on cardboard, cutting through the contrast limits and weighing the pieces (L and T+M, as given in Table 1, are therefore in grams, their absolute values being of no concern). Any agreement between the predicted and the observed preferences is shaded in grey. This can be readily checked. For instance, if the fish had used a template for the rewarded training figure, then it should prefer the test pattern that produced the larger value of L or of L/(L+T+M) or that with the lower value of T+M, depending on which of these quantities the fish used to assess the matching. However, if the fish used a template for the unrewarded training figure, then it should prefer the test pattern that produced the smaller value of L or of L/(L+T+M) or should prefer that pattern with the larger value of T+M.

All experimental findings are compatible with the view that the fish had learned the pattern discrimination by storing the image of the rewarded figure, as seen from the fixed viewpoint, and later simply selecting the actual pattern that best matched the stored template. The findings, however, exclude the view that the fish had based their selection on a stored template for the unrewarded figure. Furthermore, they exclude the possibility that the fish assesses the degree of ‘matching’ exclusively from the actual overlap L (see Fig. 4). Rather, they require a mechanism in which the non-matching areas, T and M, are also taken into account.

Fig. 5 illustrates the outline for an experiment designed to falsify the template-matching hypothesis. If the fish had used a template of the rewarded training pattern, they should fail to discriminate the two training patterns in tests in which these are shown differently reduced in size such that both equally match the template (i.e. when L, T and M are identical for both patterns). Alternatively, if fish rated, for instance, shape, and if the reduced patterns were large enough for their difference still to be perceivable, then the fish should select the reduced version of the previously rewarded pattern.

Fig. 6 shows the results of a series of experiments with fish B and C to test these predictions. These fish had previously been successfully trained to select a black disk, 17.1° in diameter, and to avoid an equilateral triangle, side length 17.1°, tip pointing upwards. In the tests, a disk of diameter 9.5° and a triangle of side length 12.8° were shown that both produced equal values of L, T and M when matched to the putative template of the large disk used in training. As predicted, both fish chose randomly. One objection to this finding might be that the fish did see that the patterns were different from the training patterns and then chose randomly. But note that this is readily disposed of by the findings of the transfer tests described in Fig. 3. These showed, for a variety of patterns, that fish did not choose randomly when shown two patterns that differed from the training patterns. A second objection would be that the size-reduced test patterns might simply have been too small to be differentiated by the fish. This objection was disposed of with an additional training phase, subsequent to the tests, in which both fish were directly trained to discriminate the patterns with which they had been tested (Fig. 6, ‘Training’ at bottom). Both fish were clearly able to discriminate both patterns after 1 week of training, thus directly showing that the lack of preference in the tests was not because the fish could not perceive the size-reduced figures as different.

The critical test described above was designed to make a trained preference to a pattern vanish simply by changing pattern size. This idea can be extended one step further. If the template-matching hypothesis is correct, it should also be possible to make the fish reverse its preference and choose the unrewarded shape more often. Fig. 7A illustrates the outline for such an experiment. The training patterns are again reduced in size, but so that the previously unrewarded figure matches better the template and should thus be chosen more often then the previously rewarded disk. This experiment was performed with fish B (Fig. 7). First, it had to be pretrained to discriminate two larger patterns (again a disk and a triangle) so that, after the appropriate size reduction in the tests to follow, the smaller pattern would be large enough to be perceived (according to the evidence shown in Figs 2A and 6 bottom). After the fish had learned this task (Fig. 7B, ‘Training’), it was tested with size-reduced versions of the disk and the triangle in which the triangle would better match the putative template of the rewarded disk. In these tests, the fish indeed reversed its preference and chose the triangle (P<0.01) instead of the disk.

As a modification of this experiment, fish B subsequently had to choose between a black disk (the rewarded training pattern) and a black square. Two types of test were made. (i) The square matched (larger L, smaller T but same M=0; cf. Fig. 4) the putative template better than the disk. In these tests, the square was of side length 17.1° and the disk was 9.7° in diameter. (ii) The disk matched the putative template better than the square (side length of square 9.7°, disk 17.1° in diameter). Again, the fish showed no preference for the disk but preferred the square (42 out of 60 choices; P<0.01) when this matched the template of the rewarded disk better but chose the disk if that produced the better match (49 out of 60 choices; P<0.001).

A further series of experiments with fish A attempted to discover the limits of template-matching. This fish was tested to ascertain (i) whether it could discriminate visual patterns composed of several elements and, if it could, (ii) whether this ability would still be compatible with template-matching. The experiments are shown in Fig. 8A. First, the fish was trained to discriminate a single black disk (in the centre of a pattern card) from two disks (placed diagonally at the corners of a pattern card). The results of unrewarded tests with these patterns showed clearly that the fish was able to learn the task. Two types of transfer tests were then performed. In the first, the single disk was much larger; in the second, it was displaced to the upper rim of the pattern card. In both tests, the fish preferred the card that contained only one disk. This indicates that the fish had not simply learned to select the darker card or the card that contained a centrally placed figure. While many hypotheses, for instance that the fish determined the number of elements, may explain these results, they do not rule out the possibility that even in this task the fish could have been using template-matching. As illustrated in Fig. 8B, the findings conform to a template-matching mechanism in which (i) the fish aligned the image of the card showing two disks with the template according to the centre of mass of the composite pattern and (ii) assessed the degree of matching by the relative overlap L/(L+T+M).

Our results indicate that the weakly electric fish Gnathonemus petersii uses template-matching to recognize visual patterns that it had previously viewed from a fixed vantage point. Such template-matching systems classify visual patterns by comparing the actual image with a stored image and are thus not viewpoint-invariant. They are well documented in several insect species (for reviews, see Collett, 1992; Heisenberg, 1995); for example, honeybees (Wehner, 1972; Cartwright and Collett, 1983; Gould, 1985; Ronacher and Duft, 1996), wasps (Zeil, 1993; Collett, 1995) and the fruit fly Drosophila melanogaster (Dill et al., 1993; Dill and Heisenberg, 1995). It has been suggested that template-matching might be an old evolutionary trait associated with visual systems (Dill and Heisenberg, 1995; Heisenberg, 1995) and we wondered whether it could also be demonstrated in vertebrates. We selected a species in which a low visual resolution was to be expected, and chose an arrangement in which the animal could always view the patterns from a defined vantage point. The results of our transfer tests suggest that the fish stored the retinal image of a previously rewarded pattern as seen from the fixed viewpoint and simply chose that of the two test patterns that ‘matched’ best with the stored image.

In a more natural situation, both distance and the viewing angle, and thus the image of a given pattern, may change dramatically, and the obvious question is what is the relevance of template-matching systems under more natural situations. Interestingly, template-matching systems efficiently guide freely flying bees and wasps to their nest or to a feeding place. The trick appears to be that these animals perform stereotyped flights to learn the visual surroundings of their nest or of a feeding place and that they also follow these stereotyped paths in their subsequent returns. The stereotyped flights thus enable templates to be stored from a limited number of vantage points. When the insect later returns, following the same path, it can simply retrieve the stored views at the corresponding viewpoints. Thus, the insect only needs to recognize the landmarks around a nest from a limited number of viewpoints and thus needs only to store a limited number of templates for this purpose.

Of what use is template-matching for the fish Gnathonemus petersii? Recent experiments indicate (P. Moller, personal communication) that Gnathonemus petersii might use visual landmarks (‘underwater marks’) to locate, for instance, feeding places or hiding places by their visual surroundings. In doing so, a template-matching system would be useful if the fish always started its route from a defined hiding place from which the distance to the (stationary) landmarks and possibly also the viewing angle would be fixed. Even for longer routes, one could imagine that the fish would proceed through a series of intermediate stops at defined hiding places so that it would suffice for the fish to store a template at each of these hiding places and to activate the correct template at the appropriate hiding place.

An open question is whether Gnathonemus petersii has any amendments to this simple form of visual pattern classification that would allow viewpoint-invariant pattern recognition. Such additional mechanisms are known in insect species that do perform template-matching. For instance, bees show an astonishing ability to generalize learned visual patterns and are not limited to the use of template-matching systems (e.g. Srinivasan et al., 1994; Giger and Srinivasan, 1995). The fruit fly Drosophila melanogaster also has further systems at its disposal to classify patterns according to a simple set of parameters (Ernst and Heisenberg, 1999). Our findings cannot rule out such additional mechanisms. In an attempt to test for such mechanisms, we tried to train fish B in a task in which the size of the training patterns varied randomly from trial to trial (all patterns, however, subtended a visual angle larger than 6°) and only their shape was kept constant. In this task, only shape would have been a reliable cue. This attempt failed and was abandoned after 6 months. Moreover, two further unsuccessful attempts were made with two naive fish that had not been trained to visual patterns of fixed size. While these failures cannot be used to argue against the species having true visual shape recognition, they indicate that any viewpoint-invariant extraction of visual shape might not be easy in Gnathonemus petersii.

Pattern vision in Gnathonemus petersii provides only limited spatial resolution. This is indicated by the results shown in Fig. 2B, in which a reduction in the size of the training patterns resulted in a drastic reduction in the number of correct choices. These tests indicate that patterns needed to be larger than 3° to be discriminable. Interestingly, a low visual acuity of the same magnitude has been found in another group of nocturnal animals, the echolocating bats of the insectivorous genus Myotis. To gain his estimate of visual resolving power, Suthers (1966) rotated a periodically striped drum at approximately 36° s^–1 around a bat and determined the minimum stripe size that elicited clear optomotor following responses. He found the critical stripe diameter to subtend a visual angle of between 3 and 6°, which indicated a low visual acuity. In Gnathonemus petersii, a low resolution was to be expected from the extreme adaption of its vision to low ambient light levels (McEwan, 1938; Ciali et al., 1997): its photoreceptors (rods and cones) are grouped in elongated bundles that are ensheathed in light-reflecting pigment cells. Light that is not immediately absorbed by the receptors is reflected back, which might further degrade spatial resolution. Our findings suggest, moreover, that pattern vision is ineffective at a light intensity of approximately 1000 lx and works best at intensities around 10 lx.

McEwan (1938) analyzed the retina of mormyrid fish that had been kept at different ambient light levels and found no signs of any adaptation. Teyssedre and Moller (1982) showed that optomotor following responses to a moving pattern of black-and-white stripes existed only at low ambient light intensities. At an ambient light level of 540 lx, fish failed to show any consistent optomotor responses, as if the contrast between the black-and-white stripes was no longer perceived. The lack of adaptation seen both in the optomotor system and in pattern vision, together with the work of McEwan (1938), thus add confidence to our conclusion of a complete lack of any adaptation mechanism in the mormyrid retina. Perhaps the small streams in which Gnathonemus petersii lives are so overshadowed that the fish never needs to see under bright illumination; for measurements in a biotope of various other mormyrid species, see Moller et al. (1979). The low resolution and the use of template-matching as suggested here might be appropriate for a vertebrate visual system adapted to operate at low ambient light levels around 10 lx.

It is a pleasure to thank our referees for many helpful comments and suggestions that improved the manuscript. Dr Angel Caputi stimulated the preference-reversal test and Dr Peter Moller provided necessary encouragement. Drs Karl Götz, Sam Rossel and Klaus Vogt commented on earlier drafts, Dr Randy Cassada corrected the English and Mr Meinrad Hahn helped in training the fish.