Virtual reality (VR) systems are useful tools that enable users to alter environmental settings and the location of landmarks in an accurate and fast way. Primates have been shown to be able to navigate in virtual environments. For rodents, however, all previous attempts to develop VR systems in which rats behave in the same way as in corresponding 3-D environments have failed. The question arises as to whether, in principle, rodents can be trained to navigate in a properly designed virtual environment (VE), or whether this peculiarity is limited to primates and humans. We built a virtual reality set-up that takes the wide-angle visual system of rats into account. We show for the first time that rats learn spatial tasks in this VE quite readily. This set-up opens up new opportunities for investigations of information processing in navigation (e.g. the importance of optic flow or vestibular input).
Virtual reality (VR) systems are widely used in designing machines(Lapointe and Massicotte,2003), training surgeons(Liang and O'Grady, 2003), or in investigating how humans navigate in space(Gillner and Mallot, 1998; Maguire et al., 1998; Wiener and Mallot, 2003). For humans, the ability to navigate in virtual environments is quite similar to that in real space (Rieser et al.,1990; Ruddle et al.,1997). Several studies have shown that primates are also able to interpret interactive 2-D projections or large monitor presentations as virtual environment (VE), in which they can navigate(Leighty and Fragaszy, 2003; Nishijo et al., 2003; Towers et al., 2003). This is most likely due to the fact that the non-human primate visual system, the optics of the eye and the information processing in the brain, are quite similar to the human visual system (Hughes,1977; Mishkin et al.,1983; Pollen,1999; Zeki,1993).
So far it has not been possible to create VR systems for rodents. Rodents explore and navigate successfully in large areas. In a radio-tracking study,home ranges of the rat Rattus rattus are reported to vary between 0.33 and 1.83 × 105 m2 and range lengths between 86 m and 311 m (Dowding and Murphy,1994). In addition, much larger distances up to 954 m arereported for migrating single recaptured males(Hartley and Bishop, 1979). In contrast, investigations of the navigational abilities of rats have long been restricted by the limited space of laboratoryoratories and the number of landmarks that can be changed quickly and accurately without disturbing the animal (Jeffery, 1998). Attempts have been made to construct a VE that could be used to investigate the spatial navigation abilities of rodents. In one such set-up, animals were trained in a Y-maze made of six monitor screens, and the scenes on the two monitors that constitute an arm identified that particular arm. However, the experimental results suggested that the animals did not treat the presentation of scenes as VE in which to navigate, but rather as objects within the real laboratoryoratory environment (Gaffan,1998; Gaffan and Eacott,1997; for a review, see Hölscher, 2003). We assume that the reason for this result lies in the properties of the visual system of the rat. The primate visual system has a relatively small visual field (about the frontal half sphere), a fovea with color vision and high spatial two-point resolution of about 0.015° (Zeki,1993). In contrast, the rat's visual field covers a large solid angle (nearly the whole half sphere above and a large part of the half sphere below the horizon (Hughes,1977). Rats also have a considerable binocular overlap. They do not possess a fovea and have practically no color vision. They have a low visual acuity for two-point discrimination of about 1° (Hughes, 1977, 1979; Prusky et al., 2000). The rat's eye is adapted to low light intensities, the lens has a high light-collecting power (f/d about 1; Hughes, 1979) and the retina contains mostly rods (Szel and Röhlich, 1991). As a consequence, we assume that any VR system for rodents must cover a large area of the visual field but does not need high resolution and luminance.
We have developed a VR set-up that covers a large part of the rat's visual field (360° of azimuth, –20° to +60° of elevation). It is combined with a treadmill, in which the animal runs on top of an air-cushioned polystyrene sphere. Any translational movement of the animal leads to a rotation of the sphere, which is monitored and fed to the PC that controls the generation of the VE. The VE is rendered and presented to the animal in a closed action-perception loop. We developed this novel system to test whether or not rodents accept such virtual environments and can be trained to navigate in it in order to obtain food rewards. In the present paper we show that rats can indeed be accustomed to this VR system and successfully navigate in various environments for sugar water rewards.
Materials and methods
18 male Long-Evans rats Rattus norvegicus (Charles-River, Germany)weighing 150–200 g at the beginning of training were used for the experiments. Six of these animals were trained in the real environment experiment, and the remaining twelve were trained in the VR experiments. All experiments were licensed according to German and EU law regulating the use of animals in research.
In our set-up, the rat runs in the zenith of a hollow polystyrene sphere of 50 cm diameter and a mass of about 400 g. The ideal mass of the sphere is 1.5 times the mass of the animal. In this case the haltered animal has to apply the same force to accelerate the sphere as it would need to accelerate its body mass by the same amount on the ground(Dahmen, 1980). The sphere is supported by an air cushion in a half-spherical mould of 50.4 cm diameter. The animal is haltered by a soft leather harness attached to small aluminium sheets that allow the rat to lift and lower its body, to pitch it slightly and to rotate around the yaw axis. (Figs 1B, 2D). The animal rests with its full weight on top of the ball. When the rat walks it stays in the zenith and rotates the ball about a horizontal axis. The ball is prevented from rotation around the vertical axis but can easily be rotated around any horizontal axis. This is accomplished by slightly tilting the half-spherical mould (by about 7°). The polystyrene sphere is slightly pressed against two wheels (w in Figs 1A, 2A), which touch the sphere under 90° in its equator and are supported by horizontal axes with tip bearings (Dahmen, 1980). The behaviour of the animal during an experiment is monitored by two small video cameras (c in Fig. 2C). The yaw angle (heading) of the animal is monitored by an angular incremental encoder(aie in Figs 1A, 2D). In order to monitor the locomotion of the animal, any motion of the ball is registered by two x/y motion sensors (md in Fig. 2B) positioned at 90°along the equator of the sphere.
The sensors (HDNS2000 Agilent, Palo Alto, CA, USA) are motion detectors that are found in most optical PC-mouse devices. A statistical pattern of tiny black dots is applied with an airbrush onto the sphere surface. A small plastic lens (4.6 mm focal length) images this pattern, scaled down by a factor of 1/7, to the 1 mm × 1 mm light sensitive area (lsa) of the HDNS2000. The pattern is illuminated by a red LED, the intensity of which is chip-controlled. In this arrangement the ball surface may move to and away from the sensor by a few mm without degradation of movement detection. The maximum allowed speed of the surface is 2 m s-1 (30 cm s-1 on the lsa), the step resolution 0.5 mm (1/16 mm on the lsa). The signals of the detectors were fed to an incremental counter board(APCI1710, ADDI-Data, Ottersweier, Germany) implemented in the controlling PC.
The virtual environment
The rat is surrounded by a toroidal screen of 140 cm diameter and 80 cm height (grey shaded in Fig. 1A). The screen covers the visual field from –20° below to +60° above the horizon of the rat and 360° of azimuth (green shaded in Fig. 1A). The image of the VE is projected onto the screen from a DMD projector (b in Figs 1A, 2C) via two plane mirrors (p) and an angular amplification mirror (aam). With respect to the position of the animal and the aam the torus acts as a horopter: the animal sees any object on the screen under the same angle as it is projected from the aam (shaded blue in Fig. 1A). The aam is a polished aluminium surface shaped as described by Chahl and Srinivasan (1997; Fig. 2C). It has a constant`angular amplification' factor of 39 over its whole surface. (This amplification factor is defined for two rays of the same azimuth as the ratio of the divergence of the reflected beams to that of the incident ones.) Rays that hit the aam far from its symmetry axis are reflected to the `sky' of the rat, whereas rays incident near that axis are reflected to the `bottom' (red shaded in Fig. 1A). The image of the VE to be projected must be arranged in an annulus: the sky must be morphed to the outer ring, the bottom to the inner ring of the annulus. This morphing is done under the control of the graphics tool OpenGLPerformer (SGI,Mountain View, CA, USA). A 360° panoramic view of the virtual landscape seen from the momentary X/Y/Z position of the rat is rendered by 12 virtual cameras. This panorama is used as texture and applied to the annulus. Another virtual camera grabs this textured annulus and produces the image for the projector. Exploiting the hardware accelerated graphic commands of a NVIDIA-GForce3 chip set, the above procedure is fast enough to synchronize to the image repetition rate of the DMD projector.
In order to reward the animal, the haltering contains a thin brass tube that feeds sugar water to the mouth of the animal(Fig. 2D). A software triggered valve controls the sugar water flow.
Because in VR it is difficult to simulate mechanical obstacles and because we wanted to avoid conflicts between visual and tactile sensory input, we designed an environment with cylinders suspended from the ceiling. Thus, the animals could not encounter a visual obstacle that could not be sensed mechanically. In order to be able to compare the learning abilities of rats in virtual environments with those in real environments, rats were trained in a real environment first in which there were real cylinders suspended from the ceiling.
Experiment 1 in real environment
Six animals were trained in a laboratoryoratory to retrieve a cocoa flavoured cereal (Kellogg's) located below one of three cylinders (20 cm diameter, 50 cm in height). The cylinders were suspended from the ceiling about 40 cm above the ground. One cylinder was white, one was black, and the third one was of a grey colour. The cylinders formed a triangular configuration and remained in their location throughout all runs. The rats could explore a 2 m × 1.6 m area that was separated from the remaining laboratoryoratory by white barriers 40 cm in height. Animals were given four runs per day. The runs were recorded on video and analysed off-line. We recorded the time until food reward was found, and the number of visits to a 20 cm catchment area below the three cylinders.
Experiment 2 in virtual environment
In our VE the ceiling was 1 m high and textured with a pattern of irregular grey stones. The same texture was applied to the ground. The cylinders were 50 cm in diameter and 80 cm in height, suspended 20 cm above the ground. They were covered with vertical black and white stripes. When the rat entered the area below a cylinder it was rewarded with a drop of sugar water. In preliminary tests some of the rats tended to return simply to a rewarded cylinder to get another reward. We tried to prevent such `circling' around one cylinder by defining an outer radius r0, which the animal had to cross before it could get another reward at the same cylinder. The VE was repetitive and was programmed as a grid of 5 × 5 identical squares. The animal started in the middle one. When it reached a border of this inner square the environment was shifted to the corresponding point on the opposite border of this square. This quick change between two consecutive frames did not change the appearance of the environment because of its periodicity. The rendering of the scene was thus limited to the 25 squares. The animal's path coordinates were restricted to the inner square while the rat was running for hundreds of meters and the landscape looked endless to it. Afterwards the path of the rat taken in the VE was reconstructed by eliminating the well-defined trace jumps, which corresponded to border jumps in the registered data.
12 Long Evans rats were handled for 3 weeks and allowed to become accustomed to the harness, to movement on the polystyrene ball, and to the sugar water reward system. They were then trained in the above-mentioned square environment, which contained a square array of cylinders with 0.5 m diameter and 2m distance to each other. The above-mentioned r0 was 0.5 m. Each animal was allowed to run for 10 min on each of 10 consecutive days.
Experiment 3 in virtual environment
In another VR experiment starting 1 day later we presented a similar`endless' square array of identical cylinders as in the previous experiment. But now the distance between the cylinders had been increased to 10 m, and r0 was 2 m. If the animal moved underneath a cylinder, the visual angle subtended by each of the four nearest cylinders was about 4.6° of elevation and about 2.9° of azimuth. This implies that because of the limited two-point resolution of the rat eye these cylinders are at the limit of visual discrimination.
Experiment 1 in real environment
The animals quickly learned to find a food pellet below the baited white cylinder, as shown in Fig. 3A,B. Animals almost stopped visiting unbaited cylinders and learned to search under the baited cylinder (one-way repeated-measures analysis of variance, RM-ANOVA, F3,11=8.34, P<0.0001; post-hoc Bonferroni multiple comparison test, *P<0.01; N=6). The time needed by animals to find the target area below the white cylinder was reduced over trials (one-way RM-ANOVA, F3,11=6.67, P<0.001). In a reversal-learning task, the black cylinder had been baited. Animals learned to visit the black cylinder and practically stopped visiting the white cylinder(one-way RM-ANOVA, F3,11=13.7, P<0.001; post-hoc Bonferroni multiple comparison test, *P=0.01). The time needed by animals to visit the black cylinder was reduced over training sessions, which was due to the fact that the animals went directly to the rewarded spot and spent very little time on exploration behaviour (one-way RM-ANOVA, F3,11=4.98, P=0.002; Fig. 2C,D). The reduction of retrieval time was continuous over all trials.
Experiment 2 in virtual environment
Results of experiments with cylinder spacing of 2 m
As shown in Fig. 4A–C,the animals improved considerably in their orientation to and in finding of the cylinders. The numbers of rewards (hits per 10 min run) increased steadily over the course of 10 days (one-way RM-ANOVA, F9,11=40.7, P<0.0001, post-hoc Bonferroni multiple comparison test, **P<0.01; ***P<0.001; N=12; Fig. 4A). In Fig. 4B the number of hits per 2 m distance increased over training sessions (ANOVA F9,11=22.1, P<0.0001, post-hoc test **P<0.01; ***P<0.001). While the ideal path would have followed a straight line between cylinders (one reward every 2 m), the animals improved to an average of 0.76 rewards per 2 m. The path length per 10 min, i.e. the average speed of the animals, increased(Fig. 4C). The maximum path length per 10 min was 219.1 m. The average path length over all 12 animals increased from 66.5 m to 166.2 m. Fig. 4D shows the inter-individual variation of the path length averaged over all 10 days. As a general feature we observed that the animals tended to follow a more-or-less defined direction in the laboratory coordinate system over several days (Figs 5A, 7A). This direction varied from rat to rat.
Fig. 5 demonstrates in more detail how the path chosen by the animals improved over time. In Fig. 5A the complete path of animal 9 for the first day (blue) and the tenth day (red) is displayed. The total path lengths were 70.6 m and 172.3 m, respectively. Fig. 5B,C depicts details of the paths boxed in Fig. 5A. Cylinders are drawn to scale as dark grey circles, hit cylinders marked by a larger light grey circular surround. Dots on the path mark every 30th sample(about every 1.4 s). Cylinder spacing is 2 m. The much more regular path on day 10 as compared to day 1 is apparent, and Fig. 5B,C depicts the most irregular part of the paths. The higher speed on day 10 can be inferred from the larger dot distance (Fig. 5B is at about twice the scale of Fig. 5C). The path on day 10 shows a more direct approach to the nearest cylinder. Fig. 5E,G shows the angular histogram of the orientation of the animal's body long axis, i.e. of the angular incremental counter (aie) data. If the animal were to run along its body long axis from one cylinder on a straight path to one of the four nearest cylinders, we would expect histogram peaks at N, E, S, W. On day 1(Fig. 5E) we find no pronounced body orientation, while on day 10 (Fig. 5G) we find superimposed on a general orientation to SE, a peak orientation to S. In order to get an even better demonstration of the improvement of the `cylinder hit probability' of our rats, we surrounded each cylinder by a 2 m × 2 m square divided into 7 × 7 subsquares. We counted the number of trace samples in each subsquare and added up the counts in corresponding subsquares (corresponding with respect to their position relative to the central cylinder). Fig. 5D,F shows the relative number of counts in corresponding subsquares. For the trace of a rat that does not aim at the cylinders we expect an equal distribution of trace sample counts over all subsquares. The considerable increase of the `trace density' under cylinders from day 1 to day 10 is apparent.
Experiment 3 in virtual environment
Experienced rats from the previous experiment were transferred to a VE with cylinder spacing of 10 m and r0 of 2 m. The results are shown in Figs 6, 7, using the same conventions as in Figs 4, 5. In Fig. 6A the numbers of rewards(hits) per 10 min run increased over the course of 5 days (one-way RM-ANOVA, F4,11=7.3, P<0.0001, post-hocBonferroni multiple comparison test, *P<0.05; **P<0.01; ***P<0.001; n=12). In Fig. 6B the numbers of hits per 10 m distance run also increased over time (ANOVA, F4,11=5.37, P<0.002, post-hocBonferroni multiple comparison test, **P<0.01). The maximum achievable number of hits per 10 m was 1; animals reached an average value of 0.644. The average path length of 12 animals increased from 154.1 m to 176.3 m per 10 min (Fig. 6C), the maximum path length was 228.8 m. Fig. 6D shows the inter-individual fluctuation of path length averaged over 5 days.
In Fig. 7A the whole trace of animal number 9, the same as in Fig. 5A, is displayed for day 1 (blue) and day 5 (red). The trace lengths are 123.7 m and 214.4 m, respectively. Fig. 7B,C show the boxed details of these traces. The traces appear, as compared to Fig. 5C, slightly less precise in their orientation to the nearest cylinders as demonstrated in the angular histograms of Fig. 7E,G. In Fig. 7D,E we surrounded each cylinder by a square of 10 m × 10 m and subdivided it into 7 × 7 subsquares. The relative trace sample density is very high in the subsquare under the cylinder (Fig. 7D,F).
Our results show for the first time that rats can learn tasks in a virtual environment. Rats ran up to 220 m per 10 min session, showing that the motivation to collect rewards was high. It underscores the advantage of the`treadmill' design, which allows the animals to `explore' very large areas, in contrast to standard laboratory situations, in which there is very limited space for the animals to explore. The home range of rats in the wild extends to 180 000 m2, home range length to 311 m(Dowding and Murphy, 1994.) The animals most likely possess navigation abilities to oversee large and complex environments. Our set-up will be useful for investigating how rats perform when solving tasks in large environments, and what type of navigational strategies they develop.
While the good ability of rats to learn tasks in the VE is very encouraging, it should be noted that important sensory modalities such as smell or touch cannot be simulated. In addition, the virtual world is presented on a screen of fixed distance. Thus motion parallax is correct but stereo disparities are not correctly displayed. However, the importance of stereo vision to rats in navigation has not been demonstrated, and even if it is of some importance, the animals clearly adapted to the novel situation in which they were placed. We did not include acoustic characteristics of places,which might add to the realism of the rat's experience in VE. The vestibular stimuli that animals experience during ego-motion are correct for rotation but not for translation. Somatosensory force feedback to the legs because of rotational and linear acceleration and because of body weight are nearly correct. The fact that the animals did adapt to the unusual situation within the VE shows the degree of flexibility that rodents possess to adapt to unpredictable changes in their environment, and that they are able to ignore unreliable sensory input.
The time required and the distance run to obtain rewards continuously shortened during training, suggesting that the animals developed a strategy to optimise their foraging behaviour. The rat learned that running on the ball towards an object would bring the object `closer', perhaps by observing the image flow (e.g. the size of the 2-D projection of the object), which is the same as with real objects. We propose, as an explanation for our results, that the rats interpreted the 2-D projections as 3-D objects.
The strategies that the individual animals develop are diverse and range from simple running along rows of cylinders to circling strategies that increase the likelihood of finding a reward. Circling strategies were reduced by excluding an immediate second reward under the same cylinder. Most animals appeared to prefer a fixed running direction with respect to the laboratory coordinate system. This direction varied from rat to rat but was maintained over several sessions. Within the movements along that general direction, most rats clearly noticed the landmarks and navigated towards them. In the 2 m cylinder spacing experiment 2, a rat could easily see the four nearest cylinders from under the cylinder where it just had been rewarded. The cylinders covered an azimuth of about 14° and an elevation of about 22°. On day 10 many rats ran for a large part of their trace on a straight path from one cylinder to the next (see Fig. 5A,C and the angular distribution of the body length orientation Fig. 5G). The number of hits per 2 m trace length reached the average level of 0.76, which should be compared to the maximum possible value of 1. In contrast, in the 10 m cylinder spacing experiment 3, the nearest cylinders covering 4.6° of elevation and 2.9° of azimuth were not that easily visible to the rats. As a consequence, despite the training in experiment 1, traces are not so well oriented towards the nearest cylinders (see Figs 7A–C). The animals did not quite reach the level of efficiency that they achieved in the experiments with 2 m cylinder distance. They hit fewer cylinders per 10 m trace length (0.64 compared to the possible maximum of 1) and the general direction of the trace dominated the angular histogram of body orientation(Fig. 7G). But when the cylinders became more visible, rats did turn towards them, find them and get a reward. The relative frequency of trace samples in the vicinity of cylinders shows a pronounced peak in the subsquare below the cylinder (see Fig. 7D,F).
The traces never became `optimal'. There seems to be always to be a tendency to deviate from the best path even when goals are clearly visible(see Fig. 5C). Perhaps we may interpret this `noise' as an investigatory strategy that cannot be suppressed. Since there are no repetitive and predictable natural environments with a 100%reward probability, it is of adaptive value constantly to deviate slightly from the seemingly perfect strategy in order to increase the likelihood of finding new food sources that otherwise would have been missed by running past them.
In conclusion, the results presented here show for the first time that rats are capable of navigating in virtual environments, an ability that so far has only been shown in humans and primates. The apparatus will open up new possibilities such as the investigation of navigation in large spaces similar in size to spaces that are explored by animals in the wild, and the importance of specific sensory information such as visual landmarks, egomotion perception from optic flow, or vestibular input in rodent orientation.
The work was supported by the Deutsche Forschungsgemeinschaft (DFG) within the framework `Graduiertenkdleg GRK778'.