Insects modify their behaviour depending on the feedback sensor used when walking on a trackball in virtual reality

To understand how animals use sensory information, researchers have designed virtual-reality paradigms for tethered animals. Virtual reality enables tight control of the sensory stimuli to which animals are exposed, and detailed observation of behavioural responses from them as they interact with the stimuli (Dombeck and Reiser, 2012). By fixing an animal in place, tethered experiments allow measurement of the animal's reaction to precisely controlled stimuli. Closed-loop paradigms, which simulate free untethered motion, measure the tethered animal's motor output and concurrently use this to update the animal's sensory environment, permitting a careful study of the interaction of the animal with its environment (Taylor et al., 2008). A useful experimental paradigm to study the behaviour of walking animals is to place them on an air-supported trackball, where the ball's movements in reaction to an animal's leg movements are measured and used to change its sensory environment (Harvey et al., 2009; Seelig et al., 2010; Takalo et al., 2012).

Trackball motion is often measured using motion sensors repurposed from computer mice (CM; Clark et al., 2011; Takalo et al., 2012; Paulk et al., 2014), which optically measure changes in ball position by acquiring sequential images of the ball's surface and calculating the direction and magnitude of visual movement in the images (Avago Technologies, 2009). Unfortunately, CM sensors have been found to suffer from errors that alter the accuracy and precision of their measurements, such as: inconsistent registration of motion about the two measurement axes (Palacin et al., 2006; Moore et al., 2014), and effects of surface texture and pattern (Minoni and Signorini, 2006), and illumination (Tunwattana et al., 2009). In contrast, newly developed software called ‘FicTrac’ (FT) provides an alternative, computer-vision-based approach to recording trackball motion, by measuring the absolute orientation, rather than the velocity, of the trackball at each point in time (Moore et al., 2014). FT calculates ball motion with greater precision than CM sensor systems, and also performs robustly over a wide range of experimental conditions (Moore et al., 2014). When simultaneous measurements of bee behaviour from both CM and FT systems were compared in open-loop experiments, the two systems recorded measurements that were qualitatively similar, but statistically different in magnitude (Moore et al., 2014).

Closed-loop, virtual-reality experiments with Drosophila have shown that these insects can adapt to use a wide range of non-standard behavioural outputs, such as their flight thrust (Wolf and Heisenberg, 1991) and lateral leg forces (Wolf et al., 1992), to control a visual stimulus, where the stimulus would normally be controlled by wing or leg movements that induce a yaw torque (Poggio and Reichardt, 1973; Bahl et al., 2013). Given the differences that occur between the behavioural responses as measured by the CM and FT systems, we asked whether insects in closed-loop experiments would adjust their behaviour depending on the subtle differences between the feedback that is provided by the two sensor types.

To examine whether an insect's behaviour in a closed-loop virtual-reality paradigm would be influenced by the characteristics of the sensor that is used to measure that behaviour, we tested honeybees in a visual fixation paradigm (Moore et al., 2014; Paulk et al., 2014) using several feedback sensor conditions. Bees walked on a ball, the motion of which was measured using either the CM or FT system for feedback. Alternatively, the output from the FT system was adjusted to mimic a systematic error that occurred in CM sensors. In all cases, bees demonstrated an ability to fixate the stimulus, a green bar displayed on LED panels surrounding the bee. However, bees varied their walking speed and the straightness of their paths depending on the feedback sensor that was used. Honeybees made this change when it would reduce the sensitivity of the sensor system to their turning actions – in other words, when it would reduce the gain of the feedback loop – which tended to improve their ability to fixate the stimulus. The behavioural changes occurred relatively quickly, and were observable in the average parameters of experiments lasting 2 min. A comparison with data from freely walking honeybees tracking a moving object found that they behaved in a similar manner to tethered bees in the fixation task. Yet, this comparison did indicate that the mismatched inertia between honeybees and the trackball limited how quickly a tethered bee could turn the trackball, and consequently the stimulus, during the experiments.

Honeybees were able to fixate a bright vertical green bar, displayed on a LED array, in their frontal visual field when the same bees were tested with both feedback sensor systems (CM or FT) in closed-loop (Fig. 1A–C). Although the distribution of bar positions appeared to differ between sensor systems (Fig. 1D), a comparison of the mean vector lengths (Fig. 1F) between groups revealed that there was negligible difference in fixation strength (Fig. 1E; paired samples t-test: t₁₀=0.67, P=0.517), or in average fixation direction (Watson–William's test: F_1,20=0.52, P=0.478). We conclude that bees perform fixation equally well, regardless of whether their walking behaviour is measured using the CM or the FT systems to provide the closed-loop feedback.

Having established that the bees were able to fixate equally well using either sensor system, we further examined whether they changed other aspects of their behaviour depending upon which sensor system was used to provide the feedback. In each case, we examined the bees' turning rate (ω), their walking speed (w) and their motion ratio [w/(w+ω)], which provided a measure of the ‘straightness' of a bee's path (see Materials and methods). An earlier study has demonstrated that FT provides a more accurate and precise measure of the ball's rotation than do CM sensors (Moore et al., 2014). During experiments when the CM system provided feedback, video of the trackball was also recorded and subsequently analysed with FT [labelled CM (FT)]; hence, CM and CM (FT) represent measurements of the same ball movements using different sensor systems. In the other set of experiments, the FT system was used to provide feedback. This combination of experiments and measurement systems allowed us to (a) make a direct comparison of the responses that were registered by the two sensor systems during each experiment (CM versus FT); and (b) use measurements from the same sensor (FT) to examine whether the bees displayed any differences in behaviour depending on which sensor system was used to close the feedback loop [CM (FT) versus FT].

Honeybee behaviour was initially compared using the responses measured by each sensor system used for closed-loop feedback (CM versus FT). The turning rate distributions displayed zero bias with either feedback sensor condition (Fig. 2A). The walking speed distributions were long tailed in either case, with a prominent peak at or close to zero (Fig. 2C). Neither the mean absolute turning rate (Fig. 2B; t₁₀=1.38, P=0.197) nor the walking speed (Fig. 2D; Wilcoxon signed rank test : W₁₀=46, P=0.248) was significantly different between the two conditions. Interestingly, the distribution of motion ratios showed a peak at 1 (pure walking) when the CM system was used for feedback but a broad distribution (Fig. 2E) when the FT system was used, with few occurrences of a motion ratio close to 1. The mean motion ratio was significantly higher when using the CM system for feedback (Fig. 2F; t₁₀=3.31, P=0.009), indicating the CM system measured bees walking along straighter paths when it was used for feedback.

We then considered the measurements from FT of the same ball motions that the CM system measured when it was used for feedback [CM (FT) versus FT]. Although we still found no significant difference between honeybees' mean absolute turning rate in either condition (t₁₀=0.76, P=0.468), their mean walking speed was now found to be significantly higher when the CM system provided feedback (Fig. 2D; W₁₀=2.13, P=0.033). The walking speed distribution showed that bees had in fact increased how often they walked quickly when feedback came from the CM system (Fig. 2C), which was not identified in the actual measurements from that system. And whilst a significant increase in motion ratio was still found when using the CM system (Fig. 2F; t₁₀=3.62, P=0.005), the CM motion ratio distribution measured by FT showed a broader distribution with a higher peak than when the FT system provided feedback, but which did not peak strongly at 1 as the CM system had registered (Fig. 2E).

Despite our finding that the feedback sensor did not influence bees' fixation performance, it does appear to influence their method of controlling the stimulus, as bees increased both their walking speed and the straightness of their paths when the CM system provided feedback. The latter finding would have been identified using the motion ratio data from the CM sensors directly. However, the increase in walking speed would not have been detected if FT had not been used to also measure the trackball motion during all experiments.

One reason the bees may have increased their walking speed when the CM system provided feedback is the existence of a systematic error that occurs in these sensors (Palacin et al., 2006; Moore et al., 2014); CM sensors exhibit an interaction, or coupling, between the movements that they register along their two principal axes. A bee could take advantage of this artefactual coupling by walking faster to reduce the sensitivity of the sensors to its turning rate, i.e. by increasing its motion ratio. We therefore hypothesized that the bees may have changed their behaviour to adjust the turning rate sensitivity of our CM system to a value that they ‘preferred’. To test this hypothesis, we conducted an experiment where the turning rate sensitivity was deliberately and directly coupled to the walking speed in two ways (see Materials and methods). In one condition, termed the ‘decrease’ condition, the sensitivity was decreased as the walking speed increased, which mimicked the systematic error observed in the CM sensors. In the second condition, termed the ‘increase’ condition, the sensitivity was increased as walking speed increased, which tested how honeybees would respond to the opposite scenario. Prior to testing either condition, bees were given a pre-test in which the turning rate sensitivity was independent of the walking speed. Importantly, FT was used to measure the ball motion for all three conditions; thus, the measurement accuracy and latency were matched for all conditions. If bees varied their walking speed between the pre-test and either coupling condition (decrease or increase), it would demonstrate that their behaviour was influenced by the alteration.

For each condition – pre-test, decrease or increase – we found that honeybees were able to position the fixation stimulus frontally. However, in the decrease condition, bees achieved a sharper fixation distribution (Fig. 3A), and displayed significantly longer mean vector lengths (Fig. 3B,C), on average, than in the corresponding pre-test (Fig. 3B; t₉=2.51, P=0.033). Conversely, bees tested with the increase condition did not exhibit different vector lengths compared with the corresponding pre-test results (Fig. 3B,D; t₉=0.28, P=0.787). The average fixation direction was not significantly different from the corresponding pre-test in either manipulation condition (Watson–William's test; pre-test_dec versus decrease: F_1,18=2.28, P=0.149; pre-test_inc versus increase: F_1,18=0.18, P=0.677).

Analysing the behaviour further, we found that, in the decrease condition, bees did not modify their mean absolute turning rate (Fig. 4A,B; t₉=0.73, P=0.484), but significantly increased their mean walking speed (Fig. 4C,D; w₉=2.29, P=0.022) and also their mean motion ratio (Fig. 4E,F; t₉=4.16, P=0.002) relative to the pre-test. These findings indicate that bees changed their behaviour in the decrease condition by spending more time walking both faster and straighter. This change would reduce the sensitivity of the feedback sensor to their turning rate, apparently aiding the bees in achieving more accurate fixation. In contrast, in the increase condition, bees did not significantly modify their mean absolute turning rate (Fig. 4B; t₉=1.41, P=0.191), mean walking speed (Fig. 4D; W₉=1.17, P=0.241) or mean motion ratio (Fig. 4F; t₉=1.49, P=0.171) relative to the corresponding pre-test. In the increase condition, bees maintained a similar behaviour to the pre-test; this would minimize any turning rate sensitivity increase, which we would expect to hamper fixation performance.

How does honeybee behaviour on the trackball compare to their behaviour given similar stimulus conditions when freely walking in the real world? To address this question, we analysed the behaviour of untethered honeybees walking within a Petri dish placed inside our LED arena (Fig. 5B). Bees in this experiment were shown a bright green bar, identical to that used in our trackball experiment, which moved slowly across the LED display. Comparisons with freely walking bees have been previously shown by Paulk et al. (2014), where bees were found to track the bar’s position by maintaining a similar angular orientation in the arena to that of the moving bar. We further analysed the data obtained in the original study, focusing on each bee's walking speed, turning rate and heading direction relative to the green bar, and compared these data with how bees had behaved when fixating in our virtual-reality arena.

When bees were freely walking, they usually headed towards the green bar (Fig. 5A). However, their fixation was not as strong as during virtual-reality experiments, with significantly shorter mean vectors recorded when compared with experiments with either the FT (Fig. 5C; t₁₈=2.30, P=0.033) or CM systems (t₁₈=2.58, P=0.019). We found that bees were capable of making much faster turning motions during free-walking experiments than when walking on the trackball (Fig. 5D,E; free versus FT: t₁₈=4.59, P<0.001, free versus CM: t₁₈=4.89, P<0.001). Yet, the mean free-walking speeds of bees in the dish (Fig. 5F) were not significantly different from those obtained when either the FT system (Fig. 5G; W₁₈=1.14, P=0.255) or the CM system (t₁₈=0.89, P=0.387) was used in virtual reality. Although a substantial increase in the amplitude of turning rate was observed during free-walking experiments, freely walking bees showed only a small, but significant, reduction in their mean motion ratio compared with the virtual-reality experiment in which the FT system was used (t₁₈=2.74, P=0.013), and a larger reduction compared with when the CM system was used (t₁₈=4.27, P<0.001). These results indicate that, in this particular fixation paradigm, freely walking bees generally behave similarly to those in the virtual-reality experiments. However, between paradigms, there were both similarities and differences in the variables we measured (turning rate, walking speed and motion ratio) to describe how bees controlled their motion. Notably, the large increase in the average turning rate of freely walking bees, relative to those in virtual reality, suggests that tethering restricts this aspect of their movement.

Closed-loop virtual reality is an essential paradigm for studying the visuomotor mechanisms that underlie numerous types of behaviour, including learning and memory (Dill et al., 1995), attention (van Swinderen and Greenspan, 2003; Sareen et al., 2011; van Swinderen, 2012), spatial navigation (Harvey et al., 2009), orientation behaviour (Strauss et al., 1997; Schuster et al., 2002) and multimodal integration (Chow et al., 2011). The critical feature of a closed-loop system is that some output from the animal produces a change in the sensory stimulus, mimicking the effect of behaviour in a real environment. In our experiments, in which walking bees manipulated an air-supported ball to control the position of a visual stimulus, we found that the bees' behaviour was influenced by the characteristics of the sensor system that was used to monitor their behaviour and control the visual feedback. Using two separate feedback sensor systems – namely, one incorporating optical motion sensors from computer mice (CM) and another using a novel computer vision approach FicTrac (FT) – we found that the bees behaved differently when fixating a green bar. In comparing these two systems, all other components of the virtual-reality system were identical, except for the different feedback sensors. In general, bees walked both faster and along straighter paths when feedback came from the CM system as compared with the FT system. They also made similar changes when we used FT to replicate a systematic error that is produced by the optical sensors used in CM. In both cases, this reduced the rotational sensitivity of the feedback sensor to the bees' turning rate, which is the behavioural measurement that is used to control the stimulus position. Decreasing the gain of a control system – which is analogous to reducing the sensitivity of the sensor in this case – typically improves stability and controllability (Nise, 2008).

Why do bees walk faster when the CM system provides feedback? By walking faster, bees appeared to exploit a systematic error that is present in CM sensors, which reduces the sensitivity of these devices when they make simultaneous measurements of their turning movement (Moore et al., 2014). Furthermore, whilst the motion-sensing chips used in CM take measurements at a high sample rate, the measured values are surprisingly imprecise when compared with those delivered by FT at a matched sampling rate (Moore et al., 2014). Analogous errors could also be present in other types of sensors that have been used for feedback in virtual-reality paradigms (Xu and Li, 2000; Kröger et al., 2008). Thus, characterizing feedback sensors should be an important consideration for establishing how accurately and reliably an animal's motor output is recorded, and also to establish whether an opportunity exists for the animal to improve its performance by exploiting non-linearities that may be present in the sensing system. Besides sensitivity, other characteristics, such as latency and frequency response, will also vary between sensors used in closed-loop paradigms, and may influence aspects of animal behaviour.

Comparing findings from virtual-reality paradigms with data from freely behaving animals experiencing a similar stimulus is an important step in identifying the validity and limitations of virtual-reality studies where tethering is involved (Maimon et al., 2008; Paulk et al., 2014). Here, we compared the results of experiments in which tethered honeybees walked on a trackball with results from a paradigm in which bees walked freely in a Petri dish whilst exposed to a similar stimulus. We found that bees generally fixated better in virtual reality than when walking freely. This is despite the mismatch in the dynamics of the trackball relative to the dynamics of a honeybee, where the perceived rotational inertia of the ball was approximately 60 times a bee's rotational inertia, whilst the perceived mass was approximately 6 times a bee's mass. Correspondingly, bees walking in the dish made turning motions approximately twice as fast as those on a trackball, but walked at a similar speed. The difference between the trackball dynamics and those of the tethered animal walking on it would be of particular concern to studies focusing on the biomechanics of walking (Weber et al., 1981).

Although bees in either experiment were exposed to a similar stimulus, a fundamental difference between the experiments was that freely walking bees behaved naturally, whereas the tethered paradigm implicitly included operant conditioning as bees learned to interact with the stimulus by walking on the trackball. As operant conditioning includes an element of reward, it is likely to have elicited different motivational states in tethered experiments (Brembs and Heisenberg, 2000). Thus, the findings from free-walking bees indicate how they could be expected to perform in an object-tracking task, but do not necessarily represent an analogous situation to tethered fixation. Additionally, freely walking bees could vary the apparent size of the stimulus by moving towards or away from it, whereas bees in virtual reality always perceived the stimulus to be a constant size. A preferable approach for experiments with a freely moving animal is to update only selected features of the visual environment based on its movement, with all other visual features held constant (Fry et al., 2009; Stowers et al., 2014). Such tight control of the visual stimulus experienced by moving animals can be technically difficult to implement, whereas virtual-reality experiments now offer a relatively simple method of dissecting aspects of behavioural control, based on specific features of the sensory environment. Consideration should be given to the tethering-related artefacts that will be unique to any particular paradigm.

For performing relatively simple tasks in virtual-reality environments, such as visual orientation and stabilization, animals quickly become accustomed to using their behavioural output to control the stimulus (Wolf and Heisenberg, 1991). More complicated tasks, such as navigation, can require training (Harvey et al., 2009). Without any obvious reward, other than improving their fixation, the bees in our study changed their behaviour and improved their control of the stimulus. Previous studies have observed Drosophila to be capable of switching between using thrust force or yaw torque to control a rotational stimulus (Wolf and Heisenberg, 1991), and adjusting the strength of their yaw torque when the magnitude of its influence on the stimulus is changed (Wolf and Heisenberg, 1990). However, our observations of bees fine-tuning behavioural performance based on a relationship between two distinct (although bio-mechanically coupled) motor commands – linear speed to turning rate – does not appear to have been reported for other invertebrates. The ability to make complex motor adaptions would prove advantageous if, for instance, the properties of an insect's method of locomotion were unexpectedly changed, as could occur after damage to its wings (Haas and Cartar, 2008; Dukas and Dukas, 2011) or legs (Götz and Wenking, 1973; Wittlinger et al., 2007). Further studies will be required to determine the temporal dynamics when bees are adapting to changes in coupling (Wolf and Heisenberg, 1990, 1991), and the mechanisms that bees use to detect changes in coupling and choose appropriate motor adaptations. These mechanisms may involve the use of predictive models for the outcome of their actions (von Holst and Mittelstaedt, 1950; Webb, 2004).

We have shown here that, in a virtual-reality system, honeybees can adapt to the feedback sensor in complex ways. This begs the question: if the animal's behaviour in virtual reality is influenced by the accuracy of the sensors, what is the experimenter actually studying? Does the path that an animal takes through a virtual-reality environment resemble the way in which the animal would move through a real space populated with similar stimuli, or do some aspects of the observed behaviours result from the uniquely artificial set up? Our findings highlight the importance of using precise and well-characterized equipment in experiments that employ closed-loop, virtual-reality systems to study animal behaviour.

Honeybees (Apis mellifera Linnaeus 1758) were captured exiting their hive at an experimental bee flight facility at the University of Queensland in Brisbane, Australia. Bees were cold anaesthetized and tethered to thin metal rods using wax and a cautery tool (Bovie). The heads of the bees were fixed to the thorax using dental cement (Coltene Whaledent synergy D6 FLOW A3.5/B3) to ensure that the bees responded to the visual stimulus by manipulating the trackball, rather than making compensatory head movements. A previous study has shown that the fixation behaviour of tethered bees is not altered by this procedure (Paulk et al., 2014). Liquid wax was also applied to the base of the wings, to prevent beating of the wings and encourage walking behaviour. Before commencing the experiments, the tethered bees were fed sucrose solution and then placed in a humidified chamber (∼35°C) for at least 1 h to allow acclimatization to the tether.

The virtual-reality arena consisted of a visual stimulus and a motion-compensating trackball supported on a cushion of air. The visual stimulus was generated using four 32×32 tri-colour LED panels positioned panoramically around the tethered honeybee (Shenzchen Sinorad Medical Electronics Inc.; Zhou et al., 2012), covering 360 deg of the azimuthal and 54 deg of the vertical field of view. A green bar, 54 deg high and 20 deg wide was displayed to the tethered honeybee as the fixation stimulus. The peak wavelength and luminance (from a bee's location) for a single green bar were measured as 518 nm and 168 lx, respectively. The angular orientation of the bar in the arena was controlled in closed-loop by monitoring the rotation of the trackball about its vertical axis, and using this signal to control the angular position of the bar in the visual panorama (Moore et al., 2014; Paulk et al., 2014). A Python script using Vision Egg software (Straw, 2008) updated the visual stimulus at 200 Hz (Fig. 1A).

The trackball was a 50 mm diameter Styrofoam ball (mass, 1.4 g) that was supported on a low friction air cushion and constrained in a custom-designed 3D printed mould. A manipulator was used to position a tethered honeybee above the centre of the ball in a natural walking posture. The dynamics of the trackball were not well matched for a honeybee [(the approximate perceived mass of the ball was 580 mg (honeybee mass ∼100 mg) and its perceived rotational inertia was 330 nkg m² (honeybee inertia ∼5 nkg m²) – calculated based on the methods of Weber et al. (1981)]. An irregular black-and-white pattern, created on the ball using a felt-tipped pen, facilitated tracking of the motion of the ball by means of a video camera, as described below.

We used two methods to measure the walking motions of the tethered bee: (i) by tracking the movement of the ball with CM sensors or (ii) by filming the movement of the ball and computing its rotations by analysing the movie using a program called FicTrac (FT). The measurements from either the CM or FT systems, representing the position of the ball, were fed back to a Python script that updated the angular position of the visual stimulus.

The first method involved the use of sensor chips from commercial computer mice, in which a proprietary optoelectronic device measures the motion of the visual texture on the trackball. Our implementation used two laser mice (Logitech MX400; sensing chips: Avago ADNS7050) aligned orthogonally to each another, with each chip measuring the motion of the texture along two perpendicular directions (Fig. 1B; note that M1_x and M2_x are perpendicular to the plane of the paper). The CM sensors were placed approximately 3 mm from the surface of the Styrofoam ball. The measurements from the two sensors were used to determine the rotation and translation rates of the ball in the bee's frame of reference, based on the two measurements from each mouse (Fig. 1B). A custom-written C++ script sampled the accumulated movement of the CM sensors at 100 Hz on a computer running Windows XP. A non-standard driver, CPNMouse (Westergaard, 2002), was used to interface directly with the mice, bypassing the standard Windows USB interface. This system had a closed-loop latency of approximately 46 ms from the motion of the ball to the movement of the stimulus. This design is comparable to other systems that have been used to study walking behaviour in insects (fly, Clark et al., 2011; cockroach, Takalo et al., 2012; honeybee, Paulk et al., 2014).

The second method tracked the motion of the ball from video images in real time (Moore et al., 2014). Images were recorded using a Firewire camera (Point Grey, Firefly), which recorded images of the ball at 30 frames s⁻¹ in greyscale on a computer running Ubuntu 12.04. FT then calculated the orientation of the ball with a closed-loop latency of approximately 87 ms. A webcam (Logitech Pro 9000) was used to film the ball when using the CM sensors in closed-loop. The video was analysed offline with FT. Data from both cameras were time stamped along with the visual presentation program for synchronization with the CM sensor data. In all cases, the shutter speed, camera gain and brightness were set such that sharp images of the pattern were obtained, even when the bees were walking quickly. FT provided a direct measurement of the turning rate and walking speed in an insect's reference frame (Moore et al., 2014), so a coordinate transformation was not required. The relatively long closed-loop latency was a limitation for both measurement methods, as the visual stabilization reflexes of some insects, such as flies, have latencies of the order of 30 ms (Warzecha and Egelhaaf, 2000).

All experiments were conducted in closed-loop. When the bee rotated the ball about the vertical axis, the visual display rotated the fixation stimulus (a green bar) around the bee by the same angle, but in the opposite direction. This represented a feedback gain of ×1, which corresponded to walking under natural conditions. During each experiment, which ran for 2 min, bees would bring the green bar to their frontal visual field and fixate the stimulus. To ensure that bees were attempting to control the visual cue, brief open-loop displacements were introduced, in which the bar was displaced abruptly by 90 deg in either direction (at random time intervals between 3 and 15 s). This ensured that bees were paying attention to the bar and compensating for its abrupt displacements (Heisenberg and Wolf, 1984; Moore et al., 2014; Paulk et al., 2014). These test periods were excluded from the analysis of the data. Eleven bees were tested using both sensor systems in a random order, and usually walked on the trackball for several minutes between trials.

In the above equations, walking speed is measured in mm s⁻¹ of the ball's movement in a bee's reference frame; graphical representations of these functions are shown in Fig. 6B. The CM system that we used displayed an approximately linear decrease of turning sensitivity in relation to walking speed (Fig. 6A). The decrease condition was intended to approximate this relationship, whereas the increase condition aimed to approximate the opposite relationship. However, the gradient of the observed sensitivity change was set to double that observed for the CM system, such that the bees would easily be able to fully influence the sensitivity of the system within their usual walking speed range. The bounds of the gain were limited to 0.5 and 1 in the decrease and increase conditions, respectively, as bees appeared to have difficulty controlling the stimulus outside this sensitivity range. In this experiment, two separate groups of 10 bees were first exposed to the pre-test control, where the sensitivity was a constant value of 1, and were then tested in either the increase or decrease conditions shortly afterwards.

To obtain a comparison of how the metrics measured from the FT and CM systems compared to how freely walking bees behaved under similar experimental conditions, we further analysed behavioural data originally presented by Paulk et al. (2014). In these experiments, bees were placed inside a Petri dish (of 145 mm diameter and 20 mm height) lined with filter paper, within the LED arena used in this study (Fig. 5B). Nine bees were exposed to an identical stimulus to that used in trackball experiments, which slowly rotated clockwise at a speed of 6 deg s⁻¹ for 2 min. The stimulus in this experiment moved in open-loop; thus, it represented an independently moving object in the world. Note that as bees could move towards or away from the stimulus during these experiments, it could subtend between approximately 15 and 130 deg of their visual field, depending on whether the bees were at a far (140 mm distance) or near (5 mm distance) point of the Petri dish. During the experiment, bees could walk freely within the dish, and their movements were filmed using a Logitech 9000 webcam at 30 Hz, from which their position and orientation in the arena could be extracted using a computer vision algorithm described by Paulk et al. (2014). From these experiments, we analysed each bee's instantaneous walking speed, turning rate and heading relative to the stimulus at each recorded video frame, as these were directly analogous to the parameters measured in the virtual-reality experiments. For direct comparison with our trackball experiments, the motion ratio was calculated as if the bee had walked on a ball of 50 mm diameter, although this is essentially an arbitrary value for a freely moving animal.

Data on the orientation of the bar and ball movements as measured by the sensors were synchronously recorded from within the Python script. All data were down-sampled to 30 Hz for analysis, to provide a true comparison between the CM and FT sensors (which had the lowest sampling frequency in the system, 30 Hz). Data analysis was conducted using custom-written MATLAB (The MathWorks Inc.) scripts.

For each bee, the fixation performance was quantified by calculating the frequency distribution of possible bar orientations, mapped over the 360 deg panorama. These are displayed as radial distribution plots, calculated in 2.8 deg bins (which corresponded to the resolution of the LED display). As an additional comparison, a mean vector was determined using the Circular Statistics Toolbox (Berens, 2009) in MATLAB for the location of the bar around the arena for each bee. This vector is a representation of the strength of fixation (vector length) and the mean angle of fixation (direction of the vector). The average fixation directions for all bees in each condition were tested for non-uniformity using Rayleigh's test, and for differences in the mean direction between multiple conditions using the Watson–William's test (Zar, 1999; Berens, 2009).

As a measure of the relationship between the walking and turning behaviour of bees, a unitless ‘motion ratio’ was calculated as w/(w+|ω|), where w and |ω| are the walking speed and absolute turning rate, respectively (both in rad s⁻¹ in the bee's reference frame; note that this makes the w term dependent on the radius of the ball), of the ball's movement measured at each time point. Thus, a motion ratio of 1 indicates that the bee only attempted to translate its position (in any direction), whereas a ratio of 0 means that the bee only adjusted its orientation. An intermediate value indicates that the bee walked along a curvilinear path. Frequency distributions of the turning rate, walking speed and motion ratio were calculated for each trial and area-normalized such that the sum of observed values for each trial equalled 100%. The average frequency distribution for the variable during any condition was then found by averaging the individual distributions for each bee tested in that condition. Turning rate distributions were calculated based on bins of 5 deg s⁻¹ width ranging from −250 to 250 deg s⁻¹, walking speed distributions were calculated based on bins of 1 mm s⁻¹ width ranging from 0 to 100 mm s⁻¹, and motion ratio distributions were calculated based on bins of width 0.01 units ranging between 0 and 1. The parameters recorded for freely walking bees were analysed in the same manner, with the single exception being that turning rate distributions were calculated based on bins of 10 deg s⁻¹ width ranging from −500 to 500 deg s⁻¹ to reflect the increased range of this variable that was observed when the bees moved freely.

The mean values of mean vector length, walking speed, motion ratio and the absolute turning rate, obtained for each bee tested in each condition, were compared statistically. Although it is apparent that the distributions of these parameters were often not normal (i.e. the walking speed and motion ratio in Figs 2 and 4), mean parameters are often used to describe the gross behavioural output of tethered insects tested under different experimental conditions (Seelig et al., 2010; Bahl et al., 2013; Paulk et al., 2014). Hence, we tested for differences between means to provide an indication of what differences in behaviour would probably be reported with trackball experiments using either the FT or CM systems for feedback. All statistical comparisons were made using MATLAB (The MathWorks Inc.). Data were first tested for normality using the Shapiro–Wilk test, and if they were found to be normal, the comparison was made using a t-test. Otherwise, a Wilcoxon rank sum test was used (for which the standardized test statistic is reported). Where the same bees repeated two sets of experiments (comparisons between FT and CM in Figs 1 and 2, and pre-tests and manipulation conditions in Figs 3 and 4), repeated measures tests were used, specifically either a paired samples t-test or a related-sample Wilcoxon signed rank test, depending on the results of normality tests. The results are reported using a t and a W test statistic, respectively (the Rayleigh's test is written with a z test statistic). All statistical tests used a significance level of 0.05. Error bars in all figures show means±1 s.e.m., with asterisks denoting statistically significant differences (*P<0.05, **P<0.01, ***P>0.001).

The authors would like to thank Dean Soccol for fabricating components of the experimental arena, Saul Thurrowgood for programming support, and Yanqiong Zhou for assistance with developing the virtual-reality paradigm.