Lack of prediction for high-temperature exposures enhances Drosophila place learning

An organism's ability to interact with the environment, evaluate choices and produce a behavioral output is crucial for survival. It should not be surprising, therefore, that organisms of varying complexities are able to acquire information from their environment and express memories (e.g. Kandel, 2001; Squire et al., 2004; Zars, 2010a; Zhang et al., 2005). Although associative stages in memory formation are crucial, it is less clear how good or bad experiences unpaired from accurate predictors can influence associative memory formation.

Pre-exposure to appetitive and aversive stimuli can strongly inhibit or enhance later associative learning. Inhibition of associative learning following unpaired exposure to an unconditioned stimulus (US) or reinforcer has been identified in multiple species, including honeybees, rats, dogs and humans (Abramson and Bitterman, 1986; Bennett et al., 1995; Claflin and Buffington, 2006; de Brugada et al., 2004; Myers et al., 2000; Sandoz et al., 2002; Seligman, 1972). By contrast, and for as yet unknown reasons, pre-exposure to some US or reinforcers can also enhance associative learning. For example, fear conditioning in one context can be behaviorally sensitized when rats experience electric shock in a different context (Rau et al., 2005). Furthermore, the effects of the lack of prediction for aversive stimuli in mice and humans – for example, the lack of control of sensory stimuli – result in the stimuli being perceived as more aversive than expected, which, at least in the mouse, leads to activation of the amygdala (Herry et al., 2007; Whalen, 2007). Humans also show that when an individual can control, or even just perceive that they can control, the duration of aversive air puffs there are lower arousal levels, presumably as a result of a decrease in the stress response (Glass et al., 1973). Moreover, unpredictable aversive stimuli have induced a higher degree of anxiety compared with predictable stimuli in experiments with human subjects (Grillon et al., 2004). Thus far, the only clear evidence that reinforcer pre-exposure enhances later performance in associative conditioning in an invertebrate animal comes from Drosophila, where pre-exposure to aversive high temperatures enhances later place learning (Sitaraman et al., 2007). Thus, it seems that the perception of the unpredictability of aversive stimuli can have lasting effects on physiology and behavior and is a property found in widely different animal species; this perception can influence memory formation.

Pre-exposure to high temperature enhances spatial memory formation in an operant learning paradigm called the heat box (Sitaraman et al., 2007). In Drosophila place learning, a fly is free to roam inside a narrow but relatively long chamber. During training, when a fly goes across an invisible midline, the chamber is heated to a predetermined high temperature; when it returns across the midline, the chamber returns to the normally preferred 24°C (Diegelmann et al., 2006; Wustmann et al., 1996; Zars and Zars, 2006; Zars, 2001; Zars, 2009; Zars et al., 2000). The training phase is followed by a test phase in which the chamber is maintained at 24°C, and flies typically continue to avoid the side previously associated with high temperature. Interestingly, flies pre-exposed to an inescapable 41°C environment before conditioning have enhanced memory levels (Sitaraman et al., 2007). This enhancement is evident in a test phase when flies are conditioned with a relatively weak reinforcing temperature. The high-temperature pre-exposure does not alter the ability of flies to sense and avoid high temperatures, suggesting that the pre-exposure effect alters nervous system physiology in a non-peripheral fashion (Sitaraman et al., 2007).

A key experiment that showed that learned helplessness, as an example of an inhibitory pre-exposure effect, did not result from long-term changes that reduce sensory input was a yoking experiment (Seligman, 1972). In a yoking experiment, one animal (the master) is allowed to control the contingency of receiving reward or punishment through behavioral choice. The yoked animal, by contrast, receives rewards or punishment dependent on the animal that is in control. Thus, although both animals receive the same number of, say, electric shocks, the master animal can control the onset and offset of shocks. To the yoked animal, the shocks appear to come at random intervals, and no accurate predictor for the shock can be developed. In the present study, using yoking experiments (Fig. 1), we determined whether it was an exposure to high temperatures per se that enhanced associative place learning in Drosophila or whether the lack of prediction about rising and falling temperatures led to this enhancement.

The Canton S (CS) strain of Drosophila melanogaster (Meigen 1830) was used as wild-type flies. This CS stock has been maintained at the University of Missouri since 2002 and was derived from the CS strain kept in the Department of Genetics and Neurobiology, University of Wuerzburg. The flies were raised on cornmeal/yeast medium at 24°C and 60% relative humidity on a 12 h:12 h light:dark cycle. Flies used were between 2 and 7 days of age and were not anesthetized for the behavioral experiments.

Yoking, learning and thermosensitivity experiments were performed using a heat box (Wustmann et al., 1996; Zars, 2001). In this apparatus, single flies are allowed to walk in a chamber that is lined top and bottom with Peltier elements. The position of the fly was detected by a bar code reader; for conditioning, a computer coordinated rising and falling temperatures with the position of a fly (Zars et al., 2000). In one half of the experiments, high temperatures were associated with the front half of the chamber; in the other half of the experiments, high temperatures were associated with the back half of the chamber.

In yoking experiments, flies in odd-numbered chambers had control of the environment, such that every time they crossed the midline to venture into the punishment-associated zone they received a high-temperature exposure, the temperature rising from 24 to 41°C (identical to direct conditioning experiments) (Putz and Heisenberg, 2002; Zars et al., 2000). The flies in the odd-numbered chambers, termed master flies, control the environment in their own chambers and in the adjacent, even-numbered chamber (i.e. those containing the yoked flies), so that chamber one is paired with chamber two, three with four and so on. Up to eight pair of chambers were used in parallel. The total number of flies tested in each experiment is listed in the figure legends. The yoked flies received exactly the same intensity and sequence of temperature changes as the master flies. The yoked flies, however, had no control of the temperature environment. In the experiments reported here, the number of training sessions varied from one to seven and each was 6 min long. The side of the chamber associated with 41°C alternated with each session. After a 10 min delay, both master and yoked flies were provided with 4 min of training with an environment of 24°C as the preferred temperature and 30°C as the non-preferred temperature (a 24/30°C contingency). A test for memory followed directly afterward for 3 min, when the chamber temperature was set to 24°C.

We used a thermosensitivity assay to test for the ability of flies to sense and avoid a high temperature source (Zars and Zars, 2006; Zars, 2001). These tests used the same chambers as the learning experiments; however, the temperature of each chamber half was manipulated independently of fly behavior. Both chamber halves were initially held at 24°C for 1 min and then one chamber half was warmed to 30°C, then 41°C. An equal number of experiments started with the 41°C half in the front or back of the chamber.

A performance index was used to calculate place preferences during training, memory tests and thermosensitivity tests (Gioia and Zars, 2009; Putz et al., 2004; Putz and Heisenberg, 2002). The performance index was calculated as the time in the punishment-associated chamber half subtracted from the time in the non-punishment-associated chamber half, divided by the total time in a given session (Wustmann et al., 1996). The maximum performance index is 1.0 and indicates perfect avoidance of the chamber-half associated with high temperature. A performance index of zero indicates preference for neither half of the chamber.

Place memory and thermosensitivity scores were tested using non-parametric statistics (Putz et al., 2004; Putz and Heisenberg, 2002). We compared the two groups using a Mann–Whitney U-test and multiple groups using a Kruskal–Wallis ANOVA. STATISTICA software (StatSoft, Tulsa, OK, USA) was used for all tests.

We sought to address whether a difference in exposure time to 41°C enhances place memory levels or whether a lack of accurate predictions for the rising and falling temperatures in the heat-box chambers leads to this enhancement. Yoking experiments were performed to differentiate between these possibilities. In yoking experiments in the heat box, the rise and fall of temperature inside a chamber was controlled by a master fly based on spatial position, as in a typical conditioning experiment. A yoked fly, by contrast, received the same rise and fall in temperatures independent of spatial location, because the temperature change inside the yoked chamber was linked to the behavior of a master fly (Fig. 1). Flies from both master and yoked groups were then conditioned as in typical experiments and tested for place memory. In the first experiment, we conditioned master and yoked flies in five 6-min training sessions with the 24/41°C contingency. We chose these conditions because we know that, in straightforward experiments, 3 min of high-temperature exposure induces a strong pre-exposure effect and 30 min of training should lead to approximately 3 min of exposure to high temperatures (Diegelmann et al., 2006; Sitaraman et al., 2007).

Examination of the behavior of two individual flies revealed coherent avoidance behavior in only the master fly during the yoking phase of the experiment. In the later conditioning phase, when both flies had control of the space/temperature contingency, both flies avoided the chamber half that was associated with high temperature. One master fly avoided the chamber half associated with high temperatures in each of five training sessions (Tr1–5) (Fig. 2A). With a periodic probing of the chamber half associated with high temperature, this master fly avoided the high-temperature contingency within seconds. Furthermore, after a 10 min rest period, when the temperature inside the chamber was maintained at 24°C, this fly was then provided with a 24/30°C place contingency. The master fly avoided the chamber half associated with 30°C, even when the chamber temperature again permanently reverted to 24°C [testing phase (Te)]. The yoked fly in this case showed no coherent avoidance of a chamber half in the first three training sessions (Fig. 2B). In this case, the rise and fall of temperature were under the control of the master fly. In sessions 4 and 5, the yoked fly avoided one half of the chamber over the other, but this was in the opposite pattern of what was expected based on the training protocol of the master fly. Finally, after the rest period, the 24/30°C training contingency led to nearly perfect avoidance of the chamber half associated with 30°C, and this continued in the post-test phase (Te). We also note that the activity of the master and yoked flies appeared to increase at the beginning of some of the training sessions, evident in densely packed position traces. This increased activity perhaps reflects an attempt of these flies to find a behavior or position that would reduce the temperature to preferred levels.

We quantified the amount of avoidance behavior and the after-effect in master and yoked flies. Over the five training sessions, master flies typically avoided the high-temperature associated half of the chamber at a performance index of approximately 0.8 (Fig. 3A). By contrast, the yoked flies displayed no average preference for either chamber half, with performance indices close to zero in all five sessions (Fig. 3B). Calculating the average dwelling times (Dill et al., 1995) for these flies in the chamber halves associated with high and low temperatures reveals that, during each of the training sessions, flies were exposed to the high temperature for about 0.5 min and the low temperature for the remaining 5.5 min (Fig. 3C). Thus, over the entire 30 min, these flies were in the chamber with high temperatures for a total of approximately 3 min (Table 1).

Does the pre-exposure to high temperatures in the yoking experiment enhance later associative place learning? After the yoking experiment, both master and yoked flies were given a 10 min rest and then trained with the 24/30°C contingency. The master flies showed a memory level of approximately 0.2 (Fig. 3D). The memory performance index at this level was similar to memory levels of ‘naïve’ flies held in the chambers at 24°C for short (seconds) to long (tens of minutes) periods of time (Sitaraman et al., 2007; Zars and Zars, 2006). By contrast, the yoked flies had a significantly higher memory score, at nearly twice the level of the master flies. Thus, it seems that the lack of prediction about rising temperatures in yoked flies enhances new place memory formation when accurate predictors can be found.

We next investigated whether the amount of exposure to unpredicted high temperatures would influence later place memory. By altering the number of 6 min training sessions, from one to seven, master and yoked flies were exposed to high temperatures for brief to longer periods of time. Calculation of the mean dwelling times for these flies showed that they were exposed to high temperatures for approximately 0.5 to 3.5 min (Table 1). When tested for place memory using the 24/30°C contingency, a significant difference was evident between the master and yoked flies (Fig. 4), even with just two training sessions. This difference was maintained or increased as the number of training sessions increased to four and seven sessions. Thus, similar to previous findings in the straightforward, high-temperature pre-exposure experiments (Sitaraman et al., 2007), exposure to high temperatures for longer than 1 min promoted later place memory.

Could the exposure to high temperatures during the yoking experiments alter the ability of these flies to sense and avoid high temperatures? Previous experiments have indicated that up to 3 min of exposure to 41°C does not alter thermosensitivity (Sitaraman et al., 2007). We tested thermosensitivity in flies after five training sessions in the yoking experiment. We again found no significant differences in the ability of master and yoked flies to sense and avoid 30 or 41°C temperature sources (Fig. 5). Thus, the changes in memory formation after unpredicted high-temperature exposures alters memory formation in flies without altering the ability to sense high temperatures.

In an attempt to understand the influence of unexpected punishment in Drosophila place learning, we found that unpredicted exposure to aversive high temperature enhanced later tests of place memory. Flies that receive the identical amount and timing of exposure to high temperatures through yoking experiments, i.e. one set of flies with control of the temperature environment through place preference behavior (master flies) and a second group of flies without control (yoked flies), have different memory-formation levels when flies of both groups are later given control of the temperature environment. Thus, it is not the exposure to high temperatures alone that induces the pre-exposure effect. By contrast, the lack of an accurate predictor biases later learning. This uncertainty bias lasts at least 20 min after the unpredictable exposure to high temperatures (present study) (Sitaraman et al., 2007).

We have previously found that Drosophila can rapidly match conditioned behavior and memory performance levels after a switch in reinforcement intensity (Zars and Zars, 2009). That is, flies match avoidance levels during conditioning within 2 min of a switch in the temperature used for conditioning (evident in experiments where the second reinforcing temperature was higher or lower than the initial conditioning parameters). This corresponds to approximately two experiences with a changed reinforcement intensity (Diegelmann et al., 2006). Memory tests after a change in reinforcement intensity resulted in performance levels that dynamically matched the change in reinforcement intensity.

There are similarities in the experimental designs for detecting the uncertainty bias and identifying rapid matching behavior. That is, in both scenarios flies are presented with high temperatures and later tested for place memory with low-intensity reinforcers (i.e. 30 or 33°C). When the high-temperature exposure is unpredicted, memory levels are enhanced (present study) (Sitaraman et al., 2007). When flies have control of the temperature environment in the ‘pre-training’ phase, these flies reduce memory performance levels and match the now lower-intensity reinforcer. How can one reconcile these differences? We propose that when flies experience unpredicted high temperatures, the reinforcing properties of that high temperature are accumulated in a buffered system. Only when accurate predictors are identified for aversive temperatures is this buffered system allowed to discharge, and the reinforcing properties of that system induce higher than predicted memory levels for a given low reinforcement intensity. By contrast, when flies can make accurate predictions about the rise and fall of temperatures, this system does not accumulate information and memory performance levels dynamically match reinforcement intensity. The buffered system is expected to be downstream of the sensory afferent pathway because thermosensitivity is not altered in pre-exposed flies. It could be part of the reinforcement system (Sitaraman et al., 2008; Sitaraman et al., 2010), the learning process (Diegelmann et al., 2006; Gioia and Zars, 2009; Laferriere et al., 2008; Putz et al., 2004; Zars, 2010a; Zars, 2010b; Zars et al., 2000) or another parallel pathway. Systems-level investigation should elucidate the parts of the nervous system that support both matching and the uncertainty bias.