Honey bees can store and retrieve independent memory traces after complex experiences that combine appetitive and aversive associations

Animals gather and integrate information from the environment to make appropriate decisions (Tinbergen, 1951). In this process, the real value of a detected signal must be computed in the context of concurrent cues that may reinforce the meaning of the first one or contradict it and demand different actions (Lewis et al., 2015). Furthermore, the relevance of different signals can change depending on the internal state of the animal. An animal that has eaten loses interest in food signals (Yapici et al., 2016) or the effort to follow a sex-pheromone plume vanishes after mating (Zhang et al., 2016). Thus, stimuli that are important at a given moment can be irrelevant minutes later or in a different context. How are all these external and internal stimuli integrated to drive adaptive decisions constitutes fundamental questions in neurobiology (Davis, 1979; Sugrue et al., 2005).

Classic experimental approaches to study learning and memory involve in general single appetitive or aversive associations. A conditioned stimulus or action is associated with a reward or a punishment and the changes in behaviour can be linked to this single association. Thanks to such clear approaches, the instructive role that distinct neural pathways and biogenic amines play in mediating aversive and appetitive learning has been successfully studied in vertebrates and invertebrates (Aso and Rubin, 2016; Beyeler et al., 2018; Burke et al., 2012; Farooqui et al., 2003; Iordanova et al., 2021; Kaczer and Maldonado, 2009; Klappenbach et al., 2012; Knapska et al., 2006; Mirenowicz and Schultz, 1996; Mizunami and Matsumoto, 2017; Totani et al., 2019; Vergoz et al., 2007). Since then, the question of how appetitive and aversive stimuli, which often appear mixed in realistic experiences, interact during learning and memory retrieval to coordinate adaptive decisions has spanned a wide range of interests. Thus, several works have begun to study situations in which stimuli and memories of opposite valence compete during learning and retrieval (Bravo-Rivera and Sotres-Bayon, 2020; Das et al., 2014; Felsenberg et al., 2018; Jacob et al., 2021; Jacob and Waddell, 2020; Kaczer and Maldonado, 2009; Klappenbach et al., 2017; McCurdy et al., 2021; Mustard et al., 2020; Rangel et al., 2008).

Honey bees (Apis mellifera) have been used for decades as a model organism to study learning and memory thanks to their abilities to form visual and olfactory memories, and the possibility of training and testing them in conditions accessible for electrophysiology and calcium imaging (Giurfa and Sandoz, 2012; Rath et al., 2011; Strube-Bloss and Rössler, 2018). The widely used appetitive olfactory conditioning of the proboscis extension response (PER) is based on the association between a neutral odour and a sucrose reward (Bitterman et al., 1983; Takeda, 1961). After appetitive conditioning, honey bees extend the proboscis upon stimulation with the odour that anticipates the sucrose reward. However, different aversive learning paradigms have also been described in restrained honey bees. Olfactory and gustatory stimuli are used as conditioned stimuli to predict electric shocks, heat or non-edible substances. The conditioned responses are the sting extension or the suppression of the proboscis extension (Junca et al., 2014; Rangel et al., 2008; Tedjakumala and Giurfa, 2013; Vergoz et al., 2007; Wright et al., 2010). So far, experiments aimed at conditioning the bees to retract the proboscis when stimulated with an odour were performed in an appetitive context, because they combined sucrose stimulation to induce proboscis extension concomitantly with an aversive stimulus that provokes proboscis retraction (Smith et al., 1991; Wright et al., 2010). The observation that after training the bees do not extend the proboscis when stimulated with the conditioned odour was interpreted as evidence of aversive learning. However, and interestingly, because the experimental design includes competing appetitive and aversive stimuli, the questions remain as to whether honey bees do not extend the proboscis because (i) they form an aversive memory, (ii) no memory is formed when stimuli of opposite valence compete during training or (iii) they formed two opposite memories that compete during retrieval.

We performed a series of experiments aimed at exploring how appetitive and aversive information interact during acquisition and memory retrieval in honey bees. First, we evaluated memory after an aversive conditioning protocol that produces odour-induced withholding of the proboscis extension and does not involve appetitive stimuli during training, so that learning can be interpreted as a classical aversive conditioning. Then, we used this protocol in combination with appetitive conditioning to explore whether honey bees establish separate appetitive and aversive memories during a differential conditioning protocol that intermingles appetitive and aversive conditioning trials. Finally, we asked how appetitive and aversive memories that elicit opposite and mutually exclusive behaviours interact during retrieval. Interestingly, we found that in circumstances in which opposite memories compete, the motivational state of the animal was critical to modulate memory expression.

We performed four independent experiments. Experiment 1 was aimed at evaluating the formation of aversive memory after pairing an odour with the bitter substance quinine. Experiment 2 was aimed at evaluating the formation of appetitive and aversive memories after a differential conditioning protocol. Experiment 3 was aimed at evaluating the ability of animals to perceive the presence of an aversive learned odour embedded in a mixture. Experiment 4 was aimed at evaluating memory expression when appetitive and aversive memory traces compete during retrieval.

Honey bee (Apis mellifera Linnaeus 1758) pollen-foragers were collected at the entrance of two regular hives situated on the campus of the University of Buenos Aires. Bees were captured in glass vials, immobilized by shortly cooling them on ice and then restrained in individual holders. Dental wax was used to fix animal's heads in a way that they could move antennae and the proboscis. After recovery from cooling, bees were fed 5 µl of 1.0 mol l⁻¹ sucrose solution and remained undisturbed until the evening, when they were fed ad libitum. When the experiment lasted more than 1 day, bees were fed 5 µl an hour after the training or testing session, and fed ad libitum in the evening. At the laboratory, bees were kept in a humid box at room temperature (20–24°C) on a 12 h:12 h light:dark cycle. All training and testing sessions were carried out between 10:00 and 13:00 h. Thirty minutes before the first training session, all animals passed an admission test that consisted of touching the antennae with 2.0 mol l⁻¹ sucrose solution. Only animals that showed a rapid and conspicuous extension of the proboscis were used in the experiments. In addition, we discarded from the analysis three honey bees that did not respond to sucrose during the appetitive conditioning sessions. One of them belonged to a quinine treated group (n=54) that was tested 24 h after stimulation with quinine, a second belonged to the quinine treated group (n=50) that was tested 48 h after stimulation with quinine, and a third belonged to a control group (n=56) not stimulated with quinine that was tested 24 h after stimulation with the odour alone. Because of the few cases compared with the number of animals, we can discard that stimulation with quinine might affect responsiveness to sucrose hours or days later.

The odours used were 1-hexanol and acetophenone diluted 1/10 in mineral oil (all reagents from Sigma-Aldrich). Both odours were used in all experiments in a counterbalanced way. A volume of 100 µl of the odour dilution was fresh-loaded into sealed 5 ml glass vials before the experiments. The odour delivery device provided a continuous 500 ml min⁻¹ stream of charcoal-filtered air pointed toward the bee's head. During odour stimulation, a solenoid valve was activated to drive part of the airflow (50 ml min⁻¹) to the vial containing the odorant, and a fraction of the head-space inside the vial was pushed into the main airstream in a mixing chamber before reaching the animal. Odour mixtures were obtained by activating the airflow through two separate vial systems that converged in the mixing chamber. The whole system was designed and controlled to provide the same final volume and speed in the air reaching the bees; thus no change in total airflow was produced during onset or offset of odour stimulation. When a mixture was used, the final concentration of each odour was the same as when it was used alone. Odour stimulation always lasted 4 s. Odours were removed from the training arena by a continuous and gentle exhaust placed 10 cm behind the honey bee.

Honey bees were trained using olfactory conditioning of the PER (Bitterman et al., 1983; Takeda, 1961). Appetitive conditioning was used to test aversive memory and also for training in experiments 2 and 4. In all cases, each appetitive conditioning trial followed the same design. Odour was used as the conditioned stimulus and 2 mol l⁻¹ sucrose solution as the unconditioned stimulus. In each training trial, an animal was individually positioned in the training arena facing toward the odour delivery device that provided a constant stream of filtered air. The bee remained undisturbed in this position for 20 s and then the odour started and lasted 4 s. Three seconds after odour onset, the antennae were touched with a 2 mol l⁻¹ sucrose solution that elicits proboscis extension. Sucrose was manipulated and offered to the bee using a glass Gilmont GS-1200 Micrometer Syringe and a metal needle. When the proboscis was extended, the bee was allowed to lick a droplet of 0.4 µl of solution. Ingestion of this amount of solution never took longer than 4 s. Twenty seconds after the end of the reward, the bee was returned to the rest position outside the training arena until the next trial. In all cases, we used a training protocol that consisted of 5 trials.

The test sessions for appetitive memory in experiment 2 consisted of 4 s of odour presentation without reward. The response of each subject was recorded as positive if the subject extended its proboscis beyond a virtual line between the open mandibles during the stimulation with the odour and before stimulation with sucrose. The proportion of conditioned responses was calculated for each trial as the number of bees that extend the proboscis over the total number of bees.

We used an olfactory conditioning protocol based on quinine as the aversive gustatory unconditioned stimulus (Ayestaran et al., 2010; Wright et al., 2010). Bees were placed in the same training position as used for the appetitive conditioning protocol. Three seconds after odour onset, the antennae were touched with a droplet of a 10 mmol l⁻¹ quinine (quinine hydrochloride, Sigma-Aldrich Q1125) prepared in distilled water. As this normally does not elicit proboscis extension, we gently forced its extension with a needle and touched the proboscis with the solution for 4 s. As for sucrose, the quinine solution was manipulated and offered using a micrometer syringe with a metal needle. In the case of quinine, the volume was indistinct because bees did not ingest it. Twenty seconds after the end of this stimulation, the bee was returned to the rest position until the next trial. The training protocol consisted of 5 trials separated by 10 min intervals. The test for aversive memory consisted of 5 trials of the standard appetitive conditioning protocol described in the previous section (see Appetitive training).

Animals were collected and maintained as explained above. On the day of the experiment, bees were randomly divided in two groups: the ‘quinine group’ underwent an aversive training whereas the ‘control group’ remained undisturbed. Two hours after the end of the session, animals were fed with 1 mol l⁻¹ sucrose using a microsyringe (Gilmont GS-1200) until the stimulation of the antennae did not elicit proboscis extension, measuring the total intake of each bee. This procedure was repeated 24 and 48 h later.

In order to evaluate whether stimulation with quinine affects survival during the hours or days after the experiments, we compared the mortality of bees that were stimulated with quinine with that of bees that were not. Seven out of 90 quinine-stimulated bees died before the 2 h test (8%), while 9 out of 90 bees not stimulated with quinine died during this period (10%). Fifty-nine of 323 quinine-stimulated bees died before the 24 h test (18%), while 71 of 333 bees not stimulated with quinine died in the same period (21%). Finally, 57 of 279 quinine-stimulated bees died before the 48 h test (20%), while 63 of 276 bees not stimulated with quinine died during this period (23%). Thus, stimulation with quinine did not increase mortality. The mortality rates measured in these experiments represent values that are commonly observed in experiments with harnessed honey bees.

With the exception of the appetitive test shown in Fig. 2 (one-way ANOVA and Holm–Šidák multiple comparisons) and the feeding assay [repeated-measures general linear models (RM-GLMs) using experimental groups as fixed factors], data were analyzed by RM-GLMs using experimental groups and trials as fixed factors. When appropriate, Holm–Šidák multiple comparisons test between groups was performed (GraphPad Prism 8). Figures were designed with Inkscape™.

A previous study in honey bees showed that if an odour is presented paired with quinine, and after that it is used as a conditioned stimulus for appetitive conditioning, a retardation in appetitive learning is observed (Ayestaran et al., 2010). This effect was used to evaluate the deterrent nature of quinine and other non-edible substances, and importantly, it implies an association between the odour and the substance. However, whether a stable and odour-specific memory is established remains unexplored. Here, we used this phenomenon to study aversive memory. Initially, we performed the controls to validate whether pairing an odour with quinine induces the formation of an associative memory. The experiment consisted of four groups of bees (Fig. 1A). All of them underwent two sessions. The first session was different for each group. The bees in the first group were placed in the training position but did not receive olfactory or gustatory stimuli (placement group). The bees in the second group received odour presentations and no gustatory stimulus (odour only group). The bees in the third group were trained using an explicitly unpaired protocol. They received intermingled presentations of odour or quinine separated by 5 min intervals (odour–quinine group). Finally, the bees in the fourth group received odour presentation paired with quinine (odour+quinine group). Because bees did normally not extend the proboscis in response to quinine, trials with quinine were made touching the antenna with the droplet of quinine solution and then gently extending the proboscis with a needle to touch its tip with the solution. We did not observe the animals ingesting the quinine solution. The second session was performed 2, 24 and 48 h after training (Fig. 1B). The protocol was the same for all groups and consisted of 5 training trials using the same odour used in the first session, but this time it was paired with sucrose. The existence of aversive olfactory memory should become evident as a delay or impairment in appetitive learning (Rescorla, 1969). The performance of the odour+quinine group was lower than the placement and odour only groups 2 and 24 h but not 48 h after the first training (Fig. 2B, left panel 2 h: Time: F_3.05,489=107, P<0.001; Group: F_3,160=15.9, P<0.001; Time×Group: F_12,640=6.49; P<0.001; contrasts: Aq versus A, t_1,360=5.32, P<0.001; Aq versus P, t_1,368=5.85, P<0.001; A-q versus A, t_1,398=4.83, P<0.001; A-q versus P, t_1,402=4.41, P<0.001; A versus P, t_1,403=0.43, P=0.670; middle panel 24 h: Time: F_2.50,410=113, P<0.001; Group: F_3,164=7.06, P<0.001 Time×Group: F_12,656=4.12; P<0.001; contrasts: Aq versus A, t_1,399=6.46, P<0.001; Aq versus P, t_1,350=5.53, P<0.001; A-q versus A, t_1,417=0.547, P=0.783; A-q versus P, t_1,391=1.15, P=0.582; A versus P, t_1,398=0.622, P=0.783; right panel 48 h: Time: F_2.25,303=194, P<0.001; Group: F_3,135=1.87, P=0.138; Time×Group: F_12,540=1.25, P=0.247). This result indicates that memory lasts between 24 and 48 h. The placement and odour only groups showed similar performances, which is consistent with a previous report that showed that at least eight unrewarded exposures to an odour are necessary to produce an evident latent inhibition (Chandra et al., 2010). An interesting result that ruled out the possibility that the low performance of the odour+quinine group was due to toxicity or aversive sensitization is that the unpaired group showed facilitation of appetitive learning at 2 h. This effect can be explained by a positive valence or a higher salience of the odour, once the animals have learnt that the odour signals trials without the negative reinforcement (Yarali et al., 2008).

Despite the fact that the bees did not appear to ingest the quinine solution used as aversive stimulus, we performed a control experiment to explore whether the low performance during the second training session could be due to quinine toxicity, as was previously reported when honey bees ingest larger amounts of it (Ayestaran et al., 2010). We measured the amount of sucrose that harnessed bees ingested ad libitum after a training session with quinine and compared it with a control group that was handled in parallel but did not receive aversive training. Fig. 1C shows that training with quinine did not affect the ingestion 2, 24 or 48 h after training (Time: F_1.96,70.6=2.62, P=0.081; Group: F_1,36=0.082, P=0.776; Time×Group: F_2,72=0.634, P=0.534).

We evaluated the ability of bees to form appetitive and aversive memories acquired by intermingled positive and negative training trials. The protocol constitutes a differential conditioning in which an odour is associated with positive reinforcement and a second odour is associated with negative reinforcement. This kind of protocol has been used to study the ability to discriminate odours without focusing on the formation of appetitive and aversive memories (Avarguès-Weber et al., 2010; Getz et al., 1986; Getz and Smith, 1987; Wright et al., 2009, 2008). The experiment consisted of four groups of bees that differed in the first session (Fig. 2A). The first group received intermingled trials of odours A and B (5 each) in pseudorandom sequence (A/B). The second group received odour B paired with sucrose intermingled with odour A trials (A/B+). The third group received the odour A paired with quinine intermingled with odour B trials (Aq/B). The fourth group received 5 trials of odour A paired with quinine intermingled with 5 trials of odour B paired with sucrose (Aq/B+). Fig. 2B (left panel) shows the learning curves of the groups that received appetitive trials (A/B+ and Aq/B+). Only a minor level of response was observed towards odour A, indicating discriminability and odour-specific learning. Furthermore, the appetitive learning curve toward odour B in the Aq/B+ was almost as steep as in the A/B+ group despite the intermingled trials with quinine (analysis of training to odour B: Time: F_3.26,209=84.0, P<0.001; Group: F_1,64=0.87, P=0.354; Time×Group: F_4,256=1.20, P=0.312). Memory retention was tested 24 h after training. Appetitive memory was tested first with a single trial with odour B (Fig. 2B, middle panel). Both groups that had training protocols the day before without appetitive trials (A/B and Aq/B) did not show proboscis extension when tested with odour B. In contrast, appetitive memory was clear in bees in the A/B+ and Aq/B+ groups. A slight but significant impairment was observed in the double-trained group, which is consistent with the trend observed along the conditioning curve (F_3,139=53.62, P<0.001; Holm–Šidák contrast: A/B+ versus Aq/B+: t₁₃₉=3.23, P<0.01). Subsequently, all bees underwent a second training session using odour A paired with a sucrose reward, which served to assess aversive memory (Fig. 2B, right). Strikingly, no response was observed to odour A in the first trial, which argues in favour of the animals' ability to discriminate the odours. The groups that had aversive training (Aq/B and Aq/B+) showed reduced learning curves in comparison with the non-aversive trained groups (A/B+ and A/B). Interestingly, aversive memory in the double-trained group (Aq/B+) was also not as strong as in the only aversive trained group (Aq/B), thus mirroring the lower performance in appetitive memory of the same group (Time: F_2.20,305=104, P<0.001; Group: F_3,139=5.43, P<0.01; Time×Group: F_12,556=2.83, P<0.001; Holm–Šidák contrasts: A/B versus Aq/B: t_1,380=5.07, P<0.001; A/B+ versus Aq/B+: t_1,323=3.77, P<0.001; Aq/B versus Aq/B+: t_1,340=2.04, P=0.083). In conclusion, the fact that the Aq/B+ group shows odour-specific appetitive and aversive memories demonstrates that bees are able to form and express two memories of opposite valence acquired during the same training session.

A second set of bees was tested 48 h after the training session (Fig. 2C, left panel, analysis of training with odour B: Time: F_3.45,179=59.3, P<0.001; Group: F_1,52=0.57, P=0.453; Time×Group: F_4,208=1.52, P=0.198). This time, we did not find any difference in appetitive memory in groups A/B+ and Aq/B+ (Fig. 2C, middle panel, F_3,100=37.04, P<0.001; Holm–Šidák contrast: A/B+ versus Aq/B+: t₁₀₀=0.017, P=0.99). Moreover, no difference was observed among groups during the second training session with odour A (Fig. 2C, left panel, Time: F_2.25,234=91.3, P<0.001; Group: F_3,104=0.459, P=0.712; Time×Group: F_12,416=0.91, P=0.538). This lack of aversive memory in the Aq/B and Aq/B+ groups is consistent with the results obtained in experiment 1 that circumscribed the duration of this aversive memory to less than 48 h. Finally, we interpret that the slight impairment observed in appetitive memory 24 h after double training can be explained as interference during retrieval rather than during acquisition or memory storage. This conclusion is indicated by the fact that expression of appetitive memory was restored to the level observed in the A/B+ group once aversive memory has vanished. Furthermore, the fact that appetite for food is not affected after aversive training supports that the different outcomes 24 and 48 h after training are not explained by a delayed recovery from any malaise caused by quinine (Fig. 1C).

Honey bees are able to detect appetitive conditioned odours embedded in complex mixtures (Locatelli et al., 2013; Reinhard et al., 2010; Schubert et al., 2015). Here, we evaluated whether they are also able to detect aversive learned odours. We trained animals using the aversive conditioning protocol and evaluated aversive memory using a binary mixture that contained the learned and a novel odour. During the first day, one group of bees was exposed to 5 presentations of odour with no unconditioned stimulus (group A) and a second group received 5 trials of the odour paired with quinine (group Aq) (Fig. 3A). Fig. 3B shows the learning curves during the second training session performed 24 h after aversive training. The lower performance of the Aq group is consistent with aversive memory and the ability of bees to detect the learned component embedded in the mixture (Trial: F_2.53,132=33.2, P<0.001; Group: F_1,52=5.97, P<0.05; Trial×Group: F_4,208=3.01, P<0.05). Finally, as expected based on the duration of this memory, performance was not different among groups 48 h after aversive training (Fig. 3C, Trial: F_2.64,153=52, P<0.001; Group: F_1,58=0.991, P=0.324; Trial×Group: F_4,232=0.52, P=0.724).

We asked how honey bees perceive a binary mixture that contains appetitive and aversive learned odours. Fig. 4A depicts the training and testing protocol. The first training session consisted of the same four groups as in experiment 2: A/B, A/B+, Aq/B and Aq/B+. Fig. 4B (left panel) shows the training curves of the two groups that had appetitive conditioning trials during the first training. Both groups (A/B+ and Aq/B+) show steep learning curves towards odour B, and only minimal response levels towards odour A (analysis of training to odour B: Time: F_3.29,201=59.5, P<0.001; Group: F_1,61=2.78, P=0.101; Time×Group: F_4,244=1.14, P=0.340). During the second session (Fig. 4B, right panel), all bees were trained again using a binary mixture of the odours A and B. The untrained group (A/B) shows a standard acquisition curve, i.e. no bee responded to the odour during the first trial and 60% of them responded in the fifth trial. The Aq/B group showed a reduced acquisition curve, which is consistent with aversive memory. The A/B+ group showed a high response from the first trial of the second training session, which is consistent with appetitive memory. Surprisingly, the performance of the Aq/B+ group did not differ from that of the A/B+ group (blue and magenta) behaving as expressing appetitive memory [Time: F_1.98,244=39.1, P<0.001; Group: F_3,123=12.9, P<0.001; Time×Group: F_12,492=5.32, P<0.001; Holm–Šidák contrasts: A/B versus Aq/B: t_1,235=5.90, P<0.001 (aversive memory); A/B+ versus Aq/B+: t_1,250=0.52, P=0.601 (appetitive memory)]. Because we knew based on experiment 2 that honey bees are able to establish both memories after double training, we hypothesized that expression of appetitive memory could be occluding aversive memory, and if so, the expression of the latter could be recovered if the motivation to express appetitive memory were partially reduced. Therefore, we repeated the experiment, but 2 h before the second training, animals were fed with 1.5 μl of 1 mol l⁻¹ sucrose solution, which represents approximately 10% of the amount that bees would eat to repletion (see Fig. 1C). Fig. 4C (right panel) corresponds to the second training session. Bees in the A/B group (grey) showed a regular acquisition curve. In the A/B+ group (blue), 60% of the bees started responding from the first trial of the second training session, a response level that is consistent with appetitive memory. The performance of these two groups is important, as this shows that the amount of feeding did not affect appetitive behaviour (learning or retrieval). Interestingly, this time the performance of the Aq/B+ group (magenta) was significantly lower than that of the A/B+ group (blue). This change is explained by a number of bees that would have expressed appetitive memory, but because they were partially fed, they switched from expressing appetitive to expressing aversive memory. This is further supported by the fact that the bees in this group did not change their decision along the whole test session. In summary, feeding the animals unveiled aversive memory and provided evidence that appetitive and aversive memory traces are ready to be expressed depending on the internal state [Fig. 4C, left panel (first training): Time: F_3.08,145=56.5, P<0.001; Group: F_1,47=1.14, P=0.292; Time×Group: F_4,188=1.17, P=0.327; right panel (second training): Time: F_3.49,328=38.4, P<0.001; Group: F_3,94=32.8, P<0.001; Time×Group: F_12,376=9.32, P<0.001; contrasts: A/B versus Aq/B: t_1,237=6.10, P<0.001 (aversive memory); A/B+ versus Aq/B+: t_1,252=6.02, P<0.001 (aversive memory)].

In experiments 2 and 4, we concluded that honey bees are able to form appetitive and aversive memories acquired during the same training session. However, this interpretation is based on the population's behaviour, which might not represent the individual performance (Pamir et al., 2011). It might happen that a fraction of bees expressed aversive memory and a non-overlapping fraction expressed appetitive memory. Therefore, we re-analyzed the data based on individual performance to evaluate whether each animal has the capacity to form and express both memories after double training.

In experiment 2, animals were first tested for appetitive memory using one trial with odour B. Bees that extended the proboscis upon stimulation with the odour were considered as individuals expressing appetitive memory. Immediately after that, bees were trained with five appetitive conditioning trials using odour A to test aversive memory. We considered bees that did not extend the proboscis in any of the five trials as individuals expressing aversive memory. Based on these criteria, each bee of the Aq/B+ group (n=34) was assigned into one of four possible categories: bees that did not express any memory (12%), bees that expressed only appetitive memory (35%), bees that expressed only aversive memory (35%) and bees that expressed both (18%) (Fig. 5A). The first conclusion is that both memories are not mutually exclusive as is shown by the proportion of bees that managed to form and express both memories. However, this does not tell us whether both memories are independent. Notice that based on the classification used, the actual proportion of bees that showed appetitive memory was 35% (appetitive)+18% (both)=53%, and the proportion of bees that showed aversive memory was 35% (aversive)+18% (both)=53%. Thus, if appetitive and aversive memories were independent, the proportion of bees showing aversive memory from the population of bees that show appetitive should be 53% of the 53%, which gives 28% of the total number of bees. Instead, the bees that showed both memories were only 18%, which provides a distribution different to one expected if both memories were independent (χ² observed versus expected, d.f.=1, P=0.015) and points to a certain degree of interference. This interference is consistent with the observation made in experiment 2 that appetitive and aversive memories in the double-trained group were reduced (see Fig. 2). A relevant question was whether this interference occurs during acquisition or during memory retrieval. When the same analysis was extended to the 48 h test, the proportion of bees that showed aversive memory dropped from 53% 24 h after training to 33% 48 h after training, while the bees that expressed appetitive memory increased from 53% to 74% (Fig. 5B). Thus, the fact that the number of bees expressing appetitive memory increases concomitantly with a reduction in the number of bees expressing aversive memory suggests that the interference measured 24 h after training occurred during memory retrieval, rather than during its acquisition or consolidation.

In experiment 4, we cannot determine whether an animal has both memories, because the ways to express them are mutually exclusive. Thus, we counted the bees that extended the proboscis upon the first stimulation with the mixture AB as animals expressing appetitive memory, and bees that did not extend the proboscis in any of the five trials as bees expressing aversive memory. This classification showed that the proportion of bees showing appetitive memory in the double-trained group was 58% before feeding and 28% after feeding, while bees showing aversive memory were 26% before feeding and 54% after feeding (Fig. 5C). Thus, the effect of feeding provided evidence that aversive memory was present in the double-trained bees but remained occluded by the expression of appetitive memory, which in turn implies that bees had formed both memories.

In the classic appetitive olfactory conditioning of the PER in honey bees, a neutral odour is paired with sucrose solution applied to the antennae and proboscis. Once the association is established, the sole stimulation with the conditioned odour causes extension of the proboscis (Bitterman et al., 1983; Takeda, 1961). However, aversive olfactory conditioning experiments conceived to produce odour-induced suppression of the proboscis extension, were based on eliciting first the proboscis extension with sugar, and in this appetitive context, signaling the occurrence of an electric shock or quinine with an odour (Smith et al., 1991; Wright et al., 2010). Here, we used a different strategy that does not need stimulation with sucrose during aversive training. A neutral odour is presented paired with quinine. No evidence of learning can be measured during training. Whether a memory was built can be measured later, during a second training session in which the same odour is paired with sucrose. This second session constitutes a reversal learning protocol in which animals assign the odour a value that is opposite to the one learned before, and thus the appetitive learning curve is affected (Devaud et al., 2007; Hadar and Menzel, 2010; Peck and Bouton, 1990). In the study by Ayestaran et al. (2010), this phenomenon was used to evaluate the deterrent nature of quinine and other bitter or salty substances. In the present work, we used it to study aversive memory, and determined that a training protocol of five spaced trials of a neutral odour paired with quinine induces the formation of a memory that lasts between 24 and 48 h. Because bees did not ingest the quinine solution during training trials, we conclude that in the present case, aversive reinforcement relies on the gustatory modality upon touching receptors on the proboscis and/or antennae. However, because we cannot discard that minute amounts of quinine might have been ingested, pre- and post-ingestive pathways might have also signalled the negative reinforcement (Wright et al., 2010).

Interestingly, explicitly unpaired presentations of odour and quinine facilitated subsequent appetitive learning of the same odour (Fig. 1B, 2 h). This is important for two reasons. First, it rules out that the retardation in appetitive learning after a treatment with quinine is due to toxicity or malaise (Hurst et al., 2014). If this were the case, quinine should affect appetitive learning regardless of whether it was applied paired or unpaired. Second, it reinforces the conclusion that this training protocol induces associative learning and not a quinine-induced sensitization that produces unspecific suppression of the proboscis extension. Interestingly, the consequence of unpaired conditioned–unconditioned stimulus presentations resembles the observation reported by Bitterman et al. (1983) that unpaired stimulations with odour and sugar produced a conditioned inhibition that affected subsequent appetitive learning of the same odour. Furthermore, the possibility to convert an aversive learning protocol into an appetitive one by altering the timing between the conditioned stimulus and an aversive unconditioned stimulus has been studied and compared across humans, rats and flies (Andreatta et al., 2012). In flies, presenting a neutral odour shortly after a negative reinforcement provides this odour a positive valence, likely because, as in relief learning, the odour signals that the shock has finished (Aso and Rubin, 2016; König et al., 2018; Vogt et al., 2015; Yarali and Gerber, 2010). Furthermore, in the context of differential aversive conditioning, it turned out that in addition to aversive memory, flies establish a complementary safety memory that is expressed as preference toward the control odour that was presented unpaired to the shock (Jacob and Waddell, 2020).

The expression of the memory that we describe here is akin to latent inhibition, i.e. an odour-specific retardation of acquisition in a subsequent appetitive conditioning (Chandra et al., 2010). This similarity rasies the question of whether latent inhibition constitutes a sort of aversive learning, as supported by studies in flies (e.g. Jacob et al., 2021), or whether paired exposures to odour and quinine do actually accelerate latent inhibition, which normally requires at least eight exposures to the odour (Chandra et al., 2010). Previous studies in honey bees ruled out the first interpretation by showing that unrewarded odour exposures that produce latent inhibition do not convert the odour into a conditioned inhibitor (Chandra et al., 2010; Fernández et al., 2012). Regarding the second possibility, we obtained different results depending on how memory was evaluated. In experiment 1, the animals were tested with the odour that was used for aversive training, and the performance is compatible with both aversive memory and latent inhibition. In experiment 3, the odour that had been paired with quinine was presented during the second training session together with a novel odour. The observed inhibition in appetitive learning is consistent with a negative value of the odour rather than with ignoring it. Finally, in experiment 4, the aversive learned odour was presented during the second training session together with an appetitive learned odour. This time, honey bees behaved as if ignoring or avoiding the aversive learned odour depending on their satiation level, which would be consistent with latent inhibition in the first case and with aversive memory in the second. These results prompt further mechanistic studies to understand whether the ability to behave as if ignoring an aversive stimulus is the result of reducing its negative value or accepting its inherent risk.

One of our main objectives was to study the ability of animals to extract and use information from experiences that mix appetitive and aversive associations. Thus, we asked whether honey bees are able to form appetitive and aversive memories during a differential conditioning session. A previous study showed that honey bees can learn that a given odour predicts an electric shock and a different one predicts sugar, both presented in the same training session (Vergoz et al., 2007). Here, we have challenged the bees to a more difficult task, because both appetitive and aversive unconditioned stimuli involve the gustatory modality and, furthermore, the expression of both memories requires the proboscis extension in one case and its suppression in the other. Interestingly, bees in the double-trained group behaved as having formed both memories without mixing the learned value of each odour. Several observations lead us to conclude that appetitive and aversive memories are independently acquired and that they compete during expression. First, we observed a slight reduction of the appetitive learning curve during the training session of the double-trained groups. Second, during the memory test 24 h after training, aversive and appetitive memories in the double-trained group were significantly reduced compared with the single-trained groups. Up to that point, a possible interpretation could be a mutual interference during memory consolidation. However, we observed in experiment 2 that appetitive memory was fully recovered when aversive memory vanishes 48 h after training. A similar effect was observed in experiment 4, in which expression of aversive memory was recovered when honey bees were partially fed to reduce appetitive motivation. Thus, the fact that expression of appetitive or aversive memories is restored when the opposite one reduces its expression indicates that both memory traces are established after double training, and that competition exists during memory expression. Then, if the reduction in memory expression is caused by the fact that both memories cannot be expressed at the same time, why was this effect also observed when the tests with odours A and B were split (experiment 2, 24 h test)? A certain weight must be conceded to the training context as part of the conditioned stimulus. Appetitive and aversive trainings take place in the same visual and mechanosensory context that may also become a predictor of sugar and quinine. It is reasonable to expect that the expression of olfactory memory competes with context memory and affects performance (Gerber and Menzel, 2000).

The fact that honey bees in the double-trained group managed to establish a specific predictive value of each odour highlights the independence of the appetitive and aversive pathways involved in learning and memory formation. A number of studies have shown that the biogenic amine octopamine provides the internal signal for appetitive learning, whereas dopamine and serotonin are necessary for aversive learning (Farooqui et al., 2003; Hammer and Menzel, 1998; Lai et al., 2020; Vergoz et al., 2007; Wright et al., 2010). It must be considered that the time interval between appetitive and aversive training trials might have contributed to the conditioned–unconditioned stimulus specificity and the lack of interference during acquisition. Based on previous pharmacological studies, it is expected that if appetitive and aversive unconditioned stimuli occur simultaneously or close in time, interference should be expected during acquisition. Indeed, it has been shown that administration of octopamine during aversive learning affects aversive memory formation (Agarwal et al., 2011) and administration of dopamine during appetitive learning affects appetitive memory (Klappenbach et al., 2013).

Honey bees are able to find appetitive learned odours embedded in complex mixtures (Chen et al., 2015; Reinhard et al., 2010; Schubert et al., 2015). Here, we provide evidence that honey bees can also react to aversive learned odours embedded in mixtures. We found that aversive learning to an odour interferes with the subsequent appetitive conditioning of a mixture that contains that odour. This ability is particularly relevant in nature, where meaningful odours are immersed in noisy backgrounds (Conchou et al., 2019; Raguso, 2008). The mechanisms by which honey bees, and other animals, detect the presence of key odorants embedded in mixtures are a matter of intense research (Marachlian et al., 2021). Behavioural studies have shown that odour mixtures can be perceived by honey bees in elemental and configural ways (Deisig et al., 2002, 2003; Reinhard et al., 2010; Smith, 1998; Schubert et al., 2015). Physiological studies have also found elemental and configural representations of odour mixtures along the olfactory circuit (Deisig et al., 2006, 2010; Krofczik et al., 2009; Yamagata, 2009). Our results based on generalization of the learned responses from components to mixtures support the view that honey bees are able to recognize the presence of learned elements in binary mixtures, which is consistent with an analytical processing of the mixture. This does not discount that the mixture may also produce a unique or configural perception, while preserving information about the components (Deisig et al., 2003; Lei and Vickers, 2008). Interestingly, the ability to detect a learned odour embedded in a mixture correlates with experience-dependent changes in the representation of the mixtures in the antennal lobe (Marachlian et al., 2021). We have described that mixture representation in the antennal lobe changes depending on the previous experience with the components. The ensemble of projection neurons that encode a mixture is more similar to the ensemble of neurons that encodes rewarded components than the non-rewarded ones (Chen et al., 2015). Moreover, when honey bees learn to ignore an odour after a latent inhibition protocol, a reduction is observed in the contribution of that odour to the representation of a mixture (Andrione et al., 2017; Locatelli et al., 2013). Interestingly, the results obtained in experiment 4 with starved and fed bees suggest that the relative weight that each component has on the perceptual quality of the mixture is not only determined by previous experience, but also tuned by the physiological state of the animal. In line with this possibility, a study in flies has shown that starvation changes internal odour representation in that correlates with more robust food-search behaviour (Root et al., 2011).

We evaluated which valence honey bees assigned to an ambiguous stimulus that would predict appetitive and aversive consequences. We took bees that had undergone double training and tested them with a mixture of the appetitive and aversive learned odours. In this experiment, only 16% of bees showed no memory and the rest behaved as expressing appetitive or aversive memory. Thus, we conclude that mixing appetitive and aversive learned odours does not convert the mixture into a neutral stimulus, which could happen if the memory traces compete in a way in which they cancel each other out. We observed that most of the bees behaved as if reacting to the presence of the appetitive learned odour. However, if they received a minimal amount of food that slightly reduced their starvation level during training with the mixture, a fraction of them changed and behaved as if reacting to the presence of the aversive learned odour; in other words, the bees became more selective. Importantly, the amount of feeding did not affect appetitive behaviour in the A/B+ or in the AB group. If appetitive performance had been affected in these two groups, the effect in the Aq/B+ group could not have been attributed to the expression of aversive memory.

Decision-making in real-life situations must take into account appetitive and aversive consequences assessed in the context of the individual's needs. Food is rewarding in many circumstances, but it can be rejected if it involves unintended consequences, or accepted if there is no other option (Desmedt et al., 2016; Lin et al., 2017). Here, we have set experimental conditions that intend to replicate a realistic situation in which appetitive and aversive memory traces compete. All results are consistent with a model in which both memories are independently stored and can be alternatively expressed depending on the animal's requirements at the moment of retrieval. Interestingly, we reached similar conclusions in previous experiments using crabs as model animal and completely different behavioural tasks (Klappenbach et al., 2017). Results in flies also support that aversive and appetitive experiences linked to the same conditioned stimulus can be stored and retrieved as independent memories (Das et al., 2014; Felsenberg et al., 2018; Perisse et al., 2013), and pointed out that the ability of a conditioned stimulus to elicit appetitive or aversive memories is provided by the balance of inputs that confer information about the internal state (Senapati et al., 2019). The processing of competing appetitive and aversive information and the possibility to flexibly orchestrate adaptive decisions has also been intensively studied in vertebrates (Bravo-Rivera and Sotres-Bayon, 2020). The fact that appetitive and aversive memories are mainly encoded in different brain regions supports the view of separate memory traces that compete during expression. The prefrontal cortex, which receives and sends input from diverse brain areas involved in appetitive and aversive memories, has been implicated in weighing the competing information and guiding adaptive decisions (Sotres-Bayon and Quirk, 2010).

The neural changes that accompany olfactory memory formation in honey bees have been mapped all the way from the sensory neurons and the first olfactory neuropils in the insect brain, to higher integrative brain regions such as the mushroom body calyces and lobes (Chen et al., 2015; Claudianos et al., 2014; Locatelli et al., 2016, 2013; Rath et al., 2011; Strube-Bloss et al., 2011). Future studies will have to address how long-lasting neural changes related with the predictive value of a given odour, and transient changes that readjust the weight of these memories during retrieval, interact along these circuits to ensure adaptive behaviour.

We appreciate the careful reading and helpful comments by two anonymous reviewers.