Surpassing the subitizing threshold: appetitive–aversive conditioning improves discrimination of numerosities in honeybees

Quantity discrimination has been explored in a number of species through different methods (e.g. spontaneous choice versus training; limited versus extensive training), which sometimes yield different results (Agrillo and Bisazza, 2014; DeWind and Brannon, 2012; Gatto et al., 2017; Gazes et al., 2018; Miletto Petrazzini et al., 2018). The ability to discriminate between quantities varies across different species, with insects such as mealworm beetles able to discriminate ratios (calculated by dividing the lower number by the higher number, such as 1 versus 4 and 1 versus 3) of 0.25–0.33 (Carazo et al., 2009). Species that can discriminate the more difficult ratio of 0.80 include monkeys (Addessi et al., 2008; Gazes et al., 2018), jungle crows (Bogale et al., 2011), Shetland ponies (Gabor and Gerken, 2018), Clark's nutcrackers (Tornick et al., 2015) and dolphins (Jaakkola et al., 2005). Some animals including elephants (Irie-Sugimoto et al., 2009), great apes (Hanus and Call, 2007), Mexican jays (Kelly, 2016), western scrub jays (Kelly, 2016) and guppies (Bisazza et al., 2014; Lucon-Xiccato et al., 2017) can discriminate ratios over 0.80. Previous studies on quantity discrimination suggest some variation between low quantities (4 and fewer) and high quantities (4 and greater). For example, frogs (Bombina orientalis) are able to discriminate ratios of 0.67 with 4 items or fewer but only ratios of 0.5 with 4 or more items (Stancher et al., 2015). Discrimination ability can also vary with different number comparisons of the same ratio; for example, North Island robins can discriminate a ratio of 0.875 when comparing quantities of 7 versus 8, but not with quantities of 14 versus 16, despite the two comparisons having the same ratio (Garland et al., 2012).

Several species, including humans, exhibit a numerical competency threshold at 4 (Agrillo et al., 2008; Cowan, 2010; Gómez-Laplaza and Gerlai, 2011a; Jevons, 1871; Kaufman et al., 1949; Simons and Langheinrich, 1982; Trick and Pylyshyn, 1994). This threshold is observed in visual object and tactile stimulation estimation in humans (Dehaene and Cohen, 1994; Jevons, 1871; Piazza et al., 2002; Riggs et al., 2006; Starkey and Cooper, 1995), quantity discrimination in fish (Agrillo et al., 2008; Gómez-Laplaza and Gerlai, 2011a; Seguin and Gerlai, 2017), as well as proto-counting and number-generalization in honeybees (Boysen, 1988; Dacke and Srinivasan, 2008; Gross et al., 2009). The threshold phenomenon is observed across a diverse range of species and may be due to the hypothesis that quantity/numerical discrimination has two mechanisms: ‘subitizing’/object file system (accurately and quickly recognizing 4 or fewer objects) and counting (the process of sequentially incrementing the number of identified elements, theoretically without bound; Agrillo et al., 2008; Kaufman et al., 1949; Piazza et al., 2002; Tomonaga and Matsuzawa, 2002; Trick and Pylyshyn, 1994). Some non-human animal species have been shown to surpass this threshold limit of 4 following mechanisms consistent with Weber's law using the approximate number system (ANS). The ANS is a non-symbolic numerical quantification system used by humans and non-human animals for quantifying numerosities above 4 (Agrillo et al., 2008; DeWind et al., 2015; Feigenson, 2004; Vallortigara, 2017). Weber's law describes how sensory systems distinguish between two stimulus magnitudes based on their proportional difference (Akre and Johnsen, 2014; Fechner, 1965; Weber, 1978). For example, the ability of mosquitofish to discriminate between two groups of conspecifics consisting of more than 4 individuals improves as the numerical distance increases; individuals fail at 4 versus 5 but succeed at 4 versus 8 (Agrillo et al., 2008).

While many studies have explored the numerical abilities of vertebrates, relatively little research has been done on the numerical competency of invertebrates (Carazo et al., 2009; Dacke and Srinivasan, 2008). Studies testing the ability of bees to ‘proto-count’ landmarks in either open natural environments or controlled laboratory conditions found that bees were able to reliably numerically assess up to 4 landmarks. True counting requires the abstraction principle to be fulfilled, where counting should be able to be transferred between different object types (Chittka and Geiger, 1995); thus, counting in honeybees is termed proto-counting. Bees failed at proto-counting more than 4 landmarks in order to find a source of food (Chittka and Geiger, 1995; Dacke and Srinivasan, 2008). In a study using a delayed matching to sample (DMTS) procedure in a Y-maze, bees were able to accurately match specific quantities of up to 3 elements irrespective of shape, colour and pattern of the objects to receive a reward of sucrose, but were unable to do this with higher numbers (Gross et al., 2009). Bees showed a discrimination ability of 2 versus 3 and 3 versus 4 but not 4 versus 5 or 4 versus 6. These studies were important steps in determining that bees could both proto-count and match numbers, and that numerical ability was a biologically meaningful skill for either navigation (Chittka and Geiger, 1995; Dacke and Srinivasan, 2008) or numerically assessing the number of flowers in a flower patch during foraging (Gross et al., 2009).

The previous studies on number matching and proto-counting in honeybees used classic appetitive or appetitive differential conditioning frameworks, where bees received a reward of sucrose for a correct choice and no outcome for an incorrect choice (Chittka and Geiger, 1995; Dacke and Srinivasan, 2008; Gross et al., 2009). Advancements in training protocol have shown that when bees are trained on a perceptually difficult colour task, performance significantly increases when appetitive–aversive differential conditioning is used in training compared with appetitive differential conditioning (Avarguès-Weber et al., 2010; Dyer, 2012). Appetitive–aversive differential conditioning is where a bee receives a reward of sucrose for a correct choice and an aversive outcome of quinine for an incorrect choice, whereas appetitive differential conditioning rewards a bee for a correct choice and gives no outcome for an incorrect choice. There is evidence that including a distractor associated with an aversive outcome also improves the strength of the association between the correct option and the reward in vertebrates (Rescorla and Wagner, 1972), which is potentially linked to attentional resources (Avarguès-Weber et al., 2010; Dyer, 2012; Giurfa, 2004; Rescorla and Wagner, 1972). Recent work on spatial visual processing in honeybees showed that learning and behavioural outcomes can be significantly influenced by the conditioning procedure employed during training (Dyer and Griffiths, 2012; Dyer et al., 2005; Giurfa et al., 1999; Howard et al., 2017; Stach and Giurfa, 2005), and so bees have become an important model species for understanding comparative visual perception and cognition.

Recently, honeybees were shown to learn the rules of ‘less than’ and ‘greater than’ and apply these rules to the novel numerosities of 0 and 5 using appetitive–aversive differential conditioning (Howard et al., 2018). Bees demonstrated an ability to learn to discriminate between all combinations of the numerosities 0 to 6 during training in one of the experiments, a discrimination ability that has not previously been shown in honeybees (Howard et al., 2018). We also recently trained bees to learn to add or subtract 1 element from an array of elements consisting of 1 to 5 elements. Bees demonstrated significant success in novel problems and thus demonstrated simple arithmetic abilities when trained with appetitive–aversive differential conditioning (Howard et al., 2019a). Honeybees have also demonstrated the capacity to learn to match signs (abstract symbols) with quantities when trained with appetitive–aversive differential conditioning (Howard et al., 2019b). These results are probably due to the use of appetitive–aversive differential conditioning during training and thus in this study we formally tested and compared the numerical discrimination ability of bees when trained with appetitive or appetitive–aversive differential conditioning. We determined whether numerical discrimination changes with different conditioning procedures, and thus provide some insights into why different quantity discrimination results are observed across a range of animal taxa.

We aimed to determine whether there was a difference in quantity discrimination performance between bees that were trained using appetitive differential conditioning (group 1) compared with bees trained with appetitive–aversive differential conditioning (group 2). To do this, we trained bees to choose 4 elements when presented against the quantities 1 to 10. We then tested bees on their ability to discriminate between quantities of 4 versus 5, 6, 7 and 8.

We used 22 free-flying honeybees (Apis mellifera Linnaeus 1758) foragers for this experiment. All bees were marked with a coloured dot on the thorax to identify individuals. A gravity feeder was set up within 20 m of the experiment to provide 10–30% sucrose which attracted a regular number of bees for use in experiments. Bees were randomly assigned to test groups. Honeybee foragers were collected from a number of different hives and gravity feeders providing sucrose solution (5–30%), which were set up to maintain visiting foragers. The bee hives were maintained at Paul Sabatier University in Toulouse, France, where there were over 25 hives to recruit honeybee foragers from.

Individual honeybees were trained to enter a Y-maze (Fig. 1; as described in Avarguès-Weber et al., 2011). The arms of the Y-maze were 40×20×20 cm (L×W×H). Each bee had to fly through an entrance hole (5 cm in diameter) to enter an initial chamber and then fly through another hole into the decision chamber where it would be presented with two different options in each arm of the chamber, one correct and one incorrect (Fig. 1).

Stimuli were presented on grey backgrounds (20×20 cm) located 5 cm away from the decision lines to ensure each element was above the minimum resolution threshold for free-flying honeybees (Srinivasan and Lehrer, 1988). During training in group 1, a 10 μl drop of 50% sucrose solution (appetitive/rewarding outcome) was paired with a correct choice and a 10 μl drop of water (neutral unreinforced outcome) was paired with the incorrect choice (appetitive differential conditioning). During training in group 2, a 10 μl drop of 50% sucrose solution (appetitive/rewarding outcome) was paired with a correct choice and a 60 mmol l⁻¹ quinine solution (aversive outcome) was paired with an incorrect choice (appetitive–aversive differential conditioning). Each stimulus had a transparent landing pole located below it which held the drop of sucrose, quinine or water. Poles were replaced when touched by a bee and cleaned with 20% ethanol then water and dried to exclude olfactory cues. The side of correct and incorrect stimuli was randomly changed between choices. If a bee made an incorrect choice and started to imbibe the quinine, it was allowed to fly to the pole in front of the correct stimulus to collect sucrose to maintain motivation; but only the first choice was recorded for statistical analysis following standard procedures (Avarguès-Weber et al., 2015). Once the bee had finished imbibing the sucrose, it was allowed to fly back to the hive if satiated or make another decision by re-entering the maze from behind an opaque screen. During the non-reinforced tests, a drop of water was placed on each of the poles placed in front of the stimuli. Ten choices (touches of the poles) were recorded for each of the four tests to enable statistical comparisons, consistent with standard testing procedures for honeybees (Avarguès-Weber et al., 2015; Howard et al., 2018, 2017).

Each stimulus was a 6×6 cm white square card containing a number of black elements (Fig. 2) and was covered with 80 µm Lowell laminate. Elements in the learning phase could be one of five shapes: square, diamond, circle, triangle or star. Randomized shapes were used for the four tests to ensure patterns and shapes were unfamiliar to bees during tests (Fig. 3A). Stimuli ranged from 1 to 10 elements, and no stimulus was shown in more than a single bout (return to hive to off-load sucrose; approximately 2–5 choices).

There were three sets of stimuli consisting of (i) equal overall surface area (set 1; n=164; surface area 10±1 cm²), (ii) consistently equal element area (set 2; n=160; all elements were 1 cm²) or (iii) novel randomized shapes (set 3; n=20; Fig. 2). Set 1 had an average element area of 2.039 cm². The training sets (set 1 and set 2) had an overall average surface area of 1.520 cm². There were 344 stimuli in total; furthermore, stimuli were rotated to one of four orientation positions determined randomly by dice roles to provide training and testing sets with over 1000 options. All stimuli were created in Adobe Illustrator and stimuli properties (e.g. set 1 of equal overall surface area and set 2 of equal element area) were validated with the program ImageJ.

The average inter-item distance of stimuli, a measure of density, for each set is presented in Fig. S1. Another measure of density/occupancy (Bertamini et al., 2018) – the mean proportion of black pixels – of stimuli is presented in Fig. S2.

Bees were incrementally trained to enter the Y-maze and both arms of the apparatus over 30–60 min periods. Once each bee was able to fly into the entrance hole and the hole that led to the decision chamber, and could find the poles in both Y-maze arms, the experiment began.

Each bee completed either 50 appetitive differential conditioning choices (group 1) or 50 appetitive–aversive differential conditioning choices (group 2). Bees in group 1 were rewarded with sucrose for a choice of 4 elements and received no outcome (a drop of water on pole) for an incorrect choice of any other number. Bees in group 2 were rewarded with sucrose for a choice of 4 elements and received an aversive outcome, quinine, for a choice of any other number of elements ranging from 1 to 10 (excluding 4; Fig. 3A).

Once bees had completed the training, there were four tests of 10 unreinforced choices each for bees in either group. Between each of the four tests there were 10 refresher reinforced choices to maintain bee motivation (same procedure as the learning phase). The sequence of these tests was randomized. The tests were non-reinforced (no reward or punishment) and used a 10 μl drop of water (neutral outcome) instead of quinine or sucrose to motivate bees to land. Bees were shown comparisons of 4 versus 5, 4 versus 6, 4 versus 7 and 4 versus 8 elements (Fig. 3A); these tests compared quantity ratio discriminations of 0.80, 0.67, 0.57 and 0.50, respectively.

To test for the effect of training on bee performance (number of correct choices) in both groups, data from the learning phase of 50 choices were analysed with a generalized linear mixed-effect model (GLMM) with a binomial distribution using the ‘glmer’ package within the R environment for statistical analysis. We fitted a full model with trial number as a continuous predictor, and subject as a random factor to account for repeated choices of individual bees.

To determine whether bees were able to learn to choose 4 elements in tests, we analysed the test data by employing a GLMM including only the intercept term as fixed factor and subject as a random term. The proportion of ‘correct’ choices (MPCC) recorded from the tests was used as the response variable in the model. The Wald statistic (z) tested whether the mean proportion of correct choices recorded from the learning test, represented by the coefficient of the intercept term, was significantly different from chance expectation, i.e. H0: MPCC=0.5. The 95% confidence intervals (CIs) for the model coefficients were constructed using profile likelihood methods rather than calculated from the standard errors of the least-squares estimates (Faraway, 2016).

All analyses were performed within the R environment for statistical analysis.

All animal care was in accordance with institutional guidelines.

Two groups of bees were trained for 50 trials of either appetitive differential conditioning (group 1) or appetitive–aversive differential conditioning (group 2; Fig. 3A). Only bees trained using appetitive–aversive differential conditioning demonstrated significant learning over the period of 50 trials (statistical test: GLMM with a binomial distribution with trial number as a continuous predictor, and subject as a random factor to account for repeated choices of individual bees; group 1: z=0.102; P=0.918; n=10, Fig. 3B; group 2: z=5.48; P<0.001; n=12; Fig. 3C).

After the acquisition phase, honeybees were evaluated on their discrimination ability between 4 elements and higher numerosities in four tests (4 versus 5, 4 versus 6, 4 versus 7 and 4 versus 8), for 10 unreinforced choices per comparison using randomly shaped novel elements (set 3), presented in a random order. Bees trained using an appetitive-only procedure were unable to differentiate between 4 versus 5 (statistical test: logistic regression with individual as random term to test for differences between the observed proportion of bee choices and chance level, y=0.5, 51.0±3.8% mean±s.e.m., z=0.200, P=0.841, 95% CI: 0.413, 0.607), 4 versus 6 (50.0±4.90%, z=0.000, P=1.000, 95% CI: 0.397, 0.603) or 4 versus 8 (54.0±5.00%, z=0.600, P=0.549, 95% CI: 0.513, 0.702) at a level significantly different from chance, except during the test of 4 versus 7 (62.0±3.60%, z=2.18, P=0.029, 95% CI: 0.423, 0.637; Fig. 3D). In contrast, bees trained using appetitive–aversive differential conditioning were able to discriminate between 4 versus 5 (59.2±3.10%, z=2.00, P=0.046, 95% CI: 0.502, 0.677), 4 versus 6 (60.8±3.80%, z=2.35, P=0.019, 95% CI: 0.519, 0.693), 4 versus 7 (63.3±3.30%, z=2.89, P=0.004, 95% CI: 0.546, 0.716) and 4 versus 8 (64.2±3.40%, z=3.06, P=0.002, 95% CI: 0.554, 0.724; Fig. 3E) at a level significantly different from chance expectation.

Our results show that numerical discrimination above the subitizing range is possible with appetitive–aversive differential conditioning. Bees trained using this method were able to discriminate between quantities at a ratio of 0.80 (4 versus 5), a finer discrimination than was previously observed in honeybees considering appetitive conditioning (Gross et al., 2009). There is some evidence that bees trained with appetitive differential conditioning may be able to discriminate ratios of 0.57 (4 versus 7) above the threshold of 4 objects; however, this is still an open question as these bees failed at discriminating the less challenging ratio of 0.50 (4 versus 8). Honeybees are currently at a level of quantity discrimination observed in species such as African grey parrots (Al Aïn et al., 2009), capuchin monkeys (Addessi et al., 2008; Gazes et al., 2018), squirrel monkeys (Gazes et al., 2018), dolphins (Jaakkola et al., 2005), ponies (Gabor and Gerken, 2018), jungle crows (Bogale et al., 2011) and guppies (Bisazza et al., 2014). The question of whether the introduction of an aversive outcome for an incorrect choice improves performance has been asked for colour discrimination tasks in bees (Avarguès-Weber et al., 2010; Chittka et al., 2003), but this is the first time appetitive and appetitive–aversive differential conditioning have been directly compared for a numerical visual task.

A potential explanation for why bees perform better at numerosity discrimination with appetitive–aversive differential conditioning is that the attentional processes are improved as a result of the presence of an aversive outcome for an incorrect choice (Avarguès-Weber et al., 2010). When the penalty for making an incorrect decision is low, there is less motivation to ensure performance is as accurate as possible; however, when the penalty is increased, there is an increase in motivation to be more accurate and thus attention may be modulated by conditioning procedure (Avarguès-Weber et al., 2010). By pairing the incorrect option with an aversive outcome, the conditioning procedure also improves the strength of the association between the rewarding outcome and the correct option, as demonstrated in vertebrates (Rescorla and Wagner, 1972).

Through the possible modulation of attention, we show that previous studies on animal numerical ability, specifically quantity discrimination, may have underestimated the potential numeric ability of non-human animals. In the current study, we found a difference in the results considering both the training and test phases of honeybee quantity discrimination when bees were trained with the respective procedures. Our results are supported by previous quantity discrimination studies that show varied results within the same species when different methods are employed. For example, the quantity discrimination ability of guppies has been well studied (Agrillo et al., 2012; Bisazza et al., 2014; Gatto et al., 2017; Lucon-Xiccato and Dadda, 2017; Lucon-Xiccato et al., 2017, 2015; Piffer et al., 2012, 2013), but this ability appears to be dependent on the testing procedure (Agrillo and Bisazza, 2014). Guppies tested on quantity discrimination using spontaneous choice, training, extensive training or different apparatuses (e.g. central cylindrical restricted areas versus large square tanks) exhibit different limits of discrimination. Recording the spontaneous choices of guppies has resulted in a discrimination ability at a ratio of 0.75 for low (4 and fewer) number comparisons (Piffer et al., 2012) and 0.67 for high (4 and higher) number comparisons (Miletto Petrazzini and Agrillo, 2016); however, with extensive training, this has been extended to 0.80 with higher numbers (Bisazza et al., 2014), and even 0.83 using a recently developed method (Lucon-Xiccato et al., 2017). In the current study, we also show that different methods yield different results and quantity discrimination abilities; thus, perhaps non-human animals in other studies that are not as motivated to avoid incorrect choices may not have been pushed to their cognitive limits. Numerical processing is likely to be a valuable skill for animals operating successfully in complex environments (Gómez-Laplaza and Gerlai, 2011a,b; Lucon-Xiccato et al., 2017; Miletto Petrazzini and Agrillo, 2016; Nieder, 2017; Seguin and Gerlai, 2017), and we show that motivation, as modulated through conditioning, is critical to understanding what level of numerical ability an animal demonstrates.

We would like to thank the CNRS and the University of Toulouse 3 for their support.