Perception and coding of envelopes in weakly electric fishes

How the brain processes natural stimuli for the control of behavior remains, generally speaking, poorly understood. This is in part because natural stimuli have complex spatiotemporal characteristics, many of which have not been fully described. One category of signal feature, envelopes, has only recently been appreciated for its widespread significance in the control of behavior. In practice, the envelope can be viewed as the line connecting successive maxima in the stimulus waveform and represents the ‘instantaneous amplitude’ of the stimulus (Fig. 1). We now know that most natural auditory (Heil, 2003; Zeng et al., 2005), visual (Derrington and Cox, 1998; Baker, 1999) and somatosensory (Lundstrom et al., 2010) stimuli contain salient slow time-varying envelopes. These envelopes are critical for perception and are used during a variety of tasks such as second-order visual processing (Grosof et al., 1993; Mareschal and Baker, 1998), stereopsis (Langley et al., 1999; Tanaka and Ohzawa, 2006), speech perception (Calhoun and Schreiner, 1998; Smith et al., 2002; Zeng et al., 2005; Gourévitch et al., 2008; Nourski et al., 2009) and sound localization (Lohuis and Fuzessery, 2000).

Although envelopes are widely used in engineered systems such as radio communications, very little is known about how they are used and processed in biological systems. This is in part because natural envelope stimuli are complex and often cannot be captured in any simple mathematical expression (Heil and Neubauer, 2003). Further, the neural circuits that process envelopes are largely unknown.

In this review, we discuss advances made in understanding both the behavioral relevance and neural coding of envelope stimuli in the electrosensory system of wave-type Gymnotiform fishes. Specifically, electrosensory envelopes emerge as a consequence of the relative movement between fish (Yu et al., 2012), the interactions of their electric organ discharges (EODs) when in groups of three or more (Stamper et al., 2012), and the combination of movement and EOD interactions (Yu et al., 2012). Thus, these fish can influence the envelopes that they receive in two ways – through their own movement and by changing the frequency of their electric signal.

Weakly electric fish use their electrosensory system for a variety of behaviors including navigation, prey capture, refuge tracking and obstacle avoidance (Caputi and Budelli, 2006; von der Emde, 2006). In addition, they also use their electrosensory system for conspecific communication (Hopkins, 1974; Fortune, 2006; Hupé and Lewis, 2008; Triefenbach and Zakon, 2008).

Wave-type electric fish have a specialized electric organ (EO), typically along the sides of their bodies and/or along the tail, composed of electrocytes derived from modified muscle cells (myogenic) or nerve cells (neurogenic) (Bennett, 1971). The EO generates a significant quasi-sinusoidal alternating current by summing ionic currents across cells that are simultaneously generated following command signals originating from the pacemaker nucleus (Pn) in the brain. The electric signal that results is known as the EOD (Babineau et al., 2006; Kelly et al., 2008; von der Emde et al., 2010), which is an electric field that surrounds the fish and may be detectable at distances of more than 1 m around the fish (Fig. 1C, inset, black).

The spatial and temporal features of the electric field are complex (Chen et al., 2005) and vary as a function of objects in the environment and the presence of other electric fish (Kelly et al., 2008; von der Emde et al., 2010). Historically, electrosensory stimuli in gymnotiform wave-type electric fish have been characterized in terms of the amplitude modulation (AM) and frequency modulation (FM) because there are two classes of electroreceptors, one for processing each attribute (Scheich, 1973). For this review, we will consider AM stimuli exclusively.

The AMs can be characterized in terms of their frequency content (Fig. 1C, green). In this regard, small prey items typically lead to low frequency (<10 Hz) AMs (Nelson and MacIver, 1999; MacIver et al., 2001), whereas those caused by interaction with conspecifics lead to AMs with a wide frequency range (near 0 Hz to 400 Hz) (Heiligenberg, 1991; Zupanc and Maler, 1993; Ramcharitar et al., 2005; Tan et al., 2005; Ramcharitar et al., 2006).

However, these AMs can also be characterized in terms of their depth of modulation or contrast (Fig. 1C, red). In general, the depth of modulation is proportional to the distance of an object or conspecific from electroreceptors in the skin of an individual fish. The envelope, which is described in detail below, is the time-varying modulation of the depth of modulation of the AM and is a function of multiple environmental variables (Middleton et al., 2006; Middleton et al., 2007; Longtin et al., 2008; Savard et al., 2011; McGillivray et al., 2012; Stamper et al., 2012).

Electrosensory envelopes can be generated through movement, the interactions of three or more EODs of conspecifics in close proximity (<1 m), or when fish emit a particular subtype of communication signal called a type I chirp in which the AM amplitude (i.e. the envelope) decreases by 50% for a brief period of time (50 ms) (Zakon et al., 2002). Observations in the natural habitat have shown that fish often form groups of two or more conspecifics (Tan et al., 2005; Stamper et al., 2010), are also commonly in motion, especially at night, and routinely emit communication signals (Hupé and Lewis, 2008). As such, envelopes are ubiquitous features of the natural electrosensory environment.

However, envelopes in the electrosensory system differ somewhat from those found in other (e.g. visual, auditory) senses. Indeed, the envelopes found in the visual (Fig. 1A, red) and in the auditory (Fig. 1B, red) systems correspond to the AM of the signal (i.e. a first order envelope). In contrast, the envelopes in the electrosensory system refer to the depth of modulation (Fig. 1C, red) of EOD AMs and would be equivalent to second order envelopes in the auditory and visual systems. It is important to note that, unlike vision and audition, electrosensation is a sensory modality in which perturbations of a self-generated signal carry the relevant information. As such, the EOD AM (Fig. 1C, green) is actually the stimulus waveform here.

When two wave-type electric fish are in close proximity, their EODs sum and the combined signal will display AMs and FMs. The rate of these modulations occurs at the frequency difference (dF) between the two fish. The depth of modulation of the AMs is determined by the relative positions of the two fish. For fish that are stationary, the depth of modulation is constant, whereas in fish that are moving, the depth of modulation varies with the movements of the fish (Fig. 2). In this way, movement creates an envelope by changing the depth of modulation, which occurs in relation to changes in both the distance and orientation between the two fish (Yu et al., 2012). Specifically, the depth of modulation increases when the distance between the two fish decreases (Fig. 2). In general (Figs 2, 3 and 4), the envelopes (red) caused by movement tend to contain lower temporal frequencies than the AMs (green). The information extracted from the time-varying envelope signal might give information as to the relative position between the two fish (Yu et al., 2012). In contrast, the AM frequency gives information as to the difference between the EOD frequencies, and can give information as to whether the other fish is the same sex (low dF) or opposite sex (high dF) in some species including Apteronotus leptorhynchus and Eigenmannia virescens.

When three or more fish are in close proximity and are stationary, the EODs sum and the combined signal then consists of two prominent AMs [see Appendix in Stamper et al. (Stamper et al., 2012)]. If these two AMs have different frequencies (i.e. dFs), then the interaction between the two as they cycle in and out of phase with respect to one another can create an envelope that oscillates at the difference between the two AM frequencies (i.e. the ddF). Such ‘social envelopes’ do not require movement and are illustrated in Fig. 3. For example, if there are three stationary fish with EOD frequencies of 300 Hz (fish 1), 350 Hz (fish 2) and 395 Hz (fish 3), then fish 2 would experience AMs (green) with frequencies dF₁=50 and dF₂=45 Hz whose interference creates an envelope (red) at frequency ddF=5 Hz. In general, ‘social’ envelopes tend to contain higher temporal frequencies (>15 Hz) (Stamper et al., 2010).

Weakly electric fish typically move with respect to one another in their natural environment and are often found in groups of three or more. As such, these fish experience both envelopes that result from interaction between their electric fields (i.e. ‘social envelopes’) as well as those that result from movement (i.e. ‘movement envelopes’). Thus, envelope signals in the natural environment will contain power due to movement and to interaction between the EODs of three or more fish. A specific case is illustrated in Fig. 4 where the temporal frequency content of the movement envelope component is lower but still overlaps with that of the social envelope component.

Weakly electric fish also emit communication signals called chirps (Zakon et al., 2002). It is important to note that these signals typically occur on top of the AMs that are created when two fish are in close proximity to one another. Chirps are typically associated to a transient (~50 ms) increase in one of the fish's EOD frequencies. However, for one particular chirp subtype (type I), the increase in EOD frequency is also associated with a ~50% decrease in EOD amplitude. This has the net effect of creating an envelope as the depth of modulation of the AM drops during the type I chirp. Type I chirps tend to be produced during opposite sex encounters, which are typically associated with a high frequency AM (Bastian et al., 2001).

The social envelopes that have been observed in groups of three or more fish in natural habitats and in laboratory settings tend to have power at higher frequencies (>15 Hz) and not lower frequencies (Stamper et al., 2010). These data suggested that fish avoid producing low frequency social envelopes in their natural environment. A recent study has shown that weakly electric fish change their EOD frequencies in response to low frequency social envelopes (Stamper et al., 2012). In this behavior, fish shift their EOD frequency to increase the frequency of the social envelope, as discussed below.

Before describing behavioral responses to envelopes, it is useful to briefly review how some species (e.g. Eigenmannia and Apteronotus) of weakly electric fish respond to constant AMs (i.e. not envelopes) that are generated when two fish are in close proximity. When the AM frequency (dF) is low (<15 Hz), the animal's electrolocation abilities are impaired (Heiligenberg, 1974; Matsubara and Heiligenberg, 1978; Bastian, 1987) as the frequency content of the signals used in electrolocation and of the AM caused by the interaction of EODs overlap. This is known as ‘jamming’ and is thought to occur because neural responses to salient signals are then degraded (Ramcharitar et al., 2005).

Some species, including Eigenmannia and Apteronotus, solve this problem by shifting their EOD frequencies in order to increase the AM frequency (dF) to move it away from the frequency content of other behaviorally relevant electrosensory signals (e.g. those caused by prey). This behavior is known as the jamming avoidance response (JAR) (Watanabe and Takeda, 1963; Bullock et al., 1972). The underlying algorithm and the neural circuits that support the JAR have been fully elucidated (Heiligenberg, 1991; Metzner, 1993). It was previously shown that the JAR as observed in Eigenmannia requires integration of AM and FM information streams. Additionally, Eigenmannia respond to EOD AMs without the concurrent FM stimuli but this response differs from the JAR (Takizawa et al., 1999; Carlson and Kawasaki, 2007).

Recent experiments by Stamper and colleagues show that Eigenmannia will either increase or decrease their EOD frequency in response to social envelope stimuli (Stamper, et al., 2012). This behavior was termed the social envelope response (SER). Indeed, when an individual fish is presented with two AMs whose interferences gives rise to a low frequency envelope, the animal shifts its EOD frequency by 2–10 Hz. The amount by which the animal changes its EOD frequency decreases with increasing envelope frequency (Fig. 5A) and increases with stimulus amplitude (Fig. 5B). The change in EOD frequency increases the envelope frequency to between 5 and 15 Hz (Fig. 5C) (Stamper et al., 2012). This suggests that the SER may be a mechanism for avoiding low frequency social envelopes in the natural electrosensory environment, much like the proposed function of the JAR.

Additional studies are necessary to determine whether low frequency envelopes, like low frequency AMs, are detrimental to electrolocation behaviors. Alternatively, it is worth noting that in a complex envelope, in which there are both social and movement components, the movement envelope often has a lower frequency content and the social envelopes a higher frequency content (Fig. 4). The SER may also function to increase the separation between these two frequency bands to permit separate processing of movement and social envelope information in brain circuits.

An algorithm that would mediate the SER has also been determined (Stamper et al., 2012). When three or more EODs sum, plotting the amplitude and phase modulations of the EOD signal can, with additional low-pass filtering, give information as to the sign of the ddF, which is necessary in order for the fish to know whether to increase or decrease its own EOD frequency (Stamper et al., 2012). Further studies are needed to understand the neural circuits that perform the necessary computations required for the SER.

There have been several recent advances towards understanding the neural algorithms for the extraction of behaviorally relevant information from envelope stimuli in electrosensory circuits. P-type tuberous electroreceptors project to pyramidal cells within the electrosensory lateral line lobe (ELL). Pyramidal cells then project to the midbrain torus semicircularis (Ts). Ts neurons project to the nucleus electrosensorius (nE), which projects to the prepacemaker nucleus (PPn). The PPn projects to the pacemaker nucleus (Pn), which sends commands to the EO, thereby completing the sensorimotor loop. This circuitry is well described elsewhere (Heiligenberg, 1991; Metzner, 1993).

Electroreceptors of gymnotiform wave-type species are characterized by high baseline firing rates (150–600 Hz) and firing rates monotonically increase with increasing EOD amplitude. Of course, receptors have the standard neural non-linearities that can influence their responses to envelopes including cut-off (i.e. their firing rate cannot be negative) and saturation (i.e. their firing rate cannot be greater than the EOD frequency as electroreceptors can fire at most one spike per EOD cycle) (Scheich et al., 1973; Bastian, 1981; Xu et al., 1996; Gussin et al., 2007). Further analysis has shown that their gains increase with stimulus frequency (Chacron et al., 2005; Xu et al., 1996). Therefore, electroreceptors have non-linear responses to electrosensory stimuli (AMs) that have amplitudes large enough to elicit non-linear saturation and/or cut-off. Further, as a consequence of increased gain to high frequency stimuli, electroreceptors display non-linear phase locking as well as increased synchronization to higher frequency AMs (Wessel et al., 1996; Nelson et al., 1997; Benda et al., 2005; Chacron et al., 2011).

As responses to envelope stimuli require a non-linear operation, mathematical models suggested that electroreceptors could respond to envelopes provided that the envelope signals exceed a certain threshold amplitude value (Longtin et al., 2008; Yu et al., 2012). Experimental recordings have confirmed these predictions by showing that electroreceptors can respond to envelopes arising from band-pass filtered white noise when the filter band contained high (>100 Hz) frequencies (Savard et al., 2011). Further analysis revealed that electroreceptor heterogeneities played a strong role in determining whether a given electroreceptor would respond to envelopes or not. Indeed, Savard and colleagues showed that it was the electroreceptor with the lowest baseline firing rate that tended to respond to envelopes because these units displayed the greatest tendency to reach cut-off during these stimuli (Savard et al., 2011).

The fact that electroreceptors can respond to envelopes when the amplitude of envelope signals exceeds a given threshold can be demonstrated by examining their responses to a sinusoidal AM stimulus whose depth of modulation (i.e. envelope) increases linearly with time (Fig. 6A). Indeed, electroreceptors tend to respond to these AMs with modulations in firing rate that are linearly related to the stimulus when the amplitude is low and with non-linear phase-locking when the amplitude is high (Fig. 6A). This has important consequences for determining whether the electroreceptor responds to the envelope or not. If the envelope amplitude is low enough, then the increase in firing rate during one half of the AM cycle will be compensated by the decrease in firing rate during the other half of the AM cycle. As such, the average firing rate during each AM cycle is then the same (Fig. 6B). However, the situation changes when the envelope amplitude is large enough to elicit static non-linearities such as cut-off. When cut-off is elicited, the increase in firing rate during one half of the AM cycle is not fully compensated by the decrease in firing rate during the other half, and the average firing rate increases (Fig. 6B).

The amplitude level at which the firing rate starts either increasing or decreasing is therefore a threshold for the detection of envelope signals. In other words, electroreceptors do not respond to envelope stimuli when they are below the threshold and do respond to envelopes when they are above threshold due, at least in part, to non-linear phase locking (Fig. 6C). The exact threshold value is dependent on multiple factors including the baseline firing rate as well as the amplitude of the AM (Savard et al., 2011). As such, electroreceptors can either respond solely to the AM or to the AM as well as the envelope.

Electroreceptors also respond strongly to type I chirps. Indeed, the high frequency AM (i.e. constant non-zero envelope) causes non-linear synchronization in the electroreceptor population while the decrease in envelope during the chirp causes a transient desynchronization (Benda et al., 2006).

Pyramidal cells are the sole output neurons of the ELL. There are two pyramidal cell types, ON-cells that are excited (E-type) and OFF-cells that are inhibited (I-type) by increases in EOD amplitude (Saunders and Bastian, 1984). Like electroreceptors, the population of pyramidal cells is heterogeneous (Fig. 7A) (for review, see Maler, 2009). Pyramidal cells whose somata are found most superficially tend to display the largest dendritic trees and the smallest baseline firing rates while pyramidal cells whose somata are found most deeply tend to display the smallest dendritic trees and the largest baseline firing rates (Fig. 7A). This strong negative correlation between baseline firing rate and dendritic morphology allows one to easily identify a given pyramidal cell based on physiological measurements (Bastian and Nguyenkim, 2001; Bastian et al., 2004).

While it has been known for some time that pyramidal cells respond to envelopes (Middleton et al., 2006), recent experiments have shown a large heterogeneity in these responses (Marsat and Maler, 2010; McGillivray et al., 2012). Indeed, McGillivray and colleagues (McGillivray et al., 2012) showed that, while all pyramidal cells respond to both AMs and envelopes, superficial pyramidal cells tend to respond most strongly to envelopes while deep pyramidal cells tend to respond least strongly to envelopes (Fig. 7B). This is because superficial pyramidal cells, with their low baseline firing rates, tend to display the most cut-off in response to AMs (Fig. 7C). Nevertheless, all pyramidal cells respond to both carrier and envelope, but to different degrees based on heterogeneities in baseline firing rates (Fig. 7D). These findings are consistent with those of Marsat and Maler, who showed that ELL pyramidal cells respond differentially to type I chirps (Marsat and Maler, 2010).

How do pyramidal cells respond to the envelope? Envelope responses in pyramidal cells are not solely due to the threshold non-linearity described above: their subthreshold membrane activity already contains information about the envelope (Middleton et al., 2006). This subthreshold envelope information is presumably transmitted via the synaptic inputs to pyramidal cells. Middleton and colleagues showed that a class of inhibitory interneurons, called ovoid cells, responds to envelopes (Middleton et al., 2006). These responses are also most likely due to a threshold non-linearity. Middleton and colleagues (Middleton et al., 2006) concluded that inhibitory inputs from ovoid cells might be a mechanism underlying envelope responses in pyramidal cells (Fig. 7E).

While the respective contributions of inhibitory inputs and the threshold non-linearity towards determining the envelope responses of pyramidal cells have not been fully elucidated, it is possible that these mechanisms might act synergistically. This is because superficial pyramidal cells with the lowest baseline firing rates also receive the greatest proportion of inhibitory inputs, while deep pyramidal cells with the highest baseline firing rates receive the least inhibitory inputs (Bastian et al., 2002). However, further studies are necessary to fully understand the mechanisms by which pyramidal cells respond to envelopes. Again, while pyramidal cell heterogeneities can influence envelope responses, all pyramidal cells still respond to both envelope and AMs (Middleton et al., 2006; McGillivray et al., 2012).

Electrophysiological studies have shown that Ts neurons can also respond to envelope stimuli (Fig. 8A–C). However, the responses of Ts neurons fall into three distinct categories. Ts neurons can respond (1) to both the AM and the envelope (Fig. 8D, blue), or (2) selectively to only the AM (Fig. 8D, green) or (3) to the envelope (Fig. 8D, red) (McGillivray et al., 2012). Some Ts neurons can also respond selectively to type I chirps (Vonderschen and Chacron, 2011). Therefore, segregation of information about envelope and AM stimuli appears to emerge at the level of the Ts. This segregation potentially resolves ambiguities as to whether a response is caused by the AM or the envelope. This is important as behavioral studies have shown that AM and envelope stimuli can overlap in their frequency ranges (Yu et al., 2012).

How can Ts neurons respond selectively to either the envelope or the AM? Anatomical studies have shown that Ts neurons receive excitatory inputs from both ON- and OFF-type ELL pyramidal cells although it is not known whether a given Ts neuron can receive input from both ON- and OFF-type cells at the same time (Carr and Maler, 1985). McGillivray and colleagues (McGillivray et al., 2012) used spike-triggered covariance analysis in order to investigate the different sources (ON-, OFF-, or both ON- and OFF-type) of inputs received by Ts neurons. Spike-triggered averages of the stimulus showed that envelope-selective neurons apparently did not respond to features of the AM. However, covariance analysis showed that envelope-selective Ts neurons, and not the other two categories of Ts neurons, exhibited great variability in their responses that could not be explained by the variability of the stimulus. Further analysis revealed that such neurons did not respond to the AM because they spiked in response to both the increases as well as the decreases in the AM. Indeed, because those features were matched in strength, they essentially canceled out and, when averaged, resulted in no effective response to the AM. However, because the neuron is responding to both phases of the AM, changes in the depth of modulation (i.e. envelope) will result in changes in the magnitude of firing. In this way, the neuron responds to the changes in the envelope and not the AM.

McGillivray and colleagues (McGillivray et al., 2012) tested this interesting idea by building a mathematical model in which they could vary the relative balance of inputs from ON- and OFF-type ELL pyramidal cells (Fig. 8E). They found that the model neuron responded selectively to the envelope only when the two sources of input were matched in strength (Fig. 8F). When there was a mismatch, the model neuron instead responded to both AM and envelope. Moreover, this model made an important prediction. When they increased the baseline firing rate of the model ELL neurons, thereby mimicking the baseline firing rates of deep ELL pyramidal cells, they found that the model Ts neuron tended to respond most selectively to the AM and not to the envelope.

Thus, these results strongly suggest that Ts neurons acquire selectivity to the envelope stimuli by combining balanced inputs from ON- and OFF-type pyramidal cells. The algorithm is the following: ON- and OFF-type pyramidal cells display rectification in their responses but at opposite phases of the AM stimulus, but are essentially in-phase with respect to the envelope stimulus (Fig. 8H). These responses are then summed and low-pass filtered in order to give rise to a selective envelope response in ON–OFF Ts neurons (Fig. 8H). Further experiments performed by McGillivray and colleagues (McGillivray et al., 2012) show that these correspond to the ON–OFF (‘E–I’) neurons that were previously reported (Partridge et al., 1981) and that they thus most likely form a distinct cell class within the Ts. Ts neurons that respond to both AM and envelope are thought to receive unbalanced input from ON- and OFF-type ELL pyramidal cells.

Thus, the following neural circuit is most likely responsible for segregation of envelope and carrier information in Ts. Envelope-selective Ts neurons are thought to receive balanced input from ON- and OFF-type superficial ELL pyramidal cells. This is because these cells tend to give the strongest responses to envelopes. In contrast, AM-selective Ts neurons most likely receive input from deep ON-type ELL pyramidal cells (Fig. 9).

Electrosensory envelopes arise as a result of interactions between the EODs of three or more motionless fish (‘social envelopes’), movement when two or more fish are in close proximity (‘movement envelopes’), and the production of specific types of intermittent social signaling. In general, envelopes found in the natural electrosensory environment likely result from a combination of each of these categories. Some species of weakly electric fish exhibit behavioral responses to social envelopes (the SER) in order to increase the frequency content of the envelope. Finally, electrosensory circuits appear to be devoted to the processing of envelopes as some midbrain neurons respond selectively to them. However, there remain many questions related to the perception and coding of envelope stimuli by weakly electric fish.

First, it is important to realize that both Middleton and colleagues (Middleton et al., 2006) as well as McGillivray and colleagues (McGillivray et al., 2012) used stimulus intensities that did not elicit envelope responses from electroreceptors. As such, how ELL pyramidal cells and subsequently Ts neurons integrate envelope information from multiple sources is unknown and will be a fascinating area of research.

Second, how electrosensory neurons in higher order areas (e.g. nE) respond to envelopes is currently unknown. It will be critical to record from these higher order neurons in order to determine where and how information from envelope-selective Ts neurons is processed in higher order brain areas.

Third, while Stamper and colleagues (Stamper et al., 2012) have proposed an algorithm that would explain the SER, further studies should investigate whether the neural circuits that mediate the SER are indeed similar to those that mediate the JAR.

Finally, while social envelopes may interfere with the animal's electrolocation abilities, this is not the case for movement envelopes that might contain information related to environmental variables (e.g. the distance between both fish). Further studies should focus on whether these movement envelopes interfere with the animal's electrolocation abilities, whether electrosensory neurons can respond to these very low frequency signals, and whether weakly electric fish actually exhibit behavioral responses to movement envelopes.

We would like to thank M. G. Metzen for help in making Fig. 1.

 
• AM

  amplitude modulation

 
• ddF

  difference of dFs

 
• dF

  frequency difference

 
• ELL

  electrosensory lateral line lobe

 
• EO

  electric organ

 
• EOD

  electric organ discharge

 
• FM

  frequency modulation

 
• FT

  Fourier transform

 
• JAR

  jamming avoidance response

 
• nE

  nucleus electrosensorius

 
• Pn

  pacemaker nucleus

 
• PPn

  pre-pacemaker nucleus

 
• PSTH

  peri-stimulus time histogram

 
• SER

  social envelope response

 
• Ts

  torus semicircularis