Revealing sound-induced motion patterns in fish hearing structures in 4D: a standing wave tube-like setup designed for high-resolution time-resolved tomography

Modern bony fishes (teleosts) cover a wide range of auditory capabilities (e.g. in terms of auditory sensitivity and the range of detectable frequencies) alongside a great diversity in their auditory structures (Braun and Grande, 2008; Ladich and Fay, 2013; Ladich and Schulz-Mirbach, 2016). Primarily, auditory structures refer to the inner ears including the mineralized otoliths, and can further include the swim bladder or any other gas-filled bladder, especially if these bladders approach or contact the inner ears (i.e. otophysic connection; Braun and Grande, 2008) as found in otophysan fishes. Otophysans such as zebrafish, Danio rerio (Cyprinidae), are important model organisms to study deafness and balance disorders in human medicine (e.g. Abbas and Whitfield, 2010). However, identifying the factors determining otolith motion and inner ear stimulation is still challenging in species such as zebrafish that have evolved specialized ancillary auditory structures (Weberian apparatus) for hearing enhancement (e.g. Schulz-Mirbach et al., 2019). Hence, knowledge of the interaction of fish auditory structures remains elusive (Popper et al., 2005; Ladich and Schulz-Mirbach, 2016).

The most straightforward and reliable way to study fish auditory structures ‘in action’ is the direct observation of the sound-induced in situ motion of the structures in the test subject. However, the technical challenge of gaining access to these internal structures, moving at amplitudes in the range of a few micrometers and at typical sound frequencies, without altering their response as a consequence of surgical procedures has hampered such research for many years. Only a limited number of experimental studies have investigated the sound-induced motion of the saccular otoliths (de Vries, 1950; Sand and Michelsen, 1978), the swim bladder walls (Popper, 1974; Clarke et al., 1975) or the whole set of auditory structures (Cox and Rogers, 1987). Nowadays, synchrotron radiation-based techniques provide powerful approaches to perform imaging of internal structures at high spatio-temporal resolution non-invasively (Mokso et al., 2015; Rack et al., 2010; Walker et al., 2014). Recent studies using hard X-ray phase contrast imaging at the European Synchrotron Radiation Facility (ESRF, Grenoble, France) provided first insights into the sound-induced motion of fish auditory structures (Schulz-Mirbach et al., 2018, 2020). In these radiographic experiments using a standing wave tube-like tank, the motion of the swim bladder and the otoliths in the inner ears of goldfish (Carassius auratus, Otophysa) and the cichlid Etroplus canarensis was successfully visualized and motion patterns could be described qualitatively. However, the motion of the structures of interest could only be observed in 2D, i.e. in a certain orientation. In addition, ‘overlying’ structures such as gills or cranial bones, in part, hampered an undisturbed view of the moving auditory structures. Hence, any in-depth insights into the basic principles of the interaction of the auditory structures would require the visualization and characterization of the motion patterns during sound presentation covering the structures' full 3D aspect. A study on the flight motor motion of a blowfly performed at the TOMCAT beamline at the Swiss Light Source (SLS, Villigen, Switzerland) (Mokso et al., 2015; Walker et al., 2014) demonstrated that periodic motion patterns can be evaluated using tomography with high spatio-temporal resolution.

In mammals and birds, the inner ears show a rather clear division of labor consisting of a vestibular part (semicircular canals and in a wider sense the utricle and the saccule) and the portion serving the auditory sense, i.e. the lagena or cochlea (Manley and Clack, 2004; Witmer et al., 2008; Ladich, 2019). In these terrestrial vertebrates, sound pressure is the primary stimulus (Ladich, 2019; Fay and Popper, 2005). In contrast, in teleosts (and fishes in general), the otolith end organs, namely utricle, saccule and lagena, seem to serve both senses (‘mixed function hypothesis’; Platt and Popper (1981). Moreover, fish inner ears detect sound through particle motion (Rogers et al., 1988). The ability to additionally make use of the pressure component of sound is developed in several teleost groups (e.g. Otophysa, Clupeiformes, Mormyridae, Anabantiformes, several sciaenid species; Braun and Grande, 2008; Hawkins, 1993; Ramcharitar et al., 2006; Horodysky et al., 2008). Sound pressure detection in fishes is possible when the swim bladder or any gas-filled bladder acts as a pressure-to-displacement transducer (Rogers et al., 1988). This is most effective if the bladder is connected to the inner ears through anterior extensions or mechanically coupled by a chain of ossicles and ligaments as in otophysan fishes (Rogers et al., 1988; Schulz-Mirbach et al., 2019). Sound-induced pressure fluctuations provoke the compression and decompression of the gas in the bladder and result in the oscillation of the swim bladder walls. The oscillating walls then function as a secondary sound source, creating local particle motion, ultimately setting the otoliths in motion (Rogers et al., 1988; Schulz-Mirbach et al., 2019). As both sound components, acoustic particle motion and sound pressure, may play a role in stimulating fish inner ears, we set out to adapt the standing wave tube-like setup used at the ESRF for tomography at the SLS. The setup at the ESRF consisted of a horizontal 2L Plexiglas^® tube equipped with a miniature inertial shaker at each end which could be driven under different phase conditions to create either a sound pressure or particle motion maximum in the tube center. The intended setup adaption implied, among others, a miniaturization from a horizontal 2 l tank to an upright 14.1 ml (one-shaker setup) or 40.8 ml (two-shaker setup) tube, respectively.

In our study, we aimed to develop a setup that enables non-invasive experiments characterizing the 3D motion of fish auditory structures during sound presentation, such that these experiments can be performed under sound-induced particle motion or sound pressure conditions. In the following, we characterize the approach, focusing on the acoustic setup.

Initial tests to optimize the setup parameters such as sound amplitude and frequency, exposure time, X-ray energy and number of images per scan were performed with an isolated left saccular otolith from a goldfish, Carassius auratus (L. 1758) [otolith ex coll. T.S.-M., standard length (SL) 49 mm; tested in the one-shaker setup]. Subsequently, the in situ motion patterns of the intact auditory system were investigated in freshly euthanized specimens of zebrafish Danio rerio (Hamilton 1822) (N=3, SL 22 mm; tested in the one-shaker setup) and glass catfish Kryptopterus vitreolus Ng and Kottelat 2013 (N=2, SL 46 mm and 56 mm, respectively; tested in the two-shaker setup). We chose zebrafish and glass catfish because of their small size and the elongate/cylindrical body shape ideally suited for the dimensions of the small test tube. The three studied zebrafish specimens of the wild-type line ‘AB’ (provided by Prof. Anna Jazwinska Müller, University of Fribourg, Switzerland) originated from one hatching (25 October 2018). The two glass catfish individuals were obtained from a local aquarium trader (Aquarium & Teich AG, Villmergen, Switzerland). Prior to the experiments, fish were kept for 1 week in 20–30 l tanks equipped with internal filters at a temperature of 23°C and fed once a day with commercial flake food at the TOMCAT beamline. Before each experiment, the test subject was euthanized with a 0.4% solution of tricaine methanesulfonate (Sigma Aldrich) buffered with sodium bicarbonate. The fish was introduced into the X-ray beam when opercular movements had completely ceased and no vestibulo-ocular reflex was discernible. The experiments were approved by the cantonal veterinary service Aargau, Switzerland (approval no. 75725 and 75734). All experiments at the ESRF were conducted in accordance with ethical guidelines, i.e. article 3, point 1 of the EU directive 2010/63, accepted by all EU countries. As no living animals were used during the imaging procedure, no ethical approval was required.

To meet the requirements of tomographic imaging, namely the need to illuminate the sample with X-rays from all directions in the imaging plane and to be able to rotate it by at least 180 deg around a direction perpendicular to the X-ray beam, we used small transparent test tubes to minimize X-ray absorption by the tube walls and the water body in the tube. First, we conducted experiments (March and June 2019) with a one-shaker setup consisting of a transparent (Plexiglas^®) tube and one miniature inertial shaker (2002E, PCB Synotec) mounted at the bottom of the tube. The test tube had an inner diameter of 15 mm, a height of 80 mm and a wall thickness of 2 mm, resulting in a volume of 14.1 ml (Fig. 1A). Subsequently (September 2020), we upgraded the system by adding a second miniature inertial shaker (2002E, PCB Synotec) on top of a longer test tube. In this two-shaker setup (i.e. standing wave tube-like setup; Fig. 1B), the test tube had an inner diameter of 19 mm, a height of 144 mm, a wall thickness of 2 mm with a larger volume of 40.8 ml. To stabilize the upper shaker and the whole setup, the system was equipped with an outer cylinder which had an inner diameter of 53 mm, a height of 142 mm and a wall thickness of 3 mm.

Pure tone stimuli were transferred to the (degassed) water body by the (one or two) miniature inertial shaker(s) driving a short metal rod (a) fitting into the lid of a 1.5 ml Eppendorf tube that was glued to the center of a rubber membrane which sealed the bottom of the tube (one-shaker setup, Fig. 1A) or (b) with each shaker transmitting the vibration to the water body through a knurl-head screw (diameter: 16 mm, thickness: 3 mm) attached to a silicon membrane (thickness: 1 mm; two-shaker setup, Fig. 1Bi–iii). The two-shaker setup was run as a standing wave tube-like setup with the two shakers either driven 0 deg in phase or 180 deg out of phase. In an ideal standing wave tube (Fig. 1C), the condition ‘0 deg in phase’ results in maximum sound pressure in the center of the tube while there is no particle motion present. Accordingly, the 180 deg out of phase condition creates maximum particle motion in the center of the tube while there is no sound pressure (Hawkins and MacLennan, 1976).

The samples were held in place in the center of the test tube using a piece of porous foam (thickness: 10 mm) while fixing the sample inside the foam at only a few points to allow for a ‘free’ motion (Fig. 1Bii). Before fixing the samples, air bubbles were removed from the foam by squeezing it underwater.

In previous studies (Schulz-Mirbach et al., 2018, 2020), a stimulus frequency of 200 Hz resulted in a successful observation of the 2D in situ motion of otoliths applying hard X-ray phase contrast imaging. Accordingly, we tested stimuli of 100 and 200 Hz (one-shaker setup). To optimize the setup to higher frequencies more relevant in the context of the swim bladder–inner ear interaction (Myrberg and Spires, 1980), we additionally tested stimuli of 350 and 450 Hz (two-shaker setup). We used two different amplifier settings to take into account potential effects of different sound (pressure) levels on the motion of the auditory structures (Table S1).

The stimuli were generated by the software CoolEdit 2000 (Syntrillium Software Corp., Phoenix, AZ, USA) as ‘simple’ pure tones in the case of the stimuli tested in the one-shaker system. In the case of the two-shaker setup, the stimuli were generated as 0 and 180 deg phase-shifted pure tones per test frequency. Stimuli were presented using the internal sound card (Conexant 20561 SmartAudio HD and NVIDIA High-Definition Audio) of a laptop (Lenovo ThinkPad T500 and Lenovo ThinkPad P53) which was connected to the shaker(s) through an amplifier (S.M.S.L. SA 36A Pro, Shenzhen ShuangMuSanLin Electronic Co., Ltd, Shenzhen, China).

Acoustic measurements of the ambient noise in the hutch of the beamline and of the test frequencies were performed before or after the actual experiments. Simultaneous acoustic measurements were not possible because either the miniature hydrophone (Brüel & Kjær 8103, sensitivity: −211 dB re. 1 V µPa⁻¹; diameter: 9.5 mm) would have been affected by the X-ray beam or the sound pressure level (SPL) would have been evaluated in places other than the location of the sample, i.e. other than in the center of the tube. SPL was measured using the miniature hydrophone (see Fig. 1Biii) connected to a sound level meter (Brüel & Kjær 2238 Mediator) which had been calibrated with a hydrophone calibrator (Brüel & Kjær 4229) beforehand.

The ambient noise in the center of the tube was recorded using the software CoolEdit 2000 and the miniature hydrophone connected to the internal sound card of the laptop via an amplifier (36B2, constructed at the University of Vienna; including an electrical grounding). The ambient noise spectrum was then analyzed using the sound analyzing software STX 3.7.8 (Acoustics Research Institute, Austrian Academy of Sciences, Vienna, Austria). Tone stimuli in the test tube were recorded using CoolEdit 2000 with the miniature hydrophone and the 36B2-amplifier connected to the laptop while playing the tone stimuli via the same laptop using the shaker(s) and the S.M.S.L. SA 36A Pro amplifier.

At the TOMCAT beamline of the SLS, X-rays are produced by a 2.9 T superbending magnet located 25 m from the sample. The full white beam spectrum of the source was filtered to reduce the incoming X-ray intensity. A Ce-doped LuAG scintillator (thickness: 150 µm), transforming the X-ray image to visible light, was placed at distances of 300, 200, 120 and 70 mm behind the sample to test different levels of edge enhancement from the soft tissues. Finally, a scintillator distance of 70 mm was chosen to optimize the phase contrast of the radiographs. The resulting edge-enhanced visible light image of the sample on the scintillator was then magnified using a custom-made, high numerical-aperture microscope (Optique Peter, Lentilly, France) with 4-fold magnification and recorded with a GigaFRoST 12-bit detector system (Mokso et al., 2017) with a native pixel size of 11 µm, resulting in an effective pixel size of 2.75 µm. Full scans of the oscillating samples (tomography with sound, i.e. 4D stacks) were acquired at frame rates of around 2 and 5 kHz while the sample was rotated over 180 deg. Static scans of the same samples (tomography without sound, i.e. 3D stacks) were recorded with 1500 projections over 180 deg at exposure times of 1.0–1.5 ms. The experimental parameters for all dynamic samples are detailed in Table 1.

Projection-guided retrospective gating and tomographic reconstruction was performed based on the approach described by Mokso et al. (2015). Hence, for each stimulus frequency, projections taken from different angles but at the same phase during the stimulus presentation were grouped into respective time bins. This resulted in 20 time bins for the 100 Hz and 350 Hz stimuli, 10 time bins for the 200 Hz stimulus, and 16 time bins for the 450 Hz stimulus. For each time bin, a single distance propagation-based phase retrieval (δ/β=50) based on Paganin's algorithm (Paganin et al., 2002) was applied to the projections from which the 3D volume datasets were reconstructed using the gridrec reconstruction algorithm (Marone and Stampanoni, 2012).

To qualitatively evaluate the motion patterns under different setup conditions, we segmented and labeled the 3D volume of the saccular otoliths in the reconstructed datasets using Dragonfly v. 2021.1 (Object Research Systems Inc.). We focused on the otophysan saccular otoliths for two reasons. First, this structure showed a rotational movement induced by sound pressure in a previous study which could be easily identified even in radiographs (Schulz-Mirbach et al., 2020) and was also in line with the motion pattern hypothesized by experts in the last century (von Frisch, 1938; de Burlet, 1929; Wohlfahrt, 1932). Second, the 0 deg in phase and 180 deg out of phase conditions in the previous study clearly differed from each other by the evoked motion pattern of this otolith, with the rotational motion only identifiable under the 0 deg in-phase condition.

The image stacks were cropped along the x-, y- and z-axes to reduce stack size without performing resampling, i.e. the isotropic voxel size of 2.75 µm was retained for all steps. We labeled the otoliths in every third (200 Hz, 450 Hz) or every fifth (100 Hz, 350 Hz) timing bin of each 4D stack applying the same threshold-based segmentation settings (method: upper or lower Otsu). Any manual corrections of the segmented structures were kept to a minimum.

For the isolated goldfish sagitta embedded in foam, a threshold-based approach did not yield a satisfying result. We therefore trained a deep-learning U-net model in Dragonfly v. 2020.1 (type: semantic segmentation using a mask, patch size: 80, stride ratio: 1, batch size: 8, epochs: 14, loss function: categorical cross-entropy, data augmentation set to 3×) which resulted in a more accurate and reproducible labeling of the sagitta than using any threshold-based segmentation tool. However, some manual corrections were necessary, especially at the ‘free’ ventrolateral wing (see fig. 4 of von Frisch, 1938) of the sagitta. To evaluate the success and to estimate the accuracy of the segmentation process, volume and surface area of the labeled structures across the studied timing bins were compared with the values obtained from the respective structure visualized without sound exposure.

To produce the overlay of the sagitta outlines shown in Fig. 2, one sub-stack within the 4D stack was rotated applying the ‘translate/rotate tool’ until both sagittae were displayed in an exact transverse section on the x–y-plane. The transformation was then applied to the labels of the other sub-stacks. Subsequently, surfaces (‘meshes’) were generated and smoothed (number of iterations: 1) without using the resampling option.

In the two setups, the ambient noise level (one-shaker setup: 112.4, 115.7 dB; two-shaker setup:123.4 dB) was 33.6–50.6 dB (one-shaker setup) and 42.8–64.3 dB (two-shaker setup) below the SPL measured during the presentation of any of the stimulus frequencies (148.0–187.7 dB re. 1 µPa; Table S1). In the one-shaker setup, the spectrum of the recorded 100 Hz stimulus revealed peaks at 100 Hz followed by harmonics of this frequency with higher levels (Fig. S1A). In contrast, the spectrum of the recorded 200 Hz stimulus displayed a distinct peak at the stimulus frequency and harmonics at much lower levels (Fig. S1B). In the two-shaker setup, the spectra of the recorded 350 Hz (Fig. S1C₁,C₂) and the 450 Hz (Fig. S1D₁,D₂,E₁,E₂) stimuli showed clear peaks at the stimulus frequencies for both phase conditions, i.e. the 0 deg in-phase and 180 deg out-of-phase mode. However, the spectrum of the recorded 450 Hz stimulus, especially under the 0 deg in-phase condition and amplifier setting ‘b’ (SPL 187.7 dB re. 1 µPa), also revealed high harmonics of this frequency (see the peak at 900 Hz in Fig. S1E₁). In addition, the applied phase conditions differed by at least 10 dB (Table S1) with the 0 deg in-phase condition displaying higher SPLs than the 180 deg out-of-phase condition at the same stimulus frequency and using the same amplifier setting.

The isolated goldfish sagitta embedded in porous foam (Fig. 2A) showed a rotational motion (Fig. 2B) when subjected to the 100 Hz stimulus. In zebrafish, the sagittae revealed different in situ motion patterns depending on the test frequency. A simple medio-lateral shift of the two sagittae in the same direction (Fig. 2Ci) was observed when the fish were subjected to the 100 Hz stimulus. In contrast, for the 200 Hz stimulus, the zebrafish sagittae displayed a weak left–right symmetrical rotational movement (Fig. 2Cii).

The in situ motion patterns of the sagittae in the glass catfish were the same for the two test frequencies and just differed with regard to the phase condition in which the two shakers operated. When the shakers were driven 0 deg in phase, both sagittae displayed a rotational motion (Fig. 2Di,Ei; Movie 1). When the shakers were driven 180 deg out of phase, the sagittae showed only a weak (Fig. 2Dii) or no (Fig. 2Eii) rotational motion.

In our study, we aimed to miniaturize the standing wave tube-like setup used previously for 2D radiography plus time measurements at the ESRF to adapt it for high-resolution tomography (4D=three spatial dimensions + time) at the SLS. Our developed miniature standing wave tube-like setup proved to be effective in characterizing the motion patterns of fish auditory structures covering the full three-dimensionality of their morphology. In the following, we will briefly discuss the observed motion patterns with respect to both setups used in this study as well as the potential for further improvements of the miniature standing wave tube-like setup.

The rotational motion of the isolated sagitta in the one-shaker setup and of the glass catfish sagittae in situ in the two-shaker setup indicates that the water-soaked porous foam fixing the sample (sagitta or fish) provided enough degrees of freedom to allow the sample to move ‘freely’. This means that the isolated otolith or the otoliths in situ do not just move along the main axis of sound propagation (i.e. along the tube axis). For the isolated goldfish sagitta, the points at which the foam holds the sample in place directly affect the motion pattern of the otolith in the sound field. Besides the foam limiting the movement of the whole fish, the native attachment of the otolith to the sensory epithelium through the otolithic membrane and the surrounding membrane of the end organ primarily limit the degrees of freedom for the otolith to move in situ (Dunkelberger et al., 1980; Platt and Popper, 1981).

In the case of the two-shaker setup, the distinct rotational motion of the glass catfish sagittae is suggestive of a quite effective separation of sound pressure and sound-induced particle motion in the tube at the tested frequencies (350 Hz, 450 Hz). This rotational motion of the sagittae was expected based on our previous study in goldfish at the ESRF using a standing wave tube-like tank (Schulz-Mirbach et al., 2020) and according to former theoretical considerations (von Frisch, 1938; Freiin von Boutteville, 1935; de Burlet, 1929). In our study on goldfish, the 0 deg in-phase condition and thus the sound pressure maximum in the center of the tank, induced the oscillation of the swim bladder walls which, in turn, resulted in the motion of the coupled Weberian ossicles and finally in a rotation of the saccular otoliths. The motion of the Weberian ossicles was hypothesized to provoke a fluid flow in the sinus impar and the canalis transversus communicans connecting the two saccules to each other. The incoming fluid from the canalis transversus communicans then should impinge on the ‘free’ wing of the sagitta, inducing a rotational motion (von Frisch, 1938; Freiin von Boutteville, 1935; de Burlet, 1929). Accordingly, the rotational sagitta motion we observed in our miniature standing wave tube-like setup when subjecting the fish to the 0 deg in-phase condition perfectly matches this prediction.

The rather high SPLs (>148 dB re. 1 µPa) might influence the motion patterns of the auditory structures. A previous study using 2D radiography tested a series of different SPLs while subjecting fish to a 200 Hz stimulus (Schulz-Mirbach et al., 2020). The amplitudes of the moving saccular otoliths and Weberian ossicles indicated a decrease in motion with decreasing SPL values while the motion patterns themselves were the same regardless of the SPL tested. In future studies, we will conduct similar tests applying descending SPL series to investigate the effects of the moving auditory structures during 4D tomography.

The small test tubes seem to be less suitable to study the motion of fish auditory structures at low frequencies. In the one-shaker setup, this was indicated by the asymmetrical shift of both zebrafish sagittae at 100 Hz together with the spectrum displaying multiple harmonics of the stimulus frequency. In addition, we observed that SPLs measured for the 100 and 200 Hz stimuli in the two-shaker setup did not meet the expected conditions for the two conduction modes (0 deg versus 180 deg) in a standing wave tube. For these frequencies, the SPLs were similar or even slightly larger under the 180 deg out-of-phase mode compared with the 0 deg in-phase mode (data not shown). These observations are in line with theoretical considerations on the correlation between wave length and distance of sound propagation (Rogers and Cox, 1988) and former experiments on fish hearing using standing wave tubes (Myrberg and Spires, 1980; Hawkins and MacLennan, 1976). The 100 Hz and 200 Hz stimuli have long wavelengths of 14.8 m and 7.4 m compared with the dimensions of our test tubes. This is why former studies conducting experiments in standing wave tube setups used long test tubes when subjecting fishes to low frequency stimuli (Myrberg and Spires, 1980: 5 m, tested frequencies 100–800 Hz; Hawkins and MacLennan, 1976: 0.8 m, tested frequencies 10–250 Hz). We are also aware that the hydrophone itself might affect the acoustics of the test tube. To our knowledge, the miniature hydrophone used is, however, currently the smallest available device to measure underwater SPLs. Notwithstanding these issues, our two-shaker setup allowed for a sufficient separation of sound pressure and sound-induced particle motion at 350 Hz and 450 Hz, which is underlined by the differences in sagitta motion due to the 0 deg in-phase and 180 deg out-of-phase condition, respectively. In a subsequent study, we will further optimize our setup to test stimuli in the kilohertz range and at lower SPLs along with direct measurements of particle motion in the tube.

In summary, our miniature standing wave tube-like setup in combination with the high-resolution time-resolved tomography is an approach capable of capturing the motion patterns of fish auditory structures while also covering the structures' three-dimensionality. With the envisaged optimizations, our setup will be even more powerful and will allow us to test hypotheses on the impact of the morphological variation of fish auditory structures on their motion patterns in a sound field.

We thank Anna Jazwinska Müller (University of Fribourg, Switzerland) for providing the zebrafish. We are grateful to Gordan Mikuljan and Philipp Zuppiger (both SLS, TOMCAT beamline), Heinz Pfeiffer (University of Vienna, Department of Behavioral and Cognitive Biology) and Andrzej Szpetkowski (University of Vienna, Department of Neuroscience and Developmental Biology) for assisting in the design and constructing the one-shaker and standing wave tube-like setups. We acknowledge the Paul Scherrer Institut, Villigen, Switzerland, for provision of synchrotron radiation beamtime at the TOMCAT beamline X02DA of the SLS.