We present a simple, intuitive algorithm for visualizing time-varying flow fields that can reveal complex flow structures with minimal user intervention. We apply this technique to a variety of biological systems, including the swimming currents of invertebrates and the collective motion of swarms of insects. We compare our results with more experimentally difficult and mathematically sophisticated techniques for identifying patterns in fluid flows, and suggest that our tool represents an essential ‘middle ground’ allowing experimentalists to easily determine whether a system exhibits interesting flow patterns and coherent structures without resorting to more intensive techniques. In addition to being informative, the visualizations generated by our tool are often striking and elegant, illustrating coherent structures directly from videos without the need for computational overlays. Our tool is available as fully documented open-source code for MATLAB, Python or ImageJ at www.flowtrace.org.

Flow visualization is an essential technique for studying biological processes arising in many diverse areas, including ecology (Rohr et al., 2002), biomechanics (Bartol et al., 2016) and molecular biology (Boot et al., 2008; Lindken et al., 2009). Observations of the trajectories of many tracer particles – whether fluorescent beads, microbubbles, vesicles inside a moving cell, or flocking bacteria – can reveal rich information about the way that biological motion such as molecular transport, ciliary flows or active force generation is coordinated over many length and time scales (Shapiro et al., 2014; Han et al., 2015; Deforet et al., 2012; Bharadvaj et al., 1982). Additionally, flow visualization aids in characterization and secondary validation of many standard techniques, such as microfluidics and flow cytometry (Santiago et al., 1998; Eyal and Quake, 2002).

The simplest qualitative flow visualization techniques involve observations of the motion of passive scalars, like dyes or smoke, as they are advected by the flow. These experiments have the benefit of being relatively straightforward to perform, and can often yield immediate insight into the global structure and mixing properties of a flow (Merzkirch, 2012). However, at length and time scales dominated by diffusive effects, or in flows characterized by large separations in the time scales of different processes, the results of such studies can be difficult to interpret (Miyake et al., 1993; Miles and Lempert, 1997). As a result, in many contexts, quantitative flow-characterization techniques based on the motion of tracer particles are preferable (Bayraktar and Pidugu, 2006; Mercado et al., 2012; Kertzscher et al., 2008).

However, standard quantitative flow-visualization techniques – particle tracking and particle image velocimetry (PIV) – are sufficiently difficult to implement and optimize that these rigorous fluid dynamical visualization techniques remain prohibitive in many experimental contexts (Miles and Lempert, 1997). While PIV and related techniques have been widely applied and optimized for certain biological systems, such as the study of fish swimming (Stamhuis and Videler, 1995) or blood flow mechanics (Bharadvaj et al., 1982), in less-established contexts these techniques require specialized modifications of apparatus such as laser light sheets to be constructed (Lindken et al., 2009; Adrian and Westerweel, 2011). As a result, system-specific techniques are often necessary (Stamhuis et al., 2002), particularly when particle motion is only partially visible in the data, such as from image streaks (Santiago et al., 1998) or out-of-focus drift due to limited depth of field (Olsen and Adrian, 2000). This issue is even more prominent for flows in non-traditional media, such as in the collective motion of flocks and herds (Garcimartín et al., 2015; Attanasi et al., 2014a,b; Vicsek and Zafeiris, 2012).

Nonetheless, many standard concepts in fluid mechanics – such as vortices, jets and turbulence – are widely known to researchers throughout the sciences, who may recognize the likely presence of these features in their data even without resorting to quantitative flow-characterization tools (Rau et al., 2006; Hejnowicz and Kuczyńska, 1987). This suggests that simpler flow-visualization tools are necessary for systems in which dye-based qualitative techniques are unavailable, but quantitative tracer–particle studies are unnecessary.

Here, we describe Flowtrace, an algorithm and associated open-source tool that can assist in identifying characteristic flow structures in experimental videos. This primarily qualitative technique is sufficiently straightforward and intuitive to be used either as a primary analysis tool for presence/absence studies or to motivate the use of more complicated flow-quantification methods. The technique is based purely on image processing of the input data, rather than numerical reconstruction of scalar fields like vorticity, allowing it to be used as a straightforward ‘first pass’ characterization technique for biological systems where traditional techniques are either unnecessary to support qualitative observations or prohibitively difficult because of the length and time scales involved. While Flowtrace is a straightforward algorithm, to our knowledge it has not been reported or widely adopted in the literature – and, importantly, we find that it has surprising utility for studying flow patterns in a wide range of biological data sets.

Algorithm

The algorithm generalizes a common technique for generating long-exposure photographs from videos, in which the maximum (or minimum) intensity projection of a time series of images is taken in order to generate pathlines for bright objects moving against a dark background – resulting in ‘motion streaks’ across the image (Shapiro et al., 2014). This technique has previously been used to create star trails, a popular astronomical visualization generated by taking the maximum intensity projection of a stabilized video of the night sky (West and Cameron, 2006 preprint).

Flowtrace primarily extends this technique by using a single long experimental video, and then iteratively taking the maximum intensity projection of small groups of successive video frames in order to generate sequential images showing pathlines at different times. The resulting time series of pathline images reveals how the shapes of the pathlines change over time. For example, in a 100 frame raw experimental video, a Flowtrace video with 30 frame traces consists of the maximum intensity projection of frames 1–31, 2–32, 3–33, etc., and so forth. The sequence ends when frames 70–100 are projected, resulting in a 70 frame video. In addition to this basic operation, various other operations can be combined with the maximum intensity projection operator in order to yield improved results. The process is illustrated graphically in Fig. 1C.

Fig. 1.

The Flowtrace algorithm. (A) Three stills from a Flowtrace video of the feeding currents generated by the larva of the starfish Patiria miniata (Movie 1 taken from Gilpin et al., 2017; τ=3 s, time points=0, 30, 90 s, scale bar 500 μm). (B) The gyration of the protozoan Stentor sp. as it filters water containing 6 μm beads (Movie 2; τ=3 s, time points=0, 6.5, 18 s, scale bar 175 μm). (C) A false-color detail from B illustrating the ‘sliding projection’ used by Flowtrace to generate pathlines (τ=3 frames).

Fig. 1.

The Flowtrace algorithm. (A) Three stills from a Flowtrace video of the feeding currents generated by the larva of the starfish Patiria miniata (Movie 1 taken from Gilpin et al., 2017; τ=3 s, time points=0, 30, 90 s, scale bar 500 μm). (B) The gyration of the protozoan Stentor sp. as it filters water containing 6 μm beads (Movie 2; τ=3 s, time points=0, 6.5, 18 s, scale bar 175 μm). (C) A false-color detail from B illustrating the ‘sliding projection’ used by Flowtrace to generate pathlines (τ=3 frames).

Symbolically, let each frame of the movie be a vector of pixel values and locations, vij[t], where t=1,2,…N represents the index of a frame in a video consisting of N frames, and i and j denote the coordinates of a pixel in the image. Suppose that the tracer particles are brightly colored objects moving against a dark background. In this case, a series of maximum-intensity projections, p[t], may be defined in terms of a forward convolution operator:
formula
(1)
where:
formula
(2)

where M<N is some subset of the frames in the video. As the time index t ‘slides’ forward across successive indices 1,2,3,…, the maximum intensity projection is taken across successive runs of M frames that each differ by two images (the first and the last). This results in a set of maximum intensity projections, , t=1,2,…,NM, that constitute a new video generated from the original dataset. Importantly, the number of frames in the generated video (NM) is almost equal to the number of frames in the original video (N). In this paper, we refer to each subsequence of M images as a substack, and the sequence of positions taken by a single particle moving across M frames as a pathline. The parameter M, the time scale over which particle pathlines are visualized in each frame, represents the only parameter that the user must specify in order to use the tool. For a time series of images taken with a fixed time spacing Δt (equivalent to 1/frame rate), the total pathline projection time is defined as τ≡MΔt.

During convolution, the projection operator P ‘slides’ across the entire sequence of frames, operating on the video in overlapping sets of M frames. This operator can be composed with other pre-processing operations in order to achieve different effects; in the code described below, other operations defined include median subtraction (to remove slowly moving objects), color inversion (for dark objects moving against a lighter background), differential weighing (coloring or darkening each frame in the M frame sequence a different amount, in order to show a gradient across the pathlines indicating time) and pairwise differencing (to isolate objects that move faster than 1 pixel per frame). Each of these operations symbolically represents composition before convolution, such that the final image series is {(v}[t], where G is the pre-processing operation.

Software package and options

The algorithm is implemented as an open-source package for MATLAB, Python or ImageJ at www.flowtrace.org. Full tutorials and sample image sets are provided there. Table 1 summarizes the primary user-specified arguments and options available for the code; optional arguments are passed as a struct object in MATLAB, as keyword arguments in Python, and as checkboxes in a GUI for ImageJ.

Table 1.

Options and parameters for Flowtrace

Options and parameters for Flowtrace
Options and parameters for Flowtrace

The primary options for the software involve removal of background drift and crossing pathlines, which complicate interpretation of the videos. Oftentimes a dataset features two well-separated velocity scales: one for advected tracer particles and one for background drift, bulk flow, etc. In these cases, Flowtrace performs best when the projection time τ is sufficient for fast particles to travel far within the field of view, while relatively slow objects move very little. This is true for the pathlines shown for the two feeding current-generating organisms shown in Fig. 1, for which τ is long compared with the mean transit time of tracer particles, but short compared with the gradual motion of each organism's body. As a result, features of each organism's anatomy remain sharp in the image. However, in many cases particle advection time scales are not well separated from background motion (resulting in motion blur for slowly moving objects) or stationary objects and obstacles in the image obscure the pathlines. In these cases, it is useful to apply a background subtraction operation to each substack before performing the projection (the ‘take_diff’ option in the software). For objects moving slowly relative to the tracer particles, but fast enough to exhibit noticeable motion blur, the most aggressive background-subtraction option available in the software takes the pairwise differences among all consecutive images before applying the projection. However, tracer particles that move less than a pixel between successive frames will also vanish. Alternate background subtraction options in Flowtrace include subtracting either the median or the first image from the stack before projection (‘subtract_median’ or ‘subtract_first’, respectively). Occasionally, it is convenient to highlight the direction of time in the pathlines, particularly when still frames from the output time series are used for analysis. In this case, directionality can be indicated by applying a color gradient across time (‘color_series’), or by applying a linear intensity gradient (‘fade_tails’).

Software availability

The source code for Flowtrace is available for Fiji, ImageJ, Python 2, Python 3 and MATLAB. The full code base, as well as screenshots, tutorials and installation instructions, may be accessed at www.flowtrace.org. The primary user arguments and options for the code are summarized in Table 1. Optional arguments are passed as struct objects in MATLAB, as keyword arguments in Python and as GUI checkboxes in ImageJ.

The individual GitHub repositories and version histories for the three implementations are open for pull requests and forks on GitHub at https://github.com/williamgilpin/flowtrace_imagej, https://github.com/williamgilpin/flowtrace_python and https://github.com/williamgilpin/flowtrace_matlab. A gallery of videos generated using the technique is available at www.flowtrace.org/flowtrace_docs/gallery.html.

Experimental methods

For bead studies, organisms were placed in a droplet of water (or an appropriate saline buffer) on a glass slide. A 1:100 dilution mixture of 6 μm polystyrene beads was placed into the droplet and gently mixed using a pipette tip. A coverslip was prepared by dragging each of its corners through modeling clay, resulting in ‘feet’ of 500–600 μm height. The coverslip was then placed on the slide feet-down, such that the organism was confined between the slide and the coverslip. Images were captured using an ORCA C11440 CCD (for color panels, a Canon EOS T3i DSLR or an Apple iPhone 5s was used). Videos were split into single frames using Fiji, and the resulting time series of images were processed using the Flowtrace software. The three implementations (Fiji/ImageJ, MATLAB, and Python 2 and 3) yielded the same results. Methods specific to individual datasets are described below.

Movie 1: an 8 week old starfish larvae modulates its swimming currents in order to increase the vorticity it generates, and thus its feeding rate. This video is taken from Gilpin et al. (2017), which discusses this phenomenon in more detail. This video was captured at 4× magnification and 20 frames s−1 on an inverted Nikon microscope with dark field illumination; the water contains 6 μm beads. The projection time is τ=3 s, and the video is shown at 8× true speed.

Movie 2: Stentor sp. generates a large dipolar feeding current using its primary ciliary band. This current is periodically disrupted when the organism rotates its stalk to invert the position of the ciliary stripes. A full 180 deg rotation of the organism (and its associated feeding currents) is shown in the video. Images were captured at 30× magnification and 20 frames s−1 on an inverted Nikon microscope; the water contains 6 μm beads as well as algae and other detritus advected by the feeding current. In Flowtrace, ‘subtract_median’ was used to remove background objects. The projection time was τ=3 s, and the video is shown at 8× true speed. This movie was generated from a new dataset taken originally for this work.

Movie 3: a hyperbolic stagnation point represents a transport barrier for the cerebrospinal fluid in a mouse brain ventricle. Original video taken by Faubel et al. (2016) using 1 μm fluorescent spheres as tracer particles. In Flowtrace, ‘subtract_median’ was used to remove background objects and gradual variations in overall intensity across the image. The projection time was τ=0.67 s, and the video is shown at 1× true speed.

Movie 4: a sea anemone pumps seawater into its body cavity, creating a short-lived jet that entrains particles. The animal was suspended in filtered seawater containing 6 μm beads. Videos were taken at 1 frame s−1 on an ORCA C11440 CCD and Nikon microscope with 1× magnification. In Flowtrace, ‘subtract_median’ was used to remove background objects, and the final projected movies were color inverted to ease visualization. The projection time was τ=240 s, and the video is shown at 48× true speed. This movie was generated from a new dataset taken originally for this work.

Movie 5: a 2 day old veliger larva of a moon snail generating a steady dipolar feeding current, punctuated by brief interruptions. The animal was suspended in filtered seawater with red 6 μm beads. Images were taken at 30 frames s−1 on a Canon EOS T3i DSLR and Nikon microscope with 10× magnification. In Flowtrace, ‘subtract_median’ was used to remove background objects. The projection time was τ=2 s, and the video is shown at 8× true speed. This movie was generated from a new dataset taken originally for this work.

Movie 6: a swarm of flying midges gradually tightens in shape. Original video taken at 170 frames s−1 by Attanasi et al. (2014a,b). In Flowtrace, ‘subtract_median’ and ‘color_series’ were enabled in order to remove background objects and color code the resulting images by time. The projection time was τ=333 ms, and the video is shown at 1/6× true speed.

Movie 7: a school of 70 fish undergoes a spontaneous transition from a ‘milling’ to a ‘swarming’ behavioral state. Original video taken by Tunstrøm et al. (2013). In Flowtrace, ‘subtract_median’ and ‘fade_tails’ were enabled in order to remove background objects and intensity code the resulting images by time. The projection time was τ=5.33 s.

Movie 8: The feeding current of the sessile, predatory protozoan Stentor in a sample of pond water. The large dipolar entrainment flow field generated by the organism captures some particles, but some smaller algae and other swimming organisms in the water appear to easily escape the vortices. This video was captured at 4× magnification and 20 frames s−1 on an inverted Nikon microscope with dark field illumination. The projection time was τ=3 s, and the video is shown at 8× true speed. This movie was generated from a new dataset taken originally for this work.

Comparison and validation with other techniques

Quantitative flow visualization involves using image analysis techniques to take discrete integral transforms of a dataset, either for the sake of performing convolution in order to extract a velocity field (PIV) or in order to establish unique identities of objects (as in particle tracking velocimetry, PTV). In these methods, an underlying model of the flow is assumed by the technique, and coherent structures may be visualized either by numerically integrating trajectories or by defining a spatially resolved scalar field (such as the strain or vorticity) and plotting contours. The techniques compared here (PIV and PTV) are subject to the basic drawbacks: they can be computationally demanding, and require the experimental dataset to have certain properties (such as narrow depth of field or high tracer particle density) in order to be well posed.

Fig. 2 compares pathlines generated by Flowtrace with several common methods of detecting coherent structures based on PIV data. A video on larval starfish swimming published in a previous study is used as a test dataset (Gilpin et al., 2017). In the referenced video, Flowtrace shows a stable arrangement of slowly varying vortices around the periphery of a starfish larva held stationary (see fig. 1 and video 1 of Gilpin et al., 2017).

Fig. 2.

Comparison of Flowtrace with other methods for identifying coherent structures. (A) Flowtrace frame; (B) backward time finite-time Lyapunov exponent (FTLE); (C) vorticity; (D) forward time FTLE. All color plots have been rescaled so that the extremal absolute value of the scalar field being plotted corresponds to an intensity of 1, such that all color maps have the same range. Deep red regions in backward-time FTLE (B) correspond to attracting coherent structures, whereas deep red regions in forward-time FTLE (D) correspond to repelling structures. All colored scalar fields have been median smoothed with a spatial kernel of a size smaller than the particle image velocimetry (PIV) mesh.

Fig. 2.

Comparison of Flowtrace with other methods for identifying coherent structures. (A) Flowtrace frame; (B) backward time finite-time Lyapunov exponent (FTLE); (C) vorticity; (D) forward time FTLE. All color plots have been rescaled so that the extremal absolute value of the scalar field being plotted corresponds to an intensity of 1, such that all color maps have the same range. Deep red regions in backward-time FTLE (B) correspond to attracting coherent structures, whereas deep red regions in forward-time FTLE (D) correspond to repelling structures. All colored scalar fields have been median smoothed with a spatial kernel of a size smaller than the particle image velocimetry (PIV) mesh.

Vorticity

One simple type of flow visualization involves isocontours of vorticity and other scalar fields derived from PIV. For the analysis in Fig. 2C, the vorticity correctly localizes to regions of the flow corresponding to steady vortices, and the color and intensity of the shaded regions match the apparent local rotation directions and intensity based on the length and direction of the pathlines. Thus, for persistent vortex structures, the vorticity field and the pathlines agree.

However, it is apparent in Fig. 2C that, as a metric based on taking the spatial derivative of experimental data, the vorticity has large spatial noise, despite the velocity field having been averaged across time and space to reduce correlation errors in the PIV (Batteen and Han, 1981). Whether noise comes from the precision of the PIV measurement or true fluctuations in the amplitude of the velocity field across space, this noise complicates interpretation of basic qualitative questions, such as the exact locations and relative sizes of the vortical regions. Because calculation of vorticity relies upon some underlying assumptions regarding the flow in the form of the relative mesh size and spatial and temporal averaging applied to the PIV dataset, vorticity and similar finite-difference metrics can misplace the center point or relative scale of vortex structures (Foucaut and Stanislas, 2002). Thus, for qualitative observations in slowly varying flows, Flowtrace may be preferable for identifying coherent structures like vortices.

However, a case where vorticity plots and Flowtrace yield different qualitative visualizations arises in quickly time-varying flows, in which vorticity may be short lived enough that particles do not have sufficient time to circumscribe vortices. For example, studies of the swimming flagellate Chlamydomonas report closed streamlines both in the instantaneous velocity field (Guasto et al., 2010) and in the velocity field averaged across multiple flagellar beat cycles (Drescher et al., 2010). However, because of the time-varying structure of this field during the beat, areas of high vorticity move between the fore and aft of the organism during the swimming stroke (Guasto et al., 2010), and so pathlines generated by Flowtrace would show non-closed and potentially overlapping paths. Thus, plotting a scalar field from PIV (like vorticity) may be preferable when the time scale of particle advection within the field of view is comparable to the time scale of flow variation. A similar argument would apply to other common scalars computed as the finite differences of PIV data, such as the strain and shear (Colin et al., 2010; Kiørboe and Visser, 1999), as well as more sophisticated techniques based on computing Eulerian quantities like the local acceleration (Kasten et al., 2011; Van Gelder, 2012).

Finite-time Lyapunov exponents

More sophisticated tools for the identification of structures in time-varying flows are based on the detection of Lagrangian coherent structures (LCS), which are bounded regions of a flow with dynamics that are qualitatively distinct from the rest of the flow.

For example, at high Reynolds number, vorticity is conserved and remains localized, causing patches of vorticity that remain intact as they are advected by the flow. In this case, the patches of vorticity essentially act as tracers, even in turbulent conditions (Newton, 2013). Vortices at high Reynolds numbers thus represent examples of ‘attracting LCS’, which are bounded regions that tend to pull trajectories of neighboring particles towards themselves, and in some cases are mathematically equivalent to the ‘islands of stability’ observed in the solutions of classical dynamical systems exhibiting chaos (Beron-Vera et al., 2010). Based on this analogy, LCS can be identified using the finite-time Lyapunov exponent (FTLE), a computationally efficient approximation of the classical Lyapunov exponent, which measures the tendency of trajectories originating from a given location to diverge or converge over time (Shadden et al., 2005). The derivation and properties of the FTLE are discussed in detail in many recent reviews (Haller, 2015; Peacock and Dabiri, 2010); for our purposes, it is a scalar field defined across space that can be used to identify regions in a flow that attract or repel trajectories.

While LCS are fundamentally a property of the underlying velocity field present in a system, they can often be visualized through the manner in which they advect passive tracers – hence, coherent flow structures such as smoke rings or whirlpools are easily visualized (Haller, 2015). Thus, the FTLE field should detect key features visible in the pathline output of Flowtrace, such as vortices and stagnation points.

Fig. 2B,D shows the result of computing the FTLE field for the starfish dataset. First, PIV was applied to the original video in order to generate an estimate of the velocity field as a function of time at various points on a fixed spatial mesh (Taylor et al., 2010). Then, the FTLE field was generated by interpolating this field and numerically integrating trajectories originating from various points in the image, using a tool developed by Shadden et al. (2006). The FTLE field was then smoothed with a median filter with a width equal to the size of the lattice on which the PIV field was calculated, in order to remove artifacts in the field arising from discretization of space. The resulting scalar field contains both positive (red) and negative (blue) values, and in Fig. 2 it is overlaid on the first frame of a Flowtrace video for the same dataset.

Fig. 2 shows FTLE fields for both forwards (Fig. 2D) and backwards (Fig. 2B) integration schemes, which respectively detect repelling and attracting LCS in the flow field. In both cases, the regions between adjacent vortices display the highest absolute intensity. Diverging streamlines produce regions with high positive FTLE values (red regions in the forward time plot), while converging streamlines produce regions with large negative FTLE values (red regions in the backward time plot). The ‘zero FTLE’ contours clearly run orthogonally to the pathlines shown by Flowtrace, as would be expected for a quasi-steady flow. The peaks and minima of the FTLE field roughly correspond to the locations of stagnation points along the boundary of the animal, with stagnation points that result in jet-like ejections of water from the surface having large positive FTLE regions, and stagnation points that pull water into the surface having large negative FTLE regions. The FTLE field and the pathlines shown by Flowtrace thus yield good agreement.

However, the FTLE field by itself is not straightforward to interpret – isocontours of the computed field do not necessarily indicate separatrices in the flow field. This is due, in part, to the lack of guaranteed coincidence of FTLE ridges with true material lines (transport barriers) in finite-time simulations (Johnson and Meneveau, 2015). In general, there is some ambiguity regarding the optimal algorithm for the definition of ‘ridges’ in an FTLE field (Haller, 2015, 2011). However, for a quasi-static flow field such as the one shown in Fig. 2, Flowtrace is able to clearly delineate pathlines belonging to different vortex regions, because the pathlines show true transport in the system.

However, recently, more sophisticated topology-based techniques based on the adjacency matrix associated with neighboring trajectories (instead of changes in the Euclidean distance) have allowed coherent structures to be determined from sparser datasets (such as those generated by particle tracking experiments) (Schlueter-Kuck and Dabiri, 2017). However, identification of structures with high spatial resolution still requires interpolation of the velocity field and subsequent integration of trajectories, leading it to be susceptible to the same limitations as the above.

Finally, the quality of the FTLE field and the associated ridges and minima that signal the presence of LCS is highly dependent on the quality and resolution of the PIV data from which the field is generated. Thus, LCS detection does not solve the original issue that motivates the use of Flowtrace – that of creating a simple and straightforward qualitative visualization technique that requires comparatively less experimental optimization. However, it does confirm that, at least for quasi-static cases, Flowtrace is capable of identifying transport barriers in a flow, which are essential for gaining qualitative understanding.

Application of Flowtrace to a variety of biological systems provides surprising insight into the complicated dynamics of unsteady flows. By varying the projection time interval, τ (and thus the pathline lengths), biological phenomena may be investigated over a wide range of length and time scales. Detailed methods for each of the experiments and Movies are discussed in Materials and Methods.

Fig. 1A shows three representative frames from a Flowtrace movie of a starfish larva (Patiria miniata), which in previous work we have shown creates dynamic vortex arrays around its body as it continuously adjusts its feeding currents (Gilpin et al., 2017). The full Flowtrace movie from which Fig. 1A is generated shows the animal smoothly alternating between distinct swimming and feeding vortex patterns, indicating how these dynamic flow patterns represent distinct behaviors controlled by the animal (Movie 1; discussed further in Gilpin et al., 2017). Fig. 1B shows similar ciliary flows generated during stationary filter feeding by the protozoan Stentor sp. (size ∼50 μm), which generates a toroidal current that draws small prey towards its stalk. The videos and images (τ=3 s) capture the helical motion of algae particles as the organism slowly rotates its stalk (Movie 2), and a larger field of view shows the effect of the feeding current on microorganisms swimming nearby (Movie 8).

Flowtrace can be applied to any standard experimental data in which tracer particles (such as fluorescent beads) travel for an extended period in the imaging plane. Application of the tool to data from a recent study of ciliary currents generated in a mouse brain revealed the presence of a prominent hyperbolic stagnation point associated with a transport barrier in the cerebrospinal fluid, an observation identified using full particle tracking in the original study (Faubel et al., 2016) (Fig. 3A, Movie 3). A similar technique allows identification of coherent flow structures formed by the sea anemone Aiptasia pallida (∼1 mm): an inverted-color Flowtrace video (τ=4 min) shows the breakup of a water jet as the animal peristaltically pumps water into its body cavity (Fig. 3B, Movie 4). In a color DSLR video of the larva of the moon snail Crepidula fornicata (∼1 mm), Flowtrace creates a color video by projecting each channel separately, generating a true color video of the formation of the dipolar flow field created by the swimming animal (Fig. 3C, Movie 5).

Fig. 3.

Application of Flowtrace to flow visualization problems. (A) Transport of particles by the cerebrospinal fluid in a mouse's brain reveals the presence of a nearly stationary hyperbolic stagnation point flow. Because of high particle density, the color of the original video has been inverted to ease visualization (Movie 2 generated from data by Faubel et al., 2016; τ=0.67 s, time points=0, 3, 6 s, scale bar 50 μm). (B) A sea anemone taking in a jet of water containing 6 μm beads. As above, the video has been inverted to ease visualization (Movie 4; τ=4 min, time points=0, 5.4, 18 min, scale bar 1 mm). (C) A swimming moon snail larva, with 6 μm beads mixed into the water (Movie 5; τ=2 s, time points=0, 6, 20 s, scale bar 350 μm).

Fig. 3.

Application of Flowtrace to flow visualization problems. (A) Transport of particles by the cerebrospinal fluid in a mouse's brain reveals the presence of a nearly stationary hyperbolic stagnation point flow. Because of high particle density, the color of the original video has been inverted to ease visualization (Movie 2 generated from data by Faubel et al., 2016; τ=0.67 s, time points=0, 3, 6 s, scale bar 50 μm). (B) A sea anemone taking in a jet of water containing 6 μm beads. As above, the video has been inverted to ease visualization (Movie 4; τ=4 min, time points=0, 5.4, 18 min, scale bar 1 mm). (C) A swimming moon snail larva, with 6 μm beads mixed into the water (Movie 5; τ=2 s, time points=0, 6, 20 s, scale bar 350 μm).

In addition to passive fluid tracer particles, Flowtrace can be applied to active particles and ecological data. For these datasets, applying a color or brightness gradient along the pathlines proves beneficial when many pathlines overlap as a result of particles appearing in the same location at different times. Fig. 4 shows an application of two such cases taken from the collective motion literature (Attanasi et al., 2014a,b; Tunstrøm et al., 2013). In a swarm of flying midges (Dasyhelea flavifrons, flock ∼1 m wide), Flowtrace with color gradient across time allows rapid tightening of the flock to be visualized (Fig. 4A, Movie 6, raw data taken from Attanasi et al., 2014a,b). Similarly, in a video of 70 freely swimming minnows (Notemigonus crysoleucas), Flowtrace can be used to identify changes in the schooling behavior: the school undergoes a transition from a visibly rotary ‘milling’ state, to a directionally aligned collective state, to a disordered ‘swarm’ state (Fig. 4B, Movie 7, raw data taken from Tunstrøm et al., 2013). Relative to the raw video, the Flowtrace video makes it easier to visualize the onset of these transitions, which arise when a subset of individuals spontaneously polarizes and travels in a single direction.

Fig. 4.

Flowtrace applied to collective animal motion. (A) Three frames from a movie of a flock of midges, with pathlines temporally color coded from blue to orange (Movie 6 generated from data by Attanasi et al., 2014a,b; τ=333 ms, time points=0, 0.66, 1.3 s, scale bar 60 mm). (B) A transition from ‘milling’ to ‘swarming’ behavior in a school of 70 minnows (Movie 7 generated from data by Tunstrøm et al., 2013; τ=5.33 s, time points=0, 9.1, 17.1 s, scale bar 0.5 m).

Fig. 4.

Flowtrace applied to collective animal motion. (A) Three frames from a movie of a flock of midges, with pathlines temporally color coded from blue to orange (Movie 6 generated from data by Attanasi et al., 2014a,b; τ=333 ms, time points=0, 0.66, 1.3 s, scale bar 60 mm). (B) A transition from ‘milling’ to ‘swarming’ behavior in a school of 70 minnows (Movie 7 generated from data by Tunstrøm et al., 2013; τ=5.33 s, time points=0, 9.1, 17.1 s, scale bar 0.5 m).

The simple sliding projection technique used by Flowtrace appears to be largely unknown in the biological sciences and fluid dynamics literature, despite the ease with which it can be implemented. Flowtrace can reproduce the core qualitative conclusions of several studies, including our recent study on larval starfish swimming (Gilpin et al., 2017), in which the key observation of distinct feeding and swimming vortex arrays generated by the animals is readily seen in Flowtrace videos but difficult to discern using PIV, the standard method of analyzing such data (Lindken et al., 2009). The algorithm works particularly well for studies of organismal feeding currents, in which there is a wide separation between the time scales of advection and behavior-driven flow variation – such that tracer particles have sufficient time to map out the structure of the flow field before the field undergoes further variation. Moreover, feeding phenomena typically involve small length scales and long time scales, for which traditional dye advection visualization techniques would fail because of rapid diffusive mixing.

Flowtrace has been compared with other techniques for identifying structures in fluid flows, and it can qualitatively reproduce the results of more sophisticated dynamical analysis using either vorticity contours or FTLE (Haller, 2015; Shadden et al., 2006). The tool is available in efficient, multithreaded implementations for Fiji/ImageJ, Python 2 and 3, and MATLAB. A broad community – from microscopists to ecologists to fluid physicists – may use and even further improve Flowtrace, and so the full source code and documentation are available at www.flowtrace.org or for pull requests on GitHub.

The authors thank L. Y. Esherick and J. R. Pringle for providing anemones, and D. N. Clarke and C. J. Lowe for providing microscopy equipment and moon snails.

Author contributions

Conceptualization: W.G., V.N.P., M.P.; Methodology: W.G., V.N.P., M.P.; Software: W.G.; Validation: W.G.; Formal analysis: W.G.; Investigation: W.G., V.N.P., M.P.; Resources: W.G., M.P.; Data curation: W.G., V.N.P., M.P.; Writing - original draft: W.G., V.N.P., M.P.; Writing - review & editing: W.G., V.N.P., M.P.; Visualization: W.G.; Supervision: W.G., M.P.; Project administration: W.G., M.P.; Funding acquisition: W.G., M.P.

Funding

This work was supported by a National Science Foundation Graduate Research Fellowship (DGE-114747); a National Geographic Society Young Explorers Grant (to W.G.); and an Army Research Office (ARO) Multidisciplinary University Research Initiative (MURI) Grant W911NF-15-1-0358 and National Science Foundation CAREER Award (to M.P.).

Adrian
,
R. J.
and
Westerweel
,
J.
(
2011
).
Particle Image Velocimetry
, vol.
30
.
Cambridge
:
Cambridge University Press
.
Attanasi
,
A.
,
Cavagna
,
A.
,
Del Castello
,
L.
,
Giardina
,
I.
,
Grigera
,
T. S.
,
Jelić
,
A.
,
Melillo
,
S.
,
Parisi
,
L.
,
Pohl
,
O.
,
Shen
,
E.
, et al. 
(
2014a
).
Information transfer and behavioural inertia in starling flocks
.
Nat. Phys.
10
,
691
-
696
.
Attanasi
,
A
.,
Cavagna
,
A.
,
Del Castello
,
L.
,
Giardina
,
I.
,
Melillo
,
S.
,
Parisi
,
L.
,
Pohl
,
O.
,
Rossaro
,
B.
,
Shen
,
E.
,
Silvestri
,
E.
, et al. 
(
2014b
).
Collective behaviour without collective order in wild swarms of midges
.
PLoS Comput. Biol.
10
,
e1003697
.
Bartol
,
I. K.
,
Krueger
,
P. S.
,
Jastrebsky
,
R. A.
,
Williams
,
S.
and
Thompson
,
J. T.
(
2016
).
Volumetric flow imaging reveals the importance of vortex ring formation in squid swimming tail-first and arms-first
.
J. Exp. Biol.
219
,
392
-
403
.
Batteen
,
M. L.
and
Han
,
Y.-J.
(
1981
).
On the computational noise of finite-difference schemes used in ocean models
.
Tellus
33
,
387
-
396
.
Bayraktar
,
T.
and
Pidugu
,
S. B.
(
2006
).
Characterization of liquid flows in microfluidic systems
.
Int. J. Heat Mass Transf.
49
,
815
-
824
.
Beron-Vera
,
F. J.
,
Olascoaga
,
M. J.
,
Brown
,
M. G.
,
Koçak
,
H.
and
Rypina
,
I. I.
(
2010
).
Invariant-tori-like lagrangian coherent structures in geophysical flows
.
Chaos
20
,
017514
.
Bharadvaj
,
B.
,
Mabon
,
R.
and
Giddens
,
D.
(
1982
).
Steady flow in a model of the human carotid bifurcation. part i--flow visualization
.
J. Biomech.
15
,
349
-
362
.
Boot
,
M. J.
,
Westerberg
,
C. H.
,
Sanz-Ezquerro
,
J.
,
Cotterell
,
J.
,
Schweitzer
,
R.
,
Torres
,
M.
and
Sharpe
,
J.
(
2008
).
In vitro whole-organ imaging: 4d quantification of growing mouse limb buds
.
Nat. Methods
5
,
609
-
612
.
Colin
,
S. P.
,
Costello
,
J. H.
,
Hansson
,
L. J.
,
Titelman
,
J.
and
Dabiri
,
J. O.
(
2010
).
Stealth predation and the predatory success of the invasive ctenophore mnemiopsis leidyi
.
Proc. Natl Acad. Sci. USA
107
,
17223
-
17227
.
Deforet
,
M.
,
Parrini
,
M. C.
,
Petitjean
,
L.
,
Biondini
,
M.
,
Buguin
,
A.
,
Camonis
,
J.
and
Silberzan
,
P.
(
2012
).
Automated velocity mapping of migrating cell populations (AVeMap)
.
Nat. Methods
9
,
1081
-
1083
.
Drescher
,
K.
,
Goldstein
,
R. E.
,
Michel
,
N.
,
Polin
,
M.
and
Tuval
,
I.
(
2010
).
Direct measurement of the flow field around swimming microorganisms
.
Phys. Rev. Lett.
105
,
168101
.
Eyal
,
S.
and
Quake
,
S. R.
(
2002
).
Velocity-independent microfluidic flow cytometry
.
Electrophoresis
23
,
2653
-
2657
.
Faubel
,
R.
,
Westendorf
,
C.
,
Bodenschatz
,
E.
and
Eichele
,
G.
(
2016
).
Cilia-based flow network in the brain ventricles
.
Science
353
,
176
-
178
.
Foucaut
,
J.-M.
and
Stanislas
,
M.
(
2002
).
Some considerations on the accuracy and frequency response of some derivative filters applied to particle image velocimetry vector fields
.
Meas. Sci. Technol.
13
,
1058
.
Garcimartín
,
A.
,
Pastor
,
J. M.
,
Ferrer
,
L. M.
,
Ramos
,
J. J.
,
Martín-GóMez
,
C.
and
Zuriguel
,
I.
(
2015
).
Flow and clogging of a sheep herd passing through a bottleneck
.
Phys. Rev. E
91
,
022808
.
Gilpin
,
W.
,
Prakash
,
V. N.
and
Prakash
,
M.
(
2017
).
Vortex arrays and ciliary tangles underlie the feeding-swimming trade-off in starfish larvae
.
Nat. Phys.
13
,
380
-
386
.
Guasto
,
J. S.
,
Johnson
,
K. A.
and
Gollub
,
J. P.
(
2010
).
Oscillatory flows induced by microorganisms swimming in two dimensions
.
Phys. Rev. Lett.
105
,
168102
.
Haller
,
G.
(
2011
).
A variational theory of hyperbolic lagrangian coherent structures
.
Phys. D
240
,
574
-
598
.
Haller
,
G.
(
2015
).
Lagrangian coherent structures
.
Annu. Rev. Fluid Mech.
47
,
137
-
162
.
Han
,
S. J.
,
Oak
,
Y.
,
Groisman
,
A.
and
Danuser
,
G.
(
2015
).
Traction microscopy to identify force modulation in subresolution adhesions
.
Nat. Methods
12
,
653
-
656
.
Hejnowicz
,
Z.
and
Kuczyńska
,
E. U.
(
1987
).
Occurrence of circular vessels above axillary buds in stems of woody plants
.
Acta Soc. Bot. Pol.
56
,
415
-
419
.
Johnson
,
P. L.
and
Meneveau
,
C.
(
2015
).
Large-deviation joint statistics of the finite-time lyapunov spectrum in isotropic turbulence
.
Phys. Fluids
27
,
085110
.
Kasten
,
J.
,
Reininghaus
,
J.
,
Hotz
,
I.
and
Hege
,
H.-C.
(
2011
).
Two-dimensional time-dependent vortex regions based on the acceleration magnitude
.
IEEE Trans. Vis. Comput. Graph
17
,
2080
-
2087
.
Kertzscher
,
U.
,
Berthe
,
A.
,
Goubergrits
,
L.
and
Affeld
,
K.
(
2008
).
Particle image velocimetry of a flow at a vaulted wall
.
Proc. Inst. Mech. Eng. H
222
,
465
-
473
.
Kiørboe
,
T.
and
Visser
,
A. W.
(
1999
).
Predator and prey perception in copepods due to hydromechanical signals
.
Mar. Ecol. Prog. Ser.
179
,
81
-
95
.
Lindken
,
R.
,
Rossi
,
M.
,
Große
,
S.
and
Westerweel
,
J.
(
2009
).
Micro-particle image velocimetry (∝piv): recent developments, applications, and guidelines
.
Lab. Chip
9
,
2551
-
2567
.
Mercado
,
J. M.
,
Prakash
,
V. N.
,
Tagawa
,
Y.
,
Sun
,
C.
and
Lohse
,
D.
(
2012
).
Lagrangian statistics of light particles in turbulence
.
Phys. Fluids
24
,
055106
.
Merzkirch
,
W.
(
2012
).
Flow Visualization
.
Amsterdam
:
Elsevier
.
Miles
,
R. B.
and
Lempert
,
W. R.
(
1997
).
Quantitative flow visualization in unseeded flows
.
Annu. Rev. Fluid Mech.
29
,
285
-
326
.
Miyake
,
R.
,
Lammerink
,
T. S.
,
Elwenspoek
,
M.
and
Fluitman
,
J. H.
(
1993
).
Micro mixer with fast diffusion
.
In Micro Electro Mechanical Systems, 1993, MEMS'93, Proceedings An Investigation of Micro Structures, Sensors, Actuators, Machines and Systems. IEEE
.
248
-
253
.
Newton
,
P. K.
(
2013
).
The N-Vortex Problem: Analytical Techniques
, vol.
145
.
New York
:
Springer Science & Business Media
.
Olsen
,
M.
and
Adrian
,
R.
(
2000
).
Out-of-focus effects on particle image visibility and correlation in microscopic particle image velocimetry
.
Exp. Fluids
29
,
S166
-
S174
.
Peacock
,
T.
and
Dabiri
,
J.
(
2010
).
Introduction to focus issue: Lagrangian coherent structures
.
Chaos
20
,
017501
.
Rau
,
K. R.
,
Quinto-Su
,
P. A.
,
Hellman
,
A. N.
and
Venugopalan
,
V.
(
2006
).
Pulsed laser microbeam-induced cell lysis: time-resolved imaging and analysis of hydrodynamic effects
.
Biophys. J.
91
,
317
-
329
.
Rohr
,
J.
,
Hyman
,
M.
,
Fallon
,
S.
and
Latz
,
M. I.
(
2002
).
Bioluminescence flow visualization in the ocean: an initial strategy based on laboratory experiments
.
Deep Sea Res. I
49
,
2009
-
2033
.
Santiago
,
J. G.
,
Wereley
,
S. T.
,
Meinhart
,
C. D.
,
Beebe
,
D.
and
Adrian
,
R. J.
(
1998
).
A particle image velocimetry system for microfluidics
.
Exp. Fluids
25
,
316
-
319
.
Schlueter-Kuck
,
K. L.
and
Dabiri
,
J. O.
(
2017
).
Coherent structure coloring: identification of coherent structures from sparse data using graph theory
.
J. Fluid Mech
.
811
,
468
-
486
.
Shadden
,
S. C.
,
Lekien
,
F.
and
Marsden
,
J. E.
(
2005
).
Definition and properties of lagrangian coherent structures from finite-time lyapunov exponents in two-dimensional aperiodic flows
.
Phys.
212
,
271
-
304
.
Shadden
,
S. C.
,
Dabiri
,
J. O.
and
Marsden
,
J. E.
(
2006
).
Lagrangian analysis of fluid transport in empirical vortex ring flows
.
Phys. Fluids
18
,
047105
.
Shapiro
,
O. H.
,
Fernandez
,
V. I.
,
Garren
,
M.
,
Guasto
,
J. S.
,
Debaillon-Vesque
,
F. P.
,
Kramarsky-Winter
,
E.
,
Vardi
,
A.
and
Stocker
,
R.
(
2014
).
Vortical ciliary flows actively enhance mass transport in reef corals
.
Proc. Natl Acad. Sci. USA
111
,
13391
-
13396
.
Stamhuis
,
E.
and
Videler
,
J.
(
1995
).
Quantitative flow analysis around aquatic animals using laser sheet particle image velocimetry
.
J. Exp. Biol.
198
,
283
-
294
.
Stamhuis
,
E.
,
Videler
,
J.
,
Van Duren
,
L.
and
Müller
,
U.
(
2002
).
Applying digital particle image velocimetry to animal-generated flows: Traps, hurdles and cures in mapping steady and unsteady flows in re regimes between 10–2 and 105
.
Exp. Fluids
33
,
801
-
813
.
Taylor
,
Z. J.
,
Gurka
,
R.
,
Kopp
,
G. A.
and
Liberzon
,
A.
(
2010
).
Long-duration time-resolved piv to study unsteady aerodynamics
.
IEEE Trans. Instrum. Meas.
59
,
3262
-
3269
.
Tunstrøm
,
K.
,
Katz
,
Y.
,
Ioannou
,
C. C.
,
Huepe
,
C.
,
Lutz
,
M. J.
and
Couzin
,
I. D.
(
2013
).
Collective states, multistability and transitional behavior in schooling fish
.
PLoS Comput. Biol.
9
,
e1002915
.
Van Gelder
,
A.
(
2012
).
Vortex core detection: back to basics
.
Proc. SPIE
8294
,
Visualization and Data Analysis 2012
,
829413
.
Vicsek
,
T.
and
Zafeiris
,
A.
(
2012
).
Collective motion
.
Phys. Rep.
517
,
71
-
140
.
West
,
J. L.
and
Cameron
,
I. D.
(
2006
).
Using the medical image processing package, imagej, for astronomy
.
arXiv
preprint astro-ph/0611686
.

Competing interests

The authors declare no competing or financial interests.