Development of a deep neural network for automated electromyographic pattern classification

Surface electromyography (sEMG) is a non-invasive method for recording muscle excitation. Electrodes are placed on the skin surface atop muscle bellies and form a conducting medium through which the electric potential of the underlying depolarizing motor units is measured. The signal measured at the skin surface is produced by a combination of many depolarizing muscle fibres, acquired only after the signal has travelled through various tissues (e.g. connective tissue, fats and skin). Mechanical perturbations of the skin–electrode interface, known as movement artefact, can distort the sEMG (Raez et al., 2006). Furthermore, the signal can be contaminated by electro-technical noise from electronic equipment, ambient noise caused by electromagnetic radiation from power sources (Amrutha and Arul, 2017) and by unequal skin–electrode impedance between bipolar electrodes (Merletti et al., 2001). Handling these sources of noise, while preserving signal integrity for subsequent interpretation, is challenging.

Electromyography (EMG) is widely used in clinical (e.g. characterization of neuromuscular pathologies), biomedical (e.g. modern human machine interfaces) and neurophysiological (e.g. study of human motor control) applications. For example, neuromusculoskeletal models used to study human movement use sEMG to estimate muscle forces and tissue loads inside the human body (Pizzolato et al., 2015). However, neuromusculoskeletal models rely on valid measures of muscle excitation to produce physiologically plausible outputs. Given the inherent potential for signal contamination, sEMG quality must be evaluated prior to use. Signal properties (e.g. signal-to-noise ratio, frequency spectrum, bursts of activation) are commonly evaluated after data acquisition by visual inspection (Raez et al., 2006). However, visual inspection is time demanding and subjective, and requires trained observers. Development of an automated procedure to evaluate sEMG quality both during and following data acquisition would significantly reduce the burden for researchers and clinicians.

A robust solution for automated sEMG evaluation could be achieved using artificial neural networks (ANNs), which are increasingly used for pattern recognition and classification in the field of machine learning (Russakovsky et al., 2015). An ANN is an information-processing system that simulates the function of biological neurons, and consists of multiple interconnected layers and connection weights (Basheer and Hajmeer, 2000). To classify the dataset in the desired manner, the ANN must be trained, during which connection weights and layer variables are adjusted using features extracted from input signals (Subasi et al., 2006). Feature extraction can be performed either by the user, also known as supervised learning, or automatically by the ANN, known as unsupervised learning (Bengio, 2009), with the latter showing increasingly superior performance in recent years (Schmidhuber, 2015). By combining training and prior manual classification, ANNs are subsequently able to choose the best pattern with which to categorize inputs; however, the exact way an ANN learns differs depending on their type, architecture and application. Once an ANN is trained, it can classify new signals by applying the previously learned relationship between input and output.

The paramount performance metric of an ANN is accuracy. In the field of machine learning, accuracy is a technical term referring to the percentage of classifications performed by the ANN that match the true manual classification. Using ANNs, researchers have successfully distinguished between different wrist and thumb movements used to control an active hand prosthesis (Arvetti et al., 2007), detected different phases of the gait cycle (Joshi et al., 2013) and differentiated between sEMG acquired from healthy and neuropathic individuals (Sadikoglu et al., 2017), thus demonstrating the potential of ANNs for human movement applications.

In the year 2017, there were 732 publications identified in PubMed using a key word search of ‘surface electromyography’. We anticipate that an automated method to evaluate sEMG quality and exclude spurious signals will have widespread practical application. The aims of this study were first to evaluate the performance of five ANNs used to classify sEMG signal quality and, second, to build an automated and robust sEMG classification tool using the best-performing ANN. We hypothesized that ANNs for sEMG classification using unsupervised learning would have superior classification performance in comparison to ANNs using supervised learning, and sEMG evaluation could be automated using an ANN to achieve human-like accuracy (∼95%) (Russakovsky et al., 2015).

Three datasets [1: SCOPEX – The University of Melbourne Human Research Ethics Committee (HREC) 11377168; 2: load sharing trials – Department of Defence and Veterans' Affairs HREC 756-14; 3: ARF – Charles Sturt University HREC 2013/185] were combined, resulting in 87,000 individual sEMG signals from various trunk and lower-limb muscles, including rectus abdominis, vastus lateralis, rectus femoris, semitendinosus, biceps femoris, gastrocnemii and gluteus maximus. The five ANNs evaluated were ANFIS, PNN, AlexNet, VGG16 and ResNet50 (described in detail below). Recordings from different trunk and lower-limb muscles, data acquisition systems (1: Noraxon Telemyo 900, 1200 Hz; 2: Noraxon Telemyo 900, 1000 Hz; 3: Trigno Delsys, 1000 Hz) and biomechanics laboratories were included to increase robustness of the ANNs and prevent overfitting to one distinct experimental or laboratory condition.

A standard approach was used for sEMG signal processing. First, a zero-offset correction was performed to remove any direct current value that may be present due to poor grounding. Data were then filtered using a zero-lag second-order Butterworth bandpass filter (30–300 Hz) to remove low-frequency noise caused by movement artefacts and high-frequency noise caused by higher harmonics from the mains, motors or light sources (Lloyd and Besier, 2003; Raez et al., 2006). The bandpass-filtered signal was full-wave rectified and low-pass filtered with a zero-lag second-order Butterworth low-pass filter (f_c=6 Hz) to create a linear envelope (Lloyd and Besier, 2003; White and Winter, 1992).

The sEMG data were manually classified into four categories by one operator as ‘good’, ‘usable’, ‘noise’ and ‘no signal’ (Fig. 1). Clearly defined signals with a high signal-to-noise ratio were classified as good. Signals where the signal-to-noise ratio was low, but individual bursts of excitation were still distinguishable were classified as usable. Signals where individual bursts of muscle excitation were not distinguishable from noise were classified as noise. Signals with an absence of any noise or bursts of excitation were classified as no signal. To prevent bias towards one category during the ANN training and in subsequent categorizations, an equal number of samples (n=10,000) from all signal categories was used to train all ANNs. This meant that all five ANNs were trained using the same 40,000 of the available 87,000 sEMG signals. These 40,000 samples with equally distributed sEMG quality classifications were then randomized and split into 70% (n=28,000) intended for training and 30% (n=12,000) for performance evaluation. Separating training from evaluation data is a standard procedure in the field of machine learning, and is designed to prevent erroneously high evaluations of ANN accuracy caused by testing performance on the same data used for training (Dobbin and Simon, 2011). The remaining 47,000 sEMG with uncontrolled classification distribution were used for additional performance testing.

Five ANNs were trained using MATLAB 2017b (MathWorks, Natick, MA, USA). Two were supervised ANNs: (i) adaptive neuro-fuzzy inference system (ANFIS) (Jang, 1993) and (ii) pattern recognition neural network (PNN) (MathWorks). Three were unsupervised ANNs: (i) AlexNet (Krizhevsky et al., 2012), (ii) VGG16 (Simonyan and Zisserman, 2015) and (iii) ResNet50 (He et al., 2016). These five ANNs were selected as they have previously shown promise in delineating motor control tasks via sEMG classification (Arvetti et al., 2007; Joshi et al., 2013; Sadikoglu et al., 2017). The ANFIS is a multi-layer feedforward neural network that works on fuzzy logic using ‘If–Then’ rules as a way of dealing with uncertainty when deciding on the output class of a signal. The combination of fuzzy logic and If–Then rules approximates human information processing and leads to an improved classification accuracy in certain ANN applications (Güler and Ubeyli, 2005; Jang, 1993). The PNN is a feedforward network-designing tool included in the proprietary programming language MATLAB as part of the Neural Network Toolbox. Here, the number of neurons and the training algorithm can be adjusted to fit the specific dataset, creating a highly customized neural network. AlexNet, VGG16 and ResNet50 are convolutional neural networks (CNNs), i.e. unsupervised neural networks specifically designed for image recognition. An overview of the five architectures can be found in Fig. S1. A CNN classifies images by automatically extracting discriminable features and recognizing patterns therein (Krizhevsky et al., 2012). To be classified by the three CNNs, the sEMG recordings were converted to images (.jpg file format).

For all three CNNs, the linear envelope was superimposed onto the bandpass-filtered sEMG signal and subsequently converted to an image (Fig. 1). All five ANNs were trained using a combination of manual classification and extracted features or created images, depending on their machine learning category.

Performance of the ANNs was assessed by four outcome measures: accuracy, false-positives, false-negatives and time demand of classification. Accuracy is the percentage of sEMG signal classifications by the ANN that match a manual classification, and is considered the primary performance outcome metric for this study. False-positives and false-negatives are represented as the percentage of sEMG falsely classified as better or worse quality, respectively. Time demand of classification is the computational time (s) required by the ANN to classify 1000 samples of sEMG on an Intel Core i5-3570 processor with 16 GB of RAM, which are computer specifications a user is likely to have. The computational time demand of classification is important for the practical implementation of the classification tool.

The computational time demand of classification for the ANFIS and PNN was low (Table 1), but performance of both was unsatisfactory compared with that of the three CNNs. Accuracy was low (60.0%, 70.9%) and both false-positives (30.3%, 15.9%) and false-negatives (9.7%, 13.2%) were high.

The CNNs produced better results than either of the supervised learning ANNs (Table 1). Accuracy was >98%, and both false-positives and false-negatives were reduced to <2% in all three CNNs. AlexNet and ResNet50 yielded comparable classification performance, and both were superior to VGG16; thus, they were considered for the automated sEMG classification tool. The performance of AlexNet and ResNet50 is shown in Fig. 2. Generally, the noise and no signal classes had better accuracy than the good and usable classes. False-positives for classification of noise to good amounted to 0.03%. The usable class showed the largest classification uncertainty of the four classification categories (combined false-positives and false-negatives: 1.77–2.73%).

Implementing and testing the two best-performing CNNs on the set of sEMG data not included in ANN training (i.e. the remaining 47,000 manually classified sEMG) confirmed that AlexNet and ResNet50 both had human-like classification accuracy (∼95%) (Russakovsky et al., 2015). AlexNet was chosen for use in the automated sEMG classification tool because ResNet50 had a comparatively high computational time demand (23.47 s for ResNet50, 2.44 s for AlexNet).

The sEMG classification tool functions in five steps. First, data stored in c3d file format (commonly used to record synchronized motion and analog data containing sEMG signals acquired in biomechanics laboratories: C3D.org 2018) are imported into the tool and sEMG signals are extracted. Second, sEMG from each c3d file is filtered as described above. Third, images of the processed sEMG recordings are created. Fourth, those images are classified into four categories (i.e. good, usable, noise and no signal). Last, images are sorted into folders based on their category, and classification results are saved in an easily interpretable American Standard Code for Information Interchange (ASCII) file.

The aims of this study were, first, to evaluate the performance of five different ANNs used to automatically classify sEMG quality and, second, to build an automated and robust sEMG classification tool using the best-performing candidate ANN, thereby streamlining data processing, minimizing subjectivity and reducing the burden on researchers and clinicians using sEMG. Unsupervised learning ANNs showed superior classification performance in comparison to supervised learning ANNs, confirming our first hypothesis. The best-performing ANNs, AlexNet and ResNet50, which each demonstrated >99% accuracy, were used to build and test two separate sEMG classification tools. Despite their comparable performance, the computational time demand for ResNet50 was much higher than for AlexNet, and therefore AlexNet was chosen for the final version of the classification tool. The results demonstrate that sEMG classification, which is normally time demanding, tedious and subjective, can be automated using a CNN without compromising human-like classification accuracy. We encourage those interested to contribute manually classified sEMG data to our dataset and to give feedback regarding use of the sEMG classification tool. Contribution from the scientific and clinical community will help to improve the robustness of this tool.

The results demonstrate that ANNs using unsupervised learning, specifically the CNNs AlexNet and ResNet50, can achieve human-like performance when classifying sEMG quality. This novel observation confirmed our second hypothesis and highlights that labour-intensive, subjective sEMG evaluation can be automated without compromising accuracy. The two ANNs using supervised learning, ANFIS and PNN, showed poor performance metrics due to their respective computational architectures. The ANFIS can handle uncertainty in classification but is unable to account for the high signal variability present in sEMG, which led to a substantial decrease in accuracy. The PNN is specifically designed to fit the sEMG classification task and accounts for this inherent signal variability in the sEMG dataset. This design feature contributed to superior accuracy and fewer false-positives compared with the ANFIS, yet performance of the PNN remained insufficiently robust and did not approach human-like accuracy. The CNNs performed considerably better than both the ANFIS and PNN because the feature extraction and selection from the sEMG signal was performed by the network itself rather than by the user. This aspect of CNNs differs from supervised ANNs, where performance is highly dependent on the appropriateness of selected and extracted features. Hence, a CNN learns directly from the dataset and extracts a high number of relevant features that are empirically determined to be important to the classification of the signal.

Classification accuracy for AlexNet and ResNet50 was similar and excellent. The usable category showed the greatest classification uncertainty with combined false-positives and false-negatives amounting to 1.77–2.73% of the classifications. This observation is unsurprising given that visual inspection of the usable class is challenging and ambiguous for an expert human operator. Computational time demand differences between the two CNNs can be explained by their architectural complexity, i.e. AlexNet consists of eight layers whereas ResNet50 has 50 layers. This time demand means that classifying a set of 1000 images takes ResNet50 approximately 10 times longer than AlexNet, which in turn decreases the practical usefulness of ResNet50 as a within-laboratory or within-clinic tool.

Previous studies using CNNs achieved an accuracy of 99.40% in iris recognition (Minaee and Wang, 2017), 99.95% in palm-print recognition (Minaee et al., 2016) and 90.00% in seizure detection (Acharya et al., 2018). The CNNs in this study showed a performance similar to those of these prior reports, and provide further evidence that robust classification automation can be achieved using unsupervised machine learning. To our knowledge, this is the first study to use CNNs for evaluating sEMG quality and, in combination with the aforementioned studies, highlights that CNNs can be implemented for a variety of clinical applications with excellent outcomes. Limitations of the automated sEMG classification tool presented in this paper should be considered. Because of limited sEMG data availability, the ANNs were trained on datasets from only three different laboratories, and a selection of muscles, experimental tasks and conditions; thus, we cannot guarantee the ANN performance using other datasets acquired in different laboratories or experimental conditions. Although the tool can reliably classify sEMG, its performance for intramuscular or high-density EMG has yet to be evaluated. This study's limitations can be addressed by expanding the training dataset to include samples from different electrode manufacturers, configurations and types applied to different muscles, acquired at different laboratories. To encourage this development, we plan to make the tool public, with a process for data curation, enabling us to assist other researchers by sharing computational resources.

In conclusion, signal quality from sEMG can be evaluated with >99% accuracy using an automated classification tool. Superior classification performance was achieved by the AlexNet and ResNet50 CNNs compared with the tested ANNs that used supervised learning. AlexNet showed a smaller computational time demand than ResNet50 and consequently was used to build a tool for automated sEMG quality evaluation. This tool demonstrated reliable performance and can be used to streamline data processing without compromising human-like classification accuracy. If implemented during data acquisition, it may also have the potential to quickly identify and thus reduce the number of sEMG recordings with low signal-to-noise ratio. Expansion of the training dataset could further improve the versatility and evaluation performance of the tool.