Muscle Fatigue Analysis and Stress Detection from Surface EMG and ECG Data Obtained Using Deep Learning for Upper-Limb Trauma Rehabilitation

Introduction

An upper arm injury has a significant impact on a person's performance, as most activities of daily living are highly dependent on the hand. Road traffic accidents (RTAs), war or conflict, and sudden falls are the main causes of upper arm injuries [1]. In Ethiopia, injuries are the third leading cause of hospital admission [2]. To restore the hand to its working condition, physical rehabilitation is the most effective way to treat upper arm injury through repeated isolated movements [3]. Due to the nature of the brain's plasticity, which is the capacity to alter and grow over time in response to its environment by forming new neural connections, increasing the therapeutic dosage, intensity of exercise, and execution of task-oriented exercises can promote plasticity and functional recovery [4]. Since rehabilitation exercises require controlled, repetitive movements, frequent exercise can result in muscular fatigue. Fatigue is described as a decline in physical performance as a result of a task or exercise being too intense [5]. It is a common, nonspecific symptom and a source of concern for those who are receiving physical treatment or training [6]. Muscular fatigue causes a change in stance and movement that raises the risk of injury, chronic fatigue syndrome, overtraining syndrome, and immune dysfunction [7,8]. It is also common for trauma-injured patients to experience mental stress [9,10]. Loss of concentration is the most common mental reaction to trauma [11].

Studies are being performed to improve the efficacy of physical therapy by monitoring muscle fatigue. Traditional machine learning has been widely used in previous studies. In recent years, deep learning has been applied for the analysis of fatigue. Wang et al. used long short-term memory (LSTM) as a classifier for lower extremity muscle fatigue classification after performing a manual feature extraction method, and the model achieved an accuracy of 95.18% [12]. Another study used long short- term memory (LSTM), an RNN, and a binary feedforward NN (BFNN) to predict different gesture fatigue levels in a variety of channels via the Daubechies 3rd-order wavelet to extract features of the EMG signal, and the results showed that the LSTM model performed better [13]. A study of the upper arm muscles during weight uplift was performed using 16 features of the EMG signal and the FNN, and an accuracy of 88% was found [14]. In the study of mental stress, Bi-LSTM models were used in emotion recognition to classify four emotion classes from the EEG signal with an accuracy of 84.2% [15].

The common approach to assessing muscle fatigue during physical rehabilitation is patient-report-based. However, this method does not provide complete information about the patient's muscle condition and is highly dependent on patient experience and physiotherapist expertise. The limited number of physiotherapy experts and neurologists, especially in third- world countries, results in severe challenges. Additionally, the psychological conditions, such as mental stress, of traumatized patients, which may result in cognitive fatigue, are forgotten. The existing studies focus on physical fatigue that occurs in daily living activities using single physiological parameters. Moreover, end-to-end deep learning has not been widely applied. Hence, muscle fatigue that occurs during trauma injury rehabilitation exercises needs further analysis to determine whether this fatigue is due to the intensity of physical rehabilitation or due to the mental stress that the patient is experiencing, as this can also cause physical fatigue. Therefore, the aim of this study was to implement deep learning-based muscle fatigue and mental stress assessments for trauma injury rehabilitation.

Methods

A Bi-LSTM model was developed to detect muscle fatigue and mental stress from EMG and ECG signals, respectively. To develop the model, the data were collected, prepared, pre- processed, and subsequently input to the training algorithm. The test data were subsequently input to the classifier, whose performance was assessed using a variety of metrics. Figure 1 shows the general block diagram of the method used in this study.

Experiment Setup

Subjects were informed about the experimental procedure and consented to confidentiality before the setup. The SCU-7 EMG system with reusable electrodes was used for data recording. Warm-up exercises were performed to prevent cramps or injury, and electrodes were subsequently placed on the flexor and extensor digitorum muscles of the dominant arm, as shown in Figure 2. Isometric exercises were selected because they can be easily performed and increase muscle mass [16]. For healthy subjects, the hand grasp was set at 25 kg for males and 15 kg for females.

Figure 2: EMG signal Recording and Electrode Placement

Dataset Preparation

A study involving 40 healthy subjects aged 21–32 years and 7 with moderate severity score trauma injury aged 32–38 years was selected for EMG data recording during isometric contraction using a hand grip. The subjects had no muscle-related injuries, cardiovascular or metabolic diseases, or mental disorders. The study excluded pregnant patients, smokers, and patients who had prostheses or orthoses. The moderate severity scale was chosen to prevent patient risk during the experimental procedure. A total of 188 myoelectric signals were obtained from 40 healthy subjects and 7 injured datasets for each fatigue class. The wearable stress and affection dataset (WESAD) was utilized for mental stress analysis using various annotation methods and stress generation techniques based on arousal and valence [17]. The study involved 15 subjects aged 24-35 years, with the exclusion criteria for pregnancy, heavy smoking, mental disorders, and chronic diseases. The biosignal data included ECG, blood volume, pulse, electrodermal activity, EMG, respiration, and temperature data. The data were labelled using positive and negative affect schedules, self-report questionnaires, a state trait anxiety inventory, and self-assessment manikins. From WESAD recordings, 238 ECG datasets were obtained, consisting of 223 normal states and 15 stress classes.

Signal Pre-Processing

The EMG signals consisted of four classes: nonfatigue, low- level fatigue, medium-level fatigue, and high-level fatigue. The signal is filtered using a bandpass Butterworth 4th-order filter, a notch filter, and a 3rd-order Butterworth bandpass filter to remove artefacts.

The WESAD dataset includes baseline, stress, amusement, and meditation classes and includes six sensor datasets. The data points in each class are labelled as normal, while stress signals are used under stress conditions. The dataset was balanced using the synthetic minority oversampling technique (SMOTE), which duplicates data points from the minority class to create a balance between classes. SMOTE was applied only to the training dataset to reduce overfitting problems. The class distributions of the ECG data for normal and understress patients before and after SMOTE was applied are shown in Figure 3.

Figure 3: The ECG Data Class Distributions (a) Before Applying SMOTE and (b) After Applying

Signal Analysis

A random selection of five people was used to determine how muscle fatigue affects the root mean square (RMS) and median frequency (MDF) of temporal and frequency domain characteristics. Figures 4(a) and 4(b) show the MDF and RMS feature plots, respectively, in relation to the labelled classes. These two characteristics were selected because they are commonly used to analyse muscle fatigue based on EMG signals [80].

Figure 4: (a) Box plot of the RMS and (b) MDF of Five Subjects at Each Fatigue Level

The amplitude change in the average RMS and the frequency change in the average MDF were determined as muscle fatigue increased. As shown in Figure 4, an increase in fatigue results in an increase in the amplitude of the RMS and a decrease in the MDF.

Deep Learning Model

After the signals were properly preprocessed, the dataset was successfully divided into training, validation, and testing datasets at a ratio of 60:20:20 using the holdout strategy. After smoothing, the preprocessed signals were eventually fed to the Bi-LSTM model.

Additionally, 1D CNN and MLP models were developed to compare the accuracy of each model and choose the model that performed the best. With an Adam optimizer, categorical cross-entropy, a learning rate of 0.01, and 50 iterations, the hyperparameters were changed. An overview of the suggested model development is shown in Figure 5. The model's performance was evaluated using the accuracy, precision, recall, F-measure, confusion matrix, and specificity curve metrics. Figure 5 shows the general deep learning architecture implemented in this study.

Figure 5: Shows the Structure of the Bi-LSTM Model

Sixty percent of the ECGs in the WESAD dataset were used as the training dataset [18]. The distributions of the two-class training datasets before and after using the SMOTE approach are shown in Table 1. The same procedure was used for the EMG model. However, the hyperparameters are modified with a reduced number of layers, an Adam optimizer, a sigmoid activation function, and binary cross-entropy at epochs of 50.

                                                              Table 1: Class Distribution of the WESAD Stress Training Dataset

The classification of an input signal into one of several classes, together with a specific likelihood percentage of 100%, is ultimately made possible via a precisely designed graphical user interface within the streamlit environment.

Experimental Results and Discussion

Results

Experiment Results of the Deep Learning Model

The proposed Bi-LSTM model was found to be the most ideal model at 50 epochs for the classification of muscle fatigue and mental stress, as shown in Table 3. The accuracy, precision, and F1 score of the Bi-LSTM model for muscle fatigue classification are described in the confusion matrix in Table 2.

Table 2: Accuracy, Precision, and F1 Score of the Muscle Fatigue Bi-LSTM Model

The test results of the accuracy and loss plot of the EMG deep learning model are shown in Figure 6.
Figure 6: The Accuracy and Loss Curve of the EMG Signal

Table 3. The F1 score, recall, and precision of the mental stress Bi-LSTM model with the confusion matrix are shown in Table 3.

                                          Table 3. F1 Score, Recall, and Precision on the Confusion Matrix of the Bi-LSTM ECG Model

Figure 7 shows the accuracy and loss plot of the mental stress deep learning models at 25 epochs for binary classification. The precision, recall, and F1-score results are also shown in Table 3.

                                 Figure 7: Shows the Accuracy and Loss Curve of the ECG Signal

Table 4 shows the outcomes of 100% of the tests conducted using the proposed Bi-LSTM, 1D CNN, and MLP models.

Table 4: The Summarized Accuracy and F1 Score of the Muscle Fatigue and Mental Stress Models

Web-Based System Development

An overview of the model user interface is shown in Figure 8.

Figure 8. General Overview of the Developed GUI on the (a) Home Page, (b) EMG Classification Panel, and (c) EMG Classification Panel

Discussion

The common approach to assessing muscle fatigue during physical rehabilitation is patient-report-based, and existing studies focus on physical fatigue that occurs in daily living activities without including psychological conditions such as mental stress in the traumatized patient. This study proposes the development of a deep learning model for the detection of muscle fatigue and the assessment of mental stress by using EMG and ECG signals during physical rehabilitation. A Bi- LSTM model was developed with an accuracy of 95% in multiclass muscle fatigue classification and 99% accuracy in mental stress classification.

In this study, good accuracy was obtained in four-stage fatigue classification by using an end-to-end deep learning method with optimal deep layers to help the model learn detailed features and avoid maximum voluntary contraction (MVC)- based measurements, which cannot always be performed in clinical settings. A study that employed MVC-based upper limb isometric contraction to detect low-level fatigue by using wavelet decomposition based on selected features was performed with an accuracy of 83% [19]. Another study that was performed based on EMG signals in cyclo-ergometric exercise by using an SVM for a set of nine features obtained an accuracy of 82% [8].

Conclusion

Even though physical therapy is the most effective treatment for traumatic injuries, individuals with posttraumatic mental stress and/or muscle fatigue may experience a slower recovery.

Unfortunately, existing clinical muscle fatigue evaluation systems are manual or subject report-based, and the mental condition of traumatized patients is not treated by considering posttraumatic mental stress, which is common in traumatized patients. Furthermore, the present muscle fatigue research focused on work-related muscular fatigue, leaving a gap in understanding the causes of fatigue.

In this study, a method for identifying mental stress and muscle exhaustion in trauma patients during physical rehabilitation was built using deep learning and EMG and ECG signals. To detect muscle fatigue and the presence of mental stress in traumatized patients during physical rehabilitation, the system addresses earlier shortcomings and now offers the ability to include other parameters that can induce muscle fatigue.

Declarations

1. Ethics Approval and Consent to Participate

An ethical approval was obtained from the Institutional Review Board of the Jimma University Institute of Health Science. All methods were carried out in accordance with the ethical standards established in the 1964 Declaration of Helsinki and its later amendments or comparable ethical standards.

2. Consent for Publication

Informed consent was not obtained; thus, the Jimma University Institute of Health Science ethical review committee waived this requirement. Participants' anonymity and privacy were upheld, and the data were sufficiently anonymized.

3. Availability of Data and Materials

The datasets used during the current study are available from the corresponding author upon reasonable request.

4. Competing Interests

The authors declare no competing interests.

5. Funding

The author received no financial support for the research, authorship, and/or publication of this article.

6. Authors' Contributions

All the authors made significant contributions to the work reported, whether by conception, study design, execution, acquisition of data, analysis and interpretation, or in all these areas; by drafting, revising or critically reviewing the article; by giving final approval of the version to be published; by agreeing on the journal to which the article has been submitted; and by agreeing to be accountable for all aspects of the work.

7. Acknowledgements

The physiotherapy clinics at Jimma University and Droga Physiotherapy Clinic in Addis Abeba provided the materials needed to carry out the study. A preprint has previously been published [20].

References

1. Alenko, A., Berhanu, H., Abera Tareke, A., Reta, W., Bariso, M., Mulat, E., ... & Girma, S. (2019). Posttraumatic stress disorder and associated factors among drivers surviving road traffic crashes in Southwest Ethiopia. Neuropsychiatric disease and treatment, 3501-3509.
2. F.  D.  Republic,  “HEALTH  &  HEALTH-RELATEDINDICATORS Federal Democratic Republic of Ethiopia Ministry of Health,” 2017.
3. Keci, A., Tani, K., & Xhema, J. (2019). Role of rehabilitation in neural plasticity. Open access Macedonian journal of medical sciences, 7(9), 1540.
4. Molteni, F., Gasperini, G., Cannaviello, G., & Guanziroli,E. (2018). Exoskeleton and end-effector robots for upper and lower limbs rehabilitation: narrative review. PM&R, 10(9), S174-S188.
5. Tornero-Aguilera, J. F., Jimenez-Morcillo, J., Rubio- Zarapuz, A., & Clemente-Suárez, V. J. (2022). Central and peripheral fatigue in physical exercise explained: A narrative review. International journal of environmental research and public health, 19(7), 3909.
6. Abd-Elfattah, H. M., Abdelazeim, F. H., & Elshennawy, S. (2015). Physical and cognitive consequences of fatigue: A review. Journal of advanced research, 6(3), 351-358.
7. Cowley, J. C., & Gates, D. H. (2017). Proximal and distal muscle fatigue differentially affect movement coordination. PloS one, 12(2), e0172835.
8. Ramos, G., Vaz, J. R., Mendonça, G. V., Pezarat-Correia, P., Rodrigues, J., Alfaras, M., & Gamboa, H. (2020). Fatigue evaluation through machine learning and a global fatigue descriptor. Journal of healthcare engineering, 2020.
9. Almomani, F., Alghwiri, A. A., Alghadir, A. H., Al-Momani,A., & Iqbal, A. (2019). Prevalence of upper limb pain and disability and its correlates with demographic and personal factors. Journal of Pain Research, 2691-2700.
10. van der Vlegel, M., Polinder, S., Toet, H., Panneman, M. J., Geraerds, A. J. L. M., & Haagsma, J. A. (2022). Anxiety, depression and post-traumatic stress symptoms among injury patients and the association with outcome after injury. European journal of psychotraumatology, 13(1), 2023422.
11. S. G. of V. Department of Health, “Trauma - reaction and recovery,” Better Heal. Channel, pp. 1–9, 2016, [Online]. Available: https://www.betterhealth.vic.gov. au/health/conditionsandtreatments/trauma-reaction-and- recovery#physical-reactions-to-trauma.
12. Wang, J., Sun, S., & Sun, Y. (2021). A muscle fatigue classification model based on LSTM and improved wavelet packet threshold. Sensors, 21(19), 6369.
13. Ma, F., Song, F., Liu, Y., & Niu, J. (2020). sEMG-based neural network prediction model selection of gesture fatigue and dataset optimization. Computational Intelligence and Neuroscience, 2020.
14. Elshafei, M., & Shihab, E. (2021). Towards detecting biceps muscle fatigue in gym activity using wearables. Sensors, 21(3), 759.
15. Karnam, N. K., Dubey, S. R., Turlapaty, A. C., & Gokaraju,B. (2022). EMGHandNet: A hybrid CNN and Bi-LSTM architecture for hand activity classification using surface EMG signals. Biocybernetics and biomedical engineering, 42(1), 325-340.
16. Khosrojerdi, H., Tajabadi, A., Amadani, M., Akrami, R., & Tadayonfar, M. (2018). The effect of isometric exercise on pain severity and muscle strength of patients with lower limb fractures: A randomized clinical trial study. Medical- Surgical Nursing Journal, 7(1).
17. Abegaz, T., & Gebremedhin, S. (2019). Magnitude of road traffic accident related injuries and fatalities in Ethiopia. PloS one, 14(1), e0202240.
18. Schmidt, P., Reiss, A., Duerichen, R., Marberger, C., & Van Laerhoven, K. (2018, October). Introducing wesad, a multimodal dataset for wearable stress and affect detection. In Proceedings of the 20th ACM international conference on multimodal interaction (pp. 400-408).
19. Zhang, G., Morin, E., Zhang, Y., & Etemad, S. A. (2018, July). Non-invasive detection of low-level muscle fatigue using surface EMG with wavelet decomposition. In 2018 40th annual international conference of the ieee engineering in medicine and biology society (embc) (pp. 5648-5651). IEEE.
20. Kassaw, E., Worassa, E., Fetene, K., & Aboye, G. (2023). Muscle Fatigue Analysis and Stress Detection from Surface EMG and ECG Using Deep Learning for Upper-Limb Trauma Rehabilitation.