Deep learning has become powerful and yet versatile tool that allows for the extraction of complex patterns from rich datasets. One field that can benefits from this advancement is human gait analysis. Conventional gait analysis requires a specialized laboratory setup with markers positioned at anatomical landmarks. It also demands accurate muscle identification and electrode placement. This work overcomes these limitations by proposing a Convolutional Neural Network (CNN) that processes tokenized data measured by the wearable Inertial Measurement Units (IMUs) to estimate muscle activities during walking. Gait data were gathered from 65 participants aged 19-73 years old. The proposed method was compared against the other deep learning models. The result of the proposed method was better that the alternatives. The results demonstrated that test dataset achieved Root Mean Squared Error (RMSE) less than 12% and Correlation Coefficient (r) greater than 90% for all muscles. It performed particularly well on unseen data, with most muscle activities showing RMSE below 20% and r above 80%. These findings indicate that the proposed technique can efficiently extract relevant gait data and estimate the muscle activity, potentially simplifying gait analysis and making it more affordable and accessible to a wider range of users.

Keywords

CNN; CNN-LSTM; IMU; LSTM; Muscle activity.

1. Trabassi, D., Serrao, M., Varrecchia, T., Ranavolo, A., Coppola, G., De Icco, R., Tassorelli, C., Castiglia, S. F., Tatarelli, A., & Di Somma, A. (2022). Machine learning approach to support the detection of Parkinson’s disease in IMU-based gait analysis. Sensors, 22(10), Article 3700. https://doi.org/10.3390/s22103700
2. Lin, C. H., Chen, C. H., & Yang, K. C. (2022). Early detection of Parkinson’s disease by neural network models. IEEE Access, 10, 19033–19044. https://doi.org/10.1109/ACCESS.2022.3150774
3. Mundt, M., Koeppe, A., David, S., Bamer, F., Potthast, W., & Markert, B. (2020). Estimation of gait mechanics based on simulated and measured IMU data using an artificial neural network. Frontiers in Bioengineering and Biotechnology, 8, Article 41. https://doi.org/10.3389/fbioe.2020.00041
4. Sung, J., Park, S., & Kim, Y. (2021). Prediction of lower extremity multi-joint angles during overground walking by using a single IMU with a low frequency based on an LSTM recurrent neural network. Sensors, 22(1), Article 53. https://doi.org/10.3390/s22010053
5. Topham, L. K., Khan, W., Al-Jumeily, D., & Hussain, A. (2022). Human body pose estimation for gait identification: A comprehensive survey of datasets and models. ACM Computing Surveys, 55(6), Article 120. https://doi.org/10.1145/3533384
6. Saboor, A., Khezri, M., Kouzani, A. Z., & Mahmoodi, S. (2020). Latest research trends in gait analysis using wearable sensors and machine learning: A systematic review. IEEE Access, 8, 167830–167864. https://doi.org/10.1109/ACCESS.2020.3022818
7. Karatsidis, A., Bellusci, G., Schepers, H. M., de Zee, M., Veltink, P. H., & Luinge, H. J. (2017). Estimation of ground reaction forces and moments during gait using only inertial motion capture. Sensors, 17(1), Article 75. https://doi.org/10.3390/s17010075
8. Ancillao, A., Tedesco, S., Barton, J., & O’Flynn, B. (2018). Indirect measurement of ground reaction forces and moments by means of wearable inertial sensors: A systematic review. Sensors, 18(8), Article 2564. https://doi.org/10.3390/s18082564
9. Mundt, M., Koeppe, A., David, S., Bamer, F., Potthast, W., & Markert, B. (2020). Estimation of gait mechanics based on simulated and measured IMU data using an artificial neural network. Frontiers in Bioengineering and Biotechnology, 8, Article 41. https://doi.org/10.3389/fbioe.2020.00041
10. Khant, M., Gouwanda, D., Gopalai, A. A., Lim, K. H., & Foong, C. C. (2023). Estimation of lower extremity muscle activity in gait using the wearable inertial measurement units and neural network. Sensors, 23(1), Article 556. https://doi.org/10.3390/s23010556
11. Camargo, J., Ramanathan, A., Flanagan, W., & Young, A. (2021). A comprehensive, open-source dataset of lower limb biomechanics in multiple conditions of stairs, ramps, and level-ground ambulation and transitions. Journal of Biomechanics, 119, Article 110320. https://doi.org/10.1016/j.jbiomech.2021.110320
12. Moghadam, S. M., Yeung, T., & Choisne, J. (2023). A comparison of machine learning models’ accuracy in predicting lower-limb joints’ kinematics, kinetics, and muscle forces from wearable sensors. Scientific Reports, 13(1), Article 5046. https://doi.org/10.1038/s41598-023-31906-z
13. Trinler, U., Leboeuf, F., Hollands, K., Jones, R., & Baker, R. (2018). Estimation of muscle activation during different walking speeds with two mathematical approaches compared to surface EMG. Gait & Posture, 64, 266–273. https://doi.org/10.1016/j.gaitpost.2018.06.021
14. Zabre-Gonzalez, E. V., Amieva-Alvarado, D., & Beardsley, S. A. (2021). Prediction of EMG activation profiles from gait kinematics and kinetics during multiple terrains. In 2021 43rd Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC) (pp. 6326–6329). IEEE. https://doi.org/10.1109/EMBC46163.2021.9630097
15. Ketkar, N., & Moolayil, J. (2021). Deep learning with Python: Learn best practices of deep learning models with PyTorch (2nd ed.). Apress. https://doi.org/10.1007/978-1-4842-5364-0
16. Khan, S., Rahmani, H., Shah, S. A. A., & Bennamoun, M. (2018). A guide to convolutional neural networks for computer vision. Morgan & Claypool Publishers. https://doi.org/10.2200/S00822ED1V01Y201712COV015
17. Kim, Y. W., Joa, K. L., Jeong, H. Y., & Lee, S. (2021). Wearable IMU-based human activity recognition algorithm for clinical balance assessment using 1D-CNN and GRU ensemble model. Sensors, 21(22), Article 7628. https://doi.org/10.3390/s21227628
18. Negi, S., Sharma, S., & Sharma, N. (2021). FSR and IMU sensors-based human gait phase detection and its correlation with EMG signal for different terrain walk. Sensor Review, 41(3), 235–245. https://doi.org/10.1108/SR-10-2020-0249
19. Wang, J., Dai, Y., & Si, X. (2021). Analysis and recognition of human lower limb motions based on electromyography (EMG) signals. Electronics, 10(20), Article 2473. https://doi.org/10.3390/electronics10202473
20. Lu, Y., Wang, H., Zhou, B., Wei, C., & Xu, S. (2022). Continuous and simultaneous estimation of lower limb multi-joint angles from sEMG signals based on stacked convolutional and LSTM models. Expert Systems with Applications, 203, Article 117340. https://doi.org/10.1016/j.eswa.2022.117340
21. SENIAM. (n.d.). SENIAM recommendations for determination of sensor location. http://seniam.org/
22. Adams, J. M., & Cerny, K. (2018). Observational gait analysis: A visual guide. SLACK Incorporated.
23. Levine, D., Richards, J., & Whittle, M. W. (Eds.). (2012). Whittle’s gait analysis (5th ed.). Churchill Livingstone/Elsevier.
24. Petry, N. M. (2002). A comparison of young, middle-aged, and older adult treatment-seeking pathological gamblers. The Gerontologist, 42(1), 92–99. https://doi.org/10.1093/geront/42.1.92
25. Banks, J. J., Chang, W. R., Xu, X., & Chang, C. C. (2015). Using horizontal heel displacement to identify heel strike instants in normal gait. Gait & Posture, 42(1), 101–103. https://doi.org/10.1016/j.gaitpost.2015.03.015
26. Zafar, I., Tzanidou, G., Burton, R., Patel, N., & Araujo, L. (2018). Hands-on convolutional neural networks with TensorFlow: Solve computer vision problems with modeling in TensorFlow and Python. Packt Publishing.
27. Samek, W., Montavon, G., Vedaldi, A., Hansen, L. K., & Müller, K. R. (Eds.). (2019). Explainable AI: Interpreting, explaining and visualizing deep learning. Springer. https://doi.org/10.1007/978-3-030-28954-6
28. Lachman, M. E. (2001). Adult development, psychology of. In N. J. Smelser & P. B. Baltes (Eds.), International encyclopedia of the social & behavioral sciences (pp. 135–139). Pergamon. https://doi.org/10.1016/B0-08-043076-7/01683-1
29. Berrar, D. (2019). Cross-validation. In S. Ranganathan, M. Gribskov, K. Nakai, & C. Schönbach (Eds.), Encyclopedia of bioinformatics and computational biology (pp. 542–545). Academic Press. https://doi.org/10.1016/B978-0-12-809633-8.20349-X
30. Akoglu, H. (2018). User’s guide to correlation coefficients. Turkish Journal of Emergency Medicine, 18(3), 91–93. https://doi.org/10.1016/j.tjem.2018.08.001
31. Bejani, M. M., & Ghatee, M. (2021). A systematic review on overfitting control in shallow and deep neural networks. Artificial Intelligence Review, 54(8), 6391–6438. https://doi.org/10.1007/s10462-021-09975-1
32. Levine, D., Richards, J., & Whittle, M. W. (2012). Whittle’s gait analysis (5th ed.). Elsevier Health Sciences.
33. Schmitz, A., Silder, A., Heiderscheit, B., Mahoney, J., & Thelen, D. G. (2009). Differences in lower-extremity muscular activation during walking between healthy older and young adults. Journal of Electromyography and Kinesiology, 19(6), 1085–1091. https://doi.org/10.1016/j.jelekin.2008.10.008
34. Strazza, A., Mengarelli, A., Fioretti, S., & Agostini, V. (2017). Surface-EMG analysis for the quantification of thigh muscle dynamic co-contractions during normal gait. Gait & Posture, 51, 228–233. https://doi.org/10.1016/j.gaitpost.2016.11.003
35. Van Criekinge, T., Saeys, W., Hallemans, A., Velghe, S., Vereeck, L., De Hertogh, W., & Truijen, S. (2018). Age-related differences in muscle activity patterns during walking in healthy individuals. Journal of Electromyography and Kinesiology, 41, 124–131. https://doi.org/10.1016/j.jelekin.2018.05.008
36. Garcia, F. A., Pérez-Ibarra, J. C., Terra, M. H., & Siqueira, A. A. G. (2022). Adaptive algorithm for gait segmentation using a single IMU in the thigh pocket. IEEE Sensors Journal, 22(13), 13251–13261. https://doi.org/10.1109/JSEN.2022.3177951
37. Ruiz-Ruiz, L., García-Domínguez, J. J., & Jiménez, A. R. (2024). Foot-forward segmentation algorithm for improving IMU-based gait analysis. IEEE Transactions on Instrumentation and Measurement, 73, Article 3449951. https://doi.org/10.1109/TIM.2024.3449951
38. Mirelman, A., Bonato, P., Camicioli, R., Ellis, T. D., Giladi, N., Hamilton, J. L., Hass, C. J., Hausdorff, J. M., Pelosin, E., & Almeida, Q. J. (2021). Detecting sensitive mobility features for Parkinson’s disease stages via machine learning. Movement Disorders, 36(9), 2144–2155. https://doi.org/10.1002/mds.28631
39. Shawen, N., O’Brien, M. K., Venkatesan, L., Lonini, L., Simuni, T., Hamilton, J. L., Ghaffari, R., Rogers, J. A., & Jayaraman, A. (2020). Role of data measurement characteristics in the accurate detection of Parkinson’s disease symptoms using wearable sensors. Journal of NeuroEngineering and Rehabilitation, 17(1), Article 52. https://doi.org/10.1186/s12984-020-00684-4
40. De Vos, M., Prince, J., Buchanan, T., FitzGerald, J. J., & Antoniades, C. A. (2020). Discriminating progressive supranuclear palsy from Parkinson’s disease using wearable technology and machine learning. Gait & Posture, 77, 257–263. https://doi.org/10.1016/j.gaitpost.2020.02.007
41. Pogorelc, B., Bosnić, Z., & Gams, M. (2012). Automatic recognition of gait-related health problems in the elderly using machine learning. Multimedia Tools and Applications, 58(2), 333–354. https://doi.org/10.1007/s11042-011-0786-1
42. Mehdizadeh, S., Nabavi, H., Sanches, M. G., & Iaboni, A. (2021). Concurrent validity of human pose tracking in video for measuring gait parameters in older adults: A preliminary analysis with multiple trackers, viewing angles, and walking directions. Journal of NeuroEngineering and Rehabilitation, 18(1), Article 139. https://doi.org/10.1186/s12984-021-00933-0
43. Topham, L. K., Khan, W., Al-Jumeily, D., Waraich, A., & Hussain, A. J. (2022). Gait identification using limb joint movement and deep machine learning. IEEE Access, 10, 100113–100127. https://doi.org/10.1109/ACCESS.2022.3207836
44. Andersson, R., Bermejo-García, J., Agujetas, R., Cronhjort, M., & Chilo, J. (2024). Smartphone IMU sensors for human identification through hip joint angle analysis. Sensors, 24(15), Article 4769. https://doi.org/10.3390/s24154769