Multi-Task Learning for Sports Training Monitoring via Multimodal Fusion of IMU and Human-Body Capacitance Signals

Wang Hailong; Guan  Xinru; Jiang  Sheng; Zhang  Lijun

doi:10.6180/jase.202608_31.038

Multi-Task Learning for Sports Training Monitoring via Multimodal Fusion of IMU and Human-Body Capacitance Signals

Wang Hailong¹, Guan Xinru², Jiang Sheng², and Zhang Lijun¹This email address is being protected from spambots. You need JavaScript enabled to view it.

¹School of Physical Education, Jiamusi University, Jiamusi 154007, China

²College of Information and Electronic Technology, Jiamusi University, Jiamusi 154007, China

Received: December 25, 2025
Accepted: January 17, 2026
Publication Date: March 1, 2026

Copyright The Author(s). This is an open access article distributed under the terms of the Creative Commons Attribution License (CC BY 4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are cited.

Download Citation: ||https://doi.org/10.6180/jase.202608_31.038

For training monitoring in free-living fitness scenarios, this paper addresses the limitations of inertial measurement units (IMUs) in cross-body information perception, accumulation of integration drift, and limited performance in recognizing low-amplitude movements. We introduce human-body capacitance signals (HBC) as a complementary modality and propose a channel-level graph attention–temporal convolution multimodal multi-task learning framework. Unlike approaches relying solely on local inertial measurements, HBC signals capture the overall movement correlation between distal body parts (such as the lower limbs or weighted equipment) and the torso or sensor-worn locations through body electrostatic coupling. This allows explicit modeling of cross-limb dependencies, compensating for the IMU’s limitations in characterizing full-body coordinated movements under local sensing conditions. The proposed method models each IMU and HBC channel as graph nodes, combining physical priors and data-driven edge sets. A graph attention network (GAT) adaptively learns cross-channel and cross-modality dependency structures, while a temporal convolutional network extracts multi-scale rhythmic features. The unified framework jointly optimizes three tasks: activity recognition, repetition counting, and training intensity estimation. Cross-subject (LOSO) evaluation results on the RecGym dataset show that the model achieves an average recognition accuracy of 0.894 (Kappa 0.871) across 10 subjects. In subsets involving low-amplitude movements (such as light dumbbell exercises and small-range joint adjustments), the fusion model demonstrates more stable recognition performance, maintaining an accuracy around 0.9, outperforming the IMU-only model. Additionally, the model achieves MAE values of 0.234 for counting and 0.156 for intensity estimation. Further modality comparisons and attention analysis reveal that the fusion of IMU and HBC not only enhances recognition robustness in low-dynamic scenarios but also provides a physically interpretable distribution of channel importance, offering an efficient and interpretable multimodal solution for wearable sports training monitoring.

Keywords: Sports Training Monitoring; Multi-Task Learning; IMU; Artificial Intelligence; Multimodal Fusion

[1] D. Chen, R. Cui, C. Huang, Z. Wang, and L. Niu, (2025) “Wearable Mechanoluminescent Triboelectric Sensors for Real-Time Monitoring of Nighttime Sports Activities" ACS Applied Materials & Interfaces 17(12): 18844–18851. DOI: 10.1021/acsami.4c22118.
[2] S. Bian, B. Zhou, H. Bello, and P. Lukowicz. “A wear able magnetic field based proximity sensing system for monitoring COVID-19 social distancing”. In: Proceedings of the 2020 ACM International Symposium on Wearable Computers. 2020, 22–26. DOI: 10.1145/3410531.3414313.
[3] H. Espinosa, A. Mears, A. Stamm, Y. Ohgi, and C. Coniglio, (2025) “Wearable sensor technology in sports monitoring" Sports Engineering 28(1): 4. DOI: 10.1007/s12283-025-00485-9.
[4] P. Kumar, S. Chauhan, and L. K. Awasthi, (2024) “Hu man activity recognition (har) using deep learning: Re view, methodologies, progress and future research directions" Archives of Computational Methods in Engineering 31(1): 179–219. DOI: 10.1007/s11831-023-09986-x.
[5] T. S. Qureshi, M. H. Shahid, A. A. Farhan, and S. Alamri, (2025) “A systematic literature review on human activity recognition using smart devices: advances, challenges, and future directions" Artificial Intelligence Review 58(9): 276. DOI: 10.1007/s10462-025-11275-x.
[6] S. Bian, R. Strahinja, P. Schilk, C. Marc-André, S. Cortesi, K. Dheman, E. Reinschmidt, and M. Magno. “Earable and wrist-worn setup for accurate step counting utilizing body-area electrostatic sensing”. In: Companion of the 2024 on ACM International Joint Conference on Pervasive and Ubiquitous Computing. 2024, 904–910. DOI: 10.1145/3675094.3678468.
[7] E. El-Adawi, E. Essa, M. Handosa, and S. Elmougy, (2024) “Wireless body area sensor networks based human activity recognition using deep learning" Scientific Re ports 14(1): 2702. DOI: 10.1038/s41598-024-53069-1.
[8] Z. Li, (2025) “Innovation and transformation of sports event management driven by artificial intelligence" Journal of Social Science Humanities and Literature 8(3): 16–20. DOI: 10.53469/jsshl.2025.08(03).03.
[9] D. Nahavandi, R. Alizadehsani, A. Khosravi, and U. R. Acharya, (2022) “Application of artificial intelligence in wearable devices: Opportunities and challenges" Computer Methods and Programs in Biomedicine 213: 106541. DOI: 10.1016/j.cmpb.2021.106541.
[10] R. Singh, A. K. S. Kushwaha, R. Srivastava, et al., (2023) “Recent trends in human activity recognition–A comparative study" Cognitive Systems Research 77: 30–44. DOI: 10.1016/j.cogsys.2022.10.003.
[11] Y. Xiong, L. Luo, J. Yang, J. Han, Y. Liu, H. Jiao, S. Wu, L. Cheng, Z. Feng, J. Sun, et al., (2023) “Scalable spinning, winding, and knitting graphene textile TENG for energy harvesting and human motion recognition" Nano Energy 107: 108137. DOI: 10.1016/j.nanoen.2022.108137.
[12] L. Yang, O. Amin, and B. Shihada, (2024) “Intelligent wearable systems: Opportunities and challenges in health and sports" ACM Computing Surveys 56(7): 1–42. DOI: 10.1145/3648469.
[13] A. Qazi and A. Iqbal. “ExerAide: AI-assisted multi modal diagnosis for enhanced sports performance and personalised rehabilitation”. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pat tern Recognition. 2024, 3430–3438. DOI: 10.1109/CVPRW63382.2024.00347.
[14] C. Li, L. Song, S. Chen, R. Xie, and W. Zhang, (2022) “Deep online video stabilization using IMU sensors" IEEE Transactions on Multimedia 25: 2047–2060. DOI: 10.1109/TMM.2022.3142429.
[15] S. Qiu, H. Zhao, N. Jiang, Z. Wang, L. Liu, Y. An, H. Zhao, X. Miao, R. Liu, and G. Fortino, (2022) “Multi sensor information fusion based on machine learning for real applications in human activity recognition: State-of the-art and research challenges" Information Fusion 80: 241–265. DOI: 10.1016/j.inffus.2021.11.006.
[16] J. Wu, Z. Mo, X. Gao, W. Xin, W. Shi, and J. Park, (2025) “Artificial intelligence assisted wearable flexible sensors for sports: research progress in technology integration and application" International Journal of Smart and Nano Materials 16(3): 510–548. DOI: 10.1080/19475411.2025.2519582.
[17] S. Bian, M. Liu, B. Zhou, P. Lukowicz, and M. Magno, (2024) “Body-area capacitive or electric field sensing for human activity recognition and human-computer inter action: A comprehensive survey" Proceedings of the AC Mon Interactive, Mobile, Wearable and Ubiquitous Technologies 8(1): 1–49. DOI: 10.1145/3643555.
[18] M. ElFezazi, A. Achmamad, A. Jbari, and A. Jilbab, (2025) “IoT-Based System Using IMU Sensor Fusion for Knee Telerehabilitation Monitoring" IEEE Sensors Journal: DOI: 10.1109/JSEN.2025.3542261.
[19] N.Brígida,D.Catela, C. Mercê, andM.Branco,(2025) “Precision of an Inertial System to Evaluate the Finger Tap ping Test in Women with Fibromyalgia" Sports 13(11): 373. DOI: 10.3390/sports13110373.
[20] S. Bian, V. F. Rey, S. Yuan, and P. Lukowicz, (2022) “The contribution of human body capacitance/body-area electric field to individual and collaborative activity recog nition" arXiv preprint arXiv:2210.14794: DOI: arXiv: 2210.14794.
[21] J. Zhang, R. Guo, Y. Zhu, Y. Che, Y. Zeng, L. Yu, Z. Yang, and J. Yang, (2025) “Sensor-Driven Real-Time Recognition of Basketball Goal States Using IMU and Deep Learning" Sensors 25(12): 3709. DOI: 10.3390/s25123709.
[22] S. Suh, V. F. Rey, S. Bian, Y.-C. Huang, J. M. Rožanec, H. T. Ghinani, B. Zhou, and P. Lukowicz, (2023) “Worker activity recognition in manufacturing line us ing near-body electric field" IEEE Internet of Things Journal 11(7): 11554–11565. DOI: 10.1109/JIOT.2023.3330372.
[23] A. Yunita, M. I. Pratama, M. Z. Almuzakki, H. Ra madhan, E. A. P. Akhir, A. B. F. Mansur, and A. H. Basori, (2025) “Performance analysis of neural network architectures for time series forecasting: A comparative study of RNN, LSTM, GRU, and hybrid models" MethodsX 15: 103462. DOI: 10.1016/j.mex.2025.103462.
[24] S. Bian, V. F. Rey, S. Yuan, and P. Lukowicz, (2022) “The contribution of human body capacitance/body-area electric field to individual and collaborative activity recog nition" arXiv preprint arXiv:2210.14794: DOI: arXiv: 2210.14794.
[25] N. sedaghati, S. ardebili, and A. Ghaffari, (2025) “Ap plication of human activity/action recognition: a review" Multimedia Tools and Applications: 1–30. DOI: 10.1007/s11042-024-20576-2.
[26] Z. Wu, S. Pan, F. Chen, G. Long, C. Zhang, and P. S. Yu, (2020) “A comprehensive survey on graph neural networks" IEEE transactions on neural networks and learning systems 32(1): 4–24. DOI: 10.1109/TNNLS.2020.2978386.
[27] N.X. Tung, V. H. Viet, T. Van Chien, N. T. Hoa, W. J. Hwang, et al., (2024) “HGNN: a hierarchical graph neural network architecture for joint resource management in dynamic wireless sensor networks" IEEE Sensors Journal: DOI: 10.1109/JSEN.2024.3485058.
[28] H. Gui, X. Tang, G. Li, C. Wang, and J. Lu, (2025) “A Novel Dynamic Graph Attention Aggregation Net work for Multivariate Time Series Classification" Pattern Recognition: 112732. DOI: 10.1016/j.patcog.2025. 112732.
[29] S. Bian, S. Yuan, V. F. Rey, and P. Lukowicz. “Using human body capacitance sensing to monitor leg motion dominated activities with a wrist worn device”. In: Sensor-and Video-Based Activity and Behavior Computing: Proceedings of 3rd International Conference on Activity and Behavior Computing (ABC 2021). Springer. 2022, 81–94. DOI: 10.1007/978-981-19-0361-8_5.
[30] Y. Wang, X. Wang, H. Yang, Y. Geng, H. Yu, G. Zheng, and L. Liao, (2023) “MhaGNN: A novel framework for wearable sensor-based human activity recognition combining multi-head attention and graph neural networks" IEEE Transactions on Instrumentation and Measurement 72: 1–14. DOI: 10.1109/TIM.2023.3276004.
[31] X. Wei and Z. Wang, (2024) “TCN-attention-HAR: Hu man activity recognition based on attention mechanism time convolutional network" Scientific Reports 14(1): 7414. DOI: 10.1038/s41598-024-57912-3.
[32] H.-J. Choi, H. Kim, M. Lee, M. Jeong, C. Kim, J. Yoon, and S.-K. Ko. “Trajectory imputation in multi-agent sports with derivative-accumulating self-ensemble”. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases. Springer. 2025, 336–353. DOI: 10.1007/978-3-032-06129-4_20.
[33] R. Singh, A. K. S. Kushwaha, R. Srivastava, et al., (2023) “Recent trends in human activity recognition–A comparative study" Cognitive Systems Research 77: 30–44. DOI: 10.1016/j.cogsys.2022.10.003.
[34] S. Bian, R. Strahinja, P. Schilk, C. Marc-André, S. Cortesi, K. Dheman, E. Reinschmidt, and M. Magno. “Earable and wrist-worn setup for accurate step counting utilizing body-area electrostatic sensing”. In: Companion of the 2024 on ACM International Joint Conference on Pervasive and Ubiquitous Computing. 2024, 904–910. DOI: 10.1145/3675094.3678468.
[35] O. Baños Legrán, J. M. Gálvez Gómez, M. Damas Hermoso, H. E. Pomares Cintas, I. Rojas Ruiz, et al., (2014) “Window Size Impact in Human Activity Recognition": DOI: 10.1109/ACCESS.2021.3099046.
[36] S. Bian, V. F. Rey, S. Yuan, and P. Lukowicz, (2025) “Hybrid CNN-Dilated Self-attention Model Using Inertial and Body-Area Electrostatic Sensing for Gym Workout Recognition, Counting, and User Authentification" arXiv preprint arXiv:2503.06311: DOI: 10.48550/arXiv.2503. 06311.