Application of Multimodal Fusion Algorithm in Intelligent Restoration and Immersive Experience System of Digital Intangible Cultural Heritage Art

YiChen  Du

doi:10.6180/jase.202608_31.037

Application of Multimodal Fusion Algorithm in Intelligent Restoration and Immersive Experience System of Digital Intangible Cultural Heritage Art

Research Categories

YiChen DuThis email address is being protected from spambots. You need JavaScript enabled to view it.

EDNA Joint Institute, China Academy of Art, Hangzhou 310000, Zhejiang, China

Received: December 20, 2025
Accepted: January 31, 2026
Publication Date: February 26, 2026

Copyright The Author(s). This is an open access article distributed under the terms of the Creative Commons Attribution License (CC BY 4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are cited.

Download Citation: ||https://doi.org/10.6180/jase.202608_31.037

This study proposes an intelligent restoration and immersion system for digital intangible cultural heritage, addressing the lack of multimodal data coordination. It uses a Multimodal Graph Convolutional Network (MM-GCN)tofuse images, craft texts, and dynamic action data, with spatiotemporal alignment and adaptive weight allocation to optimize modal contributions. The dual-channel architecture integrates visual restoration via MS-GANwith semantic constraints and tactile feedback through a mechanical model, achieving 4K/60fps real-time rendering. The system achieves an average structural similarity of 0.915, a feature extraction rate of 44.2fps, and a tactile feedback delay under 0.85ms. This algorithm provides a high-precision, low-latency solution for the digitization of intangible cultural heritage, promoting the living inheritance of cultural skills.

Keywords: Digital Intangible Cultural Heritage Art Restoration; Immersive Experience System; Multimodal Fusion; MM-GCN;MS-GAN

[1] Y. Hou, S. Kenderdine, D. Picca, et al., (2022) “Digitizing intangible cultural heritage embodied: State of the art" Journal on Computing and Cultural Heritage 15(3): 1–20. DOI: 10.1145/3494837.
[2] X. Jing, F. Tan, and M. Zhang, (2021) “Digital application of intangible cultural heritage from the perspective of cultural ecology" Journal of Smart Tourism 1(1): 41–52. DOI: 10.52255/smarttourism.2021.1.1.6.
[3] L. N. Amali, M. R. Katili, and W. Ismail, (2022) “Preservation of intangible and tangible cultural heritage using digital technology" Indonesian Journal of Electrical Engineering and Computer Science 28(2): 980–986. DOI: 10.11591/ijeecs.v28.i2.pp980-986.
[4] F. Wang and H. Liu, (2024) “Current status, problems and countermeasures of paper production for ancient book restoration—Based on the field investigation of intangible cultural heritage handmade paper in China" China Paper making 43(10): 122–127. DOI: 10.11980/j.issn.0254-508 X.2024.10.016.
[5] Y. Tong, (2023) “University leadership and practice of intangible cultural heritage craftsmanship in ancient book restoration—Taking the library of Henan University as an example" Journal of Hubei Open Vocational College 36(3): 29–31. DOI: 10.3969/j.issn.2096-711X.2023.03.01-2.
[6] J. Ding, (2024) “Research on digital design protection and inheritance strategy of intangible cultural heritage culture in Guangxi from the perspective of rural revitalization" Shoe Crafts and Design 4(21): 83–85. URL: https: //oversea.cnki.net/kcms2/article/abstract?v=o9f9ysaUiT7ghQZAhyh_QOu6AD8mk6MOp4OE-QQio0uxuUndzg_hbsncVIQKL6rd0U6azSuULmg4wvlXJXqo63tvj07WpsCu0-XuPqZCqvPB9n98sCO_M5zIWpcDdL1HAmzLgKUjc-Q7j9usAqN0-a0Ijch6LVS7mFHNv0_rQ-k=&uniplatform=OVERSEA.
[7] T. Lu, (2024) “Research on the talent training model of intangible cultural heritage digital media art under the perspective of “Double High Plan”—Taking Qin embroidery as an example" Textile Report 43(2): 103–105. URL: https: //oversea.cnki.net/kcms2/article/abstractv=o9f9ysaUiT6g0C4iHTV0KM8IkUn_byqVcjCVlyOl-lJN8IF5TH7TlKWolM455FKnSKeBoftnK4iizzlJCNfXE_gSmTE-NvZgGO1NsxBEQKtVk4GoEh898I1Mm8FGyRGpLNRQweerWiza89YJgKw9sqAd000H5nRiOS1BSeYiM=&uniplatform=OVERSEA.
[8] P. Dhiman, A. Kaur, Y. Hamid, E. Alabdulkreem, H. Elmannai, and N. Ababneh, (2023) “Smart disease detec tion system for citrus fruits using deep learning with edge computing" Sustainability 15(5): 4576. DOI: 10.3390/su15054576.