Journal of Applied Science and Engineering

Published by Tamkang University Press

ESCI jase impact factor scopus logo open access rate of Scopus journal

A Federated Learning Approach for Cross-Institutional Intangible Cultural Heritage Multimedia Content Retrieval and Sensitive Information Protection in the Digital Cultural and Creative Industry

 2026

 Volume 35

Qiku Bao

Jilin Animation Institute, 130000, Jilin, China

Received: July 21, 2026
Accepted: August 20, 2026
Publication Date: September 06, 2026

上傳圖片

Convergence Curves of Different Federated Retrieval Methods

 Copyright The Author(s). This is an open access article distributed under the terms of the Creative Commons Attribution License (CC BY 4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are cited.

Download Citation: BibTeX | http://dx.doi.org/10.6180/jase.202612_35.020

Download PDF

In the digital cultural and creative industry, ICH multimedia resources (images, videos, audio, text, metadata) are scattered across cultural institutions and often involve sensitive information (identity, location, craft knowledge, ritual scenes), so centralized retrieval cannot balance sharing, efficiency, and privacy. This study proposes a federated learning framework combining local multimodal encoding, federated contrastive learning for cross-modal alignment, gradient pruning/DP/secure aggregation against inference risks, and retrieval combining prototype routing, local indexing, sensitivity assessment, and result anonymization for privacy-protected Top-K retrieval. On 31,500 cross-institutional non-IID ICH samples, the method achieves Recall@5 of 0.807, mAP of 0.752, and NDCG@10 of 0.793 (beating FedAvg, FedProx, FedCMR, FedCMR-DP), maintains Recall@5 of 0.738 at non-IID degree 0.9, reduces sensitive-information exposure from 18.6% to 4.1%, lowers member/attribute-inference success to 38.6%/33.7%, and achieves 126 ms latency and 472 queries/s throughput at 20 nodes.

Keywords: federated learning; intangible cultural heritage; multimedia retrieval; sensitive information protection; distributed index; scalable information system; differential privacy

  1. [1] S. Münster, F. Maiwald, I. di Lenardo, J. Henriksson, A. Isaac, M. M. Graf, C. Beck, and J. Oomen, (2024) “Artificial Intelligence for Digital Heritage Innovation: Setting up a R&D Agenda for Europe” Heritage 7(2): 794-816. DOI: https://doi.org/10.3390/heritage7020038.
  2. [2] F. Ju, (2024) “Mapping the Knowledge Structure of Image Recognition in Cultural Heritage: A Scientometric Analysis Using CiteSpace, VOSviewer, and Bibliometrix” Journal of Imaging 10(11): 272. DOI: https://doi.org/10.3390/jimaging10110272.
  3. [3] J. Li, X. Zheng, I. Watanabe, and Y. Ochiai, (2024) “A Systematic Review of Digital Transformation Technologies in Museum Exhibition” Computers in Human Behavior 161: 108407. DOI: https://doi.org/10.1016/j.chb.2024.108407.
  4. [4] L. Tsipi, D. Vouyioukas, G. Loumos, A. Kargas, and D. Varoutas, (2023) “Digital Repository as a Service (D-RaaS): Enhancing Access and Preservation of Cultural Heritage Artifacts” Heritage 6(10): 6881-6900. DOI: https://doi.org/10.3390/heritage6100359.
  5. [5] X. Xia, G. Dong, F. Li, L. Zhu, and X. Ying, (2023) “When CLIP Meets Cross-Modal Hashing Retrieval: A New Strong Baseline” Information Fusion 100: 101968. DOI: https://doi.org/10.1016/j.inffus.2023.101968.
  6. [6] Y. Sun, Z. Ren, P. Hu, D. Peng, and X. Wang, (2024) “Hierarchical Consensus Hashing for Cross-Modal Retrieval” IEEE Transactions on Multimedia 26: 824-836. DOI: https://doi.org/10.1109/TMM.2023.3272169.
  7. [7] H. Cai, B. Zhang, J. Li, B. Hu, and J. Chen, (2024) “Unsupervised Dual Hashing Coding (UDC) on Semantic Tagging and Sample Content for Cross-Modal Retrieval” IEEE Transactions on Multimedia 26: 9109-9120. DOI: https://doi.org/10.1109/TMM.2024.3385986.
  8. [8] J. Liu, J. Huang, Y. Zhou, X. Li, S. Ji, H. Xiong, and D. Dou, (2022) “From Distributed Machine Learning to Federated Learning: A Survey” Knowledge and Information Systems 64(4): 885-917. DOI: https://doi.org/10.1007/s10115-022-01664-x.
  9. [9] K. Zhang, X. Song, C. Zhang, and S. Yu, (2022) “Challenges and Future Directions of Secure Federated Learning: A Survey” Frontiers of Computer Science 16(5): 165817. DOI: https://doi.org/10.1007/s11704-021-0598-z.
  10. [10] V. Mothukuri, R. M. Parizi, S. Pouriyeh, Y. Huang, A. Dehghantanha, and G. Srivastava, (2021) “A Survey on Security and Privacy of Federated Learning” Future Generation Computer Systems 115: 619-640. DOI: https://doi.org/10.1016/j.future.2020.10.007.
  11. [11] Y.-M. Lin, Y. Gao, M.-G. Gong, S.-J. Zhang, Y.-Q. Zhang, and Z.-Y. Li, (2023) “Federated Learning on Multimodal Data: A Comprehensive Survey” Machine Intelligence Research 20(4): 539-553. DOI: https://doi.org/10.1007/s11633-022-1398-0.
  12. [12] F. Liberti, D. Berardi, and B. Martini, (2024) “Federated Learning in Dynamic and Heterogeneous Environments: Advantages, Performances, and Privacy Problems” Applied Sciences 14(18): 8490. DOI: https://doi.org/10.3390/app14188490.
  13. [13] P. Qi, D. Chiaro, and F. Piccialli, (2023) “FL-FD: Federated Learning-Based Fall Detection with Multimodal Data Fusion” Information Fusion 99: 101890. DOI: https://doi.org/10.1016/j.inffus.2023.101890.
  14. [14] G. Feretzakis, K. Papaspyridis, A. Gkoulalas-Divanis, and V. S. Verykios, (2024) “Privacy-Preserving Techniques in Generative AI and Large Language Models: A Narrative Review” Information 15(11): 697. DOI: https://doi.org/10.3390/info15110697.
  15. [15] Q. Li, Y. Diao, Q. Chen, and B. He, (2022) “Federated Learning on Non-IID Data Silos: An Experimental Study” IEEE 38th International Conference on Data Engineering (ICDE): 965-978. DOI: https://doi.org/10.1109/ICDE53745.2022.00077.