{"id":2420,"date":"2026-04-06T13:21:21","date_gmt":"2026-04-06T05:21:21","guid":{"rendered":"https:\/\/iweb20wp-b205b.url.tku.edu.tw\/jase\/?post_type=tkuisotope&p=2420"},"modified":"2026-05-24T15:16:10","modified_gmt":"2026-05-24T07:16:10","slug":"dividual-a-disentangled-visible-and-infrared-image-fusion-contrastive-learning-method","status":"publish","type":"tkuisotope","link":"\/jase\/?tkuisotope=dividual-a-disentangled-visible-and-infrared-image-fusion-contrastive-learning-method","title":{"rendered":"DIVIDUAL: A Disentangled Visible and Infrared Image Fusion Contrastive Learning Method"},"content":{"rendered":"\n
\n
\n

<\/i> 2025<\/a><\/p>\n<\/div>\n\n\n\n

\n

<\/i> Volume 28, Issue 5<\/a><\/p>\n<\/div>\n\n\n\n

\n
\n
\n
\n

Shaoqi Yang1<\/sup> and Dan He2<\/sup><\/i><\/a><\/p>\n\n\n\n

1<\/sup>School of Safety Engineering, Shenyang Aerospace University, No.37 Daoyi South Avenue, Shenbei New Area, Shenyang, 110136, China<\/p>\n\n\n\n

2<\/sup>Dalian University of Finance and Economics, No. 80 Renwen Street, Jinzhou New Area, Dalian, 116622, China<\/p>\n<\/div>\n\n\n\n

\n

Received: March 15, 2024
Accepted: April 14, 2024
Publication Date: April 6, 2026<\/p>\n<\/div>\n<\/div>\n\n\n\n

\"\u4e0a\u50b3\u5716\u7247\"\n\n\n

Architecture of the ConvBlock.<\/p>\n<\/div>\n<\/div>\n\n\n\n

<\/i> Copyright <\/strong>The Author(s). This is an open access article distributed under the terms of the Creative Commons Attribution License (CC BY 4.0)<\/a>, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are cited.<\/p>\n\n\n\n

Download Citation:\u00a0 BibTeX <\/a>| http:\/\/dx.doi.org\/10.6180\/jase.202505_28(5).0005<\/a>\u00a0\u00a0<\/p>\n\n\n\n

Download PDF<\/a><\/p>\n\n\n\n

<\/div>\n\n\n\n

Visible and Infrared Image Fusion (VIIF), as a vital fundamental component in vehicle applications, has been attracting plenty of attention from the academic and industrial communities over past few years. Various deep learning based methods have been proposed to effectively fuse visible and infrared images for improving the comprehensiveness of vehicle sensing and monitoring capabilities. However, due to the complex coupling property of the vehicle observational environment, it is still a challenging problem to effectively decouple and fusion visible and infrared images. To address this problem, we propose a DIsentangled Visible and InfrareD image fUsion contrAstive Learning method (DIVIDUAL). For capturing common and complementary information between the two domains, DIVIDUAL contains a self-supervised decoupling framework to separate domain-invariant and domain-specific representations. Meanwhile, for removing the noise in domain-specific representations and extracting clean domain-invariant representations, DIVIDUAL deploys a decoupling contrastive loss to effectively separate noise information and retain critical information in domains. Finally, DIVIDUAL generates fused images in an end-to-end manner. Extensive performance and generalization experiments on TNO and RoadScene datasets demonstrate that DIVIDUAL has superior visual results.<\/p>\n\n\n\n

Keywords: Disentangled Learning; Image Fusion; Contrastive Learning; Self-unsupervised Learning<\/em><\/p>\n\n\n\n

<\/div>\n\n\n\n
\n
    \n
  1. [1] J. Pisane, S. Azarian, M. Lesturgie, and J. Verly, (2014) \u201cAutomatic target recognition for passive radar” IEEE Transactions on Aerospace and Electronic Systems
    50(1): 371\u2013392. DOI: 10.1109\/TAES.2013.120486.<\/li>\n
  2. [2] C. Sun, C. Zhang, and N. Xiong, (2020) \u201cInfrared and visible image fusion techniques based on deep learning: A review” Electronics 9(12): 2162. DOI: 10.3390\/electronics9122162.<\/li>\n
  3. [3] Y. Zou, X. Liang, and T. Wang, (2013) \u201cVisible and infrared image fusion using the lifting wavelet” TELKOMNIKA Indonesian Journal of Electrical Engineering 11(11): 6290\u20136295.<\/li>\n
  4. [4] Q. Zhang and X. Maldague, (2016) \u201cAn adaptive fusion approach for infrared and visible images based on NSCT and compressed sensing” Infrared Physics & Technology 74: 11\u201320. DOI: 10.1016\/j.infrared.2015.11.003.<\/li>\n
  5. [5] Y. Liu, X. Yang, R. Zhang, M. K. Albertini, T. Celik, and G. Jeon, (2020) \u201cEntropy-based image fusion with joint sparse representation and rolling guidance filter” Entropy 22(1): 118. DOI: 10.3390\/e22010118.<\/li>\n
  6. [6] B. Cheng, L. Jin, and G. Li, (2018) \u201cGeneral fusion method for infrared and visual images via latent low-rank representation and local non-subsampled shearlet transform” Infrared Physics & Technology 92: 68\u201377. DOI: 10.1016\/j.infrared.2018.05.006.<\/li>\n
  7. [7] Y. Huang and K. Yao. \u201cMulti-Exposure Image Fusion Method Based on Independent Component Analysis\u201d. In: Proceedings of the 2020 International Conference on Pattern Recognition and Intelligent Systems. 2020, 1\u20136.<\/li>\n
  8. [8] P. Li, J. Gao, J. Zhang, S. Jin, and Z. Chen, (2023) \u201cDeep Reinforcement Clustering” IEEE Transactions on Multimedia 25: 8183\u20138193.<\/li>\n
  9. [9] J. Gao, P. Li, A. A. Laghari, G. Srivastava, T. R. Gadekallu, S. Abbas, and J. Zhang, (2024) \u201cIncomplete Multiview Clustering via Semidiscrete Optimal Transport for Multimedia Data Mining in IoT” ACM Transactions on Multimedia Computing, Communications and Applications 20(6): 158:1\u2013158:20. DOI: 10.1145\/3625548.<\/li>\n
  10. [10] J. Gao, M. Liu, P. Li, J. Zhang, and Z. Chen, (2023) \u201cDeep Multiview Adaptive Clustering With Semantic Invariance” IEEE Transactions on Neural Networks and Learning Systems: 1\u201314. DOI: 10.1109\/TNNLS.2023.3265699.<\/li>\n
  11. [11] P. J. Burt and E. H. Adelson. \u201cThe Laplacian pyramid as a compact image code\u201d. In: Readings in computer vision. Elsevier, 1987, 671\u2013679.<\/li>\n
  12. [12] W. Kong, Y. Lei, and H. Zhao, (2014) \u201cAdaptive fusion method of visible light and infrared images based on non-subsampled shearlet transform and fast non-negative matrix factorization” Infrared Physics & Technology 67: 161\u2013172. DOI: 10.1016\/j.infrared.2014.07.019.<\/li>\n
  13. [13] Y. Liu, X. Chen, J. Cheng, H. Peng, and Z. Wang, (2018) \u201cInfrared and visible image fusion with convolutional neural networks” International Journal of Wavelets, Multiresolution and Information Processing 16(03): 1850018.<\/li>\n
  14. [14] Z. Wang, J. Wang, Y. Wu, J. Xu, and X. Zhang, (2021) \u201cUNFusion: A unified multi-scale densely connected network for infrared and visible image fusion” IEEE Transactions on Circuits and Systems for Video Technology 32(6): 3360\u20133374.<\/li>\n
  15. [15] Y. Liu, C. Miao, J. Ji, and X. Li, (2021) \u201cMMF: A Multiscale MobileNet based fusion method for infrared and visible image” Infrared Physics & Technology 119: 103894.<\/li>\n
  16. [16] J. Ma, W. Yu, P. Liang, C. Li, and J. Jiang, (2019) \u201cFusionGAN: A generative adversarial network for infrared and visible image fusion” Information fusion 48: 11\u201326. DOI: 10.1016\/j.inffus.2018.09.004.<\/li>\n
  17. [17] J. Ma, H. Xu, J. Jiang, X. Mei, and X.-P. Zhang, (2020) \u201cDDcGAN: A dual-discriminator conditional generative adversarial network for multi-resolution image fusion” IEEE Transactions on Image Processing 29: 4980\u20134995. DOI: 10.1109\/TIP.2020.2977573.<\/li>\n
  18. [18] H. Zhang, H. Xu, Y. Xiao, X. Guo, and J. Ma. \u201cRethinking the image fusion: A fast unified image fusion network based on proportional maintenance of gradient and intensity\u201d. In: Proceedings of the AAAI conference on artificial intelligence. 34. 07. 2020, 12797\u201312804.<\/li>\n
  19. [19] S. Hao, T. He, B. An, X. Ma, H. Wen, and F. Wang, (2022) \u201cVDFEFuse: A novel fusion approach to infrared and visible images” Infrared Physics & Technology 121: 104048. DOI: 10.1016\/j.infrared.2022.104048.<\/li>\n
  20. [20] X. Liu, H. Gao, Q. Miao, Y. Xi, Y. Ai, and D. Gao, (2022) \u201cMFST: Multi-modal feature self-adaptive transformer for infrared and visible image fusion” Remote Sensing 14(13): 3233.<\/li>\n
  21. [21] W. Tang, F. He, Y. Liu, Y. Duan, and T. Si, (2023) \u201cDatfuse: Infrared and visible image fusion via dual attention transformer” IEEE Transactions on Circuits and Systems for Video Technology:<\/li>\n
  22. [22] H.-Y. Lee, H.-Y. Tseng, J.-B. Huang, M. Singh, and M.-H. Yang. \u201cDiverse image-to-image translation via disentangled representations\u201d. In: Proceedings of the
    European conference on computer vision (ECCV). 2018, 35\u201351.<\/li>\n
  23. [23] H.-Y. Lee, H.-Y. Tseng, Q. Mao, J.-B. Huang, Y.-D. Lu, M. Singh, and M.-H. Yang, (2020) \u201cDrit++: Diverse image-to-image translation via disentangled representations” International Journal of Computer Vision 128: 2402\u20132417.<\/li>\n
  24. [24] L. Tran, X. Yin, and X. Liu. \u201cDisentangled representation learning gan for pose-invariant face recognition\u201d. In: Proceedings of the IEEE conference on computer vision and pattern recognition. 2017, 1415\u20131424.<\/li>\n
  25. [25] D. Yang, S. Huang, H. Kuang, Y. Du, and L. Zhang. \u201cDisentangled representation learning for multimodal emotion recognition\u201d. In: Proceedings of the 30th ACM International Conference on Multimedia. 2022, 1642\u20131651.<\/li>\n
  26. [26] Q. Wang, Y. Zhang, Y. Zheng, P. Pan, and X.-S. Hua, (2022) \u201cDisentangled representation learning for textvideo retrieval” arXiv preprint arXiv:2203.07111:<\/li>\n
  27. [27] P. Li, A. A. Laghari, M. Rashid, J. Gao, T. R. Gadekallu, A. R. Javed, and S. Yin, (2023) \u201cA Deep Multimodal Adversarial Cycle-Consistent Network for Smart Enterprise System” IEEE Transactions on Industrial Informatics 19(1): 693\u2013702. DOI: 10.1109\/TII.2022.3197201.<\/li>\n
  28. [28] H. Xu, X. Wang, and J. Ma, (2021) \u201cDRF: Disentangled representation for visible and infrared image fusion” IEEE Transactions on Instrumentation and Measurement 70: 1\u201313.<\/li>\n
  29. [29] Y. Gao, S. Ma, and J. Liu, (2022) \u201cDCDR-GAN: A densely connected disentangled representation generative adversarial network for infrared and visible image fusion” IEEE Transactions on Circuits and Systems for Video Technology 33(2): 549\u2013561.<\/li>\n
  30. [30] M. Caron, I. Misra, J. Mairal, P. Goyal, P. Bojanowski, and A. Joulin, (2020) \u201cUnsupervised learning of visual features by contrasting cluster assignments” Advances in neural information processing systems 33: 9912\u20139924.<\/li>\n
  31. [31] URL: https:\/\/figshare.com\/articles\/dataset\/TNO_ Image_Fusion_Dataset\/1008029.<\/li>\n
  32. [32] URL: https:\/\/www.flir.com\/oem\/adas\/adas-dataset-form\/.<\/li>\n
  33. [33] J. W. Roberts, J. A. Van Aardt, and F. B. Ahmed, (2008) \u201cAssessment of image fusion procedures using entropy, image quality, and multispectral classification” Journal of Applied Remote Sensing 2(1): 023522.<\/li>\n
  34. [34] G. Qu, D. Zhang, and P. Yan, (2002) \u201cInformation measure for performance of image fusion” Electronics letters 38(7): 1.<\/li>\n
  35. [35] Y. Han, Y. Cai, Y. Cao, and X. Xu, (2013) \u201cA new image fusion performance metric based on visual information fidelity” Information fusion 14(2): 127\u2013135.<\/li>\n
  36. [36] Y. J. Rao, (1997) \u201cIn-fibre Bragg grating sensors” Measurement Science & Technology 8(4): 355\u2013375.<\/li>\n
  37. [37] G. Cui, H. Feng, Z. Xu, Q. Li, and Y. Chen, (2015) \u201cDetail preserved fusion of visible and infrared images using regional saliency extraction and multi-scale image decomposition” Optics Communications 341: 199\u2013209.<\/li>\n
  38. [38] A. M. Eskicioglu and P. S. Fisher, (1995) \u201cImage quality measures and their performance” IEEE Transactions on communications 43(12): 2959\u20132965.<\/li>\n
  39. [39] H. Li and X.-J. Wu, (2018) \u201cDenseFuse: A fusion approach to infrared and visible images” IEEE Transactions on Image Processing 28(5): 2614\u20132623.<\/li>\n
  40. [40] X. Luo, Y. Gao, A. Wang, Z. Zhang, and X.-J. Wu, (2021) \u201cIFSepR: A general framework for image fusion based on separate representation learning” IEEE Transactions on Multimedia:<\/li>\n
  41. [41] H. Li, X.-J. Wu, and J. Kittler, (2021) \u201cRFN-Nest: An end-to-end residual fusion network for infrared and visible images” Information Fusion 73: 72\u201386.<\/li>\n
  42. [42] H. Xu, J. Ma, J. Jiang, X. Guo, and H. Ling, (2020) \u201cU2Fusion: A unified unsupervised image fusion network” IEEE Transactions on Pattern Analysis and Machine
    Intelligence 44(1): 502\u2013518.<\/li>\n
  43. [43] D. Wang, J. Liu, X. Fan, and R. Liu, (2022) \u201cUnsupervised misaligned infrared and visible image fusion via cross-modality image generation and registration” arXiv preprint arXiv:2205.11876:<\/li>\n
  44. [44] L. Tang, J. Yuan, H. Zhang, X. Jiang, and J. Ma, (2022) \u201cPIAFusion: A progressive infrared and visible image fusion network based on illumination aware” Information Fusion 83: 79\u201392.<\/li>\n
  45. [45] L. Tang, J. Yuan, and J. Ma, (2022) \u201cImage fusion in the loop of high-level vision tasks: A semantic-aware real-time infrared and visible image fusion network” Information Fusion 82: 28\u201342. DOI: 10.1016\/j.inffus.2021.12.004.<\/li>\n
  46. [46] Z. Zhu, X. Yang, R. Lu, T. Shen, X. Xie, and T. Zhang, (2022) \u201cClf-Net: Contrastive learning for infrared and visible image fusion network” IEEE Transactions on
    Instrumentation and Measurement 71: 1\u201315. DOI: 10.1109\/TIM.2022.3203000.<\/li>\n
  47. [47] L. Tang, H. Zhang, H. Xu, and J. Ma, (2023) \u201cRethinking the necessity of image fusion in high-level vision tasks: A practical infrared and visible image fusion network based on progressive semantic injection and scene fidelity” Information Fusion: 101870. DOI: 10.1016\/j.inffus.2023.101870.<\/li>\n<\/ol>\n<\/div>\n\n\n\n

    <\/p>\n","protected":false},"author":3,"template":"wp-custom-template-detail-4-aricles","meta":{"_uag_custom_page_level_css":""},"categories":[9,6,268],"tags":[359],"acf":[],"uagb_featured_image_src":[],"uagb_author_info":{"display_name":"\u6797\u923a\u6db5","author_link":"\/jase\/?author=3"},"uagb_comment_info":0,"uagb_excerpt":" Copyright The Author(s). This is an open access article distributed under the terms of the Creative Commons Attribution License (CC BY 4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are cited. Download Citation:\u00a0 BibTeX | http:\/\/dx.doi.org\/10.6180\/jase.202505_28(5).0005\u00a0\u00a0 Download PDF Visible and Infrared Image Fusion (VIIF), as a vital fundamental component…","_links":{"self":[{"href":"\/jase\/index.php?rest_route=\/wp\/v2\/tkuisotope\/2420"}],"collection":[{"href":"\/jase\/index.php?rest_route=\/wp\/v2\/tkuisotope"}],"about":[{"href":"\/jase\/index.php?rest_route=\/wp\/v2\/types\/tkuisotope"}],"author":[{"embeddable":true,"href":"\/jase\/index.php?rest_route=\/wp\/v2\/users\/3"}],"wp:attachment":[{"href":"\/jase\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=2420"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"\/jase\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=2420"},{"taxonomy":"post_tag","embeddable":true,"href":"\/jase\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=2420"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}