Journal of Applied Science and Engineering

Published by Tamkang University Press

ESCI jase impact factor scopus logo open access rate of Scopus journal

MSDF-Net: Macro-Guided Static–Dynamic Fusion for Micro Expression Recognition

 2026

 Volume 35

Yan Lv, Nannan Hu, Fan Zhou, Yang Wang

Shenyang Ligong University, Shenyang, Liaoning 110159, China

Received: July 29, 2026
Accepted: September 11, 2026
Publication Date: September 25, 2026

上傳圖片

Confusion matrices of MSDF-Net under the three-class setting on CASME II, SAMM, and SMIC-HS.

 Copyright The Author(s). This is an open access article distributed under the terms of the Creative Commons Attribution License (CC BY 4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are cited.

Download Citation: BibTeX | http://dx.doi.org/10.6180/jase.202612_35.067

Download PDF

Micro-expression recognition is challenging because facial changes are brief and weak, training sets are small and imbalanced, and optical flow is vulnerable to alignment and non-expression motion. This paper proposes MSDF-Net, a subject-independent macro-guided static-dynamic recognition system. An apex-ADM branch captures appearance and local onset-apex variation, while a shared optical-flow encoder represents micro- and macro-expression motion. Unlike macro-guided methods that directly concatenate or transfer macro features, MSDF-Net converts a motion-matched macro feature into adaptive channel weights and applies them residually to the target micro-expression representation, exploiting strong macro motion while limiting contamination caused by intensity mismatch. Static-guided cross-modal attention then integrates recalibrated motion with static cues, and a similarity-weighted uncertainty-aware objective reduces the influence of imperfect macro-micro correspondence. Under LOSO evaluation, MSDF-Net obtains UF1 scores of 0.9962 and 0.9591 on the three- and five-class CASME II tasks, 0.9545 and 0.8612 on the corresponding SAMM tasks, and 0.8990 on SMIC-HS. Controlled comparisons and ablations support the contributions of complementary representation, macro-guided recalibration, and cross-modal fusion. On ECMSE, the model achieves 0.8471 Accuracy and 0.8435 UF1, indicating preliminary cross-scenario applicability. The results demonstrate that selective macro guidance and complementary static-dynamic modeling improve subject-independent micro-expression recognition.

Keywords: Micro-expression recognition; Static-dynamic fusion; Macro-guided recalibration; Channel recalibration; Cross-modal attention

  1. [1] W.-J. Yan, Q. Wu, J. Liang, Y.-H. Chen, and X. Fu, (2013) “How Fast Are the Leaked Facial Expressions: The Duration of Micro-Expressions” Journal of Nonverbal Behavior 37: 217-230. DOI: https://doi.org/10.1007/s10919-013-0159-8.
  2. [2] X.-B. Shen, Q. Wu, and X.-L. Fu, (2012) “Effects of the Duration of Expressions on the Recognition of Microexpressions” Journal of Zhejiang University Science B 13: 221-230. DOI: https://doi.org/10.1631/jzus.B1100063.
  3. [3] X. Ben, Y. Ren, J. Zhang, S.-J. Wang, K. Kpalma, W. Meng, and Y.-J. Liu, (2022) “Video-Based Facial Micro-Expression Analysis: A Survey of Datasets, Features and Algorithms” IEEE Transactions on Pattern Analysis and Machine Intelligence 44(9): 5826-5846. DOI: https://doi.org/10.1109/TPAMI.2021.3067464.
  4. [4] G. Zhao and M. Pietikainen, (2007) “Dynamic Texture Recognition Using Local Binary Patterns with an Application to Facial Expressions” IEEE Transactions on Pattern Analysis and Machine Intelligence 29(6): 915-928. DOI: https://doi.org/10.1109/TPAMI.2007.1110.
  5. [5] Y. Fang, J. Luo, and C. Lou. “Fusion of Multi-Directional Rotation Invariant Uniform LBP Features for Face Recognition”. In: Proceedings of the 2009 Third International Symposium on Intelligent Information Technology Application. IEEE, 2009, 332-335. DOI: https://doi.org/10.1109/IITA.2009.86.
  6. [6] P. Kumar, S. L. Happy, and A. Routray. “A Real-Time Robust Facial Expression Recognition System Using HOG Features”. In: Proceedings of the 2016 International Conference on Computing, Analytics and Security Trends. IEEE, 2016, 289-293. DOI: https://doi.org/10.1109/CAST.2016.7914977.
  7. [7] A. C. Le Ngo, J. See, and R. C.-W. Phan, (2016) “Sparsity in Dynamics of Spontaneous Subtle Emotions: Analysis and Application” IEEE Transactions on Affective Computing 8(3): 396-411. DOI: https://doi.org/10.1109/TAFFC.2016.2578318.
  8. [8] Y. Li, X. Huang, and G. Zhao, (2021) “Joint Local and Global Information Learning with Single Apex Frame Detection for Micro-Expression Recognition” IEEE Transactions on Image Processing 30: 249-263. DOI: https://doi.org/10.1109/TIP.2020.3035042.
  9. [9] S.-T. Liong, Y. S. Gan, J. See, H.-Q. Khor, and Y.-C. Huang. “Shallow Triple Stream Three-Dimensional CNN for Micro-Expression Recognition”. In: Proceedings of the 2019 14th IEEE International Conference on Automatic Face and Gesture Recognition. IEEE, 2019, 1-5. DOI: https://doi.org/10.1109/FG.2019.8756567.
  10. [10] H.-Q. Khor, J. See, S.-T. Liong, R. C.-W. Phan, and W. Lin. “Dual-Stream Shallow Networks for Facial Micro-Expression Recognition”. In: Proceedings of the 2019 IEEE International Conference on Image Processing. IEEE, 2019, 36-40. DOI: https://doi.org/10.1109/ICIP.2019.8803059.
  11. [11] L. Lei, T. Chen, S. Li, and J. Li. “Micro-Expression Recognition Based on Facial Graph Representation Learning and Facial Action Unit Fusion”. In: Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. IEEE, 2021, 1571-1580. DOI: https://doi.org/10.1109/CVPRW53098.2021.00173.
  12. [12] H. Pan, L. Xie, and Z. Wang, (2023) “C3DBed: Facial Micro-Expression Recognition with Three-Dimensional Convolutional Neural Network Embedding in Transformer Model” Engineering Applications of Artificial Intelligence 123: 106258. DOI: https://doi.org/10.1016/j.engappai.2023.106258.
  13. [13] J. Li, S.-J. Wang, Y. Wang, H. Zhou, and X. Fu, (2025) “Parallel Spatiotemporal Network to Recognize Micro-Expression” Neurocomputing 636: 129891. DOI: https://doi.org/10.1016/j.neucom.2025.129891.
  14. [14] B. Zhang, X. Wang, C. Wang, and G. He. “Dynamic Stereotype Theory Induced Micro-Expression Recognition with Oriented Deformation”. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. IEEE, 2025, 10701-10711. DOI: https://doi.org/10.1109/CVPR52734.2025.01000.
  15. [15] Z. Shao, Y. Cheng, F. Li, Y. Zhou, X. Lu, Y. Xie, and L. Ma, (2025) “MOL: Joint Estimation of Micro-Expression, Optical Flow, and Landmark via Transformer-Graph-Style Convolution” IEEE Transactions on Pattern Analysis and Machine Intelligence 47(10): 8756-8768. DOI: https://doi.org/10.1109/TPAMI.2025.3581162.
  16. [16] J. He, Y. Xiao, H. Zhang, J. Cai, L. Cai, and R. Liu, (2025) “Micro_NesT: Multi-Scale Attention Enhanced Micro-Expression Recognition Framework” Expert Systems with Applications 290: 128372. DOI: https://doi.org/10.1016/j.eswa.2025.128372.
  17. [17] B. Allaert, I. M. Bilasco, and C. Djeraba, (2022) “Micro and Macro Facial Expression Recognition Using Advanced Local Motion Patterns” IEEE Transactions on Affective Computing 13(1): 147-158. DOI: https://doi.org/10.1109/TAFFC.2019.2909983.
  18. [18] C. Gan, J. Xiao, Q. Zhu, and Y. Zhu, (2026) “Macro-Expression-Guided Micro-Expression Recognition: A Motion Similarity Perspective” Pattern Recognition 171: 112237. DOI: https://doi.org/10.1016/j.patcog.2025.112237.
  19. [19] C. Zach, T. Pock, and H. Bischof. “A Duality Based Approach for Realtime TV-L1 Optical Flow”. In: Proceedings of the 29th DAGM Symposium on Pattern Recognition. Springer, 2007, 214-223. DOI: https://doi.org/10.1007/978-3-540-74936-3_22.
  20. [20] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, and I. Polosukhin. “Attention Is All You Need”. In: Advances in Neural Information Processing Systems. 30. 2017. DOI: https://doi.org/10.48550/arXiv.1706.03762.
  21. [21] T.-Y. Lin, P. Goyal, R. Girshick, K. He, and P. Dollar. “Focal Loss for Dense Object Detection”. In: Proceedings of the IEEE International Conference on Computer Vision. 2017, 2980-2988. DOI: https://doi.org/10.1109/ICCV.2017.324.
  22. [22] W.-J. Yan, S.-J. Wang, Y.-J. Liu, Q. Wu, and X. Fu. “CASME Database: A Dataset of Spontaneous Micro-Expressions Collected from Neutralized Faces”. In: Proceedings of the 2013 10th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition. IEEE, 2013, 1-7. DOI: https://doi.org/10.1109/FG.2013.6553799.
  23. [23] W.-J. Yan, X. Li, S.-J. Wang, G. Zhao, Y.-J. Liu, Y.-H. Chen, and X. Fu, (2014) “CASME II: An Improved Spontaneous Micro-Expression Database and the Baseline Evaluation” PLOS ONE 9(1): e86041. DOI: https://doi.org/10.1371/journal.pone.0086041.
  24. [24] A. K. Davison, C. Lansley, N. Costen, K. Tan, and M. H. Yap, (2018) “SAMM: A Spontaneous Micro-Facial Movement Dataset” IEEE Transactions on Affective Computing 9(1): 116-129. DOI: https://doi.org/10.1109/TAFFC.2016.2573832.
  25. [25] X. Li, T. Pfister, X. Huang, G. Zhao, and M. Pietikainen. “A Spontaneous Micro-Expression Database: Inducement, Collection and Baseline”. In: Proceedings of the 2013 10th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition. IEEE, 2013, 1-6. DOI: https://doi.org/10.1109/FG.2013.6553717.
  26. [26] Y. Lv, M. Ning, F. Zhou, P. Lv, P. Zhang, and J. Wang, (2024) “Real-Time Microexpression Recognition in Educational Scenarios Using a Dual-Branch Continuous Attention Network” The Journal of Supercomputing 80(19): 27123-27146. DOI: https://doi.org/10.1007/s11227-024-06455-5.
  27. [27] P. Lucey, J. F. Cohn, T. Kanade, J. Saragih, Z. Ambadar, and I. Matthews. “The Extended Cohn-Kanade Dataset (CK+): A Complete Dataset for Action Unit and Emotion-Specified Expression”. In: Proceedings of the 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops. IEEE, 2010, 94-101. DOI: https://doi.org/10.1109/CVPRW.2010.5543262.
  28. [28] H. Hao, S. Wang, H. Ben, Y. Hao, Y. Wang, and W. Wang, (2024) “Hierarchical Space-Time Attention for Micro-Expression Recognition” arXiv preprint arXiv:2405.03202: DOI: https://doi.org/10.48550/arXiv.2405.03202.
  29. [29] R. Li, L. Wang, T. Yang, L. Xu, B. Ma, Y. Li, and H. Wei, (2024) “Micro-Expression Recognition by Motion Feature Extraction Based on Pre-Training” arXiv preprint arXiv:2407.07345: DOI: https://doi.org/10.48550/arXiv.2407.07345.
  30. [30] L. Wu, T. Jiang, W. Duan, Y. Fang, and J. Keung, (2025) “FaceSleuth: Learning-Driven Single-Orientation Attention Verifies Vertical Dominance in Micro-Expression Recognition” arXiv preprint arXiv:2506.02695: DOI: https://doi.org/10.48550/arXiv.2506.02695.
  31. [31] F. Liu, B. Nan, X. Qian, and X. Fu, (2026) “Temporal-Spatial Cross-Fusion for Dynamic Micro Expression Recognition” Pattern Recognition 179: 113715. DOI: https://doi.org/10.1016/j.patcog.2026.113715.
  32. [32] S. Zhao, H. Tao, Y. Zhang, T. Xu, K. Zhang, Z. Hao, and E. Chen, (2021) “A Two-Stage 3D CNN Based Learning Method for Spontaneous Micro-Expression Recognition” Neurocomputing 448: 276-289. DOI: https://doi.org/10.1016/j.neucom.2021.03.058.
  33. [33] A. J. R. Kumar and B. Bhanu. “Micro-Expression Classification Based on Landmark Relations With Graph Attention Convolutional Network”. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. IEEE, 2021, 1511-1520. DOI: https://doi.org/10.1109/CVPRW53098.2021.00167.