Portuguese Framework Semantic Role Labeling Based on Multiple Attention Mechanisms and Bi-LSTM

Wenting  Fan

doi:10.6180/jase.202501_28(1).0014

Portuguese Framework Semantic Role Labeling Based on Multiple Attention Mechanisms and Bi-LSTM

Research Categories

Wenting FanThis email address is being protected from spambots. You need JavaScript enabled to view it.

School of European Language and Culture Studies, Dalian University of Foreign Languages Dalian 116044, Liaoning, China

Received: November 27, 2023
Accepted: December 25, 2023
Publication Date: March 27, 2024

Copyright The Author(s). This is an open access article distributed under the terms of the Creative Commons Attribution License (CC BY 4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are cited.

Download Citation: ||https://doi.org/10.6180/jase.202501_28(1).0014

The traditional semantic role labeling methods mostly use the combination of machine learning and feature engineering. In this kind of method, it usually relies on the artificially extracted features, and will bring problems such as complex model and sparse features. Semantic role labeling has strong dependence on syntax. The traditional CNN model is limited by the convolution kernel receptive field and can not get the global information well. Therefore, in this paper we propose a Portuguese framework semantic role labeling based on multiple attention mechanisms and Bi-LSTM. The multiple attention mechanism is introduced to capture the syntactic information of each word in the sentence. The bidirectional LSTM layer and CRF layer are added to construct a sequence labeling model, which considers context information, lexical information and frame type information at the same time. The experimental results show that the F1 value (exceeding 83%) of the model is improved on the Portuguese data set, which proves that the inclusion of the multiple attention mechanisms can improve the performance of the framework semantic role labeling model.

Keywords: Semantic role labeling; CNN model; Multiple attention mechanisms; Bi-LSTM; CRF layer

[1] J. I. Butenko, M. Galetka, and E. Sineva, (2022) “Developing a Semantic Role Labeling System in Scientific and Technical Texts on Aviation and Astronautics" Automatic Documentation and Mathematical Linguistics 56(5): 251–260. DOI: 10.3103/S0005105522050077.
[2] E. Isaeva, O. Manzhula, and O. Baiburova. “Semantic Framing for Specialized Knowledge Modelling”. In: Specialized Knowledge Mediation: Ontological & Metaphorical Modelling. Springer, 2022, 109–123. DOI: 10.1007/978-3-030-95104-7_5.
[3] C. Campagnano, S. Conia, and R. Navigli. “SRL4E– Semantic Role Labeling for Emotions: A unified evaluation framework”. In: Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2022, 4586–4601. DOI: 10.18653/v1/2022.acl-long.314.
[4] K. D. Varathan, A. Giachanou, and F. Crestani, (2017) “Comparative opinion mining: a review" Journal of the Association for Information Science and Technology 68(4): 811–829. DOI: 10.1002/asi.23716.
[5] T. T. Torrent, E. E. d. S. Matos, F. Belcavello, M. Viridiano, M. A. Gamonal, A. D. d. Costa, and M. C. Marim, (2022) “Representing context in framenet: A multidimensional, multimodal approach" Frontiers in Psychology 13: 838441. DOI: 10.3389/fpsyg.2022.838441.
[6] Z. Li, H. Zhao, J. Zhou, K. Parnow, and S. He, (2022) “Dependency and span, cross-style semantic role labeling on PropBank and NomBank" ACM Transactions on Asian and Low-Resource Language Information Processing 21(6): 1–16. DOI: 10.1145/3526214.
[7] S. W. Brown, J. Bonn, G. Kazeminejad, A. Zaenen, J. Pustejovsky, and M. Palmer, (2022) “Semantic representations for nlp using verbnet and the generative lexicon" Frontiers in artificial intelligence 5: 821697. DOI: 10.3389/frai.2022.821697.
[8] M. Palmer, D. Gildea, and P. Kingsbury, (2005) “The proposition bank: An annotated corpus of semantic roles" Computational linguistics 31(1): 71–106. DOI: 10.1162/0891201053630264.
[9] P. Thompson, S. A. Iqbal, J. McNaught, and S. Ananiadou, (2009) “Construction of an annotated corpus to support biomedical information extraction" BMC bioinformatics 10: 1–19. DOI: 10.1186/1471-2105-10-349.
[10] J. Liu, C. Liang, and J. Xu, (2022) “Document-level event argument extraction with self-augmentation and a cross-domain joint training mechanism" KnowledgeBased Systems 257: 109904. DOI: 10.1016/j.knosys.2022.109904.
[11] Z. Liu, D. Jiang, C. Zhang, H. Zhao, Q. Zhao, and B. Zhang, (2021) “A novel fireworks algorithm for the protein-ligand docking on the autodock" Mobile Networks and Applications 26: 657–668. DOI: 10.1007/s11036-019-01412-6.
[12] Y. Zhao, H. Li, and S. Yin, (2022) “A Multichannel Character Relationship Classification Model Based on Attention Mechanism" Int. J. Math. Sci. Comput.(IJMSC) 8: 28–36. DOI: 10.5815/ijmsc.2022.01.03.
[13] K. Munir, H. Zhao, and Z. Li, (2021) “Adaptive convolution for semantic role labeling" IEEE/ACM Transactions on Audio, Speech, and Language Processing 29: 782–791. DOI: 10.1109/TASLP.2020.3048665.
[14] H. Fei, M. Zhang, B. Li, and D. Ji. “End-to-end semantic role labeling with neural transition-based model”. In: Proceedings of the AAAI conference on artificial intelligence. 35. 14. 2021, 12803–12811. DOI: 10.1609/aaai.v35i14.17515.
[15] Z. Liu, C. Zhang, Q. Zhao, B. Zhang, and W. Sun. “Comparative study of evolutionary algorithms for protein-ligand docking problem on the AutoDock”. In: Simulation Tools and Techniques: 11th International Conference, SIMUtools 2019, Chengdu, China, July 8– 10, 2019, Proceedings 11. Springer. 2019, 598–607. DOI: 10.1007/978-3-030-32216-8_58.
[16] X. Meng, X. Wang, S. Yin, and H. Li, (2023) “Few-shot image classification algorithm based on attention mechanism and weight fusion" Journal of Engineering and Applied Science 70(1): 1–14. DOI: 10.1186/s44147-023-00186-9.
[17] J. Zhang, Q. He, and Y. Zhang, (2021) “Syntax grounded graph convolutional network for joint entity and event extraction" Neurocomputing 422: 118–128. DOI: 10.1016/j.neucom.2020.09.044.
[18] C. Li, G. Zhan, and Z. Li. “News text classification based on improved Bi-LSTM-CNN”. In: 2018 9th International Conference on Information Technology in Medicine and Education (ITME). IEEE. 2018, 890–893. DOI: 10.1109/ITME.2018.00199.
[19] N. Xue and M. Palmer. “Calibrating features for semantic role labeling.” In: EMNLP. 2004, 88–94.
[20] M. J. Al-Muhammed and D. W. Lonsdale, (2022) “Ontology-aware dynamically adaptable free-form natural language agent interface for querying databases" Knowledge-Based Systems 239: 108012. DOI: 10.1016/j.knosys.2021.108012.
[21] D. N. Ribeiro. “Reasoning and Structured Explanations in Natural Language via Analogical and Neural Learning". (phdthesis). Northwestern University, 2023.
[22] S. Sun, C. Luo, and J. Chen, (2017) “A review of natural language processing techniques for opinion mining systems" Information fusion 36: 10–25. DOI: 10.1016/j.inffus.2016.10.004.
[23] W. Li, D. Cheng, L. He, Y. Wang, and X. Jin, (2019) “Joint event extraction based on hierarchical event schemas from FrameNet" IEEE Access 7: 25001–25015. DOI: 10.1109/ACCESS.2019.2900124.
[24] L. Yuan, (2023) “Semantic Role Labeling Based on Valence Structure and Deep Neural Network" IETE Journal of Research: 1–9. DOI: 10.1080/03772063.2023.2220683.
[25] M. Liu, Y. Zhang, J. Xu, and Y. Chen, (2021) “Deep bi-directional interaction network for sentence matching" Applied Intelligence 51: 4305–4329.
[26] C. Zhang, J. James, and Y. Liu, (2019) “Spatial-temporal graph attention networks: A deep learning approach for traffic forecasting" IEEE Access 7: 166246–166256. DOI: 10.1109/ACCESS.2019.2953888.
[27] G. K. W. Huang and J. C. Lee. “Hyperpartisan news and articles detection using Bert and Elmo”. In: 2019 International Conference on Computer and Drone Applications (IConDA). IEEE. 2019, 29–32. DOI: 10.1109/ IConDA47345.2019.9034917.
[28] A. Yousif, Z. Niu, J. Chambua, and Z. Y. Khan, (2019) “Multi-task learning model based on recurrent convolutional neural networks for citation sentiment and purpose classification" Neurocomputing 335: 195–205. DOI: 10.1016/j.neucom.2019.01.021.
[29] Z. Wang, Y. Zhang, T. Lv, and L. Luo, (2022) “GTAINet: Graph neural network-based two-stage anomaly identification for locking wire point clouds using hierarchical attentive edge convolution" International Journal of Applied Earth Observation and Geoinformation 115: 103106. DOI: 10.1016/j.jag.2022.103106.
[30] Z. Li, F. Tang, M. Zhao, and Y. Zhu, (2022) “Emocaps: Emotion capsule based model for conversational emotion recognition" arXiv preprint arXiv:2203.13504: DOI: 10.48550/arXiv.2203.13504.
[31] Z. Yang, K. Xie, T. Li, M. Yang, W. Zhao, and Y. Ye. “Research on Color Constancy Algorithm of Weighted Fully Convolution Neural Network based on Multi-Path Feature Fusion”. In: Proceedings of the 2021 4th International Conference on Image and Graphics Processing. 2021, 143–149. DOI: 10.1145/3447587.3447608.
[32] D. Chen, N. Schneider, D. Das, and N. A. Smith. “Semafor: Frame argument resolution with log-linear models”. In: Proceedings of the 5th international workshop on semantic evaluation. 2010, 264–267.
[33] D. Das and N. A. Smith. “Semi-supervised framesemantic parsing for unknown predicates”. In: Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies. 2011, 1435–1444.
[34] E. R. Bartusiak and E. J. Delp. “Transformer-Based Speech Synthesizer Attribution in an Open Set Scenario”. In: 2022 21st IEEE International Conference on Machine Learning and Applications (ICMLA). IEEE. 2022, 329–336. DOI: 10.1109/ICMLA55696.2022.00054.
[35] F. Wan, Y. Yang, D. Zhu, H. Yu, A. Zhu, G. Che, and N. Ma, (2022) “Semantic role labeling integrated with multilevel linguistic cues and Bi-LSTM-CRF" Mathematical Problems in Engineering 2022: 1–8. DOI: 10.1155/2022/6300530.
[36] L. Xu, X. Pang, J. Wu, M. Cai, and J. Peng, (2023) “Learn from structural scope: Improving aspect-level sentiment analysis with hybrid graph convolutional networks" Neurocomputing 518: 373–383. DOI: 10.1016/j.neucom.2022.10.071.
[37] S. Yin, (2023) “Object Detection Based on Deep Learning: A Brief Review" IJLAI Transactions on Science and Engineering 1(02): 1–6.