Journal of Applied Science and Engineering

Published by Tamkang University Press

ESCI jase impact factor scopus logo open access rate of Scopus journal

IA-Transformer: Prediction and Classification of β-Lactamase Proteins Using Transformer Model with Integrated Attention Mechanism

 2026

 Volume 32

Yuankun Du1, Fengping Liu2, and Yi Hou1

1College of Big Data and Artificial Intelligence, Zhengzhou University of Science and Technology, Zhengzhou, Henan 450064, China

2School of Information Engineering, Zhengzhou University of Science and Technology, Zhengzhou, Henan 450064, China

Received:  September 29, 2025
Accepted: July 4, 2026
Publication Date: April 25, 2026

上傳圖片

ROC curves of the IA-Transformer model and comparison models for β-lactamase prediction 

 Copyright The Author(s). This is an open access article distributed under the terms of the Creative Commons Attribution License (CC BY 4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are cited.

Download Citation:  BibTeX | http://dx.doi.org/10.6180/jase.202609_32.010  

Download PDF

β-Lactamase proteins are the primary mediators of bacterial resistance to β-Lactam antibiotics, posing a severe threat to global public health. Accurate prediction and classification of β-Lactamase proteins are crucial for the development of novel antibiotics and the formulation of clinical treatment strategies. Traditional machine learning methods for β-Lactamase analysis often rely on manual feature engineering, which fails to fully capture the complex sequence patterns and contextual information of proteins. To address this limitation, this study proposes a Transformer model integrated with a multi-head attention mechanism (IA-Transformer) for the prediction and classification of β-Lactamase proteins. The IA-Transformer model innovatively integrates three attention modules: sequence-wise self-attention, residue-wise attention, and channel-wise attention. The sequence-wise self-attention captures long-range dependencies between amino acid residues in the protein sequence; the residue-wise attention emphasizes key functional residues related to β-Lactam hydrolysis; and the channel-wise attention optimizes the feature representation of different sequence motifs. Experimental results show that the IA-Transformer model achieves an accuracy of 98.2%, a sensitivity of 97.8%, a specificity of 98.5%, and an F1-score of 98.0% in β-Lactamase prediction, outperforming traditional methods such as SVM, Random Forest, and single-attention Transformer by 3.5%−7.2%.

Keywords: β-Lactamase; Transformer Model; Integrated Attention Mechanism; Protein Prediction; Protein Classification; Antibiotic Resistance

  1. [1] P. Agarwal, R. P. Kumar, L.-M. Oleksiuk, V. Crall, A. A. Petrov, E. K. McCreary, J. Holder-Murray, Y.-F. Chang, N. Agarwal, D. K. Hamilton, et al., (2025) “Non-β-lactam antibiotic use, β-lactam allergy, and surgical site infections” JAMA surgery 160(11): 1260–1267. DOI: 10.1001/jamasurg.2025.3789.
  2. [2] B. Bedenić, M. Pospišil, M. Nađ, and D. Bandić Pavlović, (2025) “Evolution of β-Lactam antibiotic resistance in proteus species: from Extended-Spectrum and Plasmid-Mediated AmpC β-Lactamases to carbapenemases” Microorganisms 13(3): 508. DOI: 10.3390/microorganisms13030508.
  3. [3] V. T. Nguyen, B. T. Birhanu, V. Miguel-Ruano, C. Kim, M. Batuecas, J. Yang, A. M. El-Araby, E. Jimenez-Faraco, V. A. Schroeder, A. Alba, et al., (2025) “Restoring susceptibility to β-lactam antibiotics in methicillin-resistant Staphylococcus aureus” Nature chemical biology 21(4): 482–489. DOI: 10.1038/s41589-024-01688-0.
  4. [4] J. Pacyńska and P. Niedzielski, (2025) “Scoping Review of Extraction Methods for Detecting β-Lactam Antibiotics in Food Products of Animal Origin” Molecules 30(9): 1937. DOI: 10.3390/molecules30091937.
  5. [5] W.-Y. Fan, X. Zhang, D.-H. Xie, K. M. Y. Leung, and G.-P. Sheng, (2025) “Cerium-based nanohydrolase for fast catalytic hydrolysis of β-lactam antibiotics in wastewater effluents” Journal of Hazardous Materials 484: 136800. DOI: 10.1016/j.jhazmat.2024.136800.
  6. [6] N. Abdelmalek, S. W. Yousief, M. S. Bojer, M. S. A. Alobaidallah, J. E. Olsen, and B. Paglietti, (2025) “The secondary resistome of methicillin-resistant Staphylococcus aureus to β-lactam antibiotics” Antibiotics 14(2): 112. DOI: 10.3390/antibiotics14020112.
  7. [7] Y. Cao, Y. Yang, W. Zhao, H. Liu, X. Zhang, H. Chen, M. Sui, and P. Ma, (2025) “SERS based determination of ceftriaxone, ampicillin, and vancomycin in serum using WS2/Au@ Ag nanocomposites and a 2D-CNN regression model” Spectrochimica Acta Part A: Molecular and Biomolecular Spectroscopy 333: 125850. DOI: 10.1016/j.saa.2025.125850.
  8. [8] T. Li. “Time-Series Batch Predictive Control Based on MIC-LSTM-ATT”. In: 2025 5th International Conference on Artificial Intelligence and Industrial Technology Applications (AIITA). IEEE. 2025, 1202–1208. DOI: 10.1109/AIITA65135.2025.11047860.
  9. [9] L. He, H. Li, R. Qi, Q. Zou, and Y. Wang, (2025) “MCT-ARG: Identification and classification of antibiotic resistance genes based on a multi-channel Transformer model” Science of the Total Environment 1006: 180848. DOI: 10.1016/j.scitotenv.2025.180848.
  10. [10] A. Zubair, M. Fazil, M. Jawad, and S. Wdidi, (2025) “The Role of Machine Learning in Addressing Antibiotic Resistance: A New Era in Infectious Disease Control” Microbiology Open 14(6): e70160. DOI: 10.1002/mbo3.70160.
  11. [11] A. E. Nolasco-Rojas, E. Cruz-Del-Agua, C. Cruz-Cruz, M. Á. Loyola-Cruz, B. A. Ayil-Gutiérrez, M. C. Tamayo-Ordóñez, Y. d. J. Tamayo-Ordóñez, A. Rojas-Bernabé, F. A. Tamayo-Ordóñez, E. M. Durán-Manuel, et al., (2025) “Microbiological risks to health associated with the release of antibiotic-resistant Bacteria and β-lactam antibiotics through hospital wastewater” Pathogens 14(5): 402. DOI: 10.3390/pathogens14050402.
  12. [12] M.-J. Yang, M.-J. Li, L.-D. Huang, X.-W. Zhang, Y.-Y. Huang, X.-Y. Gou, S.-N. Chen, J. Yan, P. Du, and A.-H. Sun, (2025) “Response regulator protein CiaR regulates the transcription of ccn-microRNAs and β-lactam antibiotic resistance conversion of Streptococcus pneumoniae” International Journal of Antimicrobial Agents 65(1): 107387. DOI: 10.1016/j.ijantimicag.2024.107387.
  13. [13] M. Labied, A. Belangour, and M. Banane, (2025) “P-GELU: A Novel Activation Function to Optimize Whisper for Darija Speech Translation” IEEE Access 13: 100198–100218. DOI: 10.1109/ACCESS.2025.3574398.
  14. [14] H. Perveen and J. Weeds, (2025) “Protein sequence classification using natural language processing techniques” Discover Artificial Intelligence 5(1): 66. DOI: 10.1007/s44163-025-00304-x.
  15. [15] B. Wang, R. Meng, Z. Li, M. Hu, X. Wang, Y. Zhao, Z. Chai, Y. Jin, J. Yue, W. Chen, et al., (2025) “Predicting antibiotic resistance genes and bacterial phenotypes based on protein language models” Frontiers in Microbiology 16: 1628952. DOI: 10.3389/fmicb.2025.1628952.
  16. [16] Y. Zhao, J. Zhang, Y. Gui, J. X. Huang, F. Xie, and H. Shen, (2025) “Probing the interaction mechanisms between three β-lactam antibiotics and penicillin-binding proteins of Escherichia coli by molecular dynamics simulations” Comparative Biochemistry and Physiology Part C: Toxicology & Pharmacology 287: 110057. DOI: 10.1016/j.cbpc.2024.110057.
  17. [17] H. S. Butman, M. A. Stefaniak, D. J. Walsh, V. S. Gondil, M. Young, A. H. Crow, A. M. Nemeth, R. J. Melander, P. M. Dunman, and C. Melander, (2025) “Phenyl urea based adjuvants for β-lactam antibiotics against methicillin resistant Staphylococcus aureus” Bioorganic & medicinal chemistry letters 121: 130164. DOI: 10.1016/j.bmcl.2025.130164.
  18. [18] A. Sharma, V. Diwakar, R. Kumar, and P. Garg, (2025) “Enzyme classification integrating LSTM and Prot-BERT sequence encoding” Applied Soft Computing: 113774. DOI: 10.1016/j.asoc.2025.113774.