Journal of Applied Science and Engineering

 Copyright The Author(s). This is an open access article distributed under the terms of the Creative Commons Attribution License (CC BY 4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are cited.

Download Citation:  BibTeX | http://dx.doi.org/10.6180/jase.202609_32.067  

Download PDF

Music information retrieval (MIR) has witnessed remarkable advancements with the proliferation of deep learning technologies, but existing approaches often treat core tasks such as music structure analysis (MSA), musicsegmentation (MS), andmusicstyletransfer (MST) as isolated objectives. This isolation leads to redundant feature extraction, limited cross-task knowledge sharing, and suboptimal performance in real-world audio processing scenarios, where multiple MIR tasks are often required simultaneously. To address these limitations, this paper proposes a novel end-to-end multi-task AI framework that unifies MSA, MS, and MST into a single cohesive architecture, leveraging inter-task correlations to enhance the performance of each individual task. The framework comprises three core components: a shared feature encoder (SFE) based on a multi-scale Transformer and convolutional neural network (CNN) hybrid structure, which efficiently extracts hierarchical audio features; task-specific decoders tailored to the unique characteristics of MSA, MS, and MST; and a cross-task knowledge distillation (CTKD) module that facilitates mutual knowledge transfer between tasks while mitigating negative transfer. For MSA, we design a structure-aware attention mechanism to capture long-range temporal dependencies and hierarchical musical structures (e.g., intro, verse, chorus). For MS, a boundary-refinement decoder with dynamic thresholding is proposed to achieve precise segment localization. For MST, a style disentanglement module based on time-varying inversion and diffusion model principles is integrated to separate content and style features, enabling high-fidelity style transfer without altering the core musical content. Extensive experiments are conducted on four benchmark datasets (SALAMI, RWC-Pop, McGill Billboard, and MAESTRO) across multiple evaluation metrics, including F1-score for segmentation, structural consistency score (SCS) for MSA, and Fréchet Audio Distance (FAD) for MST. Experimental results demonstrate that the proposed framework outperforms state-of-the-art single-task and multi-task baselines by significant margins: 5.2% higher F1-score for MS, 8.7% higher SCS for MSA, and 12.3% lower FAD for MST on average. Ablation studies validate the effectiveness of each component, confirming that cross-task knowledge sharing
and feature reuse substantially improve model generalization. The proposed framework provides a unified solution for multi-task music audio processing, with potential applications in music production, intelligent music recommendation, and digital music restoration. The source code and experimental data are publicly available to facilitate further research in the field.

Keywords: Multi-task learning; Music structure analysis; Music style transfer; Audio signal processing; Deep learning; Cross-task knowledge distillation