Journal of Applied Science and Engineering

Published by Tamkang University Press

ESCI jase impact factor scopus logo open access rate of Scopus journal

Deep Reinforcement Learning-Enhanced Multi-Objective Cold-Chain Vehicle Routing with Dynamic Order Insertion and Carbon Emission Constraints

 2026

 Volume 33

Qinqin Zhang

School of Business Administration, Zhengzhou University of Science and Technology, Zhengzhou 450064 China

Received: April 13, 2026
Accepted: May 16, 2026
Publication Date: June 27, 2026

上傳圖片

Dynamic Order Insertion Test Results

 Copyright The Author(s). This is an open access article distributed under the terms of the Creative Commons Attribution License (CC BY 4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are cited.

Download Citation:  BibTeX | http://dx.doi.org/10.6180/jase.202610_33.015  

Download PDF

Cold-chain logistics is critical for preserving perishable goods but faces challenges of high operational costs, carbon emissions, and dynamic order uncertainties. This paper proposes a Deep Reinforcement Learning (DRL)-enhanced multi-objective optimization framework for cold-chain vehicle routing problem (CCVRP) with dynamic order insertion and strict carbon emissionconstraints. We first formula tea multi-objective mathematical
model that simultaneously minimizes total distribution cost (including transportation, refrigeration, and time window penaltycosts), carbonemissions, and maximizes customer service level. Ac onstrained Markov Decision Process (CMDP) is designed to model the dynamic routing decisions under real-time order insertion and carbon quota limitations. Then, an improved Proximal Policy Optimization with Carbon Constraint Penalty (PPO-CCP) algorithm is developed, which integrates a graph attention encoder for spatial-temporal state representation and a multi-objective reward mechanism with adaptive carbon emission penalty. Extensive experiments on Solomon benchmark and real-world cold-chain datasets demonstrate that our method outperforms traditional metaheuristics (e.g., NSGA-II, ALNS) and baseline DRL algorithms in solution quality, dynamic adaptability,
and emission reduction efficiency. Specifically, it achieves 12.3% cost reduction, 15.7% emission decrease, and 28.5% faster response to dynamic orders compared to the best-performing baseline. This research provides an intelligent and sustainable decision-support tool for low-carbon cold-chain logistics operations.

Keywords: Cold-Chain Vehicle Routing Problem; Deep Reinforcement Learning; Dynamic Order Insertion; Multi-Objective Optimization; Carbon Emission Constraints

  1. [1] S.Zhang,N.Chen,X.Song,andJ.Yang,(2019)“Optimizing decision-making of regional cold chain logistics system in view of low-carbon economy ” Transportation research part A: Policy and practice 130: 844–857. DOI: 10.1016/j.tra.2019.10.004.
  2. [2] Q. Chen, J. Qian, H. Yang, and W. Wu, (2022) “Sustainable food cold chain logistics: From microenvironmental monitoring to global impact” Comprehensive reviews in food science and food safety 21(5): 4189–4209. DOI: 10.1111/1541-4337.13014.
  3. [3] Y. Zhang, S. Zhang, Y. Zhang, and Y. Yin, (2024) “A study of TMA aircraft conflict-free routing and operation: With mixed integer linear programming, multi-agent path finding, and metaheuristic-based neighborhood search” IEEE Transactions on Intelligent Transportation Systems 25(10): 13976–13990. DOI: 10.1109/TITS.2024.3385859.
  4. [4] M. Mahdavi, K. Schmitt, M. Chamana, F. Jurado, S. Bayne, E. A. Marfo, and A. Awaafo, (2024) “A mixed-integer programming model for reconfiguration of active distribution systems considering voltage dependency and type of loads and renewable sources” IEEE Transactions on Industry Applications 60(4): 5291–5303. DOI: 10.1109/TIA.2024.3383805.
  5. [5] W. Li, B. Li, S. He, X. Mao, C. Qiu, Y. Qiu, and X. Tan, (2022) “A novel milling parameter optimization method based on improved deep reinforcement learning considering machining cost” Journal of Manufacturing Processes 84: 1362–1375. DOI: 10.1016/j.jmapro.2022.11.015.
  6. [6] L. Teng, Y. Qiao, M. Shafiq, G. Srivastava, A. R. Javed, T. R. Gadekallu, and S. Yin, (2023) “FLPK-BiSeNet: Federated learning based on priori knowledge and bilateral segmentation network for image edge extraction” IEEE Transactions on Network and Service Management 20(2): 1529–1542. DOI: 10.1109/TNSM.2023.3273991.
  7. [7] X. Li, (2024) “Multi-objective multi-compartment vehicle routing problem of fresh products with the promised latest delivery time” Annals of Operations Research: 1–33. DOI: 10.1007/s10479-024-06254-4.
  8. [8] G. Qin, F. Tao, L. Li, and Z. Chen, (2019) “Optimization of the simultaneous pickup and delivery vehicle routing problem based on carbon tax” Industrial management & data systems 119(9): 2055–2071. DOI: 10.1108/IMDS-02-2019-0102.
  9. [9] L. Yang, Y. Gao, Y. Sun, and J. Li, (2024) “Two-phase hybrid search algorithm for time-dependent cold chain logistics route considering carbon emission and traffic congestion” IEEE Access 12: 95128–95151. DOI: 10.1109/ACCESS.2024.3425409.
  10. [10] Y. Sun and D. Pan, (2025) “Cold chain logistics path optimization with adaptive speed and hybrid genetic algorithm solution” Mathematics 13(12): 1981. DOI: 10.3390/math13121981.
  11. [11] Z. Zhang, Z. Ding, H. Sang, and F. Xiao, (2026) “Dynamic Safety Gating for Large Action Models: Mitigating Jailbreak Exploits in Large Language Models” ACM Transactions on Multimedia Computing, Communications and Applications: DOI: 10.1145/3803021.
  12. [12] Y. Bai, H. Tian, X. Xue, T. Su, X. Jin, and J. Kong, (2026) “Echo state network and variational autoencoder integrated with online optimization for robust pose estimation in unmanned surface vehicles” Measurement: 121454. DOI: 10.1016/j.measurement.2026.121454.
  13. [13] W. P. Nanry and J. W. Barnes, (2000) “Solving the pickup and delivery problem with time windows using reactive tabu search” Transportation Research Part B: Methodological 34(2): 107–121. DOI: 10.1016/S0191-2615(99)00016-8.
  14. [14] M. Alzaqebah, S. Abdullah, and S. Jawarneh, (2016) “Modified artificial bee colony for the vehicle routing problems with time windows” SpringerPlus 5(1): 1298. DOI: 10.1186/s40064-016-2940-8.
  15. [15] X. Li, Z. Wang, S. Gao, R. Hu, Q. Zhu, and L. Wang, (2019) “An intelligent context-aware management framework for cold chain logistics distribution” IEEE Transactions on Intelligent Transportation Systems 20(12): 4553–4566. DOI: 10.1109/TITS.2018.2889069.
  16. [16] D. Kalyanmoy, (2002) “A fast and elitist multi-objective genetic algorithm: NSGA-II” IEEE Trans. on Evolutionary Computation 6(2): 182–197. DOI: 10.1109/4235.996017.
  17. [17] V. G. M. Santos and M. A. M. de Carvalho, (2018) “Adaptive large neighborhood search applied to the design of electronic circuits” Applied Soft Computing 73: 14–23. DOI: 10.1016/j.asoc.2018.08.017.
  18. [18] Q. Hu, X. Liu, and X. Jiang, (2026) “Multi-objective collaborative optimization for low-carbon cold-chain routing with dynamic demand” Mathematics 14(5): 753. DOI: 10.3390/math14050753.
  19. [19] X. Wang and S. Wang, (2026) “Traffic forecast and hybrid optimization-based vehicle route planning for cold chain logistics” Applied Sciences 16(5): 2479. DOI: 10.3390/app16052479.
  20. [20] N. Berrahmania, E. Hammou, and L. El Fadi, (2025) “Short review in green vehicle routing problem with pick up and delivery and time windows using metaheuristics” The Open Transportation Journal 19(1): DOI: 10.2174/0126671212375850250212065147