Robot Path Planning Based on Improved A-DDQN Algorithm

Authors

Peilong Ni, School of Mechanical and Electrical Engineering, Henan University of Science and Technology, Luoyang 471003, China
Pengjun Mao, School of Mechanical and Electrical Engineering, Henan University of Science and Technology, Luoyang 471003, China
Ning Wang, School of Mechanical and Electrical Engineering, Henan University of Science and Technology, Luoyang 471003, China
Mengjie Yang, School of Mechanical and Electrical Engineering, Henan University of Science and Technology, Luoyang 471003, China

Abstract

Abstract: An improved A-DDQN algorithm is proposed to address the challenges of reward sparsity and the inability to distinguish sample importance in traditional DQN algorithms during robot path planning. Building on the original DQN, an enhancement is made by incorporating the Double-DQN approach, which updates the predictive Q-value network based on actions selected by the Q network, rather than directly using the predicted Q-values for action selection, thereby mitigating overestimation issues. Secondly, the concept of artificial potential field (APF) is introduced to design specific rewards for each step of the robot's movement, guiding the robot and addressing the problem of sparse rewards. Lastly, the prioritized experience replay (PER) mechanism is integrated, adjusting the sampling probability of experiences through priority ranking to accelerate the learning process and enhance performance. Comparative analysis of path planning before and after the algorithm improvements in a two-dimensional grid map shows that in small-scale maps, the improved A-DDQN algorithm reduces path length, iteration times, and the number of turning points by 11.5%, 23.1%, and 61.5% respectively; in large-scale maps with sparse obstacles, these reductions are 19.4%, 50.0%, and 52.9% respectively; and in large-scale maps with dense obstacles, reductions are 29.7%, 48.1%, and 64.3%. These simulation results prove that the improved algorithm achieves faster convergence and superior path planning performance.

Recommended Citation

Ni, Peilong; Mao, Pengjun; Wang, Ning; and Yang, Mengjie (2025) "Robot Path Planning Based on Improved A-DDQN Algorithm," Journal of System Simulation: Vol. 37: Iss. 9, Article 19.
DOI: 10.16182/j.issn1004731x.joss.24-0369
Available at: https://dc-china-simulation.researchcommons.org/journal/vol37/iss9/19

First Page

2420

Last Page

2430

CLC

TP242

Recommended Citation

Ni Peilong, Mao Pengjun, Wang Ning, et al. Robot Path Planning Based on Improved A-DDQN Algorithm[J]. Journal of System Simulation, 2025, 37(9): 2420-2430.

Corresponding Author

Mao Pengjun

DOI

10.16182/j.issn1004731x.joss.24-0369