A Quadrotor Trajectory Tracking Control Method Based on Deep Reinforcement Learning

Authors

Guohua Wu, School of Automation Central South University, Changsha 410083, China
Jiaheng Zeng, School of Traffic and Transportation Engineering, Central South University, Changsha 410083, China
Dezhi Wang, School of Meteorology and Oceanography, National University of Defense Technology, Changsha 410015, China
Long Zheng, Military Vocational Education Technology Service Center, National University of Defense Technology , Changsha 410015, China
Wei Zou, School of Computer Science and Engineering, Central South University, Changsha 410083, China

Abstract

Abstract: Traditional quadrotor controllers, constrained by fixed model equation structures, encounter challenges in addressing control errors stemming from variations in parameters and environmental disturbances. This paper proposes a deep reinforcement learning solution for the quadrotor trajectory following control problem. We present the PPO-SAG algorithm incorporated into the PPO framework, utilizing adaptive mechanisms and PID expert knowledge to enhance training convergence and stability. Target functions incorporating distance constraint penalties and entropy policies are designed in alignment with the characteristics of the given problem. We also devise innovative disturbance-adaptive structures and trajectory feature selection mechanisms to augment control error information and extract crucial elements from future trajectories, thereby enhancing convergence. Experiments on single and mixed trajectories indicate that the PPO-SAG algorithm achieves superior performance in both convergence and stability. Verification experiments confirm positive effects of proposed improvements. The trajectory tracking control problem of quadrotors based on deep reinforcement learning under unknown disturbances studied in this paper provides a solution for designing more robust and efficient quadrotor controllers.

Recommended Citation

Wu, Guohua; Zeng, Jiaheng; Wang, Dezhi; Zheng, Long; and Zou, Wei (2025) "A Quadrotor Trajectory Tracking Control Method Based on Deep Reinforcement Learning," Journal of System Simulation: Vol. 37: Iss. 5, Article 6.
DOI: 10.16182/j.issn1004731x.joss.24-0025
Available at: https://dc-china-simulation.researchcommons.org/journal/vol37/iss5/6

First Page

1169

Last Page

1187

CLC

TP273

Recommended Citation

Wu Guohua, Zeng Jiaheng, Wang Dezhi, et al. A Quadrotor Trajectory Tracking Control Method Based on Deep Reinforcement Learning[J]. Journal of System Simulation, 2025, 37(5): 1169-1187.

Corresponding Author

Wang Dezhi

DOI

10.16182/j.issn1004731x.joss.24-0025