Gradient-based Deep Reinforcement Learning Interpretation Methods

Authors

Yuan Wang, School of Computer Science and Engineering, Nanjing University of Science and Technology, Nanjing 210094, China; Science and Technology on Information Systems Engineering Laboratory, Nanjing 210014, ChinaFollow
Lin Xu, Science and Technology on Information Systems Engineering Laboratory, Nanjing 210014, China
Xiaoze Gong, PLA 63850 Troops, Baicheng 137001, ChinaFollow
Yongliang Zhang, Command and Control Engineering College, Army Engineering University of PLA, Nanjing 210007, China
Yongli Wang, School of Computer Science and Engineering, Nanjing University of Science and Technology, Nanjing 210094, China

Abstract

Abstract: The learning process and working mechanism of deep reinforcement learning methods such as DQN are not transparent, and their decision basis and reliability cannot be perceived, which makes the decisions made by the model highly questionable and greatly limits the application scenarios of deep reinforcement learning. To explain the decision-making mechanism of intelligent agents, this paper proposes a gradient based saliency map generation algorithm SMGG. It uses the gradient information of feature maps generated by high-level convolutional layers to calculate the importance of different feature maps. With the known structure and internal parameters of the model, starting from the last layer of the model, the weight of different feature maps relative to the saliency map is generated by calculating the gradient of feature maps; it classifies the importance of features in both positive and negative directions, and uses weights with positive influence to weight the features captured in the feature map, forming a positive interpretation of the current decision; it uses weights that have a negative impact on other categories to weight the features captured in the feature map, forming a reverse interpretation of the current decision. The saliency map of the decision is generated by the two together, and the basis for the intelligent agent's decision-making behavior is obtained. The effectiveness of this method has been demonstrated through experiments.

Recommended Citation

Wang, Yuan; Xu, Lin; Gong, Xiaoze; Zhang, Yongliang; and Wang, Yongli (2024) "Gradient-based Deep Reinforcement Learning Interpretation Methods," Journal of System Simulation: Vol. 36: Iss. 5, Article 8.
DOI: 10.16182/j.issn1004731x.joss.22-1480
Available at: https://dc-china-simulation.researchcommons.org/journal/vol36/iss5/8

First Page

1130

Last Page

1140

CLC

TP391.9

Recommended Citation

Wang Yuan, Xu Lin, Gong Xiaoze, et al. Gradient-based Deep Reinforcement Learning Interpretation Methods[J]. Journal of System Simulation, 2024, 36(5): 1130-1140.

Corresponding Author

Gong Xiaoze

DOI

10.16182/j.issn1004731x.joss.22-1480