Journal of System Simulation
Abstract
Abstract: In autonomous driving, the efficiency and accuracy of object detection are significant. Object detection based on Transformer structure has gradually become the mainstream method, eliminating the complex anchor generation and non-maximum suppression (NMS). It has problems of high computing cost and slow convergence. An object detection model of the based lightweight pooling transformer (LPT) is designed, which contains a pooling backbone network and dual pooling attention mechanism. A general knowledge distillation method is intended for the DETR (detection transformer) model, which transfers prediction results, query vector, and features extracted by the teacher as knowledge to the LPT model to improve its accuracy. To verify the application potential of the distilled LPT model in autonomous driving, extensive experiments are conducted on the MS COCO 2017 dataset. The results show that the method has great efficiency and accuracy, and is competitive with some advanced techniques.
Recommended Citation
Wang, Gaihua; Li, Kehong; Long, Qian; Yao, Jingxuan; Zhu, Bolun; Zhou, Zhengshu; and Pan, Xuran
(2024)
"Object Detection of Lightweight Transformer Based on Knowledge Distillation,"
Journal of System Simulation: Vol. 36:
Iss.
11, Article 2.
DOI: 10.16182/j.issn1004731x.joss.24-0754
Available at:
https://dc-china-simulation.researchcommons.org/journal/vol36/iss11/2
First Page
2517
Last Page
2527
CLC
TP391.9
Recommended Citation
Wang Gaihua, Li Kehong, Long Qian, et al. Object Detection of Lightweight Transformer Based on Knowledge Distillation[J]. Journal of System Simulation, 2024, 36(11): 2517-2527.
DOI
10.16182/j.issn1004731x.joss.24-0754
Included in
Artificial Intelligence and Robotics Commons, Computer Engineering Commons, Numerical Analysis and Scientific Computing Commons, Operations Research, Systems Engineering and Industrial Engineering Commons, Systems Science Commons