Virtual and Real Fusion Experiment Simulation Technology Based on Gesture Interaction

Authors

Xuyan Zou, 1. Guangdong University of Technology, Guangzhou 510006, China; ;3. Department of Artificial Intelligence and Big Data, Yibin university, Yibin 644007, China;
Hanwu He, 1. Guangdong University of Technology, Guangzhou 510006, China; ;2. Guangdong Polytechnic of Industry and Commerce, Guangzhou 510510, China; ;
Yueming Wu, 1. Guangdong University of Technology, Guangzhou 510006, China; ;
Jingwei Deng, 1. Guangdong University of Technology, Guangzhou 510006, China; ;

Abstract

Abstract: In experimental teaching, some traditional experiments cannot be carried out due to limitations of experimental environment, experimental equipment, and faculty. In order to solve the above problems, the method of constructing virtual and real fusion simulation experiment using depth cameras is explored.. The Aruco labeling algorithm is used to achieve one-to-one registration of the virtual scene and the real scene, and the experimental environment of virtual and real fusion is constructed combining the colour images and depth images collected by the depth camera in real time. In order to achieve the purpose of operating virtual equipment, a gesture interaction method is proposed based on threshold segmentation and imaging principles. Numerical experiments show that pose registration and gesture interaction methods basically meet the application requirements of virtual experiments, and provide a set of feasible technical solutions for the construction of virtual experiments

Recommended Citation

Zou, Xuyan; He, Hanwu; Wu, Yueming; and Deng, Jingwei (2021) "Virtual and Real Fusion Experiment Simulation Technology Based on Gesture Interaction," Journal of System Simulation: Vol. 33: Iss. 10, Article 22.
DOI: 10.16182/j.issn1004731x.joss.20-0558
Available at: https://dc-china-simulation.researchcommons.org/journal/vol33/iss10/22

First Page

2488

Revised Date

2020-09-22

DOI Link

https://doi.org/10.16182/j.issn1004731x.joss.20-0558

Last Page

2498

CLC

TP391.9

Recommended Citation

Zou Xuyan, He Hanwu, Wu Yueming, Deng Jingwei. Virtual and Real Fusion Experiment Simulation Technology Based on Gesture Interaction[J]. Journal of System Simulation, 2021, 33(10): 2488-2498.

DOI

10.16182/j.issn1004731x.joss.20-0558