Dynamic Hand Gesture Recognition for Human–Computer Interaction

Abstract

Hand gesture recognition (HGR) is a crucial technology for human–computer interaction, which allows users to control devices such as a smart TV by using only simple hand gestures without contacting them. However, getting real-time performance on resource-constrained devices remains a big hurdle. We introduce a keypoint-based dynamic HGR approach that leverages fixed-length sequences of 30 frames to obtain three-dimensional hand landmarks using MediaPipe Holistic. Data augmentation and standardization were used to make the model robust, and the dataset contained 10 classes of dynamic gestures. Several models, such as long short-term memory, gated recurrent unit (GRU), and one-dimensional convolutional neural network hybrids, both with and without attention mechanisms, were proposed, trained, and evaluated. Through experiments, it is discovered that GRU with attention has an optimal trade-off between classification accuracy and inference delay. To facilitate deployment onto embedded platforms, post-training quantization has been used to reduce the model size and accelerate inference. The optimized model is integrated with an Android TV application through a socket connection that provides real-time gesture-to-command mapping for navigation, selection, and volume control; evaluations of the optimized model using the emulator show that it is functional and has consistent performance characteristics, while the use of pointers does not entail a significant delay because of the small communication overhead. The results indicate that keypoint-based temporal modeling is a promising approach for real-time gesture-driven interaction with embedded systems in controlled deployment scenarios.

Received: 30 August 2025 | Revised: 11 February 2026 | Accepted: 11 June 2026

Conflicts of Interest

The authors declare that they have no conflicts of interest to this work.

Data Availability Statement

Data available on request from the corresponding author upon reasonable request.

Author Contribution Statement

Chim Tun Sian: Conceptualization, Methodology, Software, Investigation, Data curation, Writing – original draft. Suhardi Azliy Junoh: Validation, Formal analysis, Resources, Data curation, Writing – review & editing, Supervision, Project administration. Mohd Shahrimie Mohd Asaari: Methodology, Validation, Formal analysis, Investigation, Resources, Writing – review & editing, Visualization, Funding acquisition.