Real-Time Dual-Arm Teleoperation via Similarity-Transform Mapping

Abstract

Bimanual teleoperation of collaborative robots typically requires discrete gesture vocabularies, mode-switching, or offline-trained models, all of which interrupt task flow and burden the operator. This paper presents a real-time, markerless framework for continuous, synchronized dual-arm teleoperation of two Universal Robots 3e (UR3e) collaborative robots using a single Leap Motion Controller. A one-shot, closed-form similarity-transform calibration, derived from the Umeyama algorithm, aligns the sensor frame to each robot base from as few as two non-collinear point pairs in under 0.5 ms, eliminating iterative registration and per-session recalibration. A multi-threaded Python pipeline connecting hand-pose acquisition, signal filtering, and the Real-Time Data Exchange interface achieves a mean end-to-end response of 178 ms, a mean positional error of 8.1 mm, and inter-arm synchronization within 30 ms, all measured with commodity hardware rather than laboratory-grade motion capture. A dedicated gripper test further confirms that binary grasp commands are transmitted with ~98% detection reliability and sub-half-second actuation latency. A layered software safety architecture, comprising gesture-based halts, occlusion detection, and workspace clamping, operates independently of network conditions and complements the robots’ built-in collision detection. Benchmarked against recent single-arm and offline-trained alternatives, the system matches or exceeds reported accuracy and latency while additionally supporting synchronized two-arm control. The open-source measurement toolkit accompanying this work enables practitioners to evaluate gesture-controlled robotic systems without specialized instrumentation, and the low-latency, field-deployable design is directly transferable to mobile manipulators operating beyond fixed manufacturing cells.

Received: 20 March 2026 | Revised: 3 July 2026 | Accepted: 17 August 2026

Conflicts of Interest

Cheryl Q Li is an Editorial Board Member for Journal of Climbing and Walking Robots and was not involved in the editorial review or the decision to publish this article. The authors declare that they have no conflicts of interest in this work.

Data Availability Statement

Data are available from the corresponding author upon reasonable request.

Author Contribution Statement

Keerthivaasan Matheswaran: Conceptualization, Methodology, Software, Validation, Formal analysis, Investigation, Data Curation, Writing – original draft, Visualization. Cheryl Q Li: Conceptualization, Resources, Writing – review & editing, Supervision, Project administration, Funding acquisition.