A New Method for Designing Intelligent User Interfaces Through Image Analysis to Facilitate Human Interaction

Abstract

Traditional graphical user interfaces (GUIs) for human-computer interaction (HCI) often suffer from design complexity, cognitive overload, poor accessibility, long development times, and limited adaptability, typically requiring technical expertise for modifications. This study aims to democratize GUI design by introducing a novel framework called Intuitive Image-based User Interface (IIUI). The primary objective is to enable users to intuitively design interactive interfaces directly from digital images without requiring advanced programming knowledge. The IIUI system leverages state-of-the-art image recognition and computer vision algorithms to automatically detect and assign interactive widgets within user-defined regions of digital images. The system also facilitates dynamic interaction assignment through a streamlined user interface. The effectiveness of the widget assignment process was evaluated through experimental validation. Experimental validation of the IIUI system demonstrated a high widget assignment accuracy exceeding 99.8%. The IIUI approach significantly reduces the technical barrier to GUI creation, offering a faster and more inclusive design process compared to traditional methods. Practical implementations across diverse domains such as assistive technologies, industrial shape detection, and robotic control highlight the adaptability of the system. The IIUI framework represents a transformative step toward accessible and intelligent GUI design. By integrating AI-driven image recognition with user-centered design principles, it provides a highly intuitive method for interface creation. The high performance and versatility of the system suggest strong potential for widespread adoption across various industrial and social contexts.

Received: 26 August 2025 | Revised: 29 December 2025 | Accepted: 13 January 2026

Conflicts of Interest

The authors declare that they have no conflicts of interest to this work.

Data Availability Statement

Data sharing is not applicable to this article as no new data were created or analyzed in this study.

Author Contribution Statement

Naji Guedri: Conceptualization, Software, Validation, Formal analysis, Writing – original draft, Writing – review & editing. Abdessalam Guedri: Investigation, Resources, Writing – review & editing. Boulbaba Guedri: Methodology, Data curation. Rached Gharbi: Supervision, Project administration.