A Reproducible Machine Learning Framework for Detecting High-Risk Password Behavior in Healthcare Staff

Abstract

This study develops a reproducible machine learning framework to detect self-reported high-risk password behavior among healthcare staff, using structured survey data from a private hospital in Phuket. After deterministic cleaning and duplicate removal, 362 unique responses were retained from 416 initial records. The target variable was defined as a binary risk proxy indicating endorsement of at least two insecure password practices or unsafe security beliefs. Four supervised models, support vector machine (SVM), Random Forest, gradient-boosted trees, and logistic regression, including a TF-IDF-based baseline, were evaluated using stratified five-fold cross–validation, alongside rule-based and majority-class baselines. Macro-F1 was used as the primary metric to address class imbalance, with weighted F1, accuracy, and area under the receiver operating characteristic curve reported as complementary measures. Random Forest achieved the highest macro-F1 (0.678 ± 0.091) and weighted F1 (0.795 ± 0.058), indicating the best overall balance between minority and majority-class prediction. However, paired statistical tests showed that its advantage over SVM and logistic regression was not statistically significant ( p > 0.05), although it significantly outperformed weaker baselines ( p < 0.05; Cohen’s d = 1.25–2.63). Feature–group ablation showed that knowledge and perception variables contributed the largest gains in performance, whereas demographic and contextual variables had limited impact. Receiver operating characteristic analysis indicated moderate discriminative ability (area under the curve up to 0.707), and confusion matrices showed higher specificity than sensitivity. The findings demonstrate that a leak–free, statistically validated, and interpretable machine learning framework can support meaningful but cautious risk stratification from healthcare survey data.

Received: 19 April 2026 | Revised: 16 June 2026 | Accepted: 24 June 2026

Conflicts of Interest

The authors declare that they have no conflicts of interest to this work.

Data Availability Statement

The data that support the findings of this study are openly available in Github at https://github.com/pjarupunphol/Datasets/blob/main/dataset.zip.

Author Contribution Statement

Pita Jarupunphol: Conceptualization, Methodology, Validation, Formal analysis, Writing – original draft, Writing – review & editing, Supervision, Project administration. Siwatchaya Suksai: Software, Validation, Investigation, Resources, Data curation, Visualization.