Knowledge-Guided Multimodal Deep Learning for BIM-Based Construction Progress and Quality Verification from Site Images and Video

Abstract

Construction progress and quality verification require visual evidence to be interpreted against object identity, schedule state, and installation dependencies. This study proposes a knowledge-guided multimodal method that combines 32 × 32 synthetic image crops, a 15-variable building information modeling (BIM) vector, five schedule variables, neural late fusion, explicit construction-rule factors, and confidence-based abstention. A reproducible benchmark of 4,800 observations across eight synthetic project domains was evaluated using a complete seven-cell factorial analysis of image, BIM, and schedule modalities, together with recent compact visual backbones. On held-out Project 7, BIM-only achieved 43.5% accuracy and 0.309 macro-F1, schedule-only achieved 87.0% and 0.825, image + schedule achieved 89.3% and 0.859, and image + BIM + schedule achieved 87.8% and 0.833. These results show that the large schedule gain primarily recovers temporal relationships deliberately encoded in the synthetic generator and should not be interpreted as independent evidence of field-level causal superiority. The knowledge-guided output achieved 90.5% accuracy and 0.866 macro-F1 and eliminated the defined predecessor violations. At a confidence threshold of 0.65, 80.3% of observations were processed automatically at 95.6% retained accuracy. A separate synthetic defect task reached 0.958 F1 using image features, while BIM fusion reduced recall. The results support an auditable simulation-based proof of concept; field validation with independently labeled, registered site imagery remains necessary before operational deployment.

Received: 27 June 2026 | Revised: 18 August 2026 | Accepted: 30 August 2026

Conflicts of Interest

The author declares that he has no conflicts of interest related to this work.

Data Availability Statement

The data that support the findings of this study are openly available in Zenodo at https://doi.org/10.5281/zenodo.22122960.

Author Contribution Statement

Mohammed Hijazi: Conceptualization, Methodology, Software, Validation, Formal analysis, Investigation, Data curation, Writing – original draft, Writing – review & editing, Visualization, Supervision, Project administration.