Review of the Performance of CNN Models on Handwriting Recognition

Abstract

Handwritten text recognition continues to be a dynamic area of research, with practical uses such as processing bank checks, digitizing historical manuscripts, enabling handwriting-based user interfaces, and other OCR-related applications. Despite its potential, the task remains difficult because of a wide variety of handwriting styles, stroke patterns, and visual structures found across different writing systems. This research provides a comprehensive comparative study of deep convolutional neural network (CNN) architectures for handwritten character and word recognition of world popular scripts such as Roman (English), Devanagari, Bengali, Tamil, Telugu, Hiragana, and Arabic. Most recent and popular works on CNNs are considered. We conduct comprehensive benchmarking of widely used CNN architectures—such as VGG, ResNet, and Inception—on handwritten datasets spanning multiple scripts. Our experimental findings yield important information regarding comparative performances under diverse conditions, in addition to insights regarding the impact due to architectural extensions, i.e., attention mechanisms and regularization schemes, to recognition performance. Ensemble schemes, i.e., majority voting and stacking to obtain additional boost in performance, yield measurable increments in prediction faithfulness. Our investigation encompasses all training, validation, and testing stages and discovers key patterns such as overfitting tendencies, specifically for scripts with high visual complexity. These observations emphasize careful model selection and provide practical suggestions regarding designing robust, script-aware CNNs for multilingual handwritten text recognition.

Received: 30 April 2024 | Revised: 26 March 2025 | Accepted: 25 June 2025

Conflicts of Interest

Umapada Pal is an Advisory Board Member for Artificial Intelligence and Applications, and he was not involved in the editorial review or the decision to publish this article. The authors declare that they have no conflicts of interest to this work.

Data Availability Statement

The data that support the findings of this study are openly available in Ekush (Bengali) at https://doi.org/10.1007/978-981-13-9187-3_14; in the Devanagari handwritten character Database at https://archive.ics.uci.edu/dataset/389; in Kuzushiji-49 at https://github.com/rois-codh/kmnist; in the Telugu handwritten character database at https://www.pseudoaj.com/2016/05/pseudoajdataset0-telugu-handwritten.html; in EMNIST at https://www.nist.gov/itl/products-and-services/emnist-dataset; in the Arabic handwritten character dataset (AHCD) at https://github.com/AmrHendy/Arabic-Handwritten-Images-Recognition; in the Tamil handwritten word database at https://doi.org/10.1109/ICDAR.2013.162; in the English, Hindi, and Bengali handwritten word dataset at http://dx.doi.org/10.1109/ICFHR.2012.238; and in the Arabic handwritten word dataset at http://www.ifnenit.com/.

Author Contribution Statement

Sabyasachi Mazumder: Methodology, Software, Validation, Formal analysis, Investigation, Resources, Data curation, Writing – original draft, Writing – review & editing, Visualization. Sayan Neogy: Methodology, Software, Validation, Formal analysis, Investigation, Resources, Data curation, Writing – original draft, Writing – review & editing, Visualization. Sahana Das: Conceptualization, Methodology, Formal analysis, Writing – review & editing, Supervision. Kaushik Roy: Conceptualization, Methodology, Formal analysis, Resources, Writing – review & editing, Visualization, Supervision, Project administration. Umapada Pal: Conceptualization, Writing – review & editing, Supervision, Project administration.