Optimization of Deep Neural Networks for Multivariate Data: Systematic Review and Critical Analysis of Methods, Hyperparameters, and Performance (2015–2026)

Abstract

Optimizing deep neural networks for multivariate data remains difficult: features are high-dimensional, correlated, and heterogeneous, and no existing review connects optimization strategy, hyperparameter choice, and data type into a single picture. We address this gap through a Systematic Mapping Study of 107 articles published between 2015 and 2026, drawn from Scopus, ScienceDirect, SpringerLink, and Web of Science, screened with a PRISMA-style protocol and clear inclusion/exclusion criteria. Metaheuristics dominate the corpus (49% of studies), followed by hybrid approaches (24%) and learning-based methods (18%). We catalogued over 50 optimization algorithms and grouped them into a five-level taxonomy. The learning rate turned out to be the single most influential hyperparameter, cited in nearly 40% of cases, well ahead of network depth, width, or batch size. Multivariate time series was studied far more than any other data type, making up close to half of the reviewed publications; medical/biometric and financial applications came next. Most studies still report classification metrics such as accuracy and F1-score, even for time series problems where a regression measure would often fit better. A co-occurrence analysis ties optimization families to application domains, and both a chi-square test and a Kendall rank correlation suggest these patterns are not simply due to chance. Four problems keep coming up across the literature: high computational cost, overfitting, a shortage of labeled data, and limited interpretability. These gaps shape our recommendations, which include wider use of auto-tuning methods, fairer evaluation metrics, and more attention to under-studied domains such as agriculture and industrial Internet of Things. 

Received: 2 June 2026 | Revised: 11 August 2026 | Accepted: 30 August 2026

Conflicts of Interest

The authors declare that they have no conflicts of interest to this work.

Data Availability Statement

Data available on request from the corresponding author upon reasonable request.

Author Contribution Statement

Zrouri Amira: Conceptualization, Methodology, Software, Formal analysis, Investigation, Resources, Data curation, Writing – original draft, Writing – review & editing, Visualization, Project administration. EL Farissi Ilhame: Validation, Supervision, Project administration.