Trends of Optimization Algorithms from Supervised Learning Perspective

Abstract

Machine learning (ML) is rapidly evolving, leading to numerous theoretical advancements and widespread applications across multiple fields. The goal of ML is to enable machines to carry out cognitive tasks by acquiring knowledge from past encounters and resolving intricate issues despite varying circumstances that deviate from previous instances. Supervised Learning (SL) being one of the most popular type of ML has become an area of significant strategic importance due to its practical applications, data collection, and computing power's exponential growth. On the other hand, optimization is a crucial component of ML that has garnered significant attention from researchers. Numerous proposals have been made one after another for solving optimization problems or enhancing optimization techniques in the field of ML. A comprehensive review and application of optimization methods from the perspective of ML is crucial to guide the development of both optimization and ML research. This article presents information specifically on the area of SL and a wide range of optimization methods, applied in conjunction to address various scientific issues. Additionally, this article explores some of the challenges and open problems in optimizing SL models.

Received: 8 May 2023 | Revised: 29 June 2023 | Accepted: 9 July 2023

Conflicts of Interest

The authors declare that they have no conflicts of interest to this work.

Data Availability Statement

Data sharing is not applicable to this article as no new data were created or analyzed in this study.

Author Contribution Statement

Rahul Paul: Conceptualization, Methodology, Software, Validation, Formal analysis, Investigation, Resources, Data curation, Writing – original draft, Visualization, Project administration. Kedar Nath Das: Conceptualization, Methodology, Software, Validation, Investigation, Writing – review & editing, Visualization, Supervision, Project administration.