Supportive Environment for Better Data Management Stage in the Cycle of ML Process

Abstract

The objective of this study is to explore the process of developing artificial intelligence and machine learning (ML) applications to establish an optimal support environment. The primary stages of ML include problem understanding, data management (DM), model building, model deployment, and maintenance. This paper specifically focuses on examining the DM stage of ML development and the challenges it presents, as it is crucial for achieving accurate end models. During this stage, the major obstacle encountered was the scarcity of adequate data for model training, particularly in domains where data confidentiality is a concern. The work aimed to construct and enhance a framework that would assist researchers and developers in addressing the insufficiency of data during the DM stage. The framework incorporates various data augmentation techniques, enabling the generation of new data from the original dataset along with all the required files for detection challenges. This augmentation process improves the overall performance of ML applications by increasing both the quantity and quality of available data, thereby providing the model with the best possible input.

Received: 16 June 2023 | Revised: 24 July 2023 | Accepted: 9 August 2023

Conflicts of Interest

The authors declare that they have no conflicts of interest to this work.

Data Availability Statement

Data sharing is not applicable to this article as no new data were created or analyzed in this study.